An intelligent segmentation method for lesion area in ultrasound images of hemangioma

Abstract

The present invention relates to the field of medical image segmentation and recognition technology, and specifically to an intelligent segmentation method for lesion areas in hemangioma ultrasound images. A codec network based on deep learning is used to accurately extract the features of the lesion area in the ultrasound image of the hemangioma, and to achieve accurate segmentation of the hemangioma lesion area and other surrounding tissues. In view of the difficult problem that the difference between the hemangioma and the surrounding boundary in the ultrasound image of the hemangioma is very small, the present invention adopts a codec structure network based on deep learning, uses ResNet‑34 as an encoder, and extracts the features of the hemangioma lesion area in stages to obtain the features of different stages. Between the same stage of the encoder and the decoder, a new attention mechanism is proposed for connection, which greatly enhances the network's attention to the pixels in the lesion area, can obtain more detailed information of the lesion area, suppress other useless information, improve the network's ability to distinguish and express, and achieve accurate segmentation of the lesion area in the ultrasound image of the hemangioma.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation and recognition, and in particular to an intelligent segmentation method for a lesion area in a hemangioma ultrasound image. Background Art

[0002] Epidemiological statistics show that the incidence of infantile hemangiomas is 10% to 12%, mainly seen in premature infants and female infants. Ultrasound examinations are non-invasive and can provide clinicians with information such as the location, shape, and range of involvement of hemangiomas, which helps guide doctors for further treatment. Therefore, accurate segmentation of the lesion area of ​​​​the hemangioma is of great significance. However, due to the diversity of the size, texture, and structure of hemangiomas, and the very small difference between the boundary between hemangiomas and surrounding tissues on ultrasound images, there is a lack of strong contrast required for segmentation of the area, which makes it very difficult. For these reasons, there is currently no method specifically for segmenting the lesion area of ​​​​the ultrasound image of hemangiomas, and it still relies on doctors to distinguish them with the naked eye from a large number of images, which is a huge workload. Summary of the invention

[0003] The present invention aims to provide an intelligent segmentation method for the lesion area in the ultrasound image of the hemangioma, using a new deep learning-based encoding and decoding network to accurately extract the characteristics of the lesion area in the ultrasound image of the hemangioma, and realize the accurate segmentation of the hemangioma lesion area and other surrounding tissues.

[0004] To achieve the above object, the present invention adopts the following technical scheme: an intelligent segmentation method for the lesion area in the ultrasound image of hemangioma, adopts a codec structure network based on deep learning, uses ResNet-34 as an encoder, first pre-trains ResNet-34 on ImageNet, gradually extracts the features of the hemangioma lesion area in stages, and obtains the features of different stages; the shallow stage contains more image detail information; the deep stage contains more semantic information as the number of downsampling and convolution increases;

[0005] An attention mechanism is provided to connect the same stage of the encoder and decoder. The feature information between the same stages of the encoder and decoder is connected through the attention mechanism, which strengthens the network's attention to the pixels in the lesion area, integrates the features of different stages, enhances the fluidity of the feature information, and realizes the accurate segmentation of the lesion area in the ultrasound image of the hemangioma.

[0006] Preferably, as an improvement, an attention mechanism is involved in connecting the encoder and the decoder, a 3x3 and a 1x1 convolution are used in series decoding at the decoder stage, and the information between the encoder and decoder at the same level is fused using addition.

[0007] Preferably, as an improvement, first use 1x1 convolution to compress the input feature channel to reduce the amount of calculation, and then use 1x1 convolution and 7x7 convolution to generate feature maps and attention values ​​respectively; when generating attention values, use 7x7 convolution to obtain a larger receptive field.

[0008] Preferably, as an improvement, the attention value is activated using a sigmoid function, the attention value is multiplied by the corresponding pixel of the feature map, the output features of the encoder are readjusted, the required feature information is enhanced, and finally a 1x1 convolution is used to restore the channel, add it to the input, and fuse the required features.

[0009] Preferably, as an improvement, a joint loss function consisting of classification cross entropy and Dice is used, and the formula is as follows, where t represents the true category label and p represents the predicted category;

[0010]

[0011] Preferably, as an improvement, the data set is hemangioma ultrasound images, the training set image size ranges from 188 to 508, and the test set image size ranges from 258 to 672.

[0012] The principles and advantages of this solution are: the present invention proposes a new deep learning-based codec network to accurately extract the features of the lesion area in the ultrasound image of the hemangioma, and realize the accurate segmentation of the hemangioma lesion area and other surrounding tissues. In view of the difficult problem that the difference between the hemangioma and the surrounding boundary in the ultrasound image of the hemangioma is very small, the present invention adopts a codec structure network based on deep learning, uses ResNet-34 as an encoder, and gradually extracts the features of the hemangioma lesion area in stages to obtain the features of different stages. A new attention mechanism is proposed to connect the same stage of the encoder and the decoder, which greatly enhances the network's attention to the pixels in the lesion area, can obtain more detailed information about the lesion area, suppress other useless information, and improve the network's ability to distinguish and express, thereby achieving more accurate segmentation of the lesion area in the ultrasound image of the hemangioma. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 4 is a neural network diagram of an embodiment of the present invention.

[0014] Figure 2 Schematic diagram of the attention mechanism of an embodiment of the present invention.

[0015] Figure 3 This is a segmentation result diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following is further described in detail through specific implementation methods:

[0017] The specific implementation process is as follows:

[0018] The intelligent segmentation method of the lesion area in the ultrasound image of the hemangioma of the present invention adopts a codec structure network based on deep learning to solve the problem that the difference between the hemangioma and the surrounding boundary in the ultrasound image of the hemangioma is very small. ResNet-34 is used as the encoder to gradually extract the features of the hemangioma lesion area in stages to obtain the features of different stages. A new attention mechanism is proposed to connect the same stage of the encoder and the decoder, which greatly enhances the network's attention to the pixels in the lesion area, can obtain more detailed information of the lesion area, suppress other useless information, and improve the network's ability to distinguish and express, thereby achieving more accurate segmentation of the lesion area in the ultrasound image of the hemangioma.

[0019] The present invention is an effective ultrasound image segmentation method for hemangioma based on deep learning. We first pre-train the residual network (ResNet-34) on ImageNet to extract features of different stages of the image. The shallow stage contains more image detail information, while the deep stage, with the increase of downsampling and convolution times, contains more semantic information. For more accurate segmentation, the feature information between the same stage of the encoder and decoder is connected through the attention mechanism, and the features of different stages are gradually fused, which enhances the fluidity of the feature information. Ultrasound images have low contrast and severe speckle noise, complex lesion areas, and blurred boundaries. In the segmentation process, it is extremely important to enhance target-related features and suppress useless features. Therefore, the present invention patent proposes an attention mechanism connection method between the encoder and decoder. In the decoder stage, a 3x3 and a 1x1 convolution are used for serial decoding, and the information between the encoder and decoder at the same level is fused using addition, which can reduce parameters and calculations compared to splicing fusion. Figure 1 This is the neural network diagram proposed by the present invention.

[0020] The attention mechanism can allocate computing resources to more important features when resources are limited. The attention value can vary depending on the input features, which is an effective way to solve information overload. Figure 2The figure shows the new attention mechanism proposed by the present invention. First, 1x1 convolution is used to compress the input features to reduce the amount of calculation. Then, 1x1 convolution and 7x7 convolution are used to generate feature maps and attention values. When generating attention values, 7x7 convolution is used to obtain a larger receptive field, which can focus on a larger range and is more conducive to generating the weight of the attention mechanism. The attention value is activated using the sigmoid function, and the attention value and the corresponding pixel point of the feature map are multiplied to readjust the output features of the encoder and enhance the required feature information. Finally, 1x1 convolution is used to restore the channel so that it can be added to the input to further fuse the required features. This module is used to reduce the response of irrelevant and noisy ambiguities in the skip connection to highlight the salient features transmitted through the skip connection.

[0021] In order to further improve the segmentation effect of the present invention, we use a joint loss function consisting of classification cross entropy and Dice, the formula is as follows, where t represents the true category label and p represents the predicted category.

[0022]

[0023] The dataset is ultrasound images of hemangiomas. The training set contains 344 images of different cases, with image sizes ranging from 188 to 508, and the test set contains 86 images, with image sizes ranging from 258 to 672. In the experimental stage, the learning rate is 0.0001, and the poly method is used for learning rate decay. The Adam optimizer is used, and the batchsize and epoch are set to 24 and 80 respectively. The ratio of the two loss functions is 1, and the image size is uniformly scaled to 256x448.

[0024] The following table shows the experimental results:

[0025]

[0026] The present invention uses Dice and IOU for evaluation. Dice mainly calculates the similarity between the true label and the predicted category, and IOU is the intersection-over-union ratio of the true area and the predicted area. When tested on the hemangioma dataset, IOU reached 80.2%, while Dice reached 81.1%.

[0027] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. An intelligent segmentation method for lesion areas in hemangioma ultrasound images, characterized by: The deep learning-based encoding and decoding structure network was adopted, and ResNet-34 was used as the encoder. First, ResNet-34 was pre-trained on ImageNet, and the features of the hemangioma lesion area were extracted step by step in stages to obtain the features of different stages; the shallow stage contains more image detail information; the deep stage contains more semantic information as the number of downsampling and convolution increases; An attention mechanism is provided to connect the same stage of the encoder and decoder. The feature information between the same stages of the encoder and decoder is connected through the attention mechanism, which strengthens the network's attention to the pixels in the lesion area, integrates the features of different stages, enhances the fluidity of the feature information, and realizes the accurate segmentation of the lesion area in the ultrasound image of the hemangioma. An attention mechanism is used to connect the encoder and decoder. A 3x3 and a 1x1 convolution are used in series decoding at the decoder stage, and the information between the encoder and decoder at the same level is fused using addition. First, use 1x1 convolution to compress the input features to reduce the amount of calculation, and then use 1x1 convolution and 7x7 convolution to generate feature maps and attention values ​​respectively; when generating attention values, use 7x7 convolution to obtain a larger receptive field; The attention value is activated using the sigmoid function, and the attention value is multiplied by the corresponding pixel of the feature map to readjust the output features of the encoder and enhance the required feature information. Finally, a 1x1 convolution is used to restore the channel and add it to the input to fuse the required features.

2. The intelligent segmentation method for lesion area in ultrasound image of hemangioma according to claim 1, characterized in that: A joint loss function consisting of categorical cross entropy and Dice is used, and the formula is as follows, where t represents the true category label and p represents the predicted category; 3. The intelligent segmentation method for lesion area in ultrasound image of hemangioma according to claim 1, characterized in that: The dataset consists of ultrasound images of hemangiomas. The image sizes of the training set range from 188 to 508, and the image sizes of the test set range from 258 to 672.

Images