Carotid artery unstable plaque identification method based on dynamic ultrasonic image

Abstract

The invention discloses a carotid artery unstable plaque recognition method based on a dynamic ultrasonic image, which is applied to the field of carotid artery plaque recognition and comprises the following steps: carrying out dynamic positioning operation on a carotid artery plaque on a to-be-positioned image based on a similarity score of the to-be-positioned image and a previous frame image of the to-be-positioned image; based on the dynamic positioning result of the carotid plaque, training to obtain a carotid plaque dynamic positioning identification model based on a U-net network; and adopting a pre-trained DVAENet network model to identify a plaque lipid low echo region in an output result of the carotid artery plaque dynamic positioning identification model, and obtaining a carotid artery unstable plaque identification result. According to the method, the dynamic change characteristics of the carotid plaque and information such as forms and structures of lesion characteristics can be effectively represented, and the recognition precision of the unstable carotid plaque is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of carotid artery plaque recognition, and in particular to a method for identifying unstable carotid artery plaques based on dynamic ultrasonic images. Background Art

[0002] Transient ischemic attack (TIA) is a short-term and reversible neurological deficit syndrome, and is also an important risk factor for cerebral infarction. Studies have shown that the incidence of cerebral infarction after TIA is as high as 8.0% to 10.5%. The carotid artery is the main blood vessel supplying blood to the brain, and its atherosclerotic plaque is one of the main causes of TIA and cerebral infarction. The latest research shows that 15-20% of vulnerable plaques can develop TIA, so accurately identifying unstable carotid plaques is of great significance for the prevention and clinical treatment of TIA.

[0003] With the popularization of deep neural networks in the medical field, carotid plaque recognition technology based on deep neural networks has been widely used in the prevention and clinical treatment of TIA. The existing carotid plaque recognition technology has the following problems: First, plaque recognition based on static ultrasound images is insufficient in representing dynamic change characteristics; second, the model itself is not accurate enough to represent the morphology, structure and other information of lesion characteristics in detail, especially small and irregular ultrasound lesions.

[0004] Therefore, how to provide a method for identifying unstable carotid plaques based on dynamic ultrasound images that can effectively characterize the dynamic changes of carotid plaques and effectively improve the accuracy of identifying unstable carotid plaques in view of the problem that the model cannot characterize the morphology, structure and other information of the lesion characteristics in detail is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention proposes a method for identifying unstable carotid plaques based on dynamic ultrasound images.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for identifying unstable carotid plaque based on dynamic ultrasound images, comprising:

[0008] Step (1): based on the similarity score between the image to be located and the previous frame image of the image to be located, dynamically locate the carotid artery plaque on the image to be located;

[0009] Step (2): Based on the dynamic positioning results of carotid artery plaques, a carotid artery plaque dynamic positioning and recognition model based on a U-net network is trained;

[0010] Step (3): using the pre-trained DVAENet network model, identify the plaque lipid hypoechoic area in the output result of the carotid artery plaque dynamic positioning recognition model, and obtain the carotid artery unstable plaque recognition result.

[0011] Optionally, in step (1), the similarity score is calculated as follows:

[0012] Perform first feature extraction on the image to be positioned and the previous frame image respectively to obtain first feature maps of the image to be positioned and the previous frame image;

[0013] Perform channel splicing on the image to be located and the first feature map of the previous frame image, perform feature extraction on the feature map after channel splicing, and obtain a similarity measurement feature map;

[0014] The similarity score is calculated based on the two-channel values ​​corresponding to each pixel in the similarity measurement feature map, wherein the two-channel values ​​respectively represent the horizontal difference measurement value and the vertical difference measurement value of the pixel between the image to be located and the previous frame image.

[0015] Optionally, the first feature extraction is specifically:

[0016] Feature extraction is performed through two convolutional layers with a stride of 2, and the extracted feature maps are input into three parallel convolutional layers using dilated convolution operations with dilation rates of 3, 6, and 9 respectively;

[0017] The output results of the three parallel convolutional layers are spliced ​​by channel, and the summary features are extracted through a convolutional layer. The summarized feature map and the feature map after feature extraction are spliced ​​by channel, and another convolutional layer is used to extract the summary features to complete the first feature extraction.

[0018] Optionally, the similarity score is calculated based on the two-channel values ​​corresponding to each pixel in the similarity metric feature map, as follows:

[0019]

[0020] Among them, i is the horizontal index on the similarity measurement feature map; j is the vertical index on the similarity measurement feature map; I×J is the total number of pixels on the similarity measurement feature map; α ij is the horizontal difference measurement value of the pixel point with horizontal index i and vertical index j on the similarity measurement feature map; β ij It is the vertical difference measurement value of the pixel point with horizontal index i and vertical index j on the similarity measurement feature map.

[0021] Optionally, in step (1), based on the similarity score, a dynamic positioning operation of the carotid artery plaque is performed on the image to be positioned, specifically:

[0022] If the similarity score is less than a preset threshold, feature transfer is performed on the first feature map of the image to be positioned and the previous frame image, as well as the second feature map obtained by extracting the second feature of the previous frame image, and spatial attention feature fusion is performed on the result of feature transfer and the first feature map of the image to be positioned to obtain a positioning result;

[0023] If the similarity score is greater than or equal to a preset threshold, the first feature map of the image to be located and the second feature map obtained by extracting the second feature of the image to be located are subjected to spatial attention feature fusion to obtain a positioning result.

[0024] Optionally, feature transfer is performed on the first feature map of the image to be positioned and the previous frame image, and the second feature map obtained by performing second feature extraction on the previous frame image, specifically as follows:

[0025] The first feature map of the image to be positioned and the previous frame image is spliced ​​by channel, and feature fusion and down-sampling are performed in sequence through two convolutional layers with a step size of 2 to obtain a first fused feature map and a second fused feature map;

[0026] Using two convolutional layers to extract features from the first fused feature map respectively, and fusing the two feature maps after feature extraction with the second fused feature map;

[0027] The fused features are extracted and multiplied with the second feature map of the previous frame image, and feature extraction is performed again to complete feature transfer.

[0028] Optionally, the second feature extraction is specifically:

[0029] Feature extraction is performed through five convolutional layers with a stride of 2, and an attention optimization module is set after the output of the third to fifth convolutional layers, including: calculating global semantics on each channel through global pooling, convolution integral, batch normalization and activation operations, and encoding them into attention vectors that are multiplied by the input of the attention optimization module channel by channel;

[0030] The output results of the attention optimization module are upsampled by bilinear interpolation, concatenated by channel, and the features are summarized to complete the second feature extraction; the feature map obtained after summarizing the features is the same size as the output feature map of the third convolutional layer.

[0031] Optional, spatial attention feature fusion, specifically:

[0032] Summarize the features of the feature map to be fused, and extract the spatial attention features to obtain the spatial attention feature vector;

[0033] The spatial attention feature vector and the feature map after feature aggregation are fused to complete the spatial attention feature fusion.

[0034] Optionally, in step (2), the U-net network uses an encoding-decoding structure and a skip connection structure; the encoding part and the decoding part of the U-net network adopt a symmetrical structure, and a skip connection is used to merge the encoding and decoding feature maps by channel.

[0035] Optionally, the weights of the U-net and DVAENet networks are learned using a stochastic gradient variational Bayesian estimator.

[0036] It can be seen from the above technical solutions that, compared with the prior art, the present invention proposes a method for identifying unstable carotid plaques based on dynamic ultrasound images. By dynamically positioning the carotid plaque on the image to be positioned based on the similarity score between the image to be positioned and the previous frame of the image to be positioned, the effective characterization of the dynamic change characteristics of the carotid plaque is achieved, and on this basis, the design of the U-net network followed by the cascade DVAENet is used to identify unstable carotid plaques, and the weights of the two are learned through the stochastic gradient variational Bayesian estimator, which significantly improves the topological consistency of the plaque hypoechoic area segmentation, and can describe the morphology, structure and other information of the lesion characteristics in detail; the encoding and decoding symmetric design of the U-net network, and the merging of the feature map channels of the jump connection also make this method more accurate in pixel positioning, and can effectively deal with small and irregularly shaped ultrasound lesions. In summary, the present invention effectively improves the recognition accuracy of unstable carotid plaques. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Embodiment 1:

[0041] Embodiment 1 of the present invention discloses a method for identifying unstable carotid plaques based on dynamic ultrasound images, such as Figure 1 As shown, including:

[0042] Step (1): Based on the similarity score between the image to be located and the previous frame image of the image to be located, a dynamic positioning operation of the carotid artery plaque is performed on the image to be located.

[0043] The similarity score is calculated as follows:

[0044] Perform first feature extraction on the image to be positioned and the previous frame image respectively to obtain first feature maps of the image to be positioned and the previous frame image;

[0045] The first feature map of the image to be located and the previous frame image is channel-joined, and the feature map after channel-joining is subjected to feature extraction, that is, the feature map after channel-joining is successively input into two transposed convolutional layers with a step size of 2 for upsampling, and a standard convolutional layer is set after each transposed convolutional layer to extract features, reduce the number of channels of the feature map, and obtain a similarity measurement feature map;

[0046] The similarity score is calculated based on the two-channel values ​​corresponding to each pixel in the similarity measurement feature map, wherein the two-channel values ​​respectively represent the horizontal difference measurement value and the vertical difference measurement value of the pixel between the image to be located and the previous frame image.

[0047] The first feature extraction is as follows:

[0048] Feature extraction is performed through two convolutional layers with a stride of 2, and the extracted feature maps are input into three parallel convolutional layers using dilated convolution operations with dilation rates of 3, 6, and 9 respectively; the dilated convolution operation can expand the receptive field of the convolution kernel without increasing the amount of computation, extract more global semantic information, and retain the relative spatial position information of the feature map.

[0049] The output results of the three parallel convolutional layers are spliced ​​by channel, and the summary features are extracted through a convolutional layer. The summarized feature map and the feature map after feature extraction are spliced ​​by channel, and another convolutional layer is used to extract the summary features to complete the first feature extraction.

[0050] According to the two-channel values ​​corresponding to each pixel in the similarity measurement feature map, the similarity score is calculated as follows:

[0051]

[0052] Among them, i is the horizontal index on the similarity measurement feature map; j is the vertical index on the similarity measurement feature map; I×J is the total number of pixels on the similarity measurement feature map; α ijis the horizontal difference measurement value of the pixel point with horizontal index i and vertical index j on the similarity measurement feature map; β ij It is the vertical difference measurement value of the pixel point with horizontal index i and vertical index j on the similarity measurement feature map.

[0053] Based on the similarity score, the dynamic positioning operation of the carotid artery plaque is performed on the positioning image, specifically:

[0054] If the similarity score is less than a preset threshold, feature transfer is performed on the first feature map of the image to be positioned and the previous frame image, as well as the second feature map obtained by extracting the second feature of the previous frame image, and spatial attention feature fusion is performed on the result of feature transfer and the first feature map of the image to be positioned to obtain a positioning result;

[0055] If the similarity score is greater than or equal to a preset threshold, the first feature map of the image to be located and the second feature map obtained by extracting the second feature of the image to be located are subjected to spatial attention feature fusion to obtain a positioning result.

[0056] The first feature map of the image to be positioned and the previous frame image, and the second feature map obtained by extracting the second feature of the previous frame image are subjected to feature transfer, specifically:

[0057] The first feature map of the image to be positioned and the previous frame image is spliced ​​by channel, and feature fusion and down-sampling are performed in sequence through two convolutional layers with a step size of 2 to obtain a first fused feature map and a second fused feature map;

[0058] Two convolutional layers are used to extract features from the first fused feature map respectively, and the two feature maps after feature extraction are fused with the second fused feature map; one convolutional layer includes a transposed convolutional layer, and the convolution step size in the transposed convolutional layer is set to 2; the other convolutional layer includes a convolutional layer, a transposed convolutional layer and a convolutional layer, and the convolution step size in the transposed convolutional layer is set to 2.

[0059] The fused features are extracted and multiplied with the second feature map of the previous frame image, and then the feature is extracted again to complete the feature transfer. Before multiplying with the second feature map of the previous frame image, the feature map after feature extraction is upsampled by bilinear interpolation so that its size is the same as the second feature map of the previous frame image. Figure 1 To.

[0060] By adopting the feature transfer method, the amount of calculation can be reduced while ensuring the positioning accuracy.

[0061] The second feature extraction is as follows:

[0062] Feature extraction is performed through five convolutional layers with a stride of 2, and attention optimization modules are set after the outputs of the third to fifth convolutional layers, including: calculating the global semantics on each channel through global pooling, convolution integral, batch normalization and activation operations, and encoding them into attention vectors and multiplying them by channel with the input of the attention optimization module, that is, weighting the features of each channel, and using attention vectors to guide the network to learn features distributed on each channel with different importance, thereby enhancing the network's feature extraction capability.

[0063] The output results of the attention optimization module are upsampled by bilinear interpolation, concatenated by channel, and the features are summarized to complete the second feature extraction; the feature map obtained after summarizing the features is the same size as the output feature map of the third convolutional layer.

[0064] Spatial attention feature fusion, specifically:

[0065] The feature maps to be fused are summarized and spatial attention features are extracted to obtain the spatial attention feature vector. In the spatial attention layer, the maximum and average values ​​of the feature maps after feature summary in the channel dimension are calculated to obtain two feature maps, which are then concatenated by channel and processed by the Sigmoid activation function to obtain the spatial attention feature vector.

[0066] The spatial attention feature vector and the feature map after feature aggregation are fused, that is, each parameter value of the spatial attention feature vector is multiplied in the spatial direction by the parameter value of the corresponding index position of the feature map after feature aggregation to complete the spatial attention feature fusion;

[0067] Based on the spatial attention feature fusion results, the positioning results are obtained, specifically:

[0068] The feature map after feature aggregation is added to the parameter values ​​of the corresponding channels and index positions of the spatial attention feature fusion results, and two sets of convolution-upsampling layers are used to fuse features and expand the size of the feature map (here, upsampling can also use bilinear interpolation). Based on any pixel point in the feature map, if the pixel value of the first channel of the pixel point is greater than or equal to the pixel value of the second channel of the pixel point, the pixel value of the pixel point is set to 0; if the pixel value of the first channel of the pixel point is less than the pixel value of the second channel of the pixel point, the pixel value of the pixel point is set to 255. Therefore, a positioning indication mask with a channel number of 1 can be obtained, which is the positioning result. Among them, the pixel value of the above mask is 255, which means that on the ultrasound image, the pixel point at this location is a pixel point of the carotid plaque; the pixel value of the above mask is 0, which means that the pixel point at this location is not a pixel point of the carotid plaque. Therefore, the above mask can be used to display the location outline of the plaque on the ultrasound image, that is, to obtain the positioning result, and perform corresponding plaque feature annotation.

[0069] Step (2): Based on the dynamic positioning results of carotid artery plaques, a carotid artery plaque dynamic positioning and recognition model based on the U-net network is trained.

[0070] Unlike traditional fully convolutional neural networks, the U-net network uses an encoding-decoding structure and a skip connection structure; the encoding and decoding parts of the U-net network adopt a symmetrical structure, and use skip connections to merge the encoding and decoding feature maps. This operation directly transfers the feature map of the encoding part to the decoding part, making U-Net more accurate in pixel positioning. For ultrasound lesions with small shapes and irregular morphology, this model can produce better algorithm performance.

[0071] Step (3): using the pre-trained DVAENet network model, identify the plaque lipid hypoechoic area in the output result of the carotid artery plaque dynamic positioning recognition model, and obtain the carotid artery unstable plaque recognition result.

[0072] The variational autoencoder DVAENet is parameterized by a neural network and is usually called a recognition model. For ultrasound images with carotid artery plaque lesions, it can describe in detail the morphology, structure and other information of the lesion characteristics. The design of cascading DVAENet after the UNet segmentation network can significantly improve the topological consistency of the plaque hypoechoic area segmentation, and further improve the accuracy without further increasing the complexity of the model.

[0073] The weights of the U-net network and the DVAENet network are learned through the stochastic gradient variational Bayesian estimator, which can describe in detail the morphology, structure and other information of the lesion characteristics for ultrasound images with carotid artery plaque lesions.

[0074] The embodiment of the present invention discloses a method for identifying unstable carotid plaques based on dynamic ultrasound images. By dynamically positioning the carotid plaque on the image to be positioned based on the similarity score between the image to be positioned and the previous frame of the image to be positioned, the effective characterization of the dynamic change characteristics of the carotid plaque is achieved, and on this basis, the design of the U-net network followed by the cascade DVAENet is applied to identify unstable carotid plaques, and the weights of the two are learned by the stochastic gradient variational Bayesian estimator, which significantly improves the topological consistency of the plaque hypoechoic area segmentation, and can describe the morphology, structure and other information of the lesion characteristics in detail; the encoding and decoding symmetric design of the U-net network, and the merging of the feature map channels of the jump connection also make the method more accurate in pixel positioning, and can effectively deal with small and irregularly shaped ultrasound lesions. In summary, the present invention effectively improves the recognition accuracy of unstable carotid plaques.

[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0076] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying unstable carotid plaque based on dynamic ultrasound images, characterized in that: include: Step (1): based on the similarity score between the image to be located and the previous frame image of the image to be located, dynamically locate the carotid artery plaque on the image to be located; Step (2): Based on the dynamic positioning result of the carotid artery plaque, a carotid artery plaque dynamic positioning and recognition model based on a U-net network is trained; Step (3): using the pre-trained DVAENet network model, identifying the plaque lipid hypoechoic area in the output result of the carotid artery plaque dynamic positioning recognition model, and obtaining the carotid artery unstable plaque recognition result.

2. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 1, characterized in that: In step (1), the similarity score is calculated as follows: Performing first feature extraction on the image to be located and the previous frame image respectively to obtain first feature maps of the image to be located and the previous frame image; Perform channel splicing on the image to be located and the first feature map of the previous frame image, and perform feature extraction on the feature map after channel splicing to obtain a similarity measurement feature map; The similarity score is calculated according to the two-channel values ​​corresponding to each pixel point in the similarity measurement feature map; wherein the two-channel values ​​respectively represent the horizontal difference measurement value and the vertical difference measurement value of the pixel point between the image to be located and the previous frame image.

3. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 2, characterized in that: The first feature extraction is specifically: Feature extraction is performed through two convolutional layers with a stride of 2, and the extracted feature maps are input into three parallel convolutional layers using dilated convolution operations with dilation rates of 3, 6, and 9 respectively; The output results of the three parallel convolutional layers are spliced ​​by channel, and the summary features are extracted through a convolutional layer, the summarized feature map and the feature map after the feature extraction are spliced ​​by channel, and the summary features are extracted by another convolutional layer to complete the first feature extraction.

4. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 2, characterized in that: The similarity score is calculated based on the two-channel values ​​corresponding to each pixel in the similarity measurement feature map as follows: Wherein, i is the horizontal index on the similarity measurement feature map; j is the vertical index on the similarity measurement feature map; I×J is the total number of pixels on the similarity measurement feature map; α ij is the horizontal difference measurement value of the pixel point with horizontal index i and vertical index j on the similarity measurement feature map; β ij It is the longitudinal difference measurement value of the pixel point with the horizontal index i and the vertical index j on the similarity measurement feature map.

5. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 2, characterized in that: In step (1), based on the similarity score, a dynamic positioning operation of the carotid artery plaque is performed on the image to be positioned, specifically: If the similarity score is less than a preset threshold, feature transfer is performed on the first feature map of the image to be located and the previous frame image, as well as the second feature map obtained by performing second feature extraction on the previous frame image, and spatial attention feature fusion is performed on the result of the feature transfer and the first feature map of the image to be located to obtain a positioning result; If the similarity score is greater than or equal to a preset threshold, the first feature map of the image to be located and the second feature map obtained by extracting the second feature of the image to be located are subjected to spatial attention feature fusion to obtain a positioning result.

6. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 5, characterized in that: The first feature map of the image to be located and the previous frame image, and the second feature map obtained by performing second feature extraction on the previous frame image are subjected to feature transfer, specifically: The image to be located and the first feature map of the previous frame image are spliced ​​by channel, and feature fusion and down-sampling are performed in sequence through two convolutional layers with a step size of 2 to obtain a first fused feature map and a second fused feature map; Using two convolutional layers to extract features from the first fused feature map respectively, and fusing the two feature maps after feature extraction with the second fused feature map; Feature extraction is performed on the fused features, and they are multiplied with the second feature map of the previous frame image, and feature extraction is performed again to complete the feature transfer.

7. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 5, characterized in that: The second feature extraction is specifically: Feature extraction is performed through five convolutional layers with a step size of 2, and an attention optimization module is set after the output of the third to fifth convolutional layers, including: calculating global semantics on each channel through global pooling, convolution integral, batch normalization and activation operations, and encoding them into attention vectors that are multiplied by the input of the attention optimization module by channel; The output result of the attention optimization module is upsampled by bilinear interpolation, and is spliced ​​by channel to summarize the features to complete the second feature extraction; wherein the feature map obtained after summarizing the features is consistent in size with the output feature map of the third convolutional layer.

8. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 5, characterized in that: The spatial attention feature fusion is specifically: Summarize the features of the feature map to be fused, and extract the spatial attention features to obtain the spatial attention feature vector; The spatial attention feature vector and the feature map after the feature aggregation are fused to complete the spatial attention feature fusion.

9. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 1, characterized in that: In step (2), the U-net network uses an encoding-decoding structure and a skip connection structure; the encoding part and the decoding part of the U-net network adopt a symmetrical structure, and a skip connection is used to merge the encoding and decoding feature maps into channels.

10. The method for identifying unstable carotid plaques based on dynamic ultrasound images according to claim 1, characterized in that: The weights of the U-net network and the DVAENet network are learned by a stochastic gradient variational Bayesian estimator.

Images