Carotid artery ultrasonic scanning navigation system and method based on deep learning

Through the carotid artery ultrasound scan navigation system based on deep learning, the anatomical structure in ultrasound images is segmented using a lightweight image segmentation model, which solves the problems of primary medical personnel in positioning carotid artery bifurcation and judging the direction of the probe, and improves the examination Efficiency and accuracy enable rapid and accurate segmentation of multiple anatomical structures.

CN118924342APending Publication Date: 2024-11-12SHANGHAI SOUNDWISE TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411109354.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The lack of effective carotid ultrasound scanning navigation technology in the prior art has made it difficult for primary medical personnel to accurately locate the carotid bifurcation and judge the direction of the ultrasound probe during B-ultrasound examination, affecting the examination efficiency and accuracy.

Method used

Using a carotid artery ultrasound scanning navigation system based on deep learning, the anatomical structure in ultrasound images is segmented in real time using a lightweight image segmentation model (mobileNet-v2 and deeplab-v3), and the analysis and processing module determines whether the probe matches the current task and Whether to reach the carotid artery bifurcation position, provide real-time prompt information.

Benefits of technology

It improves the efficiency and accuracy of ultrasound scanning, reduces the requirements for technical experience, and can realize the segmentation of multiple anatomical structures in real time. It is suitable for mobile devices, has few computing resources and is fast, and has accurate segmentation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118924342A_ABST
    Figure CN118924342A_ABST
Patent Text Reader

Abstract

The invention provides a carotid artery ultrasonic scanning navigation system and a carotid artery ultrasonic scanning navigation method based on deep learning. The carotid artery ultrasonic scanning navigation method comprises the following steps: acquiring an ultrasonic image by an image acquisition module; the image segmentation module extracts an anatomical structure in the ultrasonic image by adopting a pre-trained lightweight image segmentation model to obtain an image segmentation result; the first analysis processing module performs first analysis on the image segmentation result to obtain a first analysis result, and generates first prompt information when the first analysis result is that the probe is not matched with the current scanning task; and when the first analysis result is that the probe is matched with the current scanning task, the second analysis processing module performs second analysis on the image segmentation result to obtain a second analysis result, and when the second analysis result is that current scanning reaches the carotid artery bifurcation, second prompt information is generated. The scanning efficiency is improved, few computing resources are used, computing is fast, and segmentation is accurate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image processing and handheld ultrasonic equipment, and in particular to a carotid artery ultrasonic scanning navigation system and method based on deep learning. Background Art

[0002] B-ultrasound is an important means of medical examination. The biggest difference between portable B-ultrasound and other imaging equipment is that it is light and easy to carry. Therefore, it is easy to promote to primary medical institutions.

[0003] B-ultrasound (ultrasound) is a common imaging diagnostic technology, but for primary care personnel who do not have rich experience and professional skills, it may be a challenge to perform accurate B-ultrasound examinations. B-ultrasound navigation algorithms can assist doctors in examinations, provide accurate image interpretation and diagnostic guidance, and reduce the requirements for technical experience. It can also help doctors locate and identify tissue structures, lesions, and abnormal areas in the patient's body more quickly and accurately. It can assist doctors in performing B-ultrasound examinations, optimize the placement and angle of the probe, and improve the efficiency and accuracy of the examination.

[0004] The general approach to navigation is to directly segment the target we need to navigate, and then use the segmentation results to guide the scanning doctor to complete the scanning task. However, since the main goal of carotid artery scanning is to find the location of the carotid bifurcation, an algorithm is needed to realize whether the current position has reached the carotid bifurcation. In addition, the scanning doctor often holds the ultrasound probe upside down during the scanning process, so it is also necessary to remind the doctor whether the probe is held upside down. Summary of the invention

[0005] Based on the above content, the present invention provides a carotid artery ultrasound scanning navigation system and method based on deep learning, aiming to solve technical problems such as the lack of effective navigation of carotid artery ultrasound scanning in the prior art.

[0006] A deep learning-based carotid artery ultrasound scanning navigation system, applied to handheld ultrasound equipment, including:

[0007] An image acquisition module, used for acquiring ultrasound images;

[0008] An image segmentation module, connected to the image acquisition module, is used to extract the anatomical structure in the ultrasound image by using a pre-trained lightweight image segmentation model to obtain an image segmentation result, wherein the anatomical structure includes the thyroid gland, the carotid artery and the trachea;

[0009] A first analysis and processing module, connected to the image segmentation module, is used to perform a first analysis on the image segmentation result to obtain a first analysis result, and generate a first prompt message when the first analysis result indicates that the probe does not match the current scanning task;

[0010] The second analysis and processing module is respectively connected to the first analysis and processing module and the image segmentation module, and is used to perform a second analysis on the image segmentation result to obtain a second analysis result when the first analysis result is that the probe matches the current scanning task, and to generate a second prompt message when the second analysis result is that the current scan has reached the carotid bifurcation.

[0011] Furthermore, the image segmentation models include the lightweight mobileNet-v2 model and the deeplab-v3 model;

[0012] The input of the mobileNet-v2 model is used to receive ultrasound images;

[0013] The output of the mobileNet-v2 model is connected to the input of the deeplab-v3 model.

[0014] The output of the deeplab-v3 model outputs the image segmentation result.

[0015] Further, the mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module, and a third downsampling module connected in sequence;

[0016] An input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256;

[0017] The first downsampling module outputs feature information of size 128×128, and the second convolution module outputs feature information of size 128×128;

[0018] The second downsampling module outputs feature information of size 64×64; the third convolution module outputs feature information of size 64×64;

[0019] The third downsampling module outputs feature information of size 32×32;

[0020] The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

[0021] Furthermore, the deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence;

[0022] The input end of the ASPP module is connected to the output end of the third down-sampling module;

[0023] The input end of the second cascade module is also connected to the output end of the second convolution module;

[0024] The first upsampling module is 4 times sampling.

[0025] Furthermore, the first analysis and processing module calculates the center point positions of the thyroid gland, the carotid artery and the trachea, and determines whether the probe matches the current scanning task based on the relationship between the center point positions of the thyroid gland, the carotid artery and the trachea.

[0026] Furthermore, when the second analysis and processing module determines that there are two connected domains in the currently scanned ultrasound image according to the image segmentation result, it is determined that the current scan has reached the carotid bifurcation.

[0027] A deep learning-based carotid artery ultrasound scanning navigation method, using the aforementioned deep learning-based carotid artery ultrasound scanning navigation system, comprising:

[0028] Step A1, collecting ultrasound images;

[0029] Step A2, using a pre-trained lightweight image segmentation model to extract the anatomical structure in the ultrasound image to obtain an image segmentation result, where the anatomical structure includes the thyroid gland, carotid artery, and trachea;

[0030] Step A3, performing a first analysis on the image segmentation result to obtain a first analysis result, and generating a first prompt message when the first analysis result indicates that the probe does not match the current scanning task;

[0031] Step A4, when the first analysis result is that the probe matches the current scanning task, a second analysis is performed on the image segmentation result to obtain a second analysis result, and when the second analysis result is that the current scan has reached the carotid bifurcation, a second prompt message is generated.

[0032] Furthermore, the image segmentation models include the lightweight mobileNet-v2 model and the deeplab-v3 model;

[0033] The input of the mobileNet-v2 model is used to receive ultrasound images;

[0034] The output of the mobileNet-v2 model is connected to the input of the deeplab-v3 model.

[0035] The output of the deeplab-v3 model outputs the image segmentation result.

[0036] Furthermore, the mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module, and a fifth convolution module connected in sequence;

[0037] An input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256;

[0038] The first downsampling module outputs feature information of size 128×128, and the second convolution module outputs feature information of size 128×128;

[0039] The second downsampling module outputs feature information of size 64×64; the third convolution module outputs feature information of size 64×64;

[0040] The third downsampling module outputs feature information of size 32×32;

[0041] The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

[0042] Furthermore, the deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence;

[0043] The input end of the ASPP module is connected to the output end of the third down-sampling module;

[0044] The input end of the second cascade module is also connected to the output end of the second convolution module;

[0045] The first upsampling module is 4 times sampling.

[0046] The beneficial technical effect of the present invention is that: based on the deep learning method, multiple anatomical structures can be segmented in real time. The doctor can judge whether the moving direction of the ultrasound probe meets the requirements according to the current segmentation results, and decide whether to end the scanning task according to whether the carotid bifurcation position is scanned, thereby improving the scanning efficiency, using less computing resources, fast calculation, and accurate segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a module of a carotid artery ultrasound scanning navigation system based on deep learning of the present invention;

[0048] Figure 2 A schematic diagram of the architecture of a carotid artery ultrasound scanning navigation system based on deep learning of the present invention;

[0049] Figure 3 It is a schematic diagram of the contours before and after filtering in a carotid artery ultrasound scanning navigation system based on deep learning of the present invention;

[0050] Figure 4 This is a flowchart of the steps of a deep learning-based carotid artery ultrasound scanning navigation method of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0053] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0054] See also Figure 1 The present invention provides a carotid artery ultrasound scanning navigation system based on deep learning, which is applied to a handheld ultrasound device, including:

[0055] An image acquisition module (1), used for acquiring ultrasound images;

[0056] An image segmentation module (2) is connected to the image acquisition module (1) and is used to extract the anatomical structure in the ultrasound image using a pre-trained lightweight image segmentation model to obtain an image segmentation result, wherein the anatomical structure includes a thyroid gland, a carotid artery and a trachea;

[0057] A first analysis and processing module (3), connected to the image segmentation module (2), is used to perform a first analysis on the image segmentation result to obtain a first analysis result, and to generate a first prompt message when the first analysis result indicates that the probe does not match the current scanning task;

[0058] The second analysis and processing module (4) is respectively connected to the first analysis and processing module (3) and the image segmentation module (2), and is used to perform a second analysis on the image segmentation result to obtain a second analysis result when the first analysis result shows that the probe matches the current scanning task, and to generate a second prompt message when the second analysis result shows that the current scanning has reached the carotid bifurcation.

[0059] The deep learning method can segment various anatomical structures in real time. Doctors can judge whether the movement direction of the ultrasound probe meets the requirements based on the current segmentation results, and decide whether to end the scanning task based on whether the carotid bifurcation is scanned, thereby improving the scanning efficiency.

[0060] The image segmentation model is a lightweight model that can be well adapted to mobile devices such as mobile phones and tablets. These mobile devices are connected to ultrasound probes, which collect ultrasound images for processing by the ultrasound probes. Of course, it can also be directly inherited in handheld ultrasound devices without the need for mobile devices.

[0061] Furthermore, the image segmentation models include the lightweight mobileNet-v2 model and the deeplab-v3 model;

[0062] The input of the mobileNet-v2 model is used to receive ultrasound images;

[0063] The output of the mobileNet-v2 model is connected to the input of the deeplab-v3 model.

[0064] The output of the deeplab-v3 model outputs the image segmentation result.

[0065] See also Figure 2 ,Further, the mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module and a third downsampling module that are sequentially connected;

[0066] An input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256;

[0067] The first downsampling module outputs feature information of size 128×128, and the second convolution module outputs feature information of size 128×128;

[0068] The second downsampling module outputs feature information of size 64×64; the third convolution module outputs feature information of size 64×64;

[0069] The third downsampling module outputs feature information of size 32×32;

[0070] The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

[0071] The first convolution module, the second convolution module and the third convolution module are depth-separable convolution modules, which perform convolution operations on the image, and the first to third downsampling modules perform downsampling operations on the feature image. The model used in the first stage of the image segmentation model is mobileNet-v2, which is a lightweight backbone. Its input is an image of size 256×256. After continuous depth-separable convolution modules and downsampling modules, feature information of sizes 128×128, 64×64, and 32×32 can be obtained. This stage is called the encoding stage of the image, which encodes the image features into depth features. After the encoding is completed, the final feature layer size is 32×32.

[0072] Depthwise Separable Convolution is an efficient convolution operation used in convolutional neural networks (CNNs) to reduce the amount of computation and the number of parameters while keeping the network performance as unchanged as possible. This convolution module was first proposed in MobileNets and has been widely used in computer vision tasks on mobile devices and other resource-constrained environments.

[0073] Furthermore, the deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence;

[0074] The input end of the ASPP module is connected to the output end of the third down-sampling module;

[0075] The input end of the second cascade module is also connected to the output end of the second convolution module;

[0076] The first upsampling module performs 4-fold sampling, and the second upsampling module performs 2-fold sampling.

[0077] The ASPP module (Atrous spatial pyramid pooling) is a technique used in convolutional neural networks (CNNs), especially for semantic segmentation tasks, to improve the network's perception of objects of different scales. ASPP effectively captures multi-scale contextual information by using atrous convolutions with different dilation rates to capture information of different scales. The core idea of ​​the ASPP module is to capture information of different scales by using atrous convolutions with different dilation rates. Atrous convolution can expand the receptive field without reducing the size of the feature map, thereby capturing a wider range of contextual information.

[0078] The second stage of the image segmentation model uses the lightweight deeplab-v3. The deeplab-v3 model uses the ASPP module to obtain 5 sets of features through 5 different convolution kernels for deep features, and then cascades these 5 sets of features through the first cascade module, and then upsamples the cascaded features by 4 times to obtain feature information with a feature size of 128×128. In order to fuse better feature information, the 128×128 feature information in the encoding stage is cascaded through the second cascade module, and then the convolution operation of the fifth convolution module is used to completely fuse the two feature information to obtain a heatmap with 4 channels (representing background, thyroid, carotid artery, and trachea, respectively), and a feature map size of 128×128.

[0079] Furthermore, the image segmentation module (2) also includes a filtering unit, which is connected to the output end of the deeplab-v3 model and is used to perform Gaussian filtering on the contour of the anatomical structure segmented by the image segmentation result output by the deeplab-v3 model to obtain the final image segmentation result.

[0080] The filtering unit is a 1-dimensional Gaussian filter.

[0081] Then, due to the limited computing resources of mobile devices, there are some differences in the training and prediction stages of the model. During the training process, a 256×256 image is input, and the output is a bilinear interpolation of 2x upsampling in the 128×128 heatmap feature map (the second sampling module), that is, the output is a 256×256 heatmap map, so two lightweight models are trained. In the actual use of the image segmentation stage, directly using the 256×256 segmentation result will take up a lot of computing resources. Therefore, in the actual use of the training process, the 128×128 heatmap map is directly output by the fifth convolution module. This can save a lot of computing resources, but it will cause the segmentation to be more jagged. Therefore, after extracting the contours of the segmentation results, a 1-dimensional Gaussian filter is used to perform Gaussian filtering on the horizontal and vertical coordinates of the contour coordinate points, and the contour coordinates after Gaussian filtering are used to obtain the final contour coordinates to obtain the final image segmentation result. The comparison effect is shown as follows. Figure 3 As shown, the left picture is the contour before filtering, and the right picture is the contour after filtering.

[0082] Furthermore, the first analysis and processing module (3) calculates the center point positions of the thyroid gland, the carotid artery and the trachea, and determines whether the probe matches the current scanning task based on the relationship between the center point positions of the thyroid gland, the carotid artery and the trachea.

[0083] The present invention is based on a deep learning method and can achieve the segmentation of multiple anatomical structures in real time. The doctor can judge the moving direction of the ultrasound probe according to the current segmentation result. And during the scanning process, there may be a situation where the probe is held upside down, so the doctor can be judged whether the probe is held upside down based on the prior knowledge of left transverse section or right transverse section to assist the doctor in making a correct decision quickly. After the image segmentation is completed, the center point positions of the thyroid gland, carotid artery and trachea are automatically calculated. At this time, the center point position of the thyroid gland or carotid artery is judged relative to the center point of the trachea to judge whether the doctor holds the probe upside down. For example, assuming that the current scanning task is a left transverse section, the center point of the thyroid gland or trachea is located to the left of the center point of the carotid artery. At this time, the probe is not held upside down and no prompt is required; if the center point of the thyroid gland or trachea is located to the right of the center point of the carotid artery, it means that the probe is held upside down. At this time, a pop-up window will remind you that the probe is held upside down, prompting the doctor to correct it and re-scan.

[0084] Furthermore, when the second analysis and processing module (4) determines that there are two connected domains in the currently scanned ultrasound image based on the image segmentation result, it is determined that the current scan has reached the carotid artery bifurcation.

[0085] Furthermore, the second analysis and processing module (4) determines, based on the image segmentation result, that there is a connected domain in the currently scanned ultrasound image and that the aspect ratio of the blood vessel is greater than a preset value, and determines that the current scan has reached the carotid bifurcation.

[0086] The carotid artery is generally circular. If the blood vessel has a long diameter and a short diameter, and the difference between the two is large, for example, the ratio exceeds 1.6, indicating that the difference is large, it is considered that the current scan has reached the carotid bifurcation.

[0087] When the carotid bifurcation is scanned, the doctor is automatically reminded that the carotid bifurcation has been scanned and the scanning task can be completed.

[0088] Furthermore, the system also includes:

[0089] An image annotation module (5), connected to the image segmentation module (2), is used to annotate the anatomical structure in the ultrasound image according to the image segmentation result;

[0090] The display module (6) is connected to the image annotation module (5) and is used to display the annotated ultrasound image in real time.

[0091] During the scanning process, the current segmentation results are displayed in real time to assist the scanning doctor in determining the current position. The doctor continues to move to the correct position based on the current segmentation results and prompts until the carotid bifurcation is scanned. The device can remind the doctor that the carotid bifurcation has been scanned and the current scanning task can be completed.

[0092] See also Figure 4The present invention also provides a carotid artery ultrasound scanning navigation method based on deep learning, characterized in that the aforementioned carotid artery ultrasound scanning navigation system based on deep learning is used, comprising:

[0093] Step A1, collecting ultrasound images;

[0094] Step A2, using a pre-trained lightweight image segmentation model to extract the anatomical structure in the ultrasound image to obtain an image segmentation result, where the anatomical structure includes the thyroid gland, carotid artery, and trachea;

[0095] Step A3, performing a first analysis on the image segmentation result to obtain a first analysis result, and generating a first prompt message when the first analysis result indicates that the probe does not match the current scanning task;

[0096] Step A4, when the first analysis result is that the probe matches the current scanning task, a second analysis is performed on the image segmentation result to obtain a second analysis result, and when the second analysis result is that the current scan has reached the carotid bifurcation, a second prompt message is generated.

[0097] The image segmentation model is a lightweight model that can be well adapted to mobile devices such as mobile phones and tablets. These mobile devices are connected to ultrasound probes, which collect ultrasound images for processing by the ultrasound probes. Of course, it can also be directly inherited in handheld ultrasound devices without the need for mobile devices.

[0098] Furthermore, the image segmentation models include the lightweight mobileNet-v2 model and the deeplab-v3 model;

[0099] The input of the mobileNet-v2 model is used to receive ultrasound images;

[0100] The output of the mobileNet-v2 model is connected to the input of the deeplab-v3 model.

[0101] The output of the deeplab-v3 model outputs the image segmentation result.

[0102] Further, the mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module, and a third downsampling module connected in sequence;

[0103] An input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256;

[0104] The first downsampling module outputs feature information of size 128×128, and the second convolution module outputs feature information of size 128×128;

[0105] The second downsampling module outputs feature information of size 64×64; the third convolution module outputs feature information of size 64×64;

[0106] The third downsampling module outputs feature information of size 32×32;

[0107] The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

[0108] The first convolution module, the second convolution module and the third convolution module are depth-separable convolution modules, which perform convolution operations on the image, and the first to third downsampling modules perform downsampling operations on the feature image. The model used in the first stage of the image segmentation model is mobileNet-v2, which is a lightweight backbone. Its input is an image of size 256×256. After continuous depth-separable convolution modules and downsampling modules, feature information of sizes 128×128, 64×64, and 32×32 can be obtained. This stage is called the encoding stage of the image, which encodes the image features into depth features. After the encoding is completed, the final feature layer size is 32×32.

[0109] Depthwise Separable Convolution is an efficient convolution operation used in convolutional neural networks (CNNs) to reduce the amount of computation and the number of parameters while keeping the network performance as unchanged as possible. This convolution module was first proposed in MobileNets and has been widely used in computer vision tasks on mobile devices and other resource-constrained environments.

[0110] Furthermore, the deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence;

[0111] The input end of the ASPP module is connected to the output end of the third down-sampling module;

[0112] The input end of the second cascade module is also connected to the output end of the second convolution module;

[0113] The first upsampling module performs 4-fold sampling, and the second upsampling module performs 2-fold sampling.

[0114] The ASPP module (Atrous spatial pyramid pooling) is a technique used in convolutional neural networks (CNNs), especially for semantic segmentation tasks, to improve the network's perception of objects of different scales. ASPP effectively captures multi-scale contextual information by using atrous convolutions with different dilation rates to capture information of different scales. The core idea of ​​the ASPP module is to capture information of different scales by using atrous convolutions with different dilation rates. Atrous convolution can expand the receptive field without reducing the size of the feature map, thereby capturing a wider range of contextual information.

[0115] The second stage of the image segmentation model uses the lightweight deeplab-v3. The deeplab-v3 model uses the ASPP module to obtain 5 sets of features through 5 different convolution kernels for deep features, and then cascades these 5 sets of features through the first cascade module, and then upsamples the cascaded features by 4 times to obtain feature information with a feature size of 128×128. In order to fuse better feature information, the 128×128 feature information in the encoding stage is cascaded through the second cascade module, and then the convolution operation of the fifth convolution module is used to completely fuse the two feature information to obtain a heatmap with 4 channels (representing background, thyroid, carotid artery, and trachea, respectively), and a feature map size of 128×128.

[0116] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A carotid artery ultrasound scanning navigation system based on deep learning, applied to a handheld ultrasound device, characterized in that: include: An image acquisition module, used for acquiring ultrasound images; An image segmentation module, connected to the image acquisition module, for extracting the anatomical structure in the ultrasound image using a pre-trained lightweight image segmentation model to obtain an image segmentation result, wherein the anatomical structure includes a thyroid gland, a carotid artery, and a trachea; a first analysis and processing module, connected to the image segmentation module, configured to perform a first analysis on the image segmentation result to obtain a first analysis result, and generate a first prompt message when the first analysis result indicates that the probe does not match the current scanning task; The second analysis and processing module is respectively connected to the first analysis and processing module and the image segmentation module, and is used to perform a second analysis on the image segmentation result to obtain a second analysis result when the first analysis result is that the probe matches the current scanning task, and to generate a second prompt message when the second analysis result is that the current scan has reached the carotid bifurcation.

2. The deep learning-based carotid artery ultrasound scanning navigation system according to claim 1, characterized in that: The image segmentation model includes a lightweight mobileNet-v2 model and a deeplab-v3 model; The input end of the mobileNet-v2 model is used to receive the ultrasound image; The output end of the mobileNet-v2 model is connected to the input end of the deeplab-v3 model. The output end of the deeplab-v3 model outputs the image segmentation result.

3. The deep learning-based carotid artery ultrasound scanning navigation system according to claim 2, characterized in that: The mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module and a third downsampling module connected in sequence; The input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256; The first downsampling module outputs feature information of a size of 128×128, and the second convolution module outputs feature information of a size of 128×128; The second downsampling module outputs feature information of a size of 64×64; the third convolution module outputs feature information of a size of 64×64; The third down-sampling module outputs feature information of size 32×32; The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

4. The deep learning-based carotid artery ultrasound scanning navigation system according to claim 3, characterized in that: The deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence; The input end of the ASPP module is connected to the output end of the third down-sampling module; The input end of the second cascade module is also connected to the output end of the second convolution module; The first upsampling module is 4 times sampling.

5. The deep learning-based carotid artery ultrasound scanning navigation system according to claim 1, characterized in that: The first analysis and processing module calculates the center point positions of the thyroid gland, the carotid artery and the trachea, and determines whether the probe matches the current scanning task based on the relationship between the center point positions of the thyroid gland, the carotid artery and the trachea.

6. The deep learning-based carotid artery ultrasound scanning navigation system according to claim 1, characterized in that: When the second analysis and processing module determines that there are two connected domains in the ultrasound image currently scanned according to the image segmentation result, it is determined that the current scan has reached the carotid artery bifurcation.

7. A carotid artery ultrasound scanning navigation method based on deep learning, characterized in that: A carotid artery ultrasound scanning navigation system based on deep learning as described in any one of claims 1 to 6, comprising: Step A1, collecting ultrasound images; Step A2, using a pre-trained lightweight image segmentation model to extract the anatomical structure in the ultrasound image to obtain an image segmentation result, wherein the anatomical structure includes a thyroid gland, a carotid artery, and a trachea; Step A3, performing a first analysis on the image segmentation result to obtain a first analysis result, and generating a first prompt message when the first analysis result indicates that the probe does not match the current scanning task; Step A4, when the first analysis result is that the probe matches the current scanning task, the image segmentation result is analyzed for the second time to obtain a second analysis result, and when the second analysis result is that the current scan has reached the carotid bifurcation, a second prompt message is generated.

8. The deep learning-based carotid artery ultrasound scanning navigation method according to claim 7, characterized in that: The image segmentation model includes a lightweight mobileNet-v2 model and a deeplab-v3 model; The input end of the mobileNet-v2 model is used to receive the ultrasound image; The output end of the mobileNet-v2 model is connected to the input end of the deeplab-v3 model. The output end of the deeplab-v3 model outputs the image segmentation result.

9. The deep learning-based carotid artery ultrasound scanning navigation method according to claim 8, characterized in that: The mobileNet-v2 model includes a first convolution module, a first downsampling module, a second convolution module, a second downsampling module, a third convolution module and a third downsampling module connected in sequence; The input end of the first convolution module receives an ultrasound image of a size of 256×256, and the first convolution module outputs feature information of a size of 256×256; The first downsampling module outputs feature information of a size of 128×128, and the second convolution module outputs feature information of a size of 128×128; The second downsampling module outputs feature information of a size of 64×64; the third convolution module outputs feature information of a size of 64×64; The third down-sampling module outputs feature information of size 32×32; The 128×128 feature information output by the second convolution module in the mobileNet-v2 model and the 32×32 feature information output by the third downsampling module are input into the deeplab-v3 model for processing, and the deeplab-v3 model outputs the image segmentation result.

10. The deep learning-based carotid artery ultrasound scanning navigation method according to claim 9, characterized in that: The deeplab-v3 model includes an ASPP module, a first cascade module, a fourth convolution module, a first upsampling module, a second cascade module, and a fifth convolution module connected in sequence; The input end of the ASPP module is connected to the output end of the third down-sampling module; The input end of the second cascade module is also connected to the output end of the second convolution module; The first upsampling module is 4 times sampling.