Real-time guiding method for B ultrasonic image based on deep learning

Abstract

The invention relates to the technical field of ultrasonic scanning, in particular to a real-time guiding method for a B-ultrasonic image based on deep learning, which comprises the following steps: S1, acquiring an input ultrasonic image, judging whether the input ultrasonic image contains a thyroid gland or not, and turning to S2 when the input ultrasonic image does not contain the thyroid gland; s2, identifying the input ultrasonic image and generating direction guide information corresponding to the thyroid gland; s3, returning to the step S1 to obtain the input ultrasonic image again until the thyroid gland is scanned; in the step S1 and the step S2, an image guide model is adopted to identify the input ultrasonic image. In order to solve the problem of poor scanning effect possibly caused by the fact that a doctor cannot correctly align a scanning part in the ultrasonic examination process in the prior art, an image guide model is introduced in the scheme to recognize an input ultrasonic image, and a closed-loop control process of prompting and adjusting the scanning direction is introduced, so that accurate adjustment of the scanning process is realized.

Description

Technical Field

[0001] The present invention relates to the field of ultrasonic scanning technology, and in particular to a real-time guidance method for B-ultrasound images based on deep learning. Background Art

[0002] Ultrasound scanning, or medical ultrasound examination, is a medical imaging diagnostic technology based on ultrasound. This technology is based on ultrasound and can achieve visualization of the patient's muscle tissue and internal organs outside the patient's body - including size, structure and pathological lesions, to assist doctors in making effective diagnosis of the patient's disease. Ultrasound scanning equipment refers to medical instruments that apply the above-mentioned ultrasound scanning technology, which uses ultrasound to image the various tissue parts of the patient to assist in medical diagnosis. In order to achieve a better diagnostic effect on the disease, it is usually necessary to compile a corresponding ultrasound scanning diagnostic process and perform the scanning process according to a specific process. For example, for thyroid scanning, it is usually necessary to design the corresponding scanning direction, sequence, and judgment branches, such as how to deal with the nodule site when scanning.

[0003] In the prior art, there are already inspection schemes based on ultrasound. For example, Chinese patent CN202010903937.5 provides an ultrasonic inspection method, which can inspect the object to be inspected even if ultrasonic waves directly arriving through the medium outside the object to be inspected are received in addition to ultrasonic waves propagated from the object to be inspected. An ultrasonic inspection method for inspecting an object to be inspected, wherein an ultrasonic transmitting element and an ultrasonic receiving element are arranged opposite to each other, an ultrasonic wave is transmitted from the ultrasonic transmitting element, and the transmitted ultrasonic wave is received by the ultrasonic receiving element through the object to be inspected, and the received ultrasonic wave is temporally divided into a first ultrasonic signal, which is a transmitted wave of the object to be inspected, and a second ultrasonic signal caused by a diffraction wave propagating in the air, and the object to be inspected is inspected based on the first ultrasonic signal.

[0004] However, in actual implementation, the inventors found that during the scanning process, the doctor needs to move the ultrasound probe to a specific position for scanning. If the doctor does not move the probe accurately enough, it is easy to cause poor scanning results. Summary of the invention

[0005] In view of the above problems existing in the prior art, a real-time guidance method of B-ultrasound images based on deep learning is provided.

[0006] The specific technical solutions are as follows:

[0007] A real-time guidance method for B-ultrasound images based on deep learning, comprising:

[0008] Step S1: acquiring an input ultrasound image, determining whether the input ultrasound image contains a thyroid gland, and turning to step S2 if the thyroid gland is not contained;

[0009] Step S2: identifying the input ultrasound image and generating direction guidance information corresponding to the thyroid gland;

[0010] Step S3: returning to step S1 to reacquire the input ultrasound image until the thyroid gland is scanned;

[0011] In the steps S1 and S2, an image-guided model is used to identify the input ultrasound image;.

[0012] In another aspect, the image-guided model comprises:

[0013] a mask module, wherein the mask module receives the input ultrasound image and generates a mask image according to the input ultrasound image;

[0014] a first prediction head, the first prediction head being connected to the mask module, the first prediction head receiving the input ultrasound image and generating a prediction feature according to the input ultrasound image;

[0015] In step S1, whether the first prediction head recognizes the prediction feature is used to determine whether the input ultrasound image contains the thyroid gland.

[0016] On the other hand, the mask module includes:

[0017] a first convolution module, the first convolution module receiving the input ultrasound image and generating a first convolution image;

[0018] A first down-sampling module, the first down-sampling module is connected to the first convolution module, and the first down-sampling module generates a first down-sampled image according to the first convolution image;

[0019] a second convolution module, the second convolution module being connected to the first downsampling module, and the second convolution module generating a second convolution image according to the first downsampling image;

[0020] a third convolution module, the third convolution module being connected to the second convolution module, and the third convolution module generating a third convolution image according to the second convolution image;

[0021] A first up-sampling module, wherein the first up-sampling module is connected to the third convolution module, and the first up-sampling module performs up-sampling processing according to the third convolution image to obtain the mask image.

[0022] On the other hand, the first prediction header comprises:

[0023] a second down-sampling module, wherein an input end of the second down-sampling module is connected to an output end of the second convolution module, and the fourth convolution module generates a second down-sampling image according to the second convolution image;

[0024] a fourth convolution module, the fourth convolution module being connected to the second down-sampling module, and the fourth convolution module generating a fourth convolution image according to the second down-sampling image;

[0025] a third down-sampling module, the third down-sampling module being connected to the fourth convolution module, and the third down-sampling module generating a third down-sampling image according to the fourth convolution image;

[0026] A multi-depth extraction module, the multi-depth extraction module is connected to the third down-sampling module, and the multi-depth extraction module extracts grouped image features from the third down-sampling image through a plurality of convolution kernels of different sizes;

[0027] A first cascade module, the first cascade module is connected to the multi-depth extraction module, and the first cascade module cascades the grouped image features to obtain a first cascade image;

[0028] a fifth convolution module, the fifth convolution module being connected to the cascade module, and the fifth convolution module generating a fifth convolution image according to the cascade image;

[0029] a second up-sampling module, the second up-sampling module being connected to the fifth convolution module, and the second up-sampling module generating a second up-sampling image according to the fifth convolution image;

[0030] A second cascade module, wherein the second cascade module connects the second up-sampling module and the third convolution module, and the second cascade module cascades the second up-sampled image and the third convolution image and then inputs the cascaded image into the first up-sampling module.

[0031] On the other hand, the image guidance module further includes:

[0032] a second prediction head, the second prediction head being connected to the mask module, and the second prediction head receiving the input ultrasound image and predicting the direction guidance information;

[0033] In the step S2, the input ultrasound image is input into the second prediction head to predict and obtain the direction guidance information.

[0034] On the other hand, the second prediction header comprises:

[0035] A flattening layer, the flattening layer is connected to the output end of the third down-sampling module, and the flattening layer flattens the third down-sampling image to obtain a one-dimensional feature image;

[0036] A first fully connected layer, the first fully connected layer is connected to the flattened layer, and the first fully connected layer generates a first predicted image according to the one-dimensional feature image;

[0037] a second fully connected layer, the second fully connected layer being connected to the first fully connected layer, and the second fully connected layer generating a second predicted image according to the first predicted image;

[0038] A third fully connected layer, wherein the third fully connected layer is connected to the second fully connected layer, and the third fully connected layer generates the direction guidance information according to the second predicted image.

[0039] On the other hand, the direction guidance information includes moving the probe upward, moving the probe downward, moving the probe leftward, and moving the probe rightward.

[0040] On the other hand, the step S1 comprises:

[0041] Step S11: acquiring the input ultrasound image, and using the image-guided model to identify the input ultrasound image;

[0042] Step S12: determining whether the mask image includes the thyroid gland;

[0043] If so, terminate the boot process;

[0044] If not, go to step S2.

[0045] On the other hand, the step S2 comprises:

[0046] Step S21: identifying the input ultrasound image according to the image guidance model to predict the direction guidance information;

[0047] Step S22: Rendering a prompt arrow on the mask image according to the direction guidance information.

[0048] The above technical solution has the following advantages or beneficial effects:

[0049] In order to solve the problem in the prior art that the scanning effect may be poor because the doctor cannot correctly aim at the scanning part during the ultrasound examination, this solution introduces an image-guided model to identify the input ultrasound image, prompt and adjust the scanning direction in a closed-loop control process, thereby achieving accurate adjustment of the scanning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0051] Figure 1is an overall schematic diagram of an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of a model in an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a second prediction head in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of a mask module in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of step S1 in an embodiment of the present invention; Figure 6 Schematic diagram of step S2 in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0058] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0059] The present invention comprises:

[0060] A real-time guidance method for B-ultrasound images based on deep learning, such as Figure 1 As shown, including:

[0061] Step S1: acquiring an input ultrasound image, determining whether the input ultrasound image contains the thyroid gland, and turning to step S2 if the thyroid gland is not contained;

[0062] Step S2: Identify the input ultrasound image and generate direction guidance information corresponding to the thyroid gland;

[0063] Step S3: Return to step S1 to reacquire the input ultrasound image until the thyroid gland is scanned;

[0064] In steps S1 and S2, an image-guided model is used to recognize the input ultrasound image.

[0065] Specifically, in order to address the problem in the prior art that the scanning effect may be poor because the doctor cannot correctly aim at the scanning part during the ultrasound examination, this solution introduces an image-guided model to identify the input ultrasound image, prompt and adjust the scanning direction in a closed-loop control process, thereby achieving accurate adjustment of the scanning process.

[0066] During the implementation process, the above technical solution is mainly configured as a software implementation in a computer device, such as an ultrasound scanning device or an ultrasound inspection workstation, for connecting an ultrasound probe through a communication interface and obtaining a real-time reconstructed input ultrasound image, and then processing and identifying the input ultrasound image according to the above-mentioned real-time guidance method, so as to determine whether there is a corresponding thyroid gland in the image, and generate direction guidance information when no thyroid gland exists, until the target is scanned.

[0067] In one embodiment, Figure 2 As shown, the image-guided model includes:

[0068] The mask module 1 receives an input ultrasound image and generates a mask image according to the input ultrasound image;

[0069] A first prediction head 2, the first prediction head 2 is connected to the mask module 1, and the first prediction head 2 receives an input ultrasound image and generates a prediction feature according to the input ultrasound image;

[0070] In step S1, it is determined whether the input ultrasound image contains the thyroid gland by determining whether the first prediction head 2 recognizes the prediction feature.

[0071] Specifically, in order to achieve a better guidance effect, in this embodiment, an image guidance model is first constructed, wherein the image guidance model includes a mask module 1 and a first prediction head 2. The mask module 1 receives the input ultrasound image and performs preprocessing, and then the first prediction head 2 receives the input ultrasound image and generates prediction features according to the input ultrasound image, and finally returns to the mask module 1 to generate a mask image together with the features obtained by the previous preprocessing, so as to mark the features identified in the image, mainly including the thyroid gland and surrounding tissues.

[0072] In this process, if the first prediction head 2 fails to identify the thyroid gland and surrounding tissues according to the input ultrasound image, the generated verification image does not contain the corresponding annotation information. At this time, it is determined that the input ultrasound image generated by the current scan does not contain the thyroid gland.

[0073] In one embodiment, Figure 3 As shown, the image-guided model also includes:

[0074] A second prediction head 3, the second prediction head 3 is connected to the first prediction head 2, and the second prediction head 3 receives an input ultrasound image and predicts and obtains direction guidance information;

[0075] In step S2, the input ultrasound image is input into the second prediction head 3 to predict and obtain direction guidance information.

[0076] Specifically, in view of the problem that the doctor may not be able to correctly align the scanning part during the ultrasound examination in the prior art, resulting in poor scanning effect, this solution also introduces a second prediction head 3, which receives the input ultrasound image and extracts the tissue features based on the input ultrasound image, thereby predicting the direction guidance information. At this time, in order to achieve better feature reuse, it is selected to perform pre-extraction on the mask module 1 and preliminary feature extraction on the first prediction head 2, and then input the extracted image features, especially the image features corresponding to the tissue area, into the second prediction head 3, so that the second prediction head 3 can directly predict the corresponding adjustment direction according to the tissue features extracted in the early stage, thereby achieving better feature reuse and shallower model depth.

[0077] To achieve the above process, the model can be implemented in the following ways:

[0078] In this embodiment, if Figure 4 As shown, the mask module 1 includes:

[0079] A first convolution module 11, the first convolution module 11 receives an input ultrasound image and generates a first convolution image;

[0080] A first down-sampling module 12, the first down-sampling module 12 is connected to the first convolution module 11, and the first down-sampling module 12 generates a first down-sampled image according to the first convolution image;

[0081] A second convolution module 13, the second convolution module 13 is connected to the first down-sampling module 12, and the second convolution module 13 generates a second convolution image according to the first down-sampling image;

[0082] A third convolution module 14, the third convolution module 14 is connected to the second convolution module 13, and the third convolution module 14 generates a third convolution image according to the second convolution image;

[0083] The first up-sampling module 15 is connected to the third convolution module 14. The first up-sampling module 15 performs up-sampling processing according to the third convolution image to obtain a mask image.

[0084] And, the first prediction head 2 comprises:

[0085] A second down-sampling module 21, wherein an input end of the second down-sampling module 21 is connected to an output end of the second convolution module 13, and the second down-sampling module 21 generates a second down-sampled image according to the second convolution image;

[0086] A fourth convolution module 22, the fourth convolution module 22 is connected to the second down-sampling module 21, and the fourth convolution module 22 generates a fourth convolution image according to the second down-sampling image;

[0087] A third down-sampling module 23, the third down-sampling module 23 is connected to the fourth convolution module 22, and the third down-sampling module 23 generates a third down-sampling image according to the fourth convolution image;

[0088] A multi-depth extraction module 24, the multi-depth extraction module 24 is connected to the third down-sampling module 23, and the multi-depth extraction module 24 extracts grouped image features from the third down-sampling image through a plurality of convolution kernels of different sizes;

[0089] A first cascade module 25, the first cascade module 25 is connected to the multi-depth extraction module 24, and the first cascade module 25 cascades the grouped image features to obtain a first cascade image;

[0090] A fifth convolution module 26, the fifth convolution module 26 is connected to the first cascade module 25, and the fifth convolution module 26 generates a fifth convolution image according to the cascade image;

[0091] A second up-sampling module 27, the second up-sampling module 27 is connected to the fifth convolution module 26, and the second up-sampling module 27 generates a second up-sampled image according to the fifth convolution image;

[0092] The second cascade module 28 is connected to the second up-sampling module 27 and the third convolution module 14 . The second cascade module 28 cascades the second up-sampled image and the third convolution image and then inputs the cascaded image into the first up-sampling module 15 .

[0093] Specifically, in order to realize the feature extraction process in the image, in this embodiment, the above-mentioned mask module 1 and the first prediction head 2 are constructed. First, for the input ultrasound image, the first convolution module 11 is used in the mask module 1 to extract the features of the input ultrasound image by using the depth-separable convolution to obtain the first convolution image; then the first downsampling module 12 performs downsampling processing according to the first convolution image to generate the first downsampling image; then, the second convolution module 13 is used to extract the features of the first downsampling image by using the depth-separable convolution to generate the second convolution image, and then transferred to the first prediction head 2, the second downsampling module 21 is used to generate the second downsampling image according to the second convolution image, the fourth convolution module 22 is used to generate the fourth convolution image according to the second downsampling image, and the third downsampling module 23 is used to generate the third downsampling image according to the fourth convolution image.

[0094] Through the above process, effective extraction of each tissue area in the ultrasound image is completed, and then the multi-depth extraction module 24 uses multiple convolution kernels of different sizes to extract grouped image features from the third down-sampled image.

[0095] Specifically, the multi-depth extraction module 24 obtains five groups of features from the depth features in the third down-sampled image through five different convolution kernels, and then cascades the grouped image features through the first cascade module 25 to obtain a first cascade image and uses the fifth convolution module 26 to generate a fifth convolution image based on the cascade image. On this basis, the second upsampling module 27 obtains a feature layer with a feature size of 128×128 by upsampling the fifth convolution image by four times, and generates a second up-sampled image.

[0096] Furthermore, in order to achieve a better fusion effect, the second cascade module 28 cascades the second upsampled image and the third convolution image, these two groups of 128×128 feature layers, and then inputs them into the first upsampling module 15. By upsampling the 128×128 features by 2 times, a segmentation result of 256×256 is obtained. Based on the segmentation result, information such as the thyroid gland and surrounding tissues contained in the image can be obtained. If it does not contain a valid segmentation result, it is considered that the corresponding position has not been scanned.

[0097] And, the second prediction head 3 comprises:

[0098] A flattening layer 31, the flattening layer 31 is connected to the output end of the third down-sampling module 23, and the flattening layer flattens the third down-sampling image to obtain a one-dimensional feature image;

[0099] A first fully connected layer 32, the first fully connected layer 32 is connected to the flattened layer 31, and the first fully connected layer 32 generates a first predicted image according to the one-dimensional feature image;

[0100] A second fully connected layer 33, the second fully connected layer 33 is connected to the first fully connected layer 32, and the second fully connected layer 33 generates a second predicted image according to the first predicted image;

[0101] The third fully connected layer 34 is connected to the second fully connected layer 32, and the third fully connected layer 34 generates direction guidance information according to the second predicted image.

[0102] Specifically, in order to achieve a better prompt effect, in this embodiment, based on the extraction of surrounding features implemented by the third downsampling module 23, the features are reduced in dimension through the flattening layer 31 to form a one-dimensional vector, and then the final direction guidance information is predicted through the first fully connected layer 32, the second fully connected layer 33 and the third fully connected layer 34.

[0103] The direction guidance information includes move the probe up, move the probe down, move the probe left, and move the probe right.

[0104] On this basis, the scanning process can be performed according to the following process.

[0105] In one embodiment, Figure 5 As shown, step S1 includes:

[0106] Step S11: obtaining an input ultrasound image, and using an image-guided model to identify the input ultrasound image;

[0107] Step S12: determining whether the recognition result includes the thyroid gland;

[0108] If so, terminate the boot process;

[0109] If not, go to step S2.

[0110] In one embodiment, Figure 6 As shown, step S2 includes:

[0111] Step S21: identifying the input ultrasound image according to the image guidance model to predict and obtain direction guidance information;

[0112] Step S22: Rendering a prompt arrow on the mask image according to the direction guidance information.

[0113] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time guidance method for B-ultrasound images based on deep learning, characterized in that: include: Step S1: acquiring an input ultrasound image, determining whether the input ultrasound image contains a thyroid gland, and turning to step S2 if the thyroid gland is not contained; Step S2: identifying the input ultrasound image and generating direction guidance information corresponding to the thyroid gland; Step S3: returning to step S1 to reacquire the input ultrasound image until the thyroid gland is scanned; In the steps S1 and S2, an image-guided model is used to identify the input ultrasound image.

2. The real-time boot method according to claim 1, characterized in that: The image-guided model includes: a mask module, wherein the mask module receives the input ultrasound image and generates a mask image according to the input ultrasound image; a first prediction head, the first prediction head being connected to the mask module, the first prediction head receiving the input ultrasound image and generating a prediction feature according to the input ultrasound image; In step S1, whether the first prediction head recognizes the prediction feature is used to determine whether the input ultrasound image contains the thyroid gland.

3. The real-time boot method according to claim 2, characterized in that: The mask module comprises: a first convolution module, the first convolution module receiving the input ultrasound image and generating a first convolution image; A first down-sampling module, the first down-sampling module is connected to the first convolution module, and the first down-sampling module generates a first down-sampled image according to the first convolution image; a second convolution module, the second convolution module being connected to the first downsampling module, and the second convolution module generating a second convolution image according to the first downsampling image; a third convolution module, the third convolution module being connected to the second convolution module, and the third convolution module generating a third convolution image according to the second convolution image; A first up-sampling module, wherein the first up-sampling module is connected to the third convolution module, and the first up-sampling module performs up-sampling processing according to the third convolution image to obtain the mask image.

4. The real-time boot method according to claim 3, characterized in that: The first prediction head comprises: a second down-sampling module, wherein an input end of the second down-sampling module is connected to an output end of the second convolution module, and the second down-sampling module generates a second down-sampled image according to the second convolution image; a fourth convolution module, the fourth convolution module being connected to the second down-sampling module, and the fourth convolution module generating a fourth convolution image according to the second down-sampling image; a third down-sampling module, the third down-sampling module being connected to the fourth convolution module, and the third down-sampling module generating a third down-sampling image according to the fourth convolution image; A multi-depth extraction module, the multi-depth extraction module is connected to the third down-sampling module, and the multi-depth extraction module extracts grouped image features from the third down-sampling image through a plurality of convolution kernels of different sizes; A first cascade module, the first cascade module is connected to the multi-depth extraction module, and the first cascade module cascades the grouped image features to obtain a first cascade image; a fifth convolution module, the fifth convolution module being connected to the cascade module, and the fifth convolution module generating a fifth convolution image according to the cascade image; a second up-sampling module, the second up-sampling module being connected to the fifth convolution module, and the second up-sampling module generating a second up-sampling image according to the fifth convolution image; A second cascade module, wherein the second cascade module connects the second up-sampling module and the third convolution module, and the second cascade module cascades the second up-sampled image and the third convolution image and then inputs the cascaded image into the first up-sampling module.

5. The real-time boot method according to claim 4, characterized in that: The image-guided model also includes: a second prediction head, the second prediction head being connected to the first prediction head, and the second prediction head receiving the input ultrasound image and predicting to obtain the direction guidance information; In the step S2, the input ultrasound image is input into the second prediction head to predict and obtain the direction guidance information.

6. The real-time boot method according to claim 5, characterized in that: The second prediction head comprises: A flattening layer, the flattening layer is connected to the output end of the third down-sampling module, and the flattening layer flattens the third down-sampling image to obtain a one-dimensional feature image; A first fully connected layer, the first fully connected layer is connected to the flattened layer, and the first fully connected layer generates a first predicted image according to the one-dimensional feature image; a second fully connected layer, the second fully connected layer being connected to the first fully connected layer, and the second fully connected layer generating a second predicted image according to the first predicted image; A third fully connected layer, wherein the third fully connected layer is connected to the second fully connected layer, and the third fully connected layer generates the direction guidance information according to the second predicted image.

7. The real-time boot method according to claim 6, characterized in that: The direction guidance information includes moving the probe upward, moving the probe downward, moving the probe leftward, and moving the probe rightward.

8. The real-time boot method according to claim 2, characterized in that: The step S1 comprises: Step S11: acquiring the input ultrasound image, and using the image-guided model to identify the input ultrasound image; Step S12: determining whether the thyroid gland is included in the recognition result; If so, terminate the boot process; If not, go to step S2.

9. The real-time boot method according to claim 2, characterized in that: The step S2 comprises: Step S21: identifying the input ultrasound image according to the image guidance model to predict the direction guidance information; Step S22: Rendering a prompt arrow on the mask image according to the direction guidance information.

Images