An ultrasonic image information extraction method and system based on deep learning

Abstract

The application belongs to the technical field of medical image processing, and particularly relates to an efficient ultrasonic image information extraction method and system based on deep learning. The system comprises: an input module for inputting original ultrasonic image data; an ultrasonic working mode recognition module for recognizing the ultrasonic working mode of the original ultrasonic image data; an image segmentation module for performing image segmentation; a parameter information structuring module for obtaining structured data; and a grouping and caching module for recognizing a scanning site, grouping based on the scanning site, and grouping and caching the structured data. The system can also perform data cleaning and screening on the image. The application further provides a method for performing ultrasonic image information extraction by using the above system. The ultrasonic image and parameter information extracted by the application have high quality and accuracy, and can provide reliable data support for subsequent diagnosis, analysis or research. Therefore, the application has a good application prospect.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a method and system for efficient ultrasound image information extraction based on deep learning. Background Technology

[0002] Ultrasonic testing is a non-destructive testing technique that uses the echoes generated when ultrasound waves propagate through a medium to detect objects. It determines the location, size, and shape of an object by emitting high-frequency sound waves and receiving the reflected echoes. Ultrasonic images, produced during ultrasonic testing, show the internal structure and features of the object being inspected. These images are generated by analyzing parameters such as the time, intensity, and frequency of the reflected sound wave signals. Ultrasonic testing is widely used in the medical field, such as obstetric examinations and cardiac monitoring, and also in industrial inspection, such as material defect analysis. Due to its non-invasiveness, safety, and cost-effectiveness, ultrasonic testing and ultrasound images have become indispensable tools in modern diagnostics and testing.

[0003] In the medical field, analyzing ultrasound images presents unique challenges, such as low image quality due to noise and artifacts, high dependence on the experience of the operator or diagnostic physician, and significant intra-observer or inter-observer variability between ultrasound systems from different institutions and manufacturers. For example, a study using ultrasound images for prenatal anomaly detection showed that the sensitivity ranged from 27.5% to 96% across different medical institutions. To address these challenges, developing advanced automated ultrasound image analysis methods can make ultrasound diagnosis and / or evaluation, as well as image-guided interventions and / or treatments, more objective, accurate, and intelligent.

[0004] The application of artificial intelligence (AI) in ultrasound image analysis is rapidly developing, currently focusing primarily on improving diagnostic efficiency and accuracy. AI technology applied to image recognition and classification effectively improves diagnostic accuracy and specificity, while reducing misdiagnosis rates. However, training AI models typically requires datasets consisting of large amounts of sample data. While some datasets are publicly available, for various reasons and research objectives, the field still needs to continuously collect and organize more ultrasound image data to provide more ultrasound image datasets. For example, some current datasets suffer from inconsistent data quality and format, which may not meet the needs of some researchers; ultrasound images exhibit diversity (e.g., multiple ultrasound working models can be used to detect various scanning sites), requiring the construction of different datasets for different ultrasound images to meet data usage needs; and for different regions and populations with different characteristics, separate datasets may need to be constructed for research, and existing datasets may not meet these unique research requirements.

[0005] Therefore, there is a need in this field to develop an efficient, accurate, and high-quality method for extracting information from ultrasound images. Summary of the Invention

[0006] To address the problems of existing technologies, this invention provides a method and system for efficient ultrasound image information extraction based on deep learning.

[0007] A high-efficiency ultrasound image information extraction system based on deep learning, comprising:

[0008] The input module is configured to input raw ultrasound image data;

[0009] The ultrasound working mode recognition module is configured to identify the ultrasound working mode of the original ultrasound image data through a first deep learning model.

[0010] The image segmentation module is configured to select an image segmentation algorithm processing flow based on the identified ultrasound working mode and perform image segmentation.

[0011] The parameter information structuring module is configured to further segment the ultrasound scanning image and ultrasound imaging parameters from the segmented image, and associate the ultrasound scanning image and ultrasound imaging parameters to obtain structured data.

[0012] The group caching module is configured to identify the scanned area, group the data based on the scanned area, and cache the structured data in groups.

[0013] Preferably, the ultrasound working mode is selected from B-mode ultrasound, C-mode ultrasound, D-mode ultrasound, or M-mode ultrasound;

[0014] The first deep learning model is selected from CNN, AlexNet, VGG, GoogLeNet, ResNet, Inception series, DenseNet, and MobileNet.

[0015] Preferably, the image segmentation algorithm is selected from threshold segmentation, edge detection, and region growing;

[0016] In the image segmentation module, image segmentation includes:

[0017] Step 3.1: Identify background or irrelevant components;

[0018] Step 3.2: Extract the area of ​​ultrasound image and ultrasound imaging parameters.

[0019] Preferably, the ultrasound imaging parameters are selected from at least one of image gain, scanning frequency, time gain control parameters, dynamic range, and harmonics;

[0020] In the parameter information structuring module, ultrasound imaging parameters are extracted from the segmented image using natural language processing technology.

[0021] Preferably, in the group caching module, the method for identifying the scanned area is as follows:

[0022] The scanning area is identified based on the ultrasound imaging parameters;

[0023] If the scanned area cannot be identified based on the ultrasound imaging parameters, the default scanned area is determined based on the ultrasound working mode.

[0024] Preferred options also include:

[0025] The image data cleaning module is configured to formulate specific screening criteria A based on the ultrasound working mode and the scanning site, and use a second deep learning model to clean the ultrasound scan images in the structured data according to the screening criteria A.

[0026] The effective frame filtering module is configured to formulate specific filtering criteria B based on the ultrasound working mode and the scanning location, and use a classifier to filter the cleaned ultrasound scanning images according to the filtering criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

[0027] Preferably, the second deep learning model is selected from recurrent neural networks, long short-term memory networks, long short-term memory network variants with attention mechanisms, combinations of BiLSTM and CRF, and bidirectional recurrent neural networks;

[0028] The screening criterion B includes limitations on the range of image features and / or parameters.

[0029] This invention also provides a method for efficient ultrasound image information extraction based on deep learning, implemented using the aforementioned efficient ultrasound image information extraction system based on deep learning, comprising:

[0030] Step 1: Input the raw ultrasound image data;

[0031] Step 2: Identify the ultrasound working mode of the original ultrasound image data using the first deep learning model;

[0032] Step 3: Based on the identified ultrasound working mode, select the image segmentation algorithm processing flow and perform image segmentation;

[0033] Step 4: Segment the ultrasound scanning image and ultrasound imaging parameters from the segmented image again, and associate the ultrasound scanning image and ultrasound imaging parameters to obtain structured data.

[0034] Step 5: Identify the scanning area, group the data based on the scanning area, and cache the structured data in each group.

[0035] Preferred options also include:

[0036] Step 6: Based on the ultrasound working mode and the scanning area, formulate specific screening criteria A, and use the second deep learning model to clean the ultrasound scanning images in the structured data according to the screening criteria A.

[0037] Step 7: Based on the ultrasound working mode and the scanning area, formulate specific screening criteria B, and use a classifier to screen the cleaned ultrasound scanning images according to the screening criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

[0038] The present invention also provides a computer-readable storage medium storing: a computer program for implementing the above-described high-efficiency ultrasound image information extraction system based on deep learning, or a computer program for implementing the above-described high-efficiency ultrasound image information extraction method based on deep learning.

[0039] In this invention, "Standard A" and "Standard B" are commonly used terms in the field of ultrasound imaging research, and their meanings in this invention are as follows:

[0040] Standard A is mainly used for preliminary screening, filtering out a large amount of useless image data during probe movement. Standard B is used for fine screening, such as selecting better image data from consecutive similar images during tissue observation.

[0041] The standard includes the following:

[0042] B-mode image standard A includes:

[0043] Spatial resolution: distinguishing between two adjacent structures;

[0044] Comparison resolution: distinguishing different gray levels;

[0045] Artifacts: Assess the presence of large-area artifacts caused by sound wave reflection, refraction, or absorption;

[0046] Noise level: Assess the degree of randomly occurring point or particulate noise in the background;

[0047] B-mode image standard B includes:

[0048] Uniformity: Brightness and contrast should be consistent throughout the entire field of view, with no obvious uneven areas;

[0049] Boundary clarity: Whether the edges of the organizational interface are clearly distinguishable;

[0050] Dynamic range: A larger dynamic range allows for better display of different tissue types, from dark to light;

[0051] The standard values ​​should be adjusted for different probe models and different scanning areas. For example, the spatial resolution of the abdominal area in a convex array should be lowered compared to the spatial resolution of small organs in a linear array.

[0052] Standard A for C, D, and M modes follows the standard of B mode, while standard B requires the addition of screening criteria based on the B mode standard, such as:

[0053] C mode adds color dynamic range, color saturation, and color noise within the sampling frame; D mode adds spectral morphology; M mode adds criteria for judging the sharpness of the time-motion curve.

[0054] This invention provides a method and system for efficient ultrasound image information extraction based on deep learning, addressing the application needs of ultrasound image data in the medical field. Applying this method and system can achieve the following beneficial technical effects:

[0055] 1. The data extraction method of the present invention introduces deep learning and natural language processing (NLP) technology, which can automatically identify the ultrasound working mode, extract key information and parameters, greatly reduce the workload of manual screening and information extraction, improve the efficiency of data processing, reduce labor costs, and avoid errors that may occur during manual screening and extraction.

[0056] 2. This invention addresses the characteristics of medical ultrasound images by designing a data processing workflow. Through image cleaning and effective frame filtering in specific steps, noise and artifacts in the images can be accurately identified and removed, ensuring data quality. Furthermore, the application of NLP technology can accurately extract ultrasound parameters, avoiding errors that may occur during manual extraction.

[0057] In summary, the method of this invention can efficiently process batch data and efficiently construct ultrasound image datasets with various modes and scanning sites. Furthermore, through the design of the method flow and the selection of data processing algorithms, the ultrasound images and parameter information of this invention possess high quality and accuracy, providing reliable data support for subsequent diagnosis, analysis, or research. Therefore, this invention has excellent application prospects.

[0058] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions or alterations can be made without departing from the basic technical concept of the present invention.

[0059] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating Embodiment 1 of the present invention;

[0061] Figure 2 This is an example diagram of step 1 in Embodiment 1 of the present invention;

[0062] Figure 3 This is an example diagram of step 3.1 in Embodiment 1 of the present invention;

[0063] Figure 4 This is an example diagram of step 3.2 in Embodiment 1 of the present invention;

[0064] Figure 5 This is an example diagram of step 4 in Embodiment 1 of the present invention;

[0065] Figure 6 This is an example diagram of step 6 in Embodiment 1 of the present invention. Detailed Implementation

[0066] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.

[0067] Example 1: A Deep Learning-Based Method and System for Information Extraction from Ultrasonic Images

[0068] The ultrasound image information extraction system provided in this embodiment includes:

[0069] The input module is configured to input raw ultrasound image data;

[0070] The ultrasound working mode recognition module is configured to identify the ultrasound working mode of the original ultrasound image data through a first deep learning model.

[0071] The image segmentation module is configured to select an image segmentation algorithm processing flow based on the identified ultrasound working mode and perform image segmentation.

[0072] The parameter information structuring module is configured to further segment the ultrasound scanning image and ultrasound imaging parameters from the segmented image, and associate the ultrasound scanning image and ultrasound imaging parameters to obtain structured data.

[0073] The group caching module is configured to identify the scanning area, group the data based on the scanning area, and cache the structured data in groups.

[0074] The image data cleaning module is configured to formulate specific screening criteria A based on the ultrasound working mode and the scanning site, and use a second deep learning model to clean the ultrasound scan images in the structured data according to the screening criteria A.

[0075] The effective frame filtering module is configured to formulate specific filtering criteria B based on the ultrasound working mode and the scanning location, and use a classifier to filter the cleaned ultrasound scanning images according to the filtering criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

[0076] Using the above system, information extraction from ultrasound images can be performed, and a dataset of ultrasound images can be constructed. The specific method and process are as follows: Figure 1 As shown, it includes:

[0077] Step 1, Input raw ultrasound image data:

[0078] Raw ultrasound image data is acquired in real time from ultrasound diagnostic equipment.

[0079] The input data format is high-definition video signal, including images from any of the following modes: B-mode ultrasound, C-mode ultrasound (color Doppler), D-mode ultrasound (pulse Doppler), and M-mode ultrasound (motion mode).

[0080] As an example, the raw ultrasound image data input in this step is as follows: Figure 2 As shown.

[0081] Step 2: Identify the ultrasound working mode of the original ultrasound image data using the first deep learning model:

[0082] In this embodiment, the first deep learning model can be selected from convolutional neural networks such as CNN, AlexNet, VGG, GoogLeNet, ResNet, Inception series, DenseNet, MobileNet, etc. As a preferred option, this embodiment selects CNN. The first deep learning model can distinguish the characteristics of different ultrasound working modes through training and output the recognition results (B mode, C mode, D mode, M mode).

[0083] Step 3: Based on the identified ultrasound working mode, select the image segmentation algorithm processing flow and perform image segmentation:

[0084] Image segmentation algorithms involve methods such as thresholding, edge detection, and region growing. Different ultrasound operating modes display different image content, and the corresponding segmentation processes also differ. The purpose of this step is to identify background or irrelevant components and extract the regions displayed by the ultrasound image and ultrasound imaging parameters.

[0085] Specifically, step 3 includes the following process:

[0086] Step 3.1: Identify background or irrelevant components; irrelevant components refer to areas useless for subsequent analysis, such as borders, image storage lists, etc. Image processing techniques are used to extract the regions displayed by the key ultrasound image information. Image processing techniques include filtering, enhancement, etc. In this embodiment, the preferred method is: first filter to remove noise from the image, then use enhancement to strengthen the boundary between regions within the image. As an example, the processing result of this step is as follows: Figure 3 As shown.

[0087] Step 3.2: Extract the ultrasound image and the area displaying ultrasound imaging parameters. Based on the above identification results, perform image segmentation on the original ultrasound image data. In this embodiment, the preferred method is as follows: First, use edge detection to identify the title information bar at the top of the image and the control function display area at the bottom. Then, grow the lower control function display area into a lower rectangular bar. Next, use threshold segmentation to separate the areas without images on the left and right sides of the middle region, and perform image segmentation. As an example, the processing result of this step is as follows: Figure 4 As shown.

[0088] Step 4: The ultrasound scanning image and ultrasound imaging parameters are further segmented from the segmented image. The ultrasound scanning image and ultrasound imaging parameters are then correlated to obtain structured data.

[0089] Using natural language processing techniques, relevant ultrasound imaging parameters, such as image gain, scanning frequency, time gain control parameters, dynamic range, and harmonics, are extracted from the resulting image obtained in step 3. These parameters are typically embedded in the image as text. The image is then segmented again to obtain ultrasound scan images and ultrasound imaging parameters. The extracted parameter information is then correlated with the corresponding ultrasound scan images to form structured data.

[0090] As an example, the result of this step is as follows: Figure 5 As shown.

[0091] Step 5: Identify the scanning area, group the data based on the scanning area, and cache the structured data by group:

[0092] The specific method for identifying the scanning area is as follows:

[0093] Identify the scanned area (such as the name and code of the scanned area) based on the ultrasound imaging parameters.

[0094] If the scanned area cannot be identified based on the ultrasound imaging parameters, the default scanned area is determined based on the ultrasound working mode.

[0095] As a preferred method, the default scanning areas for ultrasound working modes are as follows:

[0096] Linear array-B mode corresponds to superficial small organs;

[0097] Linear array-C mode corresponds to blood vessels;

[0098] Linear array-D mode corresponds to blood vessels;

[0099] Linear array-M mode corresponds to the carotid artery;

[0100] Convex array-B pattern corresponds to abdominal organs;

[0101] The convex array-C pattern corresponds to abdominal blood vessels;

[0102] Convex array-D pattern corresponds to abdominal blood vessels;

[0103] The convex array-M pattern corresponds to abdominal organs.

[0104] Step 6: Based on the ultrasound working mode and scanning area, formulate specific screening criteria A, and use a second deep learning model to clean the ultrasound scan images in the structured data according to the screening criteria A:

[0105] The second deep learning model can be selected from recurrent neural networks (RNN), long short-term memory networks (LSTM), LSM variants with attention mechanisms, a combination of BiLSTM and CRF, bidirectional recurrent neural networks, etc. As a preferred approach, this embodiment selects recurrent neural networks (RNN). The cached ultrasound images are cleaned according to screening criterion A. Through training, the model can identify quality problems such as noise and artifacts in the images, and filter out blurred and similar image data.

[0106] As an example, the result of this step is as follows: Figure 6 As shown.

[0107] Step 7: Based on the ultrasound working mode and scanning area, establish specific screening criteria B. Use a classifier to filter the cleaned ultrasound scan images according to screening criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

[0108] Screening criterion B is based on screening criterion A and is further formulated according to the characteristics of ultrasound working mode and scanning site: Screening criterion B includes more specific requirements for image features (such as texture and morphology) and parameter ranges (such as blood flow velocity range and tissue density range).

[0109] The resulting image set (set X) contains high-quality ultrasound images, and the ultrasound imaging parameter set (set Y) contains accurate ultrasound parameter information. These results can be used for subsequent diagnosis, analysis, or research. For example, these results can be used for model training and optimization in machine learning and deep learning; set X can serve as a training dataset for training various machine learning or deep learning models, and the parameters in set Y can be used to adjust the model's hyperparameters to optimize its performance. They can also be used for medical image analysis, diagnosis, and treatment planning; and for scene understanding and reconstruction in computer vision; the images in set X can be used to construct 3D scene models to achieve scene understanding and reconstruction, and the parameters in set Y can be used to adjust the accuracy and level of detail of the scene model, etc.

Claims

1. A high-efficiency ultrasound image information extraction system based on deep learning, characterized in that, include: The input module is configured to input raw ultrasound image data; The ultrasound working mode recognition module is configured to identify the ultrasound working mode of the original ultrasound image data through a first deep learning model. The image segmentation module is configured to select an image segmentation algorithm processing flow based on the identified ultrasound working mode and perform image segmentation. The parameter information structuring module is configured to further segment the ultrasound scanning image and ultrasound imaging parameters from the segmented image, and associate the ultrasound scanning image and ultrasound imaging parameters to obtain structured data. The group caching module is configured to identify the scanned area, group the data based on the scanned area, and cache the structured data in groups.

2. The efficient ultrasound image information extraction system based on deep learning according to claim 1, characterized in that: The ultrasound working mode is selected from B-mode ultrasound, C-mode ultrasound, D-mode ultrasound, or M-mode ultrasound; The first deep learning model is selected from CNN, AlexNet, VGG, GoogLeNet, ResNet, Inception series, DenseNet, and MobileNet.

3. The efficient ultrasound image information extraction system based on deep learning according to claim 1, characterized in that: The image segmentation algorithm processing flow includes threshold segmentation, edge detection, and region growing. In the image segmentation module, image segmentation includes: Step 3.1: Identify background or irrelevant components; Step 3.2: Extract the area of ​​ultrasound image and ultrasound imaging parameters.

4. The efficient ultrasound image information extraction system based on deep learning according to claim 1, characterized in that: The ultrasound imaging parameters are selected from at least one of image gain, scanning frequency, time gain control parameters, dynamic range, and harmonics. In the parameter information structuring module, ultrasound imaging parameters are extracted from the segmented image using natural language processing technology.

5. The efficient ultrasound image information extraction system based on deep learning according to claim 1, characterized in that: In the group caching module, the method for identifying the scanned area is as follows: The scanning area is identified based on the ultrasound imaging parameters; If the scanned area cannot be identified based on the ultrasound imaging parameters, the default scanned area is determined based on the ultrasound working mode.

6. The efficient ultrasound image information extraction system based on deep learning according to claim 1, characterized in that, Also includes: The image data cleaning module is configured to formulate specific screening criteria A based on the ultrasound working mode and the scanning site, and use a second deep learning model to clean the ultrasound scan images in the structured data according to the screening criteria A. The effective frame filtering module is configured to formulate specific filtering criteria B based on the ultrasound working mode and the scanning location, and use a classifier to filter the cleaned ultrasound scanning images according to the filtering criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

7. The efficient ultrasound image information extraction system based on deep learning according to claim 6, characterized in that: The second deep learning model is selected from recurrent neural networks, long short-term memory networks, long short-term memory network variants with attention mechanisms, the combination of BiLSTM and CRF, and bidirectional recurrent neural networks; The screening criterion B includes limitations on the range of image features and / or parameters.

8. A method for efficient information extraction from ultrasound images based on deep learning, characterized in that, The system is implemented using the high-efficiency ultrasound image information extraction system based on deep learning as described in any one of claims 1-7, comprising: Step 1: Input the raw ultrasound image data; Step 2: Identify the ultrasound working mode of the original ultrasound image data using the first deep learning model; Step 3: Based on the identified ultrasound working mode, select the image segmentation algorithm processing flow and perform image segmentation; Step 4: Segment the ultrasound scanning image and ultrasound imaging parameters from the segmented image again, and associate the ultrasound scanning image and ultrasound imaging parameters to obtain structured data. Step 5: Identify the scanning area, group the data based on the scanning area, and cache the structured data in each group.

9. The efficient ultrasound image information extraction method based on deep learning according to claim 8, characterized in that, Also includes: Step 6: Based on the ultrasound working mode and the scanning area, formulate specific screening criteria A, and use the second deep learning model to clean the ultrasound scanning images in the structured data according to the screening criteria A. Step 7: Based on the ultrasound working mode and the scanning area, formulate specific screening criteria B, and use a classifier to screen the cleaned ultrasound scanning images according to the screening criteria B to obtain an image set and a corresponding ultrasound imaging parameter set.

10. A computer-readable storage medium, characterized in that, It stores: a computer program for implementing the high-efficiency ultrasound image information extraction system based on deep learning as described in any one of claims 1-7, or a computer program for implementing the high-efficiency ultrasound image information extraction method based on deep learning as described in claim 8 or 9.

Images