Postoperative cardiothoracic trauma ultrasonic image automatic diagnosis system based on deep learning
Through the automated diagnostic system of postoperative cardiothoracic trauma ultrasound imaging based on deep learning, the problems of slow diagnosis speed, insufficient accuracy and relying on experience in traditional diagnostic methods are solved, and the rapid and accurate automated diagnosis of cardiothoracic trauma is achieved.
Patent Information
- Application Number
- CN202411455853.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-02-14
AI Technical Summary
Traditional methods of diagnosis of cardiothoracic trauma have problems such as slow diagnosis speed, insufficient accuracy and relying on experience, especially in emergency situations that cannot be diagnosed quickly and accurately.
The automated diagnosis system of postoperative cardiothoracic trauma ultrasound imaging based on deep learning is adopted, which includes an ultrasound imaging module, a data processing module and a deep learning diagnosis module to automatically diagnose cardiothoracic trauma through preprocessing and deep learning models.
It realizes rapid and accurate automated diagnosis of cardiothoracic trauma, and can quickly screen cardiothoracic trauma in emergency situations, improves diagnosis efficiency and consistency, and reduces the occurrence of misdiagnosis and missed diagnosis.
Smart Images
Figure CN119423839A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image analysis, and in particular to an automatic diagnosis system for postoperative cardiothoracic trauma ultrasound images based on deep learning. Background Art
[0002] Cardiothoracic trauma refers to injuries to the chest and heart caused by external forces, including but not limited to rib fractures, pericardial effusion, cardiac contusion, hemothorax, etc. This type of trauma is more common in traffic accidents, falls, violent incidents, etc., and is characterized by high morbidity and high mortality. Due to the complexity and diversity of cardiothoracic trauma, timely and accurate diagnosis is crucial to the patient's prognosis. However, traditional diagnostic methods have certain limitations, which are mainly manifested in the following aspects:
[0003] Slow diagnosis: Traditional imaging examinations such as X-rays and CT scans require a long time for image acquisition and analysis, and cannot meet the needs for rapid diagnosis in emergency situations.
[0004] Insufficient diagnostic accuracy: Due to the complexity of cardiothoracic trauma, traditional imaging examinations sometimes have difficulty accurately identifying subtle injuries, which can easily lead to misdiagnosis or missed diagnosis.
[0005] Reliance on experience: Traditional diagnostic methods are highly dependent on the doctor’s experience and expertise. The diagnostic results between different doctors may vary greatly, affecting the consistency and reliability of the diagnosis.
[0006] Ultrasound imaging, as a non-invasive, real-time imaging method, has been widely used in the diagnosis of cardiothoracic trauma in recent years. However, its application still faces some challenges. First, the quality of ultrasound images is affected by many factors, such as patient body shape, operation techniques, etc., which may cause large image noise and affect the diagnostic effect. Secondly, the interpretation of ultrasound images requires doctors to have high professional knowledge and experience, and the diagnostic results between different doctors may vary. Summary of the invention
[0007] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide an automated diagnosis system for postoperative cardiothoracic trauma ultrasound imaging based on deep learning.
[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0009] The present invention provides a deep learning-based postoperative cardiothoracic trauma ultrasonic imaging automated diagnosis system, comprising:
[0010] An ultrasound imaging module, used to obtain an ultrasound image of the cardiothoracic region, the ultrasound imaging module comprising an ultrasound probe and an image acquisition device, capable of capturing a high-resolution ultrasound image of the cardiothoracic region in real time;
[0011] The data processing module is used to pre-process the acquired ultrasound image, including image denoising, enhancement and standardization. The data processing module includes:
[0012] An image denoising unit, used for removing noise from an ultrasound image;
[0013] An image enhancement unit for improving the contrast and clarity of ultrasound images;
[0014] An image normalization unit for converting ultrasound images into a uniform format and size;
[0015] A deep learning diagnosis module, based on the preprocessed ultrasound images, uses a pre-trained deep learning model to perform automated diagnosis of cardiothoracic trauma, wherein the deep learning model includes VGG-11, 3DResNet-34, AlexNet, DenseNet, VGGNet and ResNet models, and the deep learning diagnosis module includes:
[0016] A feature extraction unit, used for extracting multi-level image features from the preprocessed ultrasound image;
[0017] a classification unit, for classifying the type of cardiothoracic trauma according to the extracted image features;
[0018] A diagnosis suggestion generating unit, used for generating a diagnosis suggestion based on the classification result;
[0019] The diagnosis result output module is used to output the diagnosis result and suggestions. The diagnosis result output module includes:
[0020] A display unit, used for displaying the diagnosis results and suggestions on a display screen;
[0021] The report generation unit is used to generate a detailed diagnosis report including diagnosis results, suggestions and related images.
[0022] As a preferred technical solution of the present invention, the deep learning diagnosis module is trained by transfer learning and image augmentation methods to overcome the influence of small samples on model training. The transfer learning includes:
[0023] Use the weights of the pre-trained model as initial parameters;
[0024] Fine-tune on the target dataset to adapt to the specific diagnostic task;
[0025] The image augmentation methods include rotation, translation, scaling, flipping and color transformation to increase the diversity of training data.
[0026] As a preferred technical solution of the present invention, the data processing module performs information desensitization and preprocessing on the ultrasound image data to eliminate the influence of confounding factors such as ultrasound machine model and image batch, and the information desensitization includes:
[0027] Remove personal patient information from images;
[0028] Randomize images to prevent data leakage;
[0029] The preprocessing includes image normalization and segmentation to ensure the consistency of input data.
[0030] As a preferred technical solution of the present invention, the deep learning diagnosis module performs model evaluation through 5-fold cross validation, with 500 iterations each time, recording the learning curve and accuracy curve of the model during the training process. The cross validation includes:
[0031] The data set is divided into 5 subsets, 4 of which are used for training each time, and 1 subset is used for validation;
[0032] Alternately use different subsets for validation to ensure the generalization ability of the model;
[0033] Record the loss value and accuracy of each validation to evaluate the model performance.
[0034] As a preferred technical solution of the present invention, the hyperparameters of the deep learning diagnosis module include network structure, learning rate, optimizer, penalty coefficient and batch size, and are adjusted according to the trend of loss decrease and accuracy increase in the validation set. The hyperparameter adjustment includes:
[0035] Dynamically adjust the learning rate based on the performance of the validation set to accelerate convergence;
[0036] Select an appropriate optimizer to improve the training efficiency of the model;
[0037] Adjust the penalty coefficient to prevent overfitting;
[0038] Adjust the Batch Size based on the size of the dataset and hardware resources to balance training speed and memory usage.
[0039] As a preferred technical solution of the present invention, the deep learning diagnosis module adopts the ResNet model in the diagnosis tasks of pericardial effusion and cardiac tamponade; and adopts the VGGNet model in the diagnosis tasks of hemothorax. The diagnosis tasks include:
[0040] Feature extraction and classification of ultrasound images of pericardial effusion and cardiac tamponade;
[0041] Feature extraction and classification of ultrasound images of hemothorax;
[0042] By comparing the AUC values of different models, the optimal model is selected for diagnosis.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] 1: The present invention can assist clinical medical personnel in quickly identifying common cardiothoracic injuries such as pericardial effusion, cardiac tamponade, hemothorax, etc. It can read cardiac ultrasound files in batches, automatically diagnose and identify, and perform visual display, thereby solving the problem of rapid screening of cardiothoracic trauma and realizing rapid and intelligent diagnosis of cardiothoracic trauma.
[0045] 2: The system of the present invention is easy to operate and does not rely on the user's clinical experience; it has high accuracy and is built on a large-scale clinical database and can be widely deployed and used. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0047] Figure 1 It is a detailed operation logic schematic diagram of the present invention;
[0048] Figure 2 is a schematic diagram of test-related performance indicators in internal test data in the embodiment;
[0049] Figure 3 is a schematic diagram of a result frame page frame in this implementation;
[0050] Figure 4 is a schematic diagram of the diagnosis prediction result in the embodiment; DETAILED DESCRIPTION
[0051] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0052] Example 1
[0053] like Figure 1-4 As shown, the present invention provides a deep learning-based postoperative cardiothoracic trauma ultrasound imaging automated diagnosis system, comprising:
[0054] An ultrasound imaging module, used to obtain ultrasound images of the cardiothoracic region, the ultrasound imaging module includes an ultrasound probe and an image acquisition device, and can capture high-resolution ultrasound images of the cardiothoracic region in real time;
[0055] The data processing module is used to pre-process the acquired ultrasound images, including image denoising, enhancement and standardization. The data processing module includes:
[0056] An image denoising unit, used for removing noise from an ultrasound image;
[0057] An image enhancement unit for improving the contrast and clarity of ultrasound images;
[0058] An image normalization unit for converting ultrasound images into a uniform format and size;
[0059] The deep learning diagnosis module uses pre-trained deep learning models to automatically diagnose cardiothoracic trauma based on pre-processed ultrasound images. The deep learning models include VGG-11, 3DResNet-34, AlexNet, DenseNet, VGGNet, and ResNet models. The deep learning diagnosis module includes:
[0060] A feature extraction unit, used for extracting multi-level image features from the preprocessed ultrasound image;
[0061] a classification unit, for classifying the type of cardiothoracic trauma according to the extracted image features;
[0062] A diagnosis suggestion generating unit, used for generating a diagnosis suggestion based on the classification result;
[0063] The diagnosis result output module is used to output the diagnosis results and suggestions. The diagnosis result output module includes:
[0064] A display unit, used for displaying the diagnosis results and suggestions on a display screen;
[0065] The report generation unit is used to generate a detailed diagnosis report including diagnosis results, suggestions and related images.
[0066] Furthermore, the deep learning diagnosis module is trained through transfer learning and image augmentation methods to overcome the impact of small samples on model training. Transfer learning includes:
[0067] Use the weights of the pre-trained model as initial parameters;
[0068] Fine-tune on the target dataset to adapt to the specific diagnostic task;
[0069] Image augmentation methods include rotation, translation, scaling, flipping, and color transformation to increase the diversity of training data.
[0070] Furthermore, the data processing module performs information desensitization and preprocessing on the ultrasound image data to eliminate the influence of confounding factors such as ultrasound machine model and image batch. Information desensitization includes:
[0071] Remove personal patient information from images;
[0072] Randomize images to prevent data leakage;
[0073] Preprocessing includes image normalization and segmentation to ensure the consistency of input data.
[0074] Furthermore, the deep learning diagnosis module performs model evaluation through 5-fold cross validation, with 500 iterations each time, recording the learning curve and accuracy curve of the model during the training process. The cross validation includes:
[0075] The data set is divided into 5 subsets, 4 of which are used for training each time, and 1 subset is used for validation;
[0076] Alternately use different subsets for validation to ensure the generalization ability of the model;
[0077] Record the loss value and accuracy of each validation to evaluate the model performance.
[0078] Furthermore, the hyperparameters of the deep learning diagnosis module include network structure, learning rate, optimizer, penalty coefficient and batch size, and are adjusted according to the trend of loss decrease and accuracy increase in the validation set. The hyperparameter adjustments include:
[0079] Dynamically adjust the learning rate based on the performance of the validation set to accelerate convergence;
[0080] Select appropriate optimizers, such as Adam, SGD, etc., to improve the training efficiency of the model;
[0081] Adjust the penalty coefficient to prevent overfitting;
[0082] Adjust the Batch Size based on the size of the dataset and hardware resources to balance training speed and memory usage.
[0083] Furthermore, in the diagnosis tasks of pericardial effusion and cardiac tamponade, the deep learning diagnosis module uses the ResNet model to extract and classify the ultrasound images of pericardial effusion and cardiac tamponade, with an AUC of 0.95; in the diagnosis task of hemothorax, the VGGNet model is used to extract and classify the ultrasound images of hemothorax, with an AUC of 0.96. The diagnosis tasks include:
[0084] Feature extraction and classification of ultrasound images of pericardial effusion and cardiac tamponade;
[0085] Feature extraction and classification of ultrasound images of hemothorax;
[0086] By comparing the AUC values of different models, the optimal model is selected for diagnosis.
[0087] Specifically, the operating system environment of the present invention is: Chinese version of Windows 10 and above, corresponding to the GPU version of CUDA and cuDNN. The software running hardware environment is: Processor: NIVDIA GeForce GTX1660 display and above, minimum graphics card memory 6Gb is required to support high-load computing requirements. Memory: Use at least 32GB RAM and 256G solid-state drive (SSD) to store and read large-scale data. Network: Offline data calculation and processing can be performed without access to Ethernet or wireless network, but the libraries required for the model need to be installed in advance. And ensure that CUDA and cuDNN provided by NIVDIA are installed to facilitate calling the GPU for calculation.
[0088] Clinical application steps:
[0089] Patient preparation:
[0090] The patient is positioned on the ultrasound table and the doctor performs the necessary preparations, including exposing the chest area and applying ultrasound coupling gel.
[0091] Ultrasound imaging:
[0092] The doctor uses an ultrasound probe to scan the patient's chest area and obtain high-resolution ultrasound images in real time. The ultrasound probe is connected to an image acquisition device, which transmits the captured ultrasound images to a data processing module.
[0093] Data processing:
[0094] The image denoising unit performs denoising on the acquired ultrasound image to remove the noise in the image. The image enhancement unit performs enhancement on the ultrasound image to improve the contrast and clarity of the image, making the image features more obvious. The image standardization unit converts the ultrasound image into a unified format and size to ensure the consistency of the input data. The preprocessed image data is stored in a large cardiac trauma ultrasound database.
[0095] Automated Diagnosis:
[0096] The preprocessed ultrasound images are transferred to the deep learning diagnosis module. The feature extraction unit extracts multi-level image features from the images using pre-trained deep learning models such as ResNet and VGGNet. The classification unit classifies the types of cardiothoracic trauma based on the extracted image features. The system automatically identifies the presence of pericardial effusion and hemothorax in the patient. The diagnosis suggestion generation unit generates diagnosis suggestions based on the classification results, suggesting that the doctor take further diagnosis and treatment measures immediately.
[0097] Result output:
[0098] The diagnosis results and suggestions are displayed on the screen in real time through the display unit for the doctor's reference. The displayed content includes the diagnosis results, image annotations and suggestions. The report generation unit generates a detailed diagnosis report, which includes the diagnosis results, suggestions and related images. The doctor can print out the report as part of the patient's medical record.
[0099] Clinical Decision Making:
[0100] Based on the diagnosis results and suggestions provided by the system, doctors can quickly make clinical decisions and arrange further examinations and treatments. Due to the efficiency and accuracy of the system, doctors can make accurate judgments on the patient's condition in a short period of time, greatly improving the efficiency and effectiveness of emergency treatment.
[0101] By using the deep learning-based automated diagnostic system for cardiothoracic trauma ultrasound imaging, doctors can quickly and accurately assess and diagnose patients' cardiothoracic trauma, providing important support for clinical decision-making. This not only improves the accuracy and efficiency of diagnosis, but also buys patients valuable treatment time.
[0102] The system has stable performance, and the relevant performance indicators are tested in the internal test data, such as Figure 2 As shown in Table 1.
[0103] Table 1. Internal dataset test evaluation results
[0104]
[0105] The software running dependencies and the Python involved are shown in Table 2:
[0106] Table 2 Software operating environment and dependencies
[0107]
[0108]
[0109] The operation of this system uses the browser to call the local user interface. There are three main objects in the operation interface:
[0110] 【1】Input box【2】Result box【3】Export button
[0111] The usage of various objects is described as follows:
[0112] 【1】Input box
[0113] S1-1: Diagram: Menu format;
[0114] S1-2: Function: used to select the folder for storing ultrasound data. After clicking, no special operation is required. The ultrasound data can be automatically imported and pre-processed. The analysis and diagnosis results will be output in the result box on the left;
[0115] S1-1: Operation: Click to select the folder where the ultrasound files are stored.
[0116] 【2】Results box
[0117] S2-1: Diagram: The page frame is in the form of Figure 3 As shown;
[0118] S2-2: Function: The prediction results will be displayed in this interface. The file name is displayed under the images_path column, the Predicted Label displays the prediction results (0 means no, 1 means yes), and the Prediction Probability [N / P] column displays the probability of the prediction results;
[0119] S2-3: Operation: Automatically calculate after selecting a folder.
[0120] 【3】Export button
[0121] S3-1: Diagram: The page frame is as shown;
[0122] S3-2: Function: to export and save the prediction results in the selected folder;
[0123] S3-3: Operation: Click the Export button and select folders layer by layer, and save the format as an Excel table.
[0124] This system is easy to operate, but it should be noted that the file reading format is inconsistent when diagnosing different types of cardiothoracic trauma. The default for pericardial effusion and cardiac tamponade diagnosis is to read DICOM format files, that is, the suffix is ".dcm", and the default for hemothorax diagnosis is to read JPEG format images, with the file suffix ".jpg". The input image must be a DICOM or JPEG format left ventricular long axis section ultrasound image with a resolution of not less than 600*500. When predicting the diagnosis of pericardial effusion, the DICOM image should contain at least one cardiac cycle. The output results of the model prediction will be displayed in the form of an Excel table on the web page and in the output folder. The detailed operation logic is as follows. Figure 1 shown.
[0125] The system operation process is:
[0126] a. Installation and initialization: This program contains three dist files, which need to be placed in a specific folder during installation.
[0127] b. Data format: Left ventricular long-axis ultrasound image in DICOM or JPEG format, with a resolution of no less than 600*500.
[0128] c. Run steps: Run the app.exe file in the dist folder and copy the web address in the pop-up window to the browser. Select the image to be input in the dialog window and click Upload to get the prediction results. The prediction results will be saved in the output folder in the form of an Excel table.
[0129] d. Prediction and result interpretation: The prediction results are saved as an Excel table. The item "images_path" is the input file name, the item "Predicted Label" is the prediction result (1 indicates positive, 0 indicates negative), and the item "PredictionProbability[N / P]" is the prediction probability in the format of [negative probability, positive probability].
[0130] The diagnostic algorithm for pericardial effusion and cardiac tamponade is:
[0131] Import the folder where ultrasound files are stored locally, automatically perform preprocessing and reasoning processes, and automatically display the diagnosis results and prediction probabilities in the output box on the left after the diagnosis model calculation is completed.
[0132] The diagnostic process of hemothorax is as follows:
[0133] Import the folder where ultrasound files are stored locally, automatically perform preprocessing and reasoning processes, and automatically display the diagnosis results and prediction probabilities in the output box on the left after the diagnosis model calculation is completed.
[0134] After clicking the Export button, select the appropriate folder and the diagnostic prediction results will be saved as an Excel file. The specific format can be seen in Figure 4 The meaning of each column is consistent with the visual interface.
[0135] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A deep learning-based automated ultrasound imaging diagnosis system for postoperative cardiothoracic trauma, characterized in that: include: An ultrasound imaging module, used to obtain an ultrasound image of the cardiothoracic region, the ultrasound imaging module comprising an ultrasound probe and an image acquisition device, capable of capturing a high-resolution ultrasound image of the cardiothoracic region in real time; The data processing module is used to pre-process the acquired ultrasound image, including image denoising, enhancement and standardization. The data processing module includes: An image denoising unit, used for removing noise from an ultrasound image; An image enhancement unit for improving the contrast and clarity of ultrasound images; An image normalization unit for converting ultrasound images into a uniform format and size; A deep learning diagnosis module, based on the preprocessed ultrasound images, uses a pre-trained deep learning model to perform automated diagnosis of cardiothoracic trauma, wherein the deep learning model includes VGG-11, 3DResNet-34, AlexNet, DenseNet, VGGNet and ResNet models, and the deep learning diagnosis module includes: A feature extraction unit, used for extracting multi-level image features from the preprocessed ultrasound image; a classification unit, for classifying the type of cardiothoracic trauma according to the extracted image features; A diagnosis suggestion generating unit, used for generating a diagnosis suggestion based on the classification result; The diagnosis result output module is used to output the diagnosis result and suggestions. The diagnosis result output module includes: A display unit, used for displaying the diagnosis results and suggestions on a display screen; The report generation unit is used to generate a detailed diagnosis report including diagnosis results, suggestions and related images.
2. According to claim 1, a deep learning-based automated ultrasound imaging diagnosis system for postoperative cardiothoracic trauma, characterized in that: The deep learning diagnosis module is trained by transfer learning and image augmentation methods to overcome the influence of small samples on model training. The transfer learning includes: Use the weights of the pre-trained model as initial parameters; Fine-tune on the target dataset to adapt to the specific diagnostic task; The image augmentation methods include rotation, translation, scaling, flipping and color transformation to increase the diversity of training data.
3. The deep learning-based postoperative cardiothoracic trauma ultrasound imaging automated diagnosis system according to claim 1, characterized in that: The data processing module performs information desensitization and preprocessing on the ultrasound image data to eliminate the influence of confounding factors such as ultrasound machine model and image batch. The information desensitization includes: Remove personal patient information from images; Randomize images to prevent data leakage; The preprocessing includes image normalization and segmentation to ensure the consistency of input data.
4. The deep learning-based postoperative cardiothoracic trauma ultrasound imaging automated diagnosis system according to claim 1, characterized in that: The deep learning diagnosis module performs model evaluation through 5-fold cross validation, with 500 iterations each time, recording the learning curve and accuracy curve of the model during the training process. The cross validation includes: The data set is divided into 5 subsets, 4 of which are used for training each time, and 1 subset is used for validation; Alternately use different subsets for validation to ensure the generalization ability of the model; Record the loss value and accuracy of each validation to evaluate the model performance.
5. The deep learning-based automated ultrasound imaging diagnosis system for postoperative cardiothoracic trauma according to claim 1, characterized in that: The hyperparameters of the deep learning diagnosis module include network structure, learning rate, optimizer, penalty coefficient and batch size, and are adjusted according to the trend of loss decrease and accuracy increase in the validation set. The hyperparameter adjustment includes: Dynamically adjust the learning rate based on the performance of the validation set to accelerate convergence; Select an appropriate optimizer to improve the training efficiency of the model; Adjust the penalty coefficient to prevent overfitting; Adjust the Batch Size based on the size of the dataset and hardware resources to balance training speed and memory usage.
6. The deep learning-based postoperative cardiothoracic trauma ultrasound imaging automated diagnosis system according to claim 1, characterized in that: The deep learning diagnosis module uses the ResNet model in the diagnosis tasks of pericardial effusion and cardiac tamponade; and uses the VGGNet model in the diagnosis tasks of hemothorax. The diagnosis tasks include: Feature extraction and classification of ultrasound images of pericardial effusion and cardiac tamponade; Feature extraction and classification of ultrasound images of hemothorax; By comparing the AUC values of different models, the optimal model is selected for diagnosis.