Training method of ICU bedside chest radiograph intelligent identification model

Abstract

The invention relates to a training method of an ICU bedside chest radiograph intelligent recognition model, and relates to the field of medical image processing. The training method comprises the following steps of: training a TransNext-Small model by using an ImageNet-1K data set to obtain an original model; extracting chest radiograph data of multiple types of candidate symptoms in the CheXpert data set, and constructing a first training data set; according to the training method, a data set specially aiming at ICU bedside chest radiography is used for training, and the data set covers nine kinds of common pathological characteristics actually existing in ICU, such as pulmonary atelectasis, cardiac hypertrophy, pulmonary abscess, edema, lung opaque, pleural effusion, pneumonia and pneumothorax and is not found. By using the data of the real world for training, the ICU bedside chest radiograph intelligent identification model can more accurately identify and identify the specific chest image features of the ICU, so that the identification accuracy is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a training method for an ICU bedside chest X-ray intelligent recognition model. Background Art

[0002] Chest X-ray is a commonly used imaging examination method that can be used to identify a variety of chest symptoms. Conventional chest X-rays are taken in the hospital's radiology department, and the shooting position is standing posteroanterior. Because ICU patients are in critical condition and have limited or lost mobility, it is difficult to maintain the correct position during the filming. Usually, only bedside chest X-rays can be used, which are taken in the supine anteroposterior position. This will increase the projection of the mediastinum and heart in the chest X-ray, reduce the visible area of ​​the lungs, and make it difficult to detect milder heart and lung field lesions. In addition, due to the influence of various testing and treatment tubes and wires carried by patients, the quality of chest X-rays taken by patients using bedside X-ray equipment is lower than that of conventional chest X-rays taken by the radiology department.

[0003] In addition, since the conditions of ICU patients are usually more complicated, about one-third of chest X-rays are interpreted independently by non-radiologists. In order to ensure the accuracy and timeliness of recognition, non-radiologists need special image analysis technology to assist in chest X-ray interpretation. However, most of the training data sets used by many existing chest X-ray analysis systems are standard chest X-rays from routine imaging departments or examination rooms. Although these data sets are extensive, they are very different from chest X-rays taken at the ICU bedside in terms of image quality, shooting conditions, and patient position. Chest X-rays in ICU are subject to more interference factors, such as limited patient position, the influence of ventilators, and shooting in emergency situations. When the system mixes these high-quality conventional chest X-rays with lower-quality ICU bedside chest X-rays for training, it is difficult for the model to effectively learn the unique features of ICU bedside chest X-rays, resulting in reduced accuracy in bedside chest X-ray detection.

[0004] Currently, no effective solution has been proposed to the problem that the model for ICU bedside chest X-ray analysis trained by the existing model training method has poor recognition of the pathological characteristics of ICU bedside chest X-rays. Summary of the invention

[0005] The present invention provides a training method for an ICU bedside chest X-ray intelligent recognition model to solve the problem that the ICU bedside chest X-ray analysis model trained by the existing model training method has poor recognition of ICU bedside chest X-ray pathological features.

[0006] The present invention provides a training method for an ICU bedside chest X-ray intelligent recognition model, which includes: using an ImageNet-1K data set to train a TransNext-Small model to obtain an original model; extracting chest X-ray data of multiple categories of candidate symptoms in a CheXpert data set to construct a first training data set; performing image preprocessing on the first training data set to obtain a second training data set; training the original model with the second training data set to obtain a first model; extracting ICU bedside chest X-ray data of multiple categories of target symptoms from a MIMIC-CXR chest X-ray database to form an ICU bedside chest X-ray data set; performing image preprocessing on the ICU bedside chest X-ray data set to obtain a third training data set; and training the first model with the third training data set to obtain an ICU bedside chest X-ray intelligent recognition model.

[0007] Furthermore, multiple categories of candidate symptoms included: atelectasis, cardiomegaly, pulmonary consolidation, edema, pericarditis and mediastinitis, fracture, lung injury, lung opacity, pleural effusion, pneumonia, pneumothorax, other thoracic symptoms, life support equipment, and no findings.

[0008] Furthermore, multiple target symptoms included: atelectasis, cardiomegaly, pulmonary consolidation, edema, lung opacity, pleural effusion, pneumonia, pneumothorax, and no findings.

[0009] Furthermore, the steps of image preprocessing include image resizing, image enhancement and image normalization:

[0010] Convert grayscale images to RGB images;

[0011] Adjust the image size to 384 x 384 x width;

[0012] Set the random variation range of image brightness to 0.3;

[0013] Set the random variation range of image contrast to 0.3;

[0014] Set the angle range of random image rotation to 30;

[0015] Set the maximum translation ratio of the image in the horizontal and vertical directions to (0.1, 0.1);

[0016] Set the random scaling of the image to a range of 0.9 to 1.1.

[0017] The normalization formula for image normalization is:

[0018]

[0019] Among them, mean is the mean of the image normalization parameter, mean = [0.485, 0.456, 0.406], std is the variance, std = [0.229, 0.224, 0.225], the values ​​in the brackets of mean and std represent the mean and variance of the corresponding red, green, and blue channels in the RGB image, respectively, input is the pixel value of the input image, and output is the normalized output.

[0020] Furthermore, the loss function L of the original model and the first model is:

[0021]

[0022] Where C is the number of target symptoms, p i represents the probability that the model predicts that sample i belongs to the positive class, γ + Represents the adjustment factor of the positive sample, y i Represents the true label of sample i, y for positive samples i =1,1-y i represents the probability of negative samples, p mi represents the corrected probability, defined as p mi =max(p i -m, 0), m represents the probability boundary, m ≥ 0, γ - Represents the adjustment factor of negative samples, w i is the weight, w i The calculation formula is:

[0023] w i =y i e 1-ρ +(1-y i ) ρ

[0024] Among them, ρ represents the proportion of positive samples in the i-th class of samples, y i represents the true label of sample i, and e is a constant.

[0025] The present invention also provides an ICU bedside chest X-ray intelligent recognition method, the auxiliary method comprising: identifying a target ICU bedside chest X-ray through an ICU bedside chest X-ray intelligent recognition model and outputting the probability of various symptoms of the target ICU bedside chest X-ray and its corresponding class activation map; wherein the ICU bedside chest X-ray intelligent recognition model is obtained through the training method of the ICU bedside chest X-ray intelligent recognition model described in any one of claims 1-5.

[0026] The present invention provides an ICU bedside chest X-ray intelligent recognition system, and the auxiliary system includes: a deep learning server and a cloud server. The deep learning server is equipped with an ICU bedside chest X-ray intelligent recognition model, and the ICU bedside chest X-ray intelligent recognition model is obtained by the training method of the ICU bedside chest X-ray intelligent recognition model described in any one of claims 1 to 5. The cloud server is used to receive the ICU bedside chest X-ray of the user end and transmit it to the deep learning server, and send the various symptom probabilities of the ICU bedside chest X-ray output by the deep learning server and the corresponding class activation map to the user end.

[0027] Furthermore, the cloud server is also used to transmit the probabilities of various symptoms of the ICU bedside chest X-ray to the MySQL database and to transmit the class activation graphs corresponding to the probabilities of various symptoms of the ICU bedside chest X-ray to the MinIO object storage.

[0028] Furthermore, the cloud server is used to manage the transmission of ICU bedside chest radiograph input and class activation map output corresponding to the probabilities of various symptoms for mobile terminals, PC terminals, and backend management.

[0029] Furthermore, the ICU bedside chest radiograph intelligent recognition system also includes: a message queue server, an OSS server, a database, and an access terminal. The message queue server is used to send the ICU bedside chest radiograph in the cloud server to the deep learning server. The OSS server is used to store the original ICU bedside chest radiograph of the patient and its corresponding class activation graph. The database includes a MySQL database, a Redis cache, and a MinIO object storage. The access terminal includes a mobile terminal, a PC terminal, and a background management. The access terminal is used for users to input ICU bedside chest radiographs and receive class activation graphs corresponding to various symptom probabilities output by the cloud server.

[0030] The present invention provides a training method for an ICU bedside chest X-ray intelligent recognition model, which has the following beneficial effects:

[0031] 1. This training method uses a dataset specifically for ICU bedside chest radiographs for training, which covers nine common pathological features that actually exist in the ICU, such as atelectasis, cardiomegaly, pulmonary consolidation, edema, pulmonary opacity, pleural effusion, pneumonia, pneumothorax, and no findings. By using these real-world data for training, the ICU bedside chest radiograph intelligent recognition model can more accurately identify and recognize ICU-specific chest imaging features, thereby significantly improving the accuracy of recognition.

[0032] 2. The loss function of this training method adds w to the original loss function i Afterwards, for different samples, w iWith corresponding values, the model's inclination towards various types of samples can be balanced during the training process, thereby effectively improving the accuracy of model training, and having good training effects for samples of multiple types and different numbers.

[0033] 3. The advantage of this ICU bedside chest X-ray intelligent recognition method is that compared with the existing model, the ICU bedside chest X-ray intelligent recognition model is more accurate in identifying ICU symptoms and ICU bedside chest X-rays, and can analyze the symptom probability and symptom class activation map for each symptom separately, which can effectively improve the efficiency and accuracy of doctors in identifying symptoms.

[0034] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the training flow chart of the ICU bedside chest radiograph intelligent recognition model;

[0036] Figure 2 This is the flow chart of the detection phase of the ICU bedside chest radiograph intelligent recognition model;

[0037] Figure 3 This is the user operation flow chart of the ICU bedside chest X-ray intelligent recognition system;

[0038] Figure 4 This is the system deployment diagram of the ICU bedside chest X-ray intelligent recognition system;

[0039] Figure 5 This is the system architecture diagram of the ICU bedside chest X-ray intelligent recognition system. DETAILED DESCRIPTION

[0040] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0041] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0042] In an embodiment of the present invention, a training method for an ICU bedside chest X-ray intelligent recognition model is provided. Figure 1 The training flow chart of the present invention shows that the training method of the ICU bedside chest X-ray intelligent recognition model includes steps S100, S200, S300, S400, S500, S600 and S700.

[0043] First, S100, the TransNext-Small model is initially trained using the ImageNet-1K dataset to obtain an original model. The ImageNet-1K dataset is an open source dataset, and the TransNext-Small model is an open source model. The TransNext-Small model is initially trained using the ImageNet-1K dataset to obtain a preliminary model for analyzing images, which is the original model.

[0044] Next, S200 extracts chest X-ray data of multiple categories of candidate symptoms in the CheXpert dataset to construct a first training dataset. The CheXpert dataset is an open source X-ray image dataset that contains various chest X-ray images and other types of X-ray images, such as leg images and hand images. This step extracts various chest X-ray images from the CheXpert dataset and combines these chest X-ray images into the first training dataset.

[0045] Specifically, the images in the first training data set (chest radiographs of multiple candidate symptoms in the CheXpert data set) are classified as follows: Atelectasis, Cardiomegaly, Consolidation, Edema, Enlarged Cardiomediastinum, Fracture, Lung Lesion, Lung Opacity, Pleural Effusion, Pneumonia, Pneumothorax, Pleural Other, Support Devices, No Finding, a total of 14 labels, the training rounds are set to 30 rounds, the learning rate learningrate = 0.0005, and the loss function is ASL. MAP and AUC-ROC are used as evaluation indicators of model training performance to obtain the best performance model.

[0046] Secondly, S300, image preprocessing is performed on the first training data set to obtain a second training data set. Among them, the execution steps of image preprocessing include: performing image size conversion and image enhancement; after image size conversion and image enhancement, performing image normalization. Furthermore, image size conversion and image enhancement include: converting the grayscale image into an RGB image; adjusting the image size to width 384*height 384*number of channels 3; setting the random variation range of image brightness to 0.3; setting the random variation range of image contrast to 0.3; setting the random rotation angle range of the image to 30; setting the maximum translation ratio of the image in the horizontal and vertical directions to (0.1, 0.1); setting the random scaling range of the image to 0.9 to 1.1.

[0047] Furthermore, the normalization process of the image after the size transformation and image enhancement includes:

[0048] Normalization processing, the normalization formula is as follows:

[0049]

[0050] Among them, mean is the mean of the image normalization parameter, mean = [0.485, 0.456, 0.406], std is the variance, std = [0.229, 0.224, 0.225], the values ​​in the brackets of mean and std respectively represent the mean and variance of the corresponding red, green and blue channels in the RGB image, input is the pixel value of the input image, and output is the normalized output. In the process of S300, the chest X-ray images in the first training data set are resized, feature image enhanced and image normalized, and finally a number of standardized chest X-ray images can be obtained. The several standardized chest X-ray images constitute the second training data set for training the original model.

[0051] Next, S400, the first model is obtained by training the original model with a second training data set. The second training data set is a set of several standardized chest radiograph images. Furthermore, the multiple rounds of training include: multiple rounds of training using a combined loss function L. The combined loss function L is defined as follows:

[0052]

[0053] Where C is the number of labels (target symptoms), p i represents the probability that the model predicts that sample i belongs to the positive class, γ + Represents the adjustment factor of the positive sample, y i Represents the true label of sample i, y for positive samples i =1,1-y i represents the probability of negative samples, p mi represents the corrected probability, defined as p mi =max(p i -m, 0), m represents the probability boundary, m ≥ 0, γ - Represents the adjustment factor of negative samples, w i is the weight, w i The calculation formula is as follows:

[0054] w i =y i e 1-ρ +(1-y i ) ρ

[0055] Among them, ρ represents the proportion of positive samples in the i-th class of samples, y i represents the true label of sample i, and e is a constant.

[0056] During multiple rounds of training, the original model and the first model will continuously learn the pathological features of several standardized chest X-ray images, thereby having the ability to initially identify pathological features. And the original model and the first model obtain p after each training. i , p i With y i By comparison, the closer the two are, the smaller the Loss value (loss value) of the combined loss function L is. When the Loss value becomes smaller and smaller, the trained model is more accurate.

[0057] It should be noted that in the field of machine learning and deep learning, the loss function is a function that measures the difference between the predicted result and the actual result. By minimizing the loss function, we can train the model to obtain more accurate training results. The model calculates the loss values ​​of multiple target symptoms and then sums them up to obtain the total loss value, thereby judging the accuracy of the model. For example, the loss value of sample one plus the loss value of sample two is obtained to obtain the final total loss value. However, there are some problems with the above method of calculating the loss value. In some practical problems, there is a large difference in the number of samples of different types. For ICU bedside chest radiographs with multiple symptoms, the number of ICU bedside chest radiographs of various symptoms is also inconsistent, resulting in the original model and the first model tending to predict samples as categories with more numbers during training. The total loss value obtained in this way will tend to the loss value of a category with the largest number of samples, rather than the loss value of each category in multiple categories being accurately calculated. The final total loss value will be biased towards the category with the largest number of samples, resulting in poor training results. In order to balance the inclination of the original model and the first model for various samples during training, a weight w of a combined loss function L is finally obtained through testing. i , add w to the original loss function i Afterwards, for different samples, w i With corresponding values, the model's inclination towards various types of samples can be balanced during training, thereby improving the accuracy of model training and achieving good training results for samples of multiple types and different numbers.

[0058] The key point is that the advantage of using the combined loss function L is that compared with the existing loss function, the combined loss function L can analyze and process a variety of data and images.

[0059] Secondly, S500, according to the actual multiple symptoms in the ICU ward, ICU bedside chest X-ray data of multiple categories of target symptoms are extracted from the MIMIC-CXR chest X-ray database to form an ICU bedside chest X-ray data set. The key point of this step is that the multiple categories of ICU bedside chest X-ray data extracted from the MIMIC-CXR chest X-ray database are screened for the ICU bedside chest X-rays corresponding to the target symptoms, and the obtained ICU bedside chest X-ray data set is highly targeted in identifying the characteristics of the target symptoms.

[0060] Secondly, S600, the ICU bedside chest X-ray data set is image preprocessed to obtain a third training data set. In this step, the ICU bedside chest X-ray data set is again subjected to image resizing and image enhancement; after image resizing and image enhancement, image normalization is performed to obtain a third training data set for ICU bedside chest X-rays. The core is that the ICU bedside chest X-rays of the third training data set can extract targeted pathological features, so that the first model can be trained with these targeted pathological features, in order to finally obtain a model that can accurately identify the pathological features of ICU bedside chest X-rays.

[0061] The third training data set is to extract 9 types of symptoms (ICU bedside chest X-rays) that intensive care unit doctors are concerned about from the MIMIC-CXR-JPG database: Atelectasis, Cardiomegaly, Consolidation, Edema, Lung Opacity, Pleural Effusion, Pneumonia, Pneumothorax, and No Finding, and select the bedside chest X-rays from them as the dedicated ICU bedside chest X-ray training set.

[0062] Finally, S700, the third training data set is used to train the first model to obtain an ICU bedside chest radiograph intelligent recognition model. In this step, the first model is trained multiple times with the third training data set, and the combined loss function L in S400 is also used for training to finally obtain an ICU bedside chest radiograph intelligent recognition model that can accurately identify the pathological characteristics of ICU bedside chest radiographs.

[0063] This training method uses a dataset specifically for ICU bedside chest radiographs for training, which covers nine common pathological features that actually exist in the ICU, such as atelectasis, cardiomegaly, pulmonary consolidation, edema, pulmonary opacity, pleural effusion, pneumonia, pneumothorax, and no findings. By using these real-world data for training, the ICU bedside chest radiograph intelligent recognition model can more accurately identify and recognize ICU-specific chest imaging features, thereby significantly improving the accuracy of recognition.

[0064] In addition, the loss function of this training method adds w to the original loss function i Afterwards, for different samples, w i With corresponding values, the model's inclination towards various types of samples can be balanced during the training process, thereby effectively improving the accuracy of model training, and having good training effects for samples of multiple types and different numbers.

[0065] For details, please refer to Figure 1 , which shows the flowchart of the deep learning model training phase, mainly including the following steps:

[0066] 1. ImageNet-1K pre-trained model: First, use the pre-trained weights on the ImageNet-1K dataset to initialize the TransNext-Small model to obtain the original model.

[0067] 2. CheXpert dataset: 14 types of chest X-ray data were extracted from the CheXpert dataset to construct the initial training dataset (i.e., the first training dataset).

[0068] 3. Data preprocessing: Preprocess the CheXpert dataset, including size transformation, image enhancement, and normalization, to better train the model.

[0069] 4. Pathological feature extraction: The model (original model) is placed on the preprocessed CheXpert data set for initial training, so that the model has a certain ability to identify chest X-ray pathological features and obtain the first model.

[0070] 5. MIMIC-CXR bedside chest X-ray: 9 types of bedside chest X-ray data were extracted from the MIMIC-CXR chest X-ray database, totaling 86,571 images constituting the critical care bedside chest X-ray data set (the second training data set).

[0071] 6. Data preprocessing: The critical care bedside chest X-ray data set obtained in step 5 is subjected to preprocessing operations such as size transformation, image enhancement, and standardization to obtain the third training data set.

[0072] 7. Extraction of ICU focus disease features: The model obtained in 4 is further put into the critical care bedside chest X-ray dataset (the third training dataset) for training, and the features of ICU focus disease features are extracted from it, and the fitting is continuously converged.

[0073] 8. Save model: Save the trained model with the best performance (ICU bedside chest X-ray intelligent recognition model) for future use.

[0074] The whole process aims to gradually refine and optimize, and eventually obtain a model that can perform well in the field of auxiliary analysis of symptoms that ICU doctors are concerned about.

[0075] It should be noted that after image preprocessing, the model is generally manually evaluated. There are two evaluation indicators, one is the mean average precision (MAP), and the other is the area under the ROC curve (AUC-ROC). The first evaluation indicator MAP (Mean Average Precision) measures the average accuracy performance of the model on multiple queries or categories. The calculation of MAP involves two main concepts: Precision and Recall.

[0076] Precision: Precision refers to the proportion of truly positive samples among all samples predicted by the model to be positive. Calculation formula:

[0077]

[0078] Among them, TP (True Positive) represents true positive examples, and FP (False Positive) represents false positive examples.

[0079] Recall: Recall refers to the proportion of all true positive samples that are correctly predicted as positive by the model. Calculation formula:

[0080]

[0081] Among them, FN (False Negative) represents a false negative example.

[0082] AP (Average Precision): AP is the average value of the precision calculated under different recall thresholds. By plotting the Precision-Recall (PR) curve, the area under the PR curve is calculated as the AP value. Simply put, AP is a measure of the detection ability of the model in a certain category.

[0083] PR curve: With recall as the horizontal axis and precision as the vertical axis, the curve shows the change in model precision under different recall rates. The higher the AP value, the better the model's precision performance under different recall rates.

[0084] MAP (Mean Average Precision):

[0085] MAP is the average of APs across multiple categories and is used to measure the detection performance of the model on the entire dataset. The specific process is as follows:

[0086] For each category, calculate its AP. Take the average of the AP of all categories to get MAP. The formula is:

[0087]

[0088] Where N is the number of categories, AP i is the AP value of the i-th category.

[0089] Evaluation indicator 2 is AUC-ROC (Area Under the Curve-Receiver Operating Characteristic), which measures the classification ability of the model by calculating the area under the ROC curve.

[0090] The second evaluation indicator is the ROC curve. The ROC curve (Receiver Operating Characteristic Curve) is a graphical tool used to show the performance of the classification model at different thresholds.

[0091] Horizontal axis: False Positive Rate (FPR), calculated as:

[0092]

[0093] Vertical axis: True Positive Rate (TPR), calculated as:

[0094]

[0095] Among them, TP (True Positive) represents a true positive example, FP (False Positive) represents a false positive example, TN (True Negative) represents a true negative example, and FN (False Negative) represents a false negative example.

[0096] AUC (Area Under the Curve) refers to the area under the ROC curve. The value range of AUC is between 0 and 1. The closer the AUC is to 1, the better the classification performance of the model.

[0097] The present invention also provides an ICU bedside chest X-ray intelligent recognition method, the method comprising: recognizing a target ICU bedside chest X-ray through an ICU bedside chest X-ray intelligent recognition model and outputting various symptom probabilities of the target ICU bedside chest X-ray and their corresponding class activation maps. The ICU bedside chest X-ray intelligent recognition model is obtained through the training method of the above-mentioned ICU bedside chest X-ray intelligent recognition model.

[0098] The advantage of this ICU bedside chest X-ray intelligent recognition method is that compared with the existing model, the ICU bedside chest X-ray intelligent recognition model is more accurate in identifying ICU symptoms and ICU bedside chest X-rays, and can analyze the symptom probability and symptom class activation map for each symptom separately, which can effectively improve the efficiency and accuracy of doctors in identifying symptoms.

[0099] The present invention also provides an ICU bedside chest X-ray intelligent recognition system, including a deep learning server and a cloud server.

[0100] The deep learning server is equipped with an ICU bedside chest X-ray intelligent recognition model, and the ICU bedside chest X-ray intelligent recognition model is obtained through the training method of the above-mentioned ICU bedside chest X-ray intelligent recognition model.

[0101] The ICU bedside chest radiograph intelligent recognition model is deployed on a dedicated inference server (deep learning server), which communicates with the cloud server through a message queue. When the backend initiates a detection request, the image-related information is packaged as a message and passed to the message queue. The deep learning server monitors this queue, automatically obtains and unpacks the message when there is a message, obtains the patient's bedside chest radiograph from the MinIO object storage based on the obtained information, uses the trained model for detection and generates the corresponding class activation map. After completion, the corresponding detection results are written to the database, and this batch of class activation maps are uploaded to the MinIO object storage.

[0102] The cloud server is used to receive data from the user end, save the processed data, and transmit it to the deep learning server, or to process the data output by the deep learning server, save it, and transmit it to the user end (receive the ICU bedside chest X-ray from the user end and transmit it to the deep learning server, and send the various symptom probabilities of the ICU bedside chest X-ray output by the deep learning server and their corresponding class activation maps to the user end).

[0103] Specifically, the cloud server is used to transfer data of the ICU bedside chest X-ray intelligent recognition model to the MySQL database, Redis cache, and MinIO object storage (the cloud server is also used to transfer the probabilities of various symptoms of the ICU bedside chest X-ray to the MySQL database and the class activation graphs corresponding to the probabilities of various symptoms of the ICU bedside chest X-ray to the MinIO object storage).

[0104] Specifically, the cloud server is used to manage the information input and output for transmission of mobile terminals, PC terminals and background management (the cloud server is used to manage the ICU bedside chest X-ray input and the class activation map output corresponding to the probabilities of various symptoms for transmission of mobile terminals, PC terminals and background management).

[0105] In addition, the cloud server has other functions: it is responsible for receiving requests from the front end and implementing corresponding services, including login service, detection service, detection record service, browsing history service and patient information service. Among them, login service: receives doctor-related information, compares it with the database, and performs identity verification. Detection service: receives requests, uses RabbitMQ message queue, and sends detection requests to the deep learning server in the form of messages. Detection record service: queries the corresponding detection records from the database and returns them to the front end. Browsing history service: queries the doctor's browsing records from the database and returns them to the front end. Patient information service: obtains the patient's chest X-ray information and electronic medical record information.

[0106] For details, please refer to Figure 2 , the cloud server transmits the original image to the ICU bedside chest radiograph intelligent recognition model (receives the bedside chest radiograph from the ICU as input); the ICU bedside chest radiograph intelligent recognition model preprocesses the original image to highlight the image features, and the preprocessing includes: adjusting the image size to 5384*384*3, normalizing the image with mean mean=[0.485, 0.456, 0.406] and variance std=[0.229, 0.224, 0.225]; the ICU bedside chest radiograph intelligent recognition model parses the preprocessed image to obtain the probabilities corresponding to 9 types of pathologies (after preprocessing, the image will be feature extracted, and this step aims to identify image patterns or abnormalities associated with specific symptoms) (perform multi-label classification) (the model extracts features, identify the probability of 9 diseases); the ICU bedside chest X-ray intelligent recognition model uses the weight information of the last layer "norm4" of the model, combined with Grad-CAM technology to generate 9 class activation maps of this chest X-ray (after completing the classification, the system will generate class activation maps, which can highlight which areas in the original image are most likely to lead to a specific recognition result, helping to understand the key basis for making recognition decisions); the cloud server saves and uploads the class activation map to MinIO object storage, and saves the probabilities corresponding to the 9 types of pathology to the MySQL database, and finally the cloud server transmits the results to the user end (finally, the cloud server will synthesize the results of the above analysis, including the probability of the identified symptoms and their location information in the original image, to form a final detection report).

[0107] See also Figure 3 , Figure 3 The flow chart for user operation is as follows:

[0108] 1. The user enters the patient's name through the system interface.

[0109] 2. The system checks whether the corresponding patient information exists. If so, it proceeds to the next step; if not, it prompts the user to re-enter the information.

[0110] 3. The user selects the chest X-ray that needs to be tested.

[0111] 4. The user selects an appropriate detection model.

[0112] 5. The system generates a detection record and saves it to the database, and sets the detection status to "waiting"; the system also generates a message to notify the deep learning server to start the detection process; in order to improve the user experience, the system notifies the user at this time that the detection request has been successfully sent; after this operation is completed, the subsequent processing tasks will be automatically completed by the deep learning server.

[0113] 6. If the deep learning server detects successfully, it saves the generated detection record to the database, rewrites the record status to "success", and saves the generated class activation map to MinIO storage.

[0114] 7. If the detection fails, the deep learning server will update the status field of the detection record to "failed" and generate a message to notify the user.

[0115] In addition, you can also design a user-side page to match this system. The user web client interface includes the following parts:

[0116] Login page: The user enters the account number and password to log in.

[0117] Chest X-ray detection page: Users can select the patient's chest X-ray image for detection.

[0118] Test record page: displays all chest X-ray test records in the database.

[0119] Patient record retrieval page: All test records of the patient in the database can be retrieved according to the patient's name.

[0120] Personal center page: displays the basic information and browsing history of the currently logged-in doctor.

[0121] Recognition result display page: displays the recognition results, including pathology probability and pathology heat map.

[0122] Electronic medical record display page: displays the patient's relevant electronic medical record data.

[0123] 404 page: The default page displayed when the browsed URL cannot be found.

[0124] This ICU bedside chest X-ray intelligent recognition system has the following beneficial effects:

[0125] 1. Improve recognition accuracy:

[0126] Specialized datasets: This system is trained using datasets specifically for ICU bedside chest radiographs, which cover nine pathological features (symptoms) that ICU doctors are concerned about, such as pleural effusion, pneumonia, pneumothorax, etc. By using these real-world data, the system can more accurately identify and recognize ICU-specific chest imaging features, significantly improving recognition accuracy.

[0127] Deep learning model: This training method uses the TransNext network as the deep learning model, adds ImageNet-1k weights, and trains with the ChexPert chest X-ray dataset, which can help the model effectively extract the features of bedside chest X-rays. The multi-category ICU bedside chest X-ray data extracted from the MIMIC-CXR chest X-ray database are obtained by screening ICU bedside chest X-rays, which are highly targeted, thereby improving the model's ability to recognize the pathological characteristics of ICU bedside chest X-rays.

[0128] 2. This system is designed for mobile devices (such as smartphones and tablets), and doctors can access chest X-ray recognition support at any time while moving freely between beds. This mobility greatly improves doctors' work efficiency, allowing them to handle multiple tasks more flexibly. The conditions of ICU patients change rapidly, and this system provides real-time image recognition support. Doctors can access and analyze patient images anytime and anywhere through mobile devices, without being restricted by location and scene, ensuring the timeliness and convenience of recognition.

[0129] 3. Unlike most existing chest X-ray analysis systems, this system is designed and optimized specifically for ICU bedside chest radiographs. Most existing systems use a mixed dataset of standard chest radiographs and other types of chest radiographs, which have a large difference in image quality and shooting conditions from ICU bedside chest radiographs, resulting in low recognition accuracy and practicality. This system, through a specialized dataset and optimized model, can better meet the needs of ICU doctors and provide more accurate and reliable recognition support.

[0130] 4. Traditional medical imaging software has a complex interface and high operating threshold, and it may be difficult for medical staff without a radiology background to master it. This system is optimized for mobile terminals and has an intuitive and easy-to-use interactive interface, so that even personnel without special training can easily get started, thereby further shortening the time cycle from shooting to obtaining recognition suggestions.

[0131] 5. This system can automatically analyze and identify chest X-rays, reducing the time doctors spend manually reviewing each image and improving work efficiency. The system can generate recognition results in a short period of time, helping doctors make decisions quickly, especially in emergency situations, which can significantly improve treatment efficiency.

[0132] 6. Use datasets specifically for ICU bedside chest radiographs to train deep learning models. These datasets contain a large number of chest radiographs actually taken in critical care environments, covering the pathological features that ICU doctors are concerned about. Unlike most existing systems that use a mixed dataset of standard chest radiographs and other types of chest radiographs, the dataset of this system is specifically for ICU bedside chest radiographs, which can more accurately identify and recognize ICU-specific chest imaging features.

[0133] 7. The system is designed for mobile devices (such as smartphones and tablets), allowing doctors to access chest X-ray recognition support functions when they move freely between beds. The design of mobile devices enables doctors to perform recognition anytime and anywhere, improving work efficiency and flexibility.

[0134] 8. The system provides real-time image recognition support. Doctors can access and analyze patient images anytime and anywhere through mobile devices, without being restricted by location and scene. Real-time feedback and convenient design ensure timely and efficient recognition, especially for patients with rapidly changing conditions in ICU environments.

[0135] 9. The system has an intuitive and easy-to-use interactive interface and supports multiple languages, making it convenient for medical staff with different language backgrounds to use. It reduces the difficulty of operation and improves user satisfaction and the practicality of the system.

[0136] See also Figure 4 and Figure 5 , ICU bedside chest radiograph intelligent recognition system also includes: load balancing server cluster for improving the concurrent processing capability of the system. Gateway server cluster for unified processing of all requests entering the system. System services for identity authentication and permission management. Management services for daily operation and maintenance management of the system.

[0137] A message queue server used for asynchronous communication and improving system response speed and service quality (sending ICU bedside chest X-rays in the cloud server to the deep learning server). Among them, the purpose of the message queue is: the system uses RabbitMQ message queues to communicate between multiple servers, and the server listens to the corresponding message queue. When there is a message, it automatically takes it out, unpacks it and performs corresponding processing. At the same time, the system also sets up a delayed message queue to save delayed messages. The delayed message is sent together with the chest X-ray detection message, and is pushed to the delayed message queue after the specified time. The back-end server listens to this delayed message queue, and obtains the corresponding patient information from it, and queries the database to check the patient's chest X-ray test results. If the patient's test results are not found in the database, the result status of this test record in the database is set to "failed".

[0138] A cache server used to speed up data access and reduce the pressure on the database. An OSS server used to store the original bedside chest X-ray data of the patient and the generated class activation graph data (used to store the original bedside chest X-ray of the patient's ICU and its corresponding class activation graph). The data layer includes data cache, read-write database, data synchronization and data message queue. The data layer is used for efficient management and transmission of data. The database includes MySQL database, Redis cache and MinIO object storage. The database is used to provide data persistence and high-speed access. The service layer includes chest X-ray detection service, electronic medical record service, detection record service, browsing history service, detection result query service and personal center service. The API interface layer includes API gateway and Nginx load balancing. The API interface layer is used to process and distribute requests. The access terminal includes mobile terminal, PC terminal and background management. The access terminal is used for user interaction and data input.

[0139] The overall architecture of the system is mainly divided into five modules: access terminal, API interface layer, service layer, data layer and operating environment. The access terminal includes mobile terminal, PC terminal and background management, which are responsible for user interaction and data input. The API interface layer contains API gateway and Nginx load balancing, which are used to process and distribute requests (for users to input ICU bedside chest X-rays and receive class activation graphs corresponding to various symptom probabilities output by the cloud server). The service layer includes chest X-ray detection service, electronic medical record service, detection record service, browsing history service, detection result query service and personal center service, providing core business functions. The data layer includes data cache, read-write database, data synchronization and data message queue to ensure efficient management and transmission of data.

[0140] The database module uses MySQL database, Redis cache and MinIO object storage to provide data persistence and high-speed access. The functions of different databases in the database module are as follows: MySQL database: stores patient chest X-ray information, electronic medical records, doctor information, test records, browsing records, etc. Redis cache database: stores the test results of patients tested on the same day, and stores login user information to improve the query efficiency of the system. Among them, the purpose of MinIO object storage is: based on MinIO object storage service, store the original image data of all patient chest X-rays, and save the class activation map generated after single image recognition in the form of folders. Each image has a unique URL, which is saved in the MySQL database for easy access to the patient's bedside chest X-ray image data.

[0141] The operating environment includes cloud servers and deep learning detection model servers, which support the system's high-performance computing and model reasoning. Permission control runs through the entire system to ensure secure access and operation of data.

[0142] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0143] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

Claims

1. A training method for an ICU bedside chest radiograph intelligent recognition model, characterized in that: include: The TransNext-Small model is trained using the ImageNet-1K dataset to obtain the original model; Extract chest X-ray data of multiple categories of candidate symptoms in the CheXpert data set to construct the first training data set; Performing image preprocessing on the first training data set to obtain a second training data set; The original model is trained using the second training data set to obtain a first model; ICU bedside chest X-ray data of multiple target symptoms are extracted from the MIMIC-CXR chest X-ray database to form the ICU bedside chest X-ray dataset; The third training data set is obtained by performing image preprocessing on the ICU bedside chest X-ray data set; The first model is trained using the third training data set to obtain an ICU bedside chest X-ray intelligent recognition model.

2. The training method of the ICU bedside chest X-ray intelligent recognition model according to claim 1 is characterized in that: The multiple candidate symptom categories include: Atelectasis, cardiomegaly, pulmonary consolidation, edema, pericarditis and mediastinitis, fracture, lung injury, lung opacity, pleural effusion, pneumonia, pneumothorax, other thoracic symptoms, life support equipment, and no findings.

3. The training method of the ICU bedside chest X-ray intelligent recognition model according to claim 1 is characterized in that: Multiple target symptoms include: Atelectasis, cardiomegaly, pulmonary consolidation, edema, lung opacity, pleural effusion, pneumonia, pneumothorax, and no findings.

4. The training method of the ICU bedside chest X-ray intelligent recognition model according to claim 1 is characterized in that: The steps of image preprocessing include image resizing, image enhancement and image normalization: Convert grayscale images to RGB images; Adjust the image size to 384 x 384 x width; Set the random variation range of image brightness to 0.3; Set the random variation range of image contrast to 0.3; Set the angle range of random image rotation to 30; Set the maximum translation ratio of the image in the horizontal and vertical directions to (0.1, 0.1); Set the random scaling range of the image to 0.9 to 1.1; The normalization formula for image normalization is: Among them, mean is the mean of the image normalization parameter, mean = [0.485, 0.456, 0.406], std is the variance, std = [0.229, 0.224, 0.225], the values ​​in the brackets of mean and std represent the mean and variance of the corresponding red, green, and blue channels in the RGB image, respectively, input is the pixel value of the input image, and output is the normalized output.

5. The training method of the ICU bedside chest X-ray intelligent recognition model according to claim 1 is characterized in that: The loss function L of the original model and the first model is: Where C is the number of target symptoms, p i represents the probability that the model predicts that sample i belongs to the positive class, γ + Represents the adjustment factor of the positive sample, y i Represents the true label of sample i, y for positive samples i =1,1-y i represents the probability of negative samples, p mi represents the corrected probability, defined as p mi =max(p i -m, 0), m represents the probability boundary, m ≥ 0, γ - Represents the adjustment factor of negative samples, w i is the weight, w i The calculation formula is: w i =y i e 1-ρ +(1-y i )e ρ Among them, ρ represents the proportion of positive samples in the i-th class of samples, y i represents the true label of sample i, and e is a constant.

6. An ICU bedside chest radiograph intelligent recognition method, characterized in that: include: The target ICU bedside chest X-ray is identified through the ICU bedside chest X-ray intelligent recognition model and the probability of various symptoms of the target ICU bedside chest X-ray and its corresponding class activation map are output; Among them, the ICU bedside chest X-ray intelligent recognition model is obtained by the training method of the ICU bedside chest X-ray intelligent recognition model described in any one of claims 1-5.

7. An ICU bedside chest X-ray intelligent recognition system, characterized in that: include: A deep learning server equipped with an ICU bedside chest X-ray intelligent recognition model, wherein the ICU bedside chest X-ray intelligent recognition model is obtained by the training method of the ICU bedside chest X-ray intelligent recognition model according to any one of claims 1 to 5; The cloud server is used to receive the ICU bedside chest X-ray from the user end and transmit it to the deep learning server, and send the various symptom probabilities of the ICU bedside chest X-ray output by the deep learning server and their corresponding class activation maps to the user end.

8. The ICU bedside chest X-ray intelligent recognition system according to claim 7, characterized in that: The cloud server is also used to transfer the probabilities of various symptoms of ICU bedside chest radiographs to the MySQL database and to transfer the class activation maps corresponding to the probabilities of various symptoms of ICU bedside chest radiographs to the MinIO object storage.

9. The ICU bedside chest X-ray intelligent recognition system according to claim 7, characterized in that: The cloud server is used to manage the transmission of ICU bedside chest X-ray input and class activation map output corresponding to the probabilities of various symptoms for mobile terminals, PC terminals and background management.

10. The ICU bedside chest X-ray intelligent recognition system according to claim 7, characterized in that: Also includes: Message queue server, used to send ICU bedside chest radiographs in the cloud server to the deep learning server; OSS server, used to store the original ICU bedside chest X-rays of patients and their corresponding class activation maps; Database, including MySQL database, Redis cache and MinIO object storage; Access terminal, including mobile terminal, PC terminal and background management. The access terminal is used by users to input ICU bedside chest X-rays and receive class activation maps corresponding to various symptom probabilities output by the cloud server.

Images