Acute infectious pneumonia intelligent monitoring system and method based on federal learning model
By adopting a federated learning model and multimodal, cross-modal intelligent diagnostic and early warning network in the diagnosis of acute infectious pneumonia, combined with CNN and Transformer networks, the problems of insufficient multimodal diagnostic capabilities and cumbersome diagnosis process in the diagnosis are solved, real-time monitoring and early warning of acute infectious pneumonia and unlearned lung diseases are achieved, and the accuracy and efficiency of diagnosis are improved.
Patent Information
- Application Number
- CN202411909902.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-25
AI Technical Summary
The lack of multimodal diagnostic ability of comprehensive imaging and clinical data in the diagnosis of acute infectious pneumonia leads to poor accuracy of qualitative diagnosis of lesions, cumbersome diagnosis process, slow speed, and lack of detection ability of unlearned data.
An intelligent monitoring system for acute infectious pneumonia based on federated learning model is adopted, and an encrypted two-way communication connection is established through the central server and the local clients of multiple hospitals. Multimodal databases, multimodal data preprocessing modules and multimodal and cross-modal intelligent diagnostic and early warning networks are used to detect multimodal case abnormalities. The system includes a CNN network and a Transformer network for feature fusion and deep feature extraction, enabling abnormal detection and early warning.
Realize instant monitoring and early warning of acute infectious pneumonia and unlearned lung diseases, improve the accuracy and efficiency of diagnosis, and overcome the problems of insufficient multimodal diagnosis capabilities and cumbersome diagnosis procedures in the prior art.
Smart Images
Figure CN119889731A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an intelligent monitoring system and method for acute infectious pneumonia based on a federated learning model, and belongs to the technical field of medical diagnosis. Background Art
[0002] Chest CT of acute infectious pneumonia has relatively specific manifestations. Therefore, in the early stage of the epidemic, when the pathogen is still uncertain or the production capacity of RT-PCR kits is limited, chest CT may be the only available detection or screening method for acute unexplained or infectious pneumonia. It has a much higher diagnostic sensitivity than RT-PCR nucleic acid detection (97% vs. 50%), and the detection rate is faster. Driven by this demand for imaging screening, chest CT artificial intelligence (AI) diagnostic models based on convolutional neural networks (CNN) have been rapidly developed and applied. However, these models generally have three shortcomings: 1. Lack of independent "one test for multiple diseases" capabilities, actual use requires the use of multiple algorithm models, and the diagnostic process is cumbersome and slow. 2. Lack of multimodal diagnostic capabilities of comprehensive imaging and clinical data, resulting in poor accuracy in qualitative diagnosis of lesions. 3. Lack of detection capabilities for unlearned data, such as pneumonia of unknown cause.
[0003] Data is the cornerstone of any AI model training. Whether or not a large number of high-quality data sets can be obtained has become one of the important factors restricting its further development. However, in the medical field, it is extremely difficult and costly to establish a large sample data set. The rapidly developing federated learning framework in recent years provides an ideal solution to the above problems. In federated learning, model training is performed independently on the client side of each medical center, and patient data is not shared or transmitted. Instead, the training results of all clients are "evaluated" through a central server, and then integrated and corrected. It can complete multi-center, large data model training without leaking user privacy data. This provides an ideal learning framework foundation for this project to establish a high-performance AI diagnosis model for multiple lung diseases. Summary of the invention
[0004] The present invention aims to solve the problems of lack of multimodal diagnostic capability of comprehensive imaging and clinical data in the diagnosis of acute infectious pneumonia, resulting in poor accuracy of qualitative diagnosis of lesions, lack of detection capability of unlearned data in the diagnostic method, cumbersome diagnostic process and slow speed, and thus proposes an intelligent monitoring system and method for acute infectious pneumonia based on a federated learning model, wherein the system includes:
[0005] Central server, local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6;
[0006] Local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 are all installed with encrypted two-way transmission network interfaces, and the central server has encrypted two-way communication connections with local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6.
[0007] Preferably, local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 all include a local multimodal database, a multimodal data preprocessing module and a multimodal, cross-modal intelligent diagnosis and early warning network;
[0008] The local multimodal database establishes a multimodal case database based on the recorded historical lung CT images, clinical text materials, and other classified clinical information;
[0009] The multimodal data preprocessing module is used to preprocess missing values in the multimodal database;
[0010] Multimodal and cross-modal intelligent diagnosis and early warning networks are used to detect anomalies in preprocessed multimodal case data.
[0011] Preferably, the multimodal and cross-modal intelligent diagnosis and early warning network includes a CNN network and a Transformer network;
[0012] The CNN network is used to perform local feature fusion on historical lung CT images, clinical text materials, and other classified clinical information in the multimodal case database;
[0013] The Transformer network is used to extract deep features from historical lung CT images, clinical text data, and other classified clinical information after fusion of local features, and map the deep features to the deep embedding space for anomaly detection.
[0014] The intelligent monitoring method for acute infectious pneumonia based on the federated learning model includes:
[0015] S1: local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 perform multimodal case anomaly detection based on a local multimodal database, a multimodal data preprocessing module and a multimodal and cross-modal intelligent diagnosis and early warning network;
[0016] S2: Local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 encrypt and transmit the multimodal case abnormality detection results to the central server, and the central server integrates the received multimodal case abnormality detection results and encrypts and sends them to each local client-hospital to update the local multimodal database;
[0017] S3: Repeat S1-S2 until the preset number of cycles is reached.
[0018] Preferably, the step of performing multimodal case anomaly detection in S1 includes:
[0019] S101: Establish a multimodal case database based on historical lung CT images, clinical text materials and other classified clinical information recorded in the local multimodal database;
[0020] S102: preprocessing the multimodal case database based on the multimodal data preprocessing module;
[0021] S103: Based on the multimodal and cross-modal intelligent diagnosis and early warning network, feature fusion and feature mapping are performed on the pre-processed multimodal case data to obtain the Gaussian distribution of the multimodal case data, and anomaly detection is performed based on the Gaussian distribution of the multimodal case data.
[0022] Preferably, the preprocessing of the multimodal case database in S102 specifically includes:
[0023] For missing data in the multimodal case database, a collaborative filtering missing value interpolation algorithm based on deep learning is used to intelligently interpolate missing data.
[0024] Preferably, the step of performing anomaly detection based on Gaussian distribution of multimodal case data in S103 includes:
[0025] S10301: Training multi-modal and cross-modal intelligent diagnosis and early warning networks;
[0026] S10302: Input historical lung CT images, clinical text materials and other classified clinical information in the multimodal case database into the trained CNN network for local feature extraction;
[0027] S10303: Extract deep features of historical lung CT images, clinical text data and other classified clinical information after fusion of local features based on Transformer network;
[0028] S10304: Map the deep features to the deep embedding space through dimensionality reduction, calculate the center and distribution range of all types of features through Gaussian distribution, and obtain the distance of the unknown sample from the center point of each classification in the deep embedding space;
[0029] S10305: Set a threshold and judge the outliers based on the distance of the unknown sample from the center point of each classification in the deep embedding space. If clustered outliers appear, an alarm message is sent to the central server.
[0030] The beneficial effects of the present invention are:
[0031] 1. The present invention screens cases from multiple hospitals and establishes a multimodal database required for intelligent monitoring of acute infectious pneumonia. Each hospital establishes an encrypted transmission connection with the local database and the central server, thereby realizing multimodal and cross-modal fusion of CT images and clinical data, providing data support for intelligent diagnosis of acute infectious pneumonia.
[0032] 2. The present invention adopts a multimodal learning network with a hybrid architecture of CNN+Transformer. As the most advanced network architecture in the field of AI in recent years, the hybrid architecture of CNN+Transformer overcomes the gap that this architecture has not been applied in the medical field. Compared with the existing CNN or Transformer, the hybrid architecture of CNN+Transformer adopted in the present invention has better trainability and excellent classification potential.
[0033] 3. The present invention realizes the real-time monitoring and early warning of acute infectious pneumonia and unlearned lung diseases, which helps to enable image AI to have rapid response and early warning capabilities, and has broad application prospects.
[0034] 4. The multimodal federated learning framework adopted by the present invention fills the gap in this framework in medical applications. The present invention can improve the problems of insufficient diagnosis and update of multiple diseases of image AI through the multimodal federated learning framework. It can realize the training of cross-regional and multi-center multimodal heterogeneous data models without leaking patient privacy data, thereby improving the diagnostic potential of image AI.
[0035] 5. The present invention enables the Center for Disease Control and Prevention to conduct cross-regional intelligent epidemiological monitoring of lung diseases, thereby preventing large-scale acute infectious pneumonia or pneumonia of unknown cause that may occur again in the future. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A framework diagram of an intelligent monitoring system for acute infectious pneumonia based on a federated learning model provided by the present invention;
[0037] Figure 2A flowchart of the intelligent monitoring method for acute infectious pneumonia based on the federated learning model provided by the present invention;
[0038] Figure 3 A flow chart of data preprocessing provided by the present invention;
[0039] Figure 4 A schematic diagram of the CNN+Transformer hybrid architecture model provided by the present invention;
[0040] Figure 5 A schematic diagram of anomaly detection provided by the present invention;
[0041] Figure 6 A schematic diagram of the structure of the federated learning framework provided by the present invention. DETAILED DESCRIPTION
[0042] Specific implementation method 1: Combination Figure 1 and Figure 6 To illustrate this embodiment, Figure 1 As shown, the intelligent monitoring system for acute infectious pneumonia based on the federated learning model described in this embodiment includes:
[0043] Central server, local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6;
[0044] Local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 are all installed with encrypted two-way transmission network interfaces, and the central server has encrypted two-way communication connections with local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6.
[0045] like Figure 6 As shown, local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 all include a local multimodal database, a multimodal data preprocessing module and a multimodal, cross-modal intelligent diagnosis and early warning network;
[0046] The local multimodal database establishes a multimodal case database based on the recorded historical lung CT images, clinical text materials, and other classified clinical information;
[0047] The multimodal data preprocessing module is used to preprocess missing values in the multimodal database;
[0048] Multimodal and cross-modal intelligent diagnosis and early warning networks are used to detect anomalies in preprocessed multimodal case data.
[0049] Multimodal and cross-modal intelligent diagnosis and early warning networks include CNN networks and Transformer networks;
[0050] The CNN network is used to perform local feature fusion on historical lung CT images, clinical text materials, and other classified clinical information in the multimodal case database;
[0051] The Transformer network is used to extract deep features from historical lung CT images, clinical text data, and other classified clinical information after fusion of local features, and map the deep features to the deep embedding space for anomaly detection.
[0052] The present invention adopts a multimodal learning network with a hybrid architecture of CNN+Transformer. As the most advanced network architecture in the field of AI in recent years, the hybrid architecture of CNN+Transformer overcomes the gap that this architecture has not been applied in the medical field. Compared with the existing CNN or Transformer, the hybrid architecture of CNN+Transformer adopted in the present invention has better trainability and excellent classification potential.
[0053] Specific implementation method 2: Combination Figure 2-5 To illustrate this embodiment, Figure 2 As shown, the steps of the intelligent monitoring method for acute infectious pneumonia based on the federated learning model described in this embodiment include:
[0054] S1: local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 perform multimodal case anomaly detection based on a local multimodal database, a multimodal data preprocessing module and a multimodal and cross-modal intelligent diagnosis and early warning network;
[0055] This implementation method independently collects case data in 8 independent medical centers (including Union Hospital Headquarters, Brain Tumor Center, West Hospital, Jinyinhu Hospital, Union Jiangnan Hospital, Union Jiangbei Hospital, Union Dongxihu Hospital and Union Red Cross Hospital). Each independent medical center has mature and independent case data and medical imaging digital and information management systems. Case collection only includes patients with various lung diseases that are clinically confirmed or pathologically confirmed (ICD-10 coded diagnosis), and their confirmed ICD-10 coded diagnosis is used as a label. A single label can allow the presence of multiple lung lesions at the same time. After the case is selected, this implementation method locally collects the patient's chest CT images and clinical data (including structured outpatient medical records, structured admission records and laboratory test results), and establishes a multimodal database based on the iCTCF architecture that has been developed. The single-center sample of the database established in this implementation method can reach 10,000 to 20,000, and at least 10 common diseases are included.
[0056] S101: Establish a multimodal case database based on historical lung CT images, clinical text materials and other classified clinical information recorded in the local multimodal database;
[0057] S102: Figure 3 As shown, the multimodal case database is preprocessed based on the multimodal data preprocessing module, and the missing data in the multimodal case database is intelligently interpolated using a collaborative filtering missing value interpolation algorithm based on deep learning;
[0058] S103: Figure 4 As shown, based on the multimodal and cross-modal intelligent diagnosis and early warning network, feature fusion and feature mapping are performed on the pre-processed multimodal case data to obtain the Gaussian distribution of the multimodal case data, and anomaly detection is performed based on the Gaussian distribution of the multimodal case data;
[0059] S10301: Training multi-modal and cross-modal intelligent diagnosis and early warning networks;
[0060] S10302: Input historical lung CT images, clinical text materials and other classified clinical information in the multimodal case database into the trained CNN network for local feature extraction;
[0061] S10303: Extract deep features of historical lung CT images, clinical text data and other classified clinical information after fusion of local features based on Transformer network;
[0062] S10304: Figure 5 As shown, the deep features are reduced in dimension and mapped to the deep embedding space, and the center and distribution range of all types of features are calculated through Gaussian distribution to obtain the distance of unknown samples from the center points of each classification in the deep embedding space;
[0063] S10305: Set a threshold and judge the outliers based on the distance of the unknown sample from the center point of each classification in the deep embedding space. If clustered outliers appear, an alarm message is sent to the central server.
[0064] S2: Local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 encrypt and transmit the multimodal case abnormality detection results to the central server, and the central server integrates the received multimodal case abnormality detection results and encrypts and sends them to each local client-hospital to update the local multimodal database;
[0065] S3: Repeat S1-S2 until the preset number of cycles is reached.
[0066] In summary, this implementation method utilizes the above-mentioned technical means to overcome the bottleneck problems existing in the current image AI diagnosis, and through the combined application of multiple networks and algorithms at the forefront of current AI, through more novel multimodal modeling and training, more accurate intelligent diagnosis and abnormality detection of multiple diseases can be achieved. This implementation method incorporates the digitization and information management of various pneumonia case materials that have been carried out by multiple hospitals since more than ten years ago, and has established various case management information databases, realized electronic case retrieval, and realized a model of medical data sharing among multiple medical centers, merging the scattered medical data of multiple centers into a whole, thereby improving the early positive detection rate of infectious diseases, and conducting cross-regional intelligent epidemiological monitoring of lung diseases to prevent large-scale acute infectious pneumonia or unexplained pneumonia epidemics that may occur again in the future.
[0067] In addition, the present invention has already adopted multimodal database establishment, CNN segmentation and differential diagnosis (such as COVID-Lesion Net, etc.), CNN+Transformer hybrid architecture classification network and federated learning network foundation in the early stage of modeling, which can meet the overall modeling requirements of the present invention and make the present invention feasible.
[0068] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.
Claims
1. The intelligent monitoring system for acute infectious pneumonia based on the federated learning model is characterized by: The structure of the acute infectious pneumonia intelligent monitoring system based on the federated learning model includes: Central server, local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6; The local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 are all installed with encrypted two-way transmission network interfaces, and the central server has encrypted two-way communication connections with the local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6.
2. According to claim 1, the intelligent monitoring system for acute infectious pneumonia based on the federated learning model is characterized in that: The local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 all include a local multimodal database, a multimodal data preprocessing module and a multimodal, cross-modal intelligent diagnosis and early warning network; The local multimodal database establishes a multimodal case database based on the recorded historical lung CT images, clinical text materials and other classified clinical information; The multimodal data preprocessing module is used to preprocess missing values in the multimodal database; The multimodal and cross-modal intelligent diagnosis and early warning network is used to perform anomaly detection on pre-processed multimodal case data.
3. The intelligent monitoring system for acute infectious pneumonia based on a federated learning model according to claim 2 is characterized in that: The multimodal and cross-modal intelligent diagnosis and early warning network includes a CNN network and a Transformer network; The CNN network is used to perform local feature fusion on historical lung CT images, clinical text materials, and other classified clinical information in the multimodal case database; The Transformer network is used to extract deep features of historical lung CT images, clinical text materials and other classified clinical information after fusion of local features, and map the deep features to the deep embedding space for anomaly detection.
4. An intelligent monitoring method for acute infectious pneumonia based on a federated learning model, applied to an intelligent monitoring system for acute infectious pneumonia based on a federated learning model according to any one of claims 1 to 3, characterized in that: The specific steps include: S1: The local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 perform multimodal case anomaly detection based on a local multimodal database, a multimodal data preprocessing module and a multimodal and cross-modal intelligent diagnosis and early warning network; S2: The local client-hospital 1, local client-hospital 2, local client-hospital 3, local client-hospital 4, local client-hospital 5 and local client-hospital 6 encrypt and transmit the multimodal case abnormality detection results to the central server, and the central server integrates the received multimodal case abnormality detection results and encrypts and sends them to each local client-hospital to update the local multimodal database; S3: Repeat S1-S2 until the preset number of cycles is reached.
5. The intelligent monitoring method for acute infectious pneumonia based on a federated learning model according to claim 4 is characterized in that: The steps of multimodal case anomaly detection in S1 include: S101: Establishing a multimodal case database based on historical lung CT images, clinical text materials and other classified clinical information recorded in the local multimodal database; S102: Preprocessing the multimodal case database based on the multimodal data preprocessing module; S103: Based on the multimodal and cross-modal intelligent diagnosis and early warning network, feature fusion and feature mapping are performed on the pre-processed multimodal case data to obtain the Gaussian distribution of the multimodal case data, and anomaly detection is performed based on the Gaussian distribution of the multimodal case data.
6. The intelligent monitoring method for acute infectious pneumonia based on a federated learning model according to claim 5 is characterized in that: The preprocessing of the multimodal case database in S102 specifically includes: For the missing data in the multimodal case database, a collaborative filtering missing value interpolation algorithm based on deep learning is used to intelligently interpolate the missing data.
7. The intelligent monitoring method for acute infectious pneumonia based on a federated learning model according to claim 5 is characterized in that: The step of performing anomaly detection based on Gaussian distribution of multimodal case data in S103 includes: S10301: Training multi-modal and cross-modal intelligent diagnosis and early warning networks; S10302: inputting historical lung CT images, clinical text materials and other classified clinical information in the multimodal case database into the trained CNN network to extract local features; S10303: Extract deep features of historical lung CT images, clinical text data and other classified clinical information after fusion of local features based on Transformer network; S10304: Map the deep features to the deep embedding space through dimensionality reduction, calculate the center and distribution range of all types of features through Gaussian distribution, and obtain the distance of the unknown sample from the center point of each classification in the deep embedding space; S10305: Set a threshold and judge the outliers based on the distance of the unknown sample from the center point of each classification in the deep embedding space. If clustered outliers appear, an alarm message is sent to the central server.