Multi-modal artificial intelligence assisted pneumonia differential diagnosis system and terminal based on chest CT image

Through a multimodal artificial intelligence system based on chest CT images, the pneumonia identification model is constructed using image and text information, and the problem of insufficient identification of pneumonia types in the existing technology is solved, and accurate identification and targeted treatment of 10 pneumonia types are achieved. , improves the therapeutic effect.

CN118942669APending Publication Date: 2024-11-12ZHONGSHAN HOSPITAL FUDAN UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410847760.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing deep learning models or identification systems only involve the identification of pneumonia types or individual pathogens, and cannot meet the clinical needs of different diagnosis and treatment strategies for different pathogens. In addition, traditional microbial detection methods have sensitivity, specificity, poor timeliness and antibacterial use. Impact issues.

Method used

A multimodal artificial intelligence-assisted differential diagnosis system for pneumonia based on chest CT images, collects 10 cases of pneumonia types through the training sample acquisition module, constructs image classifiers and non-image classifiers, establishes differential diagnosis model of pneumonia, and uses chest CT images and Clinical text information is identified.

Benefits of technology

Accurate identification of 10 types of pneumonia has been achieved, helping clinicians to formulate targeted treatment strategies, reduce unnecessary antibacterial drugs, improve treatment effect, and reduce misdiagnosis and missed diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942669A_ABST
    Figure CN118942669A_ABST
Patent Text Reader

Abstract

The invention provides a multi-modal artificial intelligence assisted pneumonia differential diagnosis system and terminal based on a chest CT image, and the method comprises the steps: constructing an image classifier and a non-image classifier through the chest CT image in a training sample containing 10 pneumonia types and clinical text information, so as to build a pneumonia differential diagnosis model; and based on the pneumonia differential diagnosis model, obtaining a corresponding pneumonia identification result according to an input chest CT image of a pneumonia case and clinical text information. According to the invention, 10 pneumonia types can be identified, and the accuracy is extremely high. Clinical doctors can formulate microorganism inspection or diagnosis strategies in a targeted manner according to the diagnosis result and implement targeted anti-infection treatment, so that unnecessary use of antibacterial drugs is remarkably reduced, the treatment effect is improved, and misdiagnosis and missed diagnosis are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of lung infection pathogen identification, and in particular to a multimodal artificial intelligence-assisted pneumonia differential diagnosis system and terminal based on chest CT images. Background Art

[0002] There are many types of pathogens of lung infection. Traditional microbial detection methods have problems such as poor sensitivity, specificity, timeliness and being affected by the use of antibiotics. More than 50% of lung infections cannot be identified. Early rapid and accurate etiological diagnosis and early targeted treatment are the key to improving the prognosis of difficult lung infections and reducing mortality. As an important technology for the diagnosis of lung infection, chest CT imaging can reflect the histopathological characteristics to a certain extent, making it possible for chest CT to evaluate the infectious pathogens at the first time. Many lung infection lesions caused by specific pathogens have certain characteristic manifestations in chest CT imaging. The diagnosis of pneumonia in real scenarios is very complicated, not just limited to viral pneumonia, bacterial pneumonia, fungal pneumonia, and tuberculosis. For example, the diagnosis and treatment strategies of Pneumocystis jiroveci pneumonia, pulmonary cryptococcosis, and pulmonary aspergillosis in fungal pneumonia are completely different.

[0003] At present, artificial intelligence or deep learning models related to chest imaging of lung infection are mainly used to distinguish COVID-19 from other types of pneumonia, or to distinguish 3-4 types of pneumonia, mainly including bacterial pneumonia, fungal pneumonia, viral pneumonia and tuberculosis. The deep learning models or identification systems proposed in the prior art only involve the identification of pneumonia types or individual pathogens. Different pathogens have completely different diagnosis and treatment strategies, and previous results cannot meet actual clinical needs. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a multimodal artificial intelligence-assisted pneumonia differential diagnosis system and terminal based on chest CT images, which is used to solve the technical problem that the deep learning model or identification system proposed in the prior art only involves the identification of pneumonia types or individual pathogens, and cannot meet the clinical needs of different diagnosis and treatment strategies for different pathogens.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images, the system comprising: a training sample acquisition module, used to collect multiple pneumonia cases covering 10 types of pneumonia, and use the chest CT images and clinical text information of each pneumonia case as training samples marked with corresponding pneumonia type labels; a model construction module, connected to the training sample acquisition module, used to construct an image classifier and a non-image classifier based on the chest CT images and clinical text information in each training sample, so as to establish a pneumonia differential diagnosis model; a diagnosis module, connected to the model construction module, used to obtain corresponding pneumonia identification results based on the pneumonia differential diagnosis model and the input chest CT images and clinical text information of the pneumonia cases.

[0006] In one embodiment of the present invention, the 10 types of pneumonia include: bacterial pneumonia, atypical pathogen pneumonia, Nocardia pneumonia, pulmonary cryptococcosis, pulmonary aspergillosis, Pneumocystis jiroveci pneumonia, pulmonary tuberculosis, pulmonary nontuberculous mycobacterial disease, pulmonary malignancy and pneumonia-like disease.

[0007] In one embodiment of the present invention, the pneumonia differential diagnosis model includes: a model input module, used to input chest CT images and clinical text information of pneumonia cases; an image classifier, connected to the model input module, used to obtain corresponding image classification results based on the chest CT images of the input pneumonia cases; wherein the image classification results include: image classification probability values ​​for each type of pneumonia; a non-image classifier, connected to the model input module, used to obtain corresponding non-image classification results based on the input clinical text information of the pneumonia cases; wherein the non-image classification results include: non-image classification probability values ​​for each type of pneumonia; a result integration module, connected to the image classifier and the non-image classifier, used to integrate the image classification results and the non-image classification results to obtain the pneumonia differential results of the corresponding pneumonia cases.

[0008] In one embodiment of the present invention, a method for constructing an image classifier based on chest CT images in each training sample includes: preprocessing the chest CT images in each input training sample; normalizing the lung window and the standard window of each preprocessed chest CT image; and inputting the images normalized by the lung window and the standard window into a 3D-DenseNet network through dual channels for training to obtain an image classifier.

[0009] In one embodiment of the present invention, the method of preprocessing the chest CT images in each input training sample includes: using a neural network to automatically segment the left and right lungs in the chest CT image to obtain the bounding boxes of the left and right lungs; extracting the lung field area from the chest CT image based on the bounding boxes of the left and right lungs to obtain multiple lung field image blocks; adjusting each lung field image block to the same pixel.

[0010] In one embodiment of the present invention, a method for constructing a non-imaging classifier based on clinical text information in each training sample includes: preprocessing the clinical text information in each training sample; and using the preprocessed clinical text information to train a GBT network to obtain a non-imaging classifier.

[0011] In one embodiment of the present invention, the clinical text information includes: gender, age, white blood cell count, neutrophil count, erythrocyte sedimentation rate, C-reactive protein, procalcitonin, carcinoembryonic antigen, T-SPOT.TB test results, symptoms and symptom duration.

[0012] To achieve the above-mentioned purpose and other related purposes, the present invention provides an electronic terminal, including: the multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images.

[0013] As described above, the present invention is a multimodal artificial intelligence-assisted pneumonia differential diagnosis system and terminal based on chest CT images, which has the following beneficial effects: the present invention uses chest CT images and clinical text information in training samples covering 10 types of pneumonia to respectively construct image classifiers and non-image classifiers to establish a pneumonia differential diagnosis model; based on the pneumonia differential diagnosis model, the corresponding pneumonia identification results are obtained according to the chest CT images and clinical text information of the input pneumonia cases. The present invention can identify 10 types of pneumonia with extremely high accuracy. Based on this diagnostic result, clinicians can formulate targeted microbial inspection or diagnostic strategies and implement targeted anti-infection treatment, thereby significantly reducing unnecessary use of antibiotics, improving treatment effects, and reducing misdiagnosis and missed diagnoses. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Shown is a schematic diagram of the structure of a multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images in one embodiment of the present invention.

[0015] Figure 2 Shown is a schematic diagram of the structure of a pneumonia differential diagnosis model in one embodiment of the present invention.

[0016] Figure 3 Shown is a schematic structural diagram of an electronic terminal in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0018] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present invention. It should be understood that other embodiments may also be used, and that mechanical composition, structural, electrical and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present invention is limited only by the claims of the published patents. The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.

[0019] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a part is said to "include" a certain constituent element, unless otherwise stated, it does not exclude other constituent elements, but means that other constituent elements may be included.

[0020] The terms first, second and third mentioned herein are used to describe various parts, components, regions, layers and / or segments, but are not limited thereto. These terms are only used to distinguish a certain part, component, region, layer or segment from other parts, components, regions, layers or segments. Therefore, the first part, component, region, layer or segment described below may refer to the second part, component, region, layer or segment within the scope of the present invention.

[0021] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.

[0022] The present invention provides a multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images. By using chest CT images and clinical text information in training samples covering 10 types of pneumonia, image classifiers and non-image classifiers are respectively constructed to establish a pneumonia differential diagnosis model; based on the pneumonia differential diagnosis model, the corresponding pneumonia identification results are obtained according to the chest CT images and clinical text information of the input pneumonia cases. The present invention can identify 10 types of pneumonia with extremely high accuracy. Clinicians can formulate microbial inspection or diagnostic strategies in a targeted manner based on this diagnostic result, and implement targeted anti-infection treatment, thereby significantly reducing unnecessary use of antibiotics, improving treatment effects, and reducing misdiagnosis and missed diagnosis.

[0023] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0024] like Figure 1 A structural schematic diagram of a multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images in an embodiment of the present invention is shown.

[0025] The system comprises:

[0026] The training sample acquisition module 1 is used to collect multiple pneumonia cases covering 10 types of pneumonia to build a comprehensive and diverse data set. In this embodiment, the 10 types of pneumonia include: bacterial pneumonia, pneumonia caused by atypical pathogens, Nocardia pneumonia, pulmonary cryptococcosis, pulmonary aspergillosis, Pneumocystis jiroveci pneumonia, tuberculosis, pulmonary non-tuberculous mycobacterial disease, lung malignancy and pneumonia-like, which not only involve most common types of pneumonia, but also include types of pneumonia caused by various specific pathogens, which can provide more accurate pathogen prediction. The training sample acquisition module 1 uses the chest CT image and clinical text information of each pneumonia case as training samples marked with corresponding pneumonia type labels; among them, the chest CT image is an important basis for the diagnosis of pneumonia cases, providing a detailed view of the lung structure, which helps to judge the distribution and degree of inflammation. At the same time, the clinical text information contains the patient's medical history, symptoms, signs, laboratory tests and other data, which provides important clinical clues for pathogen diagnosis.

[0027] A model building module 2, connected to the training sample acquisition module 1, is used to respectively build an image classifier and a non-image classifier based on the chest CT images and clinical text information in each training sample to establish a pneumonia differential diagnosis model; specifically, the chest CT images in each training sample are used to build an image classifier, and the clinical text information in each training sample is used to build a non-image classifier, thereby establishing a pneumonia differential diagnosis model;

[0028] For example, we collected a total of 1,091 pneumonia cases with clear pathogen diagnosis in 10 categories. Among these cases, we specially selected 461 cases for training, validation, and testing of image classifiers and non-image classifiers. At the same time, in order to enhance the generalization ability and accuracy of the model, we additionally included 630 chest CT images and added them to the training set of the image classifier. After completing the model performance evaluation, we conducted a detailed comparison of the model's diagnostic results with the reading results of 4 experienced senior radiologists and 4 senior respiratory physicians.

[0029] The diagnosis module 3 is connected to the model building module 2 and is used to obtain corresponding pneumonia identification results based on the pneumonia differential diagnosis model according to the chest CT images and clinical text information of the input pneumonia cases, and the pneumonia identification results include specific pneumonia types.

[0030] The present invention can quickly and accurately identify the specific type of pneumonia cases and can clearly present the identification results to users, which not only improves the efficiency and accuracy of pneumonia diagnosis, but also provides doctors with an important auxiliary diagnostic tool, helping them to develop more accurate treatment plans for patients.

[0031] In one embodiment, the clinical text information includes: gender, age, white blood cell count, neutrophil count, erythrocyte sedimentation rate, C-reactive protein, procalcitonin, carcinoembryonic antigen, T-SPOT.TB test results, symptoms and symptom duration. Specific symptoms include fever, cough, sputum, shortness of breath, dyspnea, chest pain, blood in sputum and hemoptysis.

[0032] In one embodiment, if Figure 2 As shown, the pneumonia differential diagnosis model includes:

[0033] Model input module, used to input chest CT images and clinical text information of pneumonia cases;

[0034] An image classifier, connected to the model input module, is used to obtain corresponding image classification results according to the chest CT image of the input pneumonia case; wherein the image classification results include: image classification probability values ​​of 10 types of pneumonia;

[0035] A non-image classifier, connected to the model input module, is used to obtain corresponding non-image classification results according to the clinical text information of the input pneumonia case; wherein the non-image classification results include: non-image classification probability values ​​of 10 types of pneumonia;

[0036] The result integration module is connected to the image classifier and the non-image classifier, and is used to integrate the image classification results and the non-image classification results to obtain the pneumonia identification results of the corresponding pneumonia cases.

[0037] In one embodiment, the image classifier uses a DenseNet network, and the method of constructing the image classifier based on the chest CT images in each training sample includes:

[0038] The chest CT images in each input training sample are preprocessed, for example, to improve image quality, reduce noise and artifacts, and enhance useful information in the image so that the model can learn more effectively.

[0039] The lung window and standard window are normalized for each preprocessed chest CT image; CT images usually contain a variety of tissues and structures, each of which has a different density (i.e., pixel value) in the CT image. Lung window (such as window length -400HU / , window width 1500HU) and standard window (such as window length 40HU / , window width 350HU) are two commonly used window settings, which are used to highlight tissues of different density ranges in CT images. For each window, the pixel value is normalized to a range that the model can handle, which helps the model better learn the features in the image.

[0040] The images normalized by the lung window and the standard window are input into the 3D-DenseNet network through dual channels for training to obtain an image classifier. Specifically, the images normalized by the lung window and the standard window are input into the 3D-DenseNet network as two channels respectively. This allows the network to learn the image features under two different window settings at the same time, which may improve the classification performance. 3D-DenseNet is a deep convolutional neural network structure for processing 3D images. It improves information flow and gradient propagation through dense connections (ie, each layer is directly connected to its subsequent layer). Since chest CT is 3D data, the use of a 3D network can capture spatial context information, which is very important for many medical applications.

[0041] Preferably, the image classifier is developed based on the PyTorch framework and optimized with focal loss as the loss function. During the training process, we used the ADAM optimizer and set the initial learning rate to 0.001. The model was trained from scratch for 1000 epochs. In order to monitor the performance of the model, we performed an accuracy evaluation every 20 epochs and finally selected the epoch with the highest accuracy as the final result of the model. In addition, in order to enhance the robustness of the network, we introduced data enhancement strategies including flipping, rotation, scaling, etc. during the training process.

[0042] In a specific embodiment, the method of preprocessing the chest CT images in each input training sample includes:

[0043] Using a neural network to automatically segment the left and right lungs in the chest CT image to obtain the bounding boxes of the left and right lungs; wherein the neural network used may be a network model architecture such as V-Net, U-Net, and Transformer;

[0044] Extracting the lung field region from the chest CT image based on the bounding boxes of the left and right lungs to obtain multiple lung field image blocks;

[0045] Each lung field image block is adjusted to the same pixel. For example, each lung field image block is adjusted to 192×192×192 pixels to ensure the consistency of the size of the lung field region between images.

[0046] In one embodiment, the method of constructing a non-image classifier based on the clinical text information in each training sample includes:

[0047] Preprocess the clinical text information in each training sample; for example, for various laboratory results (such as white blood cell count, neutrophil count, etc.), standardize them according to the minimum and maximum values ​​in the training set and convert them into values ​​between [0,1]. This helps to eliminate differences between different indicators due to different units or dimensions. For data of negative or positive, yes or no, etc., use 0 and 1 for encoding. This makes it easier for machine learning models to process. And the duration is expressed in days. For missing values, you can use a specific value (such as -1024) to fill them, or use other strategies (such as mean filling, median filling, interpolation, etc.). -1024 was chosen because it is a value that is unlikely to appear in the normal data range, which helps the model identify missing data.

[0048] The preprocessed clinical text information is used to train a Gradient Boosting Trees (GBT) network to obtain a non-imaging classifier. The GBT network used is, for example, XGBoost, LightGBM, or GradientBoostingClassifier of scikit-learn.

[0049] In one embodiment, integrating the image classification results and the non-image classification results to obtain the pneumonia identification results corresponding to the pneumonia cases includes:

[0050] The image classification probability value and non-image classification probability value of each pneumonia type are added and normalized to ensure that the total probability of all pneumonia types is 1;

[0051] According to the normalized probability values, the three pneumonia types with the highest probabilities are selected as the pneumonia identification results output.

[0052] It should be noted that if a certain pneumonia type is not predicted in the image classification or non-image classification (i.e., the probability is 0), but has a higher probability in another classification method, then the probability after addition may still be low. This may cause the type to be ignored in the final identification result. To avoid this situation, you can consider setting a minimum probability threshold for each type before adding to ensure that even if one classification method does not predict the type, it still has a chance to occupy a place in the final result. This scheme can also consider using more complex fusion strategies, such as weighted averaging, Bayesian fusion, etc., to more finely integrate information from different sources. These strategies can be selected according to specific application scenarios and data characteristics.

[0053] The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the electronic terminal, please refer to Figure 3 , is an optional hardware structure diagram of an electronic terminal 1000 provided in an embodiment of the present invention. The electronic terminal 1000 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The electronic terminal 1000 includes: at least one processor 1001, a memory 1002, at least one network interface 10010 and a user interface 10010. The various components in the device are coupled together through a bus system 1005. It can be understood that the bus system 1005 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1005 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 3 In the specification, various buses are labeled as bus systems.

[0054] Among them, the user interface 10010 may include a display, keyboard, mouse, trackball, click gun, keys, buttons, touch pad or touch screen, etc.

[0055] It is understood that the memory 1002 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0056] The memory 1002 in the embodiment of the present invention is used to store various categories of data to support the operation of the terminal 1000. Examples of these data include: any executable program for operating on the terminal 1000, such as an operating system 10021 and an application 10022; the operating system 10021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. Application 10022 may include various applications, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images provided in the embodiment of the present invention may be included in application 10022.

[0057] The method disclosed in the above embodiment of the present invention can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1001 or the instruction in the form of software. The above processor 1001 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 1001 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 1001 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0058] In an exemplary embodiment, the terminal 1000 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0059] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0060] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0061] In summary, the multimodal artificial intelligence-assisted pneumonia differential diagnosis system and terminal based on chest CT images of the present invention uses chest CT images and clinical text information in training samples covering 10 types of pneumonia to construct image classifiers and non-image classifiers respectively to establish a pneumonia differential diagnosis model; based on the pneumonia differential diagnosis model, the corresponding pneumonia identification results are obtained according to the chest CT images and clinical text information of the input pneumonia cases. The present invention can identify 10 types of pneumonia with extremely high accuracy. Clinicians can formulate microbial inspection or diagnostic strategies in a targeted manner based on this diagnostic result, and implement targeted anti-infection treatment, thereby significantly reducing unnecessary use of antibacterial drugs, improving treatment effects, and reducing misdiagnosis and missed diagnosis. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0062] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images, characterized in that: The system comprises: The training sample acquisition module is used to collect multiple pneumonia cases covering 10 types of pneumonia, and use the chest CT images and clinical text information of each pneumonia case as training samples marked with the corresponding pneumonia type label; A model building module, connected to the training sample acquisition module, for building an image classifier and a non-image classifier based on the chest CT images and clinical text information in each training sample, so as to establish a pneumonia differential diagnosis model; The diagnosis module is connected to the model building module and is used to obtain corresponding pneumonia identification results based on the pneumonia differential diagnosis model according to the chest CT images and clinical text information of the input pneumonia cases.

2. According to the multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images as described in claim 1, it is characterized in that: The 10 types of pneumonia include: bacterial pneumonia, pneumonia caused by atypical pathogens, Nocardia pneumonia, pulmonary cryptococcosis, pulmonary aspergillosis, Pneumocystis jiroveci pneumonia, pulmonary tuberculosis, pulmonary nontuberculous mycobacterial disease, pulmonary malignancy, and pneumo-like disease.

3. According to the multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images as described in claim 3, it is characterized in that: The pneumonia differential diagnosis model includes: Model input module, used to input chest CT images and clinical text information of pneumonia cases; An image classifier, connected to the model input module, is used to obtain corresponding image classification results according to the chest CT images of the input pneumonia cases; wherein the image classification results include: image classification probability values ​​of each pneumonia type; A non-image classifier, connected to the model input module, is used to obtain corresponding non-image classification results according to the clinical text information of the input pneumonia case; wherein the non-image classification results include: non-image classification probability values ​​of each pneumonia type; The result integration module is connected to the image classifier and the non-image classifier, and is used to integrate the image classification results and the non-image classification results to obtain the pneumonia identification results of the corresponding pneumonia cases.

4. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images according to claim 3 is characterized in that: Methods for constructing image classifiers based on chest CT images in each training sample include: Preprocessing the chest CT images in each input training sample; Normalizing the lung window and the standard window of each preprocessed chest CT image; The images normalized by the lung window and the standard window are input into the 3D-DenseNet network through dual channels for training to obtain an image classifier.

5. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images according to claim 4 is characterized in that: The method of preprocessing the chest CT images in each input training sample includes: Use a neural network to automatically segment the left and right lungs in chest CT images and obtain the bounding boxes of the left and right lungs; Extracting the lung field region from the chest CT image based on the bounding boxes of the left and right lungs to obtain multiple lung field image blocks; Adjust each lung field image block to the same pixel.

6. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images according to claim 3 is characterized in that: Methods for constructing non-imaging classifiers based on clinical text information in each training sample include: Preprocess the clinical text information in each training sample; The preprocessed clinical text information is used to train the GBT network to obtain a non-imaging classifier.

7. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images according to claim 6 is characterized in that: The clinical text information includes: gender, age, white blood cell count, neutrophil count, erythrocyte sedimentation rate, C-reactive protein, procalcitonin, carcinoembryonic antigen, T-SPOT.TB test results, symptoms and symptom duration.

8. The multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images according to claim 3 is characterized in that: The pneumonia identification results of the corresponding pneumonia cases obtained by integrating the image classification results and non-image classification results include: The image classification probability value and non-image classification probability value of each pneumonia type are added and then normalized; the three pneumonia types with the highest probability are selected according to the normalized probability values ​​as the pneumonia identification result output.

9. An electronic terminal, characterized in that: include: A multimodal artificial intelligence-assisted pneumonia differential diagnosis system based on chest CT images as described in any one of claims 1 to 8.

Citation Information

Cited By

  • Three-dimensional artificial intelligence model for pneumonia pathogenesis discrimination based on CT image

    CN121392146A