A multi-modal intelligent screening method, system and device for rare lung diseases in children

Abstract

The application discloses a kind of children's rare lung disease multi-modal intelligent screening method, system and equipment, belong to medical artificial intelligence technical field.The method includes: obtaining the medical image data, clinical text data and corresponding visit time information generated by the children at multiple time points, and the multi-modal data is standardized;Based on the multi-modal time series data after processing, construct cross-modal time series logic diagram, the cross-modal time series logic diagram is used to represent the association between different modal data and its evolution characteristics with time change;The cross-modal time series logic diagram is input into the children's rare lung disease screening model for analysis, and the risk prediction result of suffering from rare lung disease is output.The screening model includes multi-modal feature extraction module, cross-modal feature fusion module and global comprehensive prediction module, by the joint modeling of image feature, text feature and time series information, realize the intelligent screening of children's rare lung disease.The application can overcome the problem of multi-modal information fragmentation and insufficient use of time series association in the prior art, and is suitable for the auxiliary screening and risk assessment scene of children's rare lung disease.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence technology, specifically relating to an intelligent screening method, system and device for rare respiratory diseases in children, and particularly to an intelligent screening solution based on multimodal time-series data processing and analysis of medical imaging data, clinical text data and consultation time information. Background Technology

[0002] Although rare diseases have a low prevalence, their diagnosis is often lengthy and prone to misdiagnosis due to their diverse types and complex clinical manifestations. Current research indicates that patients with rare diseases often require multiple consultations and examinations before a definitive diagnosis, which can lead to missed opportunities for early intervention and negatively impact disease management outcomes.

[0003] Rare pediatric lung diseases, such as certain types of interstitial lung disease, often present with clinical symptoms similar to common respiratory illnesses and lack standardized diagnostic criteria, making early identification challenging. Limited by insufficient specialist medical resources and fragmented treatment pathways, these diseases still face a high risk of missed or misdiagnosed diagnoses in clinical practice.

[0004] Currently, artificial intelligence technology has been applied to some extent in fields such as medical imaging-assisted diagnosis. However, existing technical solutions mostly focus on a single data modality or simply combine imaging data with a small amount of clinical information, failing to effectively utilize the inherent correlations between imaging data, clinical text information, and time-series information generated during multiple medical visits in children. Due to the characteristics of rare pediatric lung diseases, such as their strong insidiousness and complex disease progression, screening methods based on a single modality or lacking time-series modeling are insufficient to comprehensively reflect disease characteristics, and the screening effectiveness still needs improvement.

[0005] Therefore, it is necessary to provide an intelligent screening method that can uniformly process multimodal data generated during multiple medical visits in children and make full use of its temporal correlation features to assist in the early identification and risk assessment of rare lung diseases in children. Summary of the Invention

[0006] This invention aims to overcome the problems of insufficient utilization of multimodal information and limited ability to model temporal correlation in the prior art, and provides a multimodal intelligent screening method, system and device for rare lung diseases in children that can jointly model medical imaging data, clinical text data and consultation time information generated during multiple medical visits.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] In a first aspect, the present invention provides a multimodal intelligent screening method for rare pulmonary diseases in children, the method comprising the following steps: First, medical imaging data, clinical text data, and corresponding consultation time information of children at multiple time points are acquired, and the multimodal data is standardized to form multimodal time-series data. Secondly, a cross-modal time-series logic graph is constructed based on the multimodal time-series data to characterize the correlation between different modal data and their evolutionary characteristics over time; Finally, the cross-modal temporal logic diagram is input into the pediatric rare lung disease screening model for analysis, and the risk prediction results of children suffering from rare lung diseases are output.

[0009] The pediatric lung rare disease screening model includes at least the following: A multimodal feature extraction module is used to extract image features and text features from the cross-modal temporal logic graph; The cross-modal feature fusion module is used to jointly model features from different modalities and generate fused features. The global comprehensive prediction module is used to output screening results based on the fused features.

[0010] Secondly, the present invention also provides a multimodal intelligent screening system for rare pediatric lung diseases that implements the above-described method, the system comprising: Data preprocessing module; Cross-modal timing logic diagram construction module; The screening model module contains a screening model for rare lung diseases in children.

[0011] Thirdly, the present invention also provides an electronic device and a computer-readable storage medium, the electronic device including a processor and a memory, the memory storing a computer program, and when the processor executes the computer program, implementing the above-mentioned multimodal intelligent screening method for rare lung diseases in children.

[0012] The present invention has the following significant beneficial effects: Compared with the prior art, the present invention has at least the following beneficial effects: (1) By uniformly processing and jointly modeling medical imaging data, clinical text data and consultation time information, the utilization efficiency of multimodal information is improved; (2) By constructing a cross-modal temporal logic diagram, the temporal change characteristics of disease-related information during multiple medical visits by children can be effectively reflected; (3) The method has a clear structure and a complete implementation process, making it easy to integrate into existing medical information systems for auxiliary screening and risk assessment of rare lung diseases in children. Attached Figure Description

[0013] Figure 1This is an overall flowchart of a multimodal intelligent screening method for rare lung diseases in children, provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the structure of a rare lung disease screening model for children in one embodiment of the present invention.

[0015] Figure 3 This is a flowchart illustrating the training process of a rare lung disease screening model for children in one embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are for illustrative purposes only and are not intended to limit the invention. Example 1

[0017] This embodiment describes a multimodal intelligent screening method for rare pediatric lung diseases, the process of which is as follows: Figure 1 As shown.

[0018] Step S1: Data preprocessing.

[0019] This step aims to transform diverse and formatted real-world medical data into high-quality, standardized model input. Specifically, it includes: Medical Image Preprocessing (S1-1): Input is a child's chest CT or X-ray image. First, an automatic image quality assessment algorithm is used to filter out low-quality images that are unrecognizable due to motion artifacts, improper exposure, or low resolution. Qualified images are then standardized, such as by uniform size and grayscale normalization.

[0020] Clinical text preprocessing (S1-2): Inputs include chief complaint, present illness, past medical history, and imaging report text. Natural language processing (NLP) techniques are used for word segmentation and removal of meaningless stop words. Key information is extracted through medical entity recognition models (such as named entity recognition for symptoms, signs, and diseases), and then transformed into structured feature vectors or standardized codes.

[0021] Time-series data integration (S1-3): Taking an individual child as the unit, the imaging data and clinical text data corresponding to each visit are aligned on the timeline according to the timestamps of the multiple visits. The time intervals are encoded, and all features are unified to the same dimension, ultimately forming a standardized data sequence containing "patient-time point-multimodal features".

[0022] Step S2: Construct cross-modal sequential logic diagram.

[0023] This step is crucial for simulating a clinician's construction of a "diagnostic thinking network." For a child patient, standardized data from multiple medical visits constitute a dynamic information network.

[0024] Node (V) generation: The depth feature vector of each patient's image is used as an "image node" by using a pre-trained image feature extraction model. The semantic temporal feature vector of each patient's text is extracted using a pre-trained text feature extraction model combined with the patient's time of visit encoding, and is used as a "text node".

[0025] Edge (E) Construction: This mechanism calculates correlation weights using node features. It calculates the correlation weights between image and text nodes within the same time point (intramodal cross-modal association) and also between nodes at different time points (cross-temporal association). For example, the model can learn a strong correlation between "ground-glass opacity on the current CT scan" and "progressive dyspnea recorded in a medical record from two weeks ago." These weights constitute the "edges" connecting the nodes.

[0026] Time sequence information (T): Each node is accompanied by a time code for its corresponding consultation time.

[0027] Ultimately, all the data of a single patient were constructed into a graph G=(V,E,T), which is a cross-modal temporal logic graph that fully encapsulates the patient's multi-dimensional and dynamic diagnostic information.

[0028] Step S3: Intelligent risk assessment based on screening model.

[0029] The constructed logic graph is input into the already trained pediatric lung rare disease screening model. For example... Figure 2 As shown, the model works internally as follows: Multimodal feature extraction module: Receives a logic graph, in which the image feature extraction model branch and the text feature extraction model branch refine the features of the image nodes and text nodes in the graph, respectively, and output the enhanced initial features.

[0030] Cross-modal feature fusion module: The cross-modal feature fusion module is used to jointly model image node features and text node features based on the association relationship between nodes in the cross-modal temporal logic graph, thereby generating a fused feature representation.

[0031] In the embodiments, the joint modeling process can be implemented through feature weight calculation, correlation modeling or other equivalent methods.

[0032] Global comprehensive prediction module: The fused features are input into a prediction head consisting of a fully connected layer and a classifier. The fully connected layer performs high-level feature synthesis and dimensionality reduction, and finally the classifier outputs a probability value, which represents the risk probability that the child has a rare lung disease (e.g., a probability > 0.85 is considered high risk, and specialist consultation is recommended).

[0033] The training process of the model (e.g.) Figure 3 (as shown) The performance of a model depends on high-quality supervised training.

[0034] Data preparation: Collect a large-scale dataset of pediatric lung diseases (including rare and common diseases) that has been diagnosed and labeled by professional doctors, and divide it into training, validation and test sets according to the proportion.

[0035] Model initialization and training: constructing such Figure 2 The model architecture is shown. The feature extraction module loads weights pre-trained on publicly available medical image and text datasets, while the remaining modules are randomly initialized. The training set data is input into the model in batches.

[0036] Forward propagation yields the predicted probability.

[0037] Calculate the cross-entropy loss between the predicted probability and the true label (whether it is a rare disease), and add an L2 regularization term to prevent overfitting.

[0038] Update all weights of the model using a backpropagation algorithm (such as the Adam optimizer).

[0039] Iteration and Validation: Repeat the training process and monitor performance metrics (such as AUC value and F1 score) on the validation set. When the performance on the validation set no longer improves, stop training and save the best-performing model as the final screening model.

[0040] Testing and evaluation: Evaluate the model's final screening performance on independent test sets to ensure that its accuracy, sensitivity, and specificity meet the basic requirements for clinical application. Example 2

[0041] Based on the same inventive concept, this embodiment provides a multimodal intelligent screening system for rare pediatric lung diseases. This system can be deployed on a hospital server or in the cloud, and includes: Data access and preprocessing module: Connects to the Hospital Information System (HIS) and the Picture Archiving System (PACS) and automatically executes step S1 in Example 1.

[0042] Logical graph computation engine: It integrates an image feature extraction module, a text feature extraction module, and a feature association computation unit to construct cross-modal temporal logical graphs.

[0043] Screening Model Service Module: Loads a pre-trained screening model for rare pediatric lung diseases and provides an API interface. After receiving the logic diagram, it executes step S3 and returns the risk assessment results.

[0044] Results Presentation and Reporting Module: Generates a visual report from the model's predicted probabilities and key evidence (such as image regions with high attention weights and text descriptions) for doctors' reference. Example 3

[0045] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the program, it implements the method described in Embodiment 1. The device may be a workstation, a server, or a computing unit integrated into a medical device.

[0046] The embodiments of the present invention have been described in detail above with reference to specific examples, but the present invention is not limited to the above embodiments. Various changes or substitutions can be made by those skilled in the art without departing from the technical concept of the present invention, and all such changes or substitutions should fall within the protection scope of the present invention.

Claims

1. A multimodal intelligent screening method, system, and device for rare pediatric lung diseases, characterized in that, include: Step S1, Multimodal data acquisition and standardization processing: Acquire multimodal medical data of children at at least two different time points during their medical visits. The multimodal medical data includes at least medical image data, clinical text data, and corresponding medical visit time information. The medical image data is subjected to quality screening and format normalization, the clinical text data is subjected to semantic cleaning and structuring, and the data of each modality is time-aligned based on the consultation time information to obtain standardized multimodal time series data. Step S2, Construction of cross-modal temporal logic graph: Based on the standardized multimodal temporal data, a cross-modal temporal logic graph is constructed. The cross-modal temporal logic graph includes at least a node set, an edge set, and temporal attributes, wherein: The node set includes at least image nodes generated from medical image data and text nodes generated from clinical text data; The edge set is used to represent the association relationship between nodes. The association relationship includes at least the cross-modal association relationship between image nodes and text nodes within the same consultation time point, and the temporal association relationship between nodes at different consultation time points. The time sequence attribute is used to encode the consultation time information corresponding to each node; The weights of the edges are used to characterize the degree of correlation between image node features and text node features; Step S3, Multimodal Feature Fusion and Risk Prediction: The cross-modal temporal logic graph is input into a pre-trained pediatric rare lung disease screening model for analysis. The pediatric rare lung disease screening model includes at least: A multimodal feature extraction module is used to extract image node features and text node features from the cross-modal temporal logic graph; A cross-modal feature fusion module is used to perform weighted fusion of image node features and text node features based on the weights of the edges to generate fused features; The global comprehensive prediction module is used to comprehensively process the fused features and output the risk prediction results of the child seeking medical treatment for rare lung diseases.

2. The multimodal intelligent screening method for rare pediatric lung diseases according to claim 1, characterized in that, The multimodal data standardization process in step S1 includes at least the following: Step S1-1: Perform quality inspection on the medical image data. The quality inspection includes at least the detection of image sharpness, artifact level or exposure consistency, and screen out the medical image data that meets the preset quality threshold. Step S1-2: Semantic cleaning and structuring processing are performed on the clinical text data. The structuring processing includes at least entity recognition and standardized representation of symptom descriptions, medical history descriptions, or examination conclusions. Steps S1-3: Based on the consultation time information, the medical image data and clinical text data are time-aligned, and the multimodal data within the same consultation time point are associated as a time series unit, thereby forming standardized multimodal time series data.

3. The multimodal intelligent screening method for rare pediatric lung diseases according to claim 1, characterized in that, The cross-modal timing logic diagram is represented using a graph structure, which includes at least: A set of nodes, the set of nodes including at least image nodes and text nodes, wherein image nodes are used to represent medical image features and text nodes are used to represent clinical text features; An edge set is used to characterize the association relationships between nodes. The association relationships include at least: cross-modal association relationships between image nodes and text nodes within the same consultation time point; temporal association relationships between nodes at different consultation time points; and temporal attributes, which are used to identify the consultation time information corresponding to each node; wherein the weight of the edge is calculated based on the correlation between the features of the corresponding nodes.

4. The multimodal intelligent screening method for rare pediatric lung diseases according to claim 1, characterized in that, The multimodal feature extraction module includes at least: The image feature extraction submodule is used to extract features from the image nodes in the cross-modal temporal logic graph to generate image feature vectors that characterize the spatial features of medical images. The text feature extraction submodule is used to extract features from the text nodes in the cross-modal temporal logic graph and generate text feature vectors that characterize the semantic features and temporal evolution features of clinical texts by combining the temporal attributes. The image feature extraction submodule is implemented using a visual feature extraction model based on a self-attention mechanism, while the text feature extraction submodule is implemented using a language model.

5. The multimodal intelligent screening method for rare pediatric lung diseases according to claim 1, characterized in that, The pediatric rare lung disease screening model was trained in the following way: Step T1: Construct an initial model framework, which includes at least a multimodal feature extraction module, a cross-modal feature fusion module, and a global comprehensive prediction module, and is configured to receive a cross-modal temporal logic graph as input; Step T2: Obtain a multimodal temporal training dataset for children labeled with disease category information. The training dataset includes at least a cross-modal temporal logic graph formed from multiple medical records. Step T3: Input the cross-modal temporal logic diagram into the initial model framework for iterative training, so that the model can learn the correlation between different modal features and the temporal features that change with the time of medical visit during the training process. Step T4: Based on the difference between the model output and the disease category information, update the model parameters in the initial model framework to obtain the trained pediatric lung rare disease screening model.

6. The multimodal intelligent screening method for rare pediatric lung diseases according to claim 5, characterized in that, During the model training process, a modal parameter initialization strategy is adopted for the multimodal feature extraction module, specifically including: Load the model parameters obtained from pre-training on medical image data into the image feature extraction submodule to initialize the image feature extraction submodule; Load the model parameters obtained from pre-training on clinical text corpus into the text feature extraction submodule to initialize the text feature extraction submodule; The parameters of the cross-modal feature fusion module and the global comprehensive prediction module are initialized randomly or semi-randomly at the beginning of training, and are jointly optimized based on the cross-modal temporal logic graph during training.

7. A multimodal intelligent screening system for rare pediatric lung diseases, characterized in that, The system for implementing the method of any one of claims 1 to 6 comprises: The data preprocessing module is used to acquire medical imaging data, clinical text data, and consultation time information of children seeking medical treatment, and to standardize the data to generate multimodal time series data. A cross-modal timing logic graph construction module is used to construct a cross-modal timing logic graph based on the multimodal timing data; The screening model module, which embeds a pediatric rare lung disease screening model, receives the cross-modal temporal logic diagram and outputs a risk prediction result for the child seeking medical attention for a rare lung disease. The screening model module is used to perform the cross-modal feature extraction, fusion, and prediction steps.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images