A method and system for dynamic early warning of pneumonia risk in influenza patients

Abstract

This invention relates to the fields of deep data fusion and machine learning technology, and provides a method and system for dynamic early warning of pneumonia risk in influenza patients. The method includes acquiring influenza-related data and constructing an influenza dataset; dividing the influenza dataset into structured data and unstructured image data; forming a standardized multimodal feature dataset; inputting the standardized multimodal feature dataset into a bidirectional modality-aware attention network and an adversarial cross-modal alignment network for deep fusion and semantic alignment processing to form an aligned unified feature representation; inputting the aligned unified feature representation into a downstream prediction network, and after risk probability calculation and threshold judgment processing, forming a risk level and corresponding personalized early warning information and clinical auxiliary suggestions. This invention improves the accuracy and practicality of early warning of pneumonia risk in influenza patients through comprehensive processing, deep fusion, accurate assessment, and personalized early warning of multimodal data.

Description

Technical Field

[0001] This invention relates to the fields of deep data fusion and machine learning technology, and in particular to a method and system for dynamic early warning of pneumonia risk in influenza patients. Background Technology

[0002] Currently, the clinical diagnosis and risk assessment of influenza complicated with pneumonia mainly rely on the physician's experience and judgment, combined with clinical symptoms, laboratory tests (such as blood routine and inflammatory markers), and imaging examinations (such as chest CT) for comprehensive judgment. Regarding intelligent assistance, some technologies already use a single data source to build predictive models, such as a predictive model for community-acquired pneumonia complicated with acute kidney injury, and a network prediction platform has been established; initial screening methods, systems, and equipment, based on clinical symptoms, blood routine, and blood biochemical indicators, construct differentiation tools, but their starting point and data modality differ from this invention. In terms of data fusion methods, existing technologies mainly include: early fusion (feature-level fusion): simply concatenating feature vectors from different modalities and inputting them into the model, ignoring the differences in feature scale and semantics between modalities, easily introducing noise and leading to the curse of dimensionality. Late fusion (decision-level fusion): training independent models for each modality and then averaging or voting on their outputs, failing to achieve intermediate-level information interaction between modalities and unable to capture complex cross-modal relationships. Fusion based on simple attention mechanisms, while capable of weighting features, is typically unidirectional or static, lacking cross-modal, dynamic, and bidirectional semantic interaction capabilities. In summary, a deep, dynamic, and interpretable multimodal fusion mechanism-based intelligent prediction and early warning system for influenza combined with pneumonia has yet to emerge. Currently, the application of artificial intelligence in the healthcare field is becoming increasingly profound. Existing prediction models often rely on single data sources, such as only laboratory indicators or only image reports, or employ simple early fusion, feature splicing, late fusion, or decision-level averaging strategies. These methods fail to fully exploit the deep correlations and complementary information between multimodal data, resulting in limited model prediction accuracy, poor interpretability, and weak generalization ability across institutional data.

[0003] Current technologies suffer from several drawbacks: shallow fusion methods fail to achieve deep semantic interaction and alignment of heterogeneous data, leading to insufficient information utilization; opaque model decision-making processes lack meaningful explanations for clinicians, hindering trust and integration into treatment workflows; significant differences in cross-institutional data distribution result in weak generalization capabilities of single-modality or simple fusion models; and a disconnect between intelligent prediction technologies and practical clinical applications, lacking convenient, real-time, and operable auxiliary tools. Therefore, this invention provides a method and system for dynamic early warning of pneumonia risk in influenza patients. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for dynamic early warning of pneumonia risk in influenza patients, comprising the following steps: A standardized multimodal feature dataset containing structured and unstructured image data from influenza datasets is input into a bidirectional modality-aware attention network and an adversarial cross-modal alignment network for deep fusion and semantic alignment processing to form an aligned unified feature representation. The aligned unified feature representation is input into the downstream prediction network. After risk probability calculation and threshold judgment, a risk level and corresponding personalized early warning information and clinical auxiliary suggestions are formed.

[0005] Optionally, the process of forming an aligned, unified feature representation includes the following steps: A standardized multimodal feature dataset is input into a bidirectional modality-aware attention network. Intramodal self-attention enhancement processing is performed on the clinical feature vectors and image depth feature vectors of the structured data in the multimodal feature dataset. The association weights between each element are calculated through the self-attention mechanism, and the original clinical features are weighted and recombined according to the weights to obtain the enhanced clinical semantic features. The image depth feature sequence is used as input, and the association weights between each element are calculated through the self-attention mechanism. The original image features are weighted and recombined according to the weights to obtain the enhanced image semantic features. The enhanced clinical semantic features and enhanced image semantic features are input into a bidirectional modality-aware attention network. The enhanced clinical semantic features are used as query signals and the enhanced image semantic features are used as key-value pairs. The clinical attention weight to the image is calculated by the degree of matching between the query and the key. The image semantic features are then weighted and recombined based on the attention weight to obtain clinically guided image features. Then, using the enhanced image semantic features as the query signal and the enhanced clinical semantic features as the key-value pair, the attention weight of the image to the clinical is calculated by the degree of matching between the query and the key. Based on the attention weight, the clinical semantic features are weighted and recombined to obtain the image-guided clinical features. The clinical-guided image features are then spliced ​​and fused with the image-guided clinical features to form a fused feature representation after bidirectional interaction. Optionally, the process of inputting the aligned, uniform feature representation into the downstream prediction network includes the following steps: The aligned unified feature representation is input into the hidden layer of the downstream prediction network. The hidden layer is composed of multiple fully connected layers connected sequentially. The unified feature representation enters each fully connected layer in turn. Each layer first performs a linear transformation on the input features of the layer, and then maps the transformation result to a new feature representation through a non-linear activation function. After passing through layer by layer, the output is a decision feature vector that has undergone deep abstraction and fusion. The decision feature vector is input into the output layer of the downstream prediction network. The output layer contains a neuron. The neuron performs a weighted summation of the decision feature vector and then converts the summation result into a value between 0 and 1 through the Sigmoid function. The value is the probability value of the patient developing influenza complicated with pneumonia. The system obtains a preset first risk threshold and a second risk threshold, and compares the calculated risk probability value with the first threshold and the second threshold in sequence. If the risk probability value is less than the first threshold, it is determined to be a low-risk level. If the risk probability value is greater than or equal to the first threshold and less than the second threshold, it is determined to be a medium-risk level. If the risk probability value is greater than or equal to the second threshold, it is determined to be a high-risk level. Based on the determined risk level, the system retrieves the corresponding early warning information template from the early warning information database and the corresponding clinical auxiliary suggestion template from the clinical auxiliary suggestion database. The patient's identity, key clinical indicator values, and imaging feature descriptions are filled into the corresponding positions in the templates to generate personalized early warning information and clinical auxiliary suggestions for the patient.

[0006] Optionally, the process of outputting the decision feature vector after deep abstraction and fusion includes the following steps: The aligned unified feature representation is input into the first fully connected layer in the hidden layer. The unified feature representation is formed by fusing clinical feature vectors and image feature vectors through bidirectional interaction and adversarial alignment, and contains a common representation of clinical indicator information and image region information. The first fully connected layer performs a linear combination operation on each element in the input feature representation, corresponding to the information of clinical indicators or image regions respectively: assigning a learnable weight coefficient to each element, summing all the weighted element values, and adding a learnable bias term to obtain the linear combination result; the linear combination result is input into a non-linear activation function for mapping, and the output is the feature vector transformed by the first fully connected layer. The feature vector output from the first fully connected layer is input into the intermediate fully connected layers in the hidden layer. The intermediate fully connected layers are composed of one or more sequentially connected fully connected layers. The feature vector enters each intermediate fully connected layer in turn. Each layer repeats the linear combination and non-linear activation operation on the elements in the input vector, which is the result of the previous layer's further integration of clinical and imaging information. After weighted summation and biasing of the input features, they are mapped by a non-linear activation function. After being passed layer by layer, the output is the feature vector abstracted by each intermediate layer. The feature vector output from the intermediate fully connected layers is input into the last fully connected layer in the hidden layers. The elements in the input vector are linearly combined and nonlinearly activated by integrating the deep correlation information between clinical and imaging. The weighted summation and bias of each element of the input feature is then mapped by a nonlinear activation function, and the output is a decision feature vector after deep abstraction and fusion of all hidden layers.

[0007] Optionally, the process of integrating deep correlation information between clinical and imaging data through linear combination and nonlinear activation includes the following steps: The feature vector output from the intermediate fully connected layer is obtained. Each element is a deep association representation formed by the layer-by-layer abstraction of clinical indicator information and image region information. The feature vector is input to each neuron of the last fully connected layer. Each neuron stores a set of weight coefficients in advance. The number of coefficients is the same as the dimension of the input feature vector. Each coefficient corresponds to an element in the input feature vector. For each neuron, its corresponding weight coefficient is multiplied by the corresponding element in the input feature vector to obtain a set of weighted components for that neuron. Within each neuron, all weighted components are summed to obtain the neuron's weighted sum; this weighted sum is then added to a pre-stored bias term within the neuron to obtain the neuron's linear combination output value; the linear combination output values ​​of all neurons together form a temporary vector. Each element in the temporary vector is input into a non-linear activation function, which maps each element independently and outputs a new value after mapping. All the new values ​​are arranged in the original order to form the final output vector of the last fully connected layer, which is the decision feature vector after deep abstraction and fusion of all hidden layers.

[0008] Optionally, the process of obtaining the linear combination output values ​​of neurons includes the following steps: The system obtains the weighted sum formed by summing the weighted components within the current neuron. The weighted sum is the comprehensive response value of clinical indicator information and image region information after being modulated by weight coefficients. At the same time, it reads the pre-configured bias terms from the neuron's internal storage unit. The bias terms are the inherent adjustment parameters determined by the neuron during training.

[0009] The weighted sum and the bias term are fed into the addition operation simultaneously. The two values ​​are combined into a new value through numerical addition. The new value is the result of the weighted sum after the bias term is adjusted. It contains the linear combination output of clinical and imaging information on the neuron. The new value obtained by the addition operation is passed to the output of the neuron as the linear response of the neuron to the clinical and imaging deep correlation information in the current input feature vector; the output value of the neuron is placed in the temporary vector at the position corresponding to the neuron, and together with the linear combination of the output values ​​of other neurons, it forms a complete temporary vector.

[0010] Optionally, the process of combining two values ​​into a new value using numerical addition includes the following steps: The weighted sum is used as the first operand, and the bias term is used as the second operand. The two operands are sent to the adder. After receiving the two operands, the adder extracts each bit of the weighted sum and the corresponding bit of the bias term. The adder starts from the least significant bit, aligns the current bit value of the weighted sum with the current bit value of the bias term, merges the two values ​​of the current bit and produces the result of this bit, and generates a carry signal based on the carry value generated during the merging process and passes it to the next bit to participate in the operation. After performing alignment, merging, and carry processing on all bits in sequence, the adder combines the bit-specific results of all bits in order from least significant bit to most significant bit to generate a complete value. This value is the new value obtained by adding the weighted sum and the bias term.

[0011] Optionally, the process of combining the bit-specific results of all bits in order from least significant bit to most significant bit includes the following steps: Get all the bit results generated by the adder after aligning and merging each bit. Each bit result corresponds to the value of one binary bit, and each bit result carries the bit position information of its own, which is determined by the order of operations in the addition process. Based on the position information carried by each bit, the bit corresponding to the least significant bit is determined as the starting bit of the entire value, the bit corresponding to the second least significant bit is determined as the subsequent bit of the starting bit, and so on, so that all bit results are arranged into a sequence in order from the least significant bit to the most significant bit. All local results in the sequence are stored in a contiguous storage area in the order of arrangement. The first local result in the sequence is stored at the beginning of the storage area, the second local result is stored at the next position immediately following it, and so on until the last local result is stored. The complete data formed in the storage area is the new value after adding the weighted sum and the bias term.

[0012] Optionally, it also includes starting the data acquisition terminal, establishing communication between the data acquisition terminal and the data information system, acquiring influenza-related data, and constructing an influenza dataset; dividing the influenza dataset into structured data and unstructured image data; cleaning, standardizing, and encoding the structured data to obtain processed structured data; preprocessing the unstructured image data; extracting deep features using a self-supervised contrastive learning network; and converting the unstructured image data into high-dimensional feature vectors; thus forming a standardized multimodal feature dataset.

[0013] The present invention provides a dynamic early warning system for the risk of pneumonia in influenza patients, comprising: The data acquisition module is used to start the data acquisition terminal, establish communication between the data acquisition terminal and the data information system, acquire influenza-related data, and construct an influenza dataset. The influenza dataset is divided into structured data and unstructured image data. The structured data is cleaned, standardized, and encoded to obtain processed structured data. The unstructured image data is preprocessed, and a self-supervised contrastive learning network is used to extract deep features and transform them into high-dimensional feature vectors. A standardized multimodal feature dataset is formed. The alignment processing module is used to input the standardized multimodal feature dataset into the bidirectional modality-aware attention network and the adversarial cross-modality alignment network for deep fusion and semantic alignment processing, forming a unified feature representation after alignment; The early warning output module is used to input the aligned unified feature representation into the downstream prediction network. After risk probability calculation and threshold judgment, it generates risk level and corresponding personalized early warning information and clinical auxiliary suggestions.

[0014] This invention utilizes data acquisition, cleaning, standardization, and encoding processes, along with image preprocessing and deep feature extraction of unstructured data, to form a standardized multimodal feature dataset, providing a high-quality data foundation. By employing a bidirectional modality-aware attention network and an adversarial cross-modal alignment network, it achieves deep fusion and semantic alignment of the multimodal feature data, improving the consistency and accuracy of feature representations. The aligned, unified feature representation is input into a downstream prediction network, and through risk probability calculation and threshold judgment, it achieves accurate assessment and classification of the pneumonia risk in influenza patients. Based on the risk level, corresponding personalized early warning information and clinical assistance suggestions are generated, providing decision support for doctors, helping to identify and intervene in high-risk patients in a timely manner, and reducing the risk of pneumonia.

[0015] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the dynamic early warning method for pneumonia risk in influenza patients in Embodiment 1 of the present invention; Figure 2This is a schematic diagram of the dynamic early warning method for pneumonia risk in influenza patients in Embodiment 1 of the present invention; Figure 3 This is a diagram illustrating the process of forming a standardized multimodal feature dataset in Embodiment 2 of the present invention. Figure 4 This is a diagram illustrating the process of forming an aligned unified feature representation in Embodiment 4 of the present invention; Figure 5 This is a diagram illustrating the process of inputting the aligned unified feature representation into the downstream prediction network in Embodiment 7 of the present invention. Figure 6 This is a diagram illustrating the training process of the downstream prediction network in Embodiment 7 of the present invention; Figure 7 This is a schematic diagram of the specific software development process in Embodiment 7 of the present invention; Figure 8 This is a block diagram of the dynamic early warning system for pneumonia risk in influenza patients in Embodiment 13 of the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "say," and "this" used in the embodiments of this application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms used herein refer to and / or include any or all possible combinations of one or more associated listed items.

[0020] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application. In the description of this application, it should be understood that the terms first, second, third, etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0021] This invention addresses the medical scenario of influenza-complicated pneumonia early warning by constructing a collaborative architecture of a bidirectional modal awareness attention network (BMA-Net) and an adversarial cross-modal alignment network (ACA-Net) to resolve the inherent and deep semantic misalignment between clinical text features and image features. This architecture allows clinical indicators, such as high fever and CRP, to accurately guide the model to focus on related pathological regions in images, such as pulmonary consolidation, while simultaneously allowing lesions discovered in images to reinforce key clinical indicators. BMA-Net is responsible for intramodal semantic extraction and deep intermodal interaction, while ACA-Net is responsible for mapping the interacted features to a unified semantic space in adversarial games. The two networks are complementary and hierarchically progressive. First, BMA-Net facilitates feature dialogue, and then ACA-Net aligns the results of this dialogue, jointly forming a unified, aligned feature representation.

[0022] Example 1: As Figure 1 As shown in the figure, this embodiment of the invention provides a method for dynamic early warning of pneumonia risk in influenza patients, comprising the following steps: S100: Start the data acquisition terminal, establish communication between the data acquisition terminal and the data information system, acquire influenza-related data, and construct an influenza dataset; divide the influenza dataset into structured data and unstructured image data; clean, standardize, and encode the structured data to obtain processed structured data; preprocess the unstructured image data, extract deep features using a self-supervised contrastive learning network, and convert them into high-dimensional feature vectors; form a standardized multimodal feature dataset; S200: The standardized multimodal feature dataset is input into a bidirectional modality-aware attention network and an adversarial cross-modal alignment network for deep fusion and semantic alignment processing to form a unified feature representation after alignment; S300: The aligned unified feature representation is input into the downstream prediction network. After risk probability calculation and threshold judgment, a risk level and corresponding personalized early warning information and clinical auxiliary suggestions are formed.

[0023] The working principle and beneficial effects of the above technical solution are as follows: The specific principle is as follows: Figure 2As shown, this embodiment uses data acquisition, cleaning, standardization, and encoding processes, along with image preprocessing and deep feature extraction of unstructured data, to form a standardized multimodal feature dataset, providing a high-quality data foundation. A bidirectional modality-aware attention network and an adversarial cross-modal alignment network are used to achieve deep fusion and semantic alignment of the multimodal feature data, improving the consistency and accuracy of feature representations. The aligned unified feature representation is input into a downstream prediction network, and through risk probability calculation and threshold judgment, accurate assessment and classification of pneumonia risk in influenza patients are achieved. Based on the risk level, corresponding personalized early warning information and clinical auxiliary suggestions are generated to provide decision support for doctors, helping to identify and intervene in high-risk patients in a timely manner and reduce the risk of pneumonia.

[0024] In summary, this embodiment improves the accuracy and practicality of pneumonia risk warning for influenza patients through comprehensive processing, deep fusion, accurate assessment, and personalized early warning of multimodal data, which helps in the early identification and management of high-risk patients in clinical practice.

[0025] In this embodiment, clinical data, laboratory test results, and imaging data of influenza patients are collected through a hospital information system to construct a high-quality influenza dataset. The influenza dataset contains 27 features, including 14 numerical features and 13 categorical features, with a sample size of approximately 2000 cases, as detailed in Table 1. The preprocessing workflow includes: cleaning, standardizing / normalizing, and encoding structured data; and preprocessing unstructured image data, using a self-supervised contrastive learning network to extract deep features and convert them into high-dimensional feature vectors. Specifically, structured data consists of clinical and laboratory indicators; cleaning includes handling missing and outlier values; standardization / normalization is applied to numerical data, which is then encoded as categorical; and unstructured image data, such as DICOM format, undergoes image preprocessing including cropping, scaling, and enhancement.

[0026] Table 1 Dataset Numerical features Categorical features age Is it due to advanced age? BMI Do you smoke? Blood white blood cells Obesity Blood neutrophils Does the patient have diabetes? blood lymphocytes Do you have high blood pressure? Blood mononuclear cells Is it a cardiovascular disease? CRP Is it hyperuricemia? Neutrophil / lymphocyte ratio (NLR) Should I get a flu shot? Monocyte / lymphocyte ratio (MLR) Do you have a cough? Platelet-to-lymphocyte ratio (PLR) Do you have a sore throat? platelets gender maximum body temperature Do you have body aches and pains? thermal process Is it a lung infection? SOFA rating This embodiment is applicable to early warning and clinical support scenarios for influenza complicated with pneumonia during peak influenza season, specifically including: Patient-side scenario or mobile / standalone device: Patients can fill in or authorize the synchronization of clinical symptoms, such as body temperature, cough, laboratory test results, etc., through WeChat mini program; the system analyzes the data in real time, assesses the risk level of pneumonia, and pushes personalized warnings and health guidance to patients. If the risk is high, it is recommended to immediately repeat a chest CT scan. In physician / hospital scenarios or multi-device collaboration: Physicians can view authorized patients' health records, risk scores, and historical trends through the system. The system assists in identifying high-risk patients and provides evidence-based treatment recommendations. The system interacts with hospital information systems (HIS / LIS / PACS) through secure interfaces, enabling automatic collection of patient information and writing back risk assessment results.

[0027] In this embodiment, the unified feature representation, processed by deep fusion with BMA-Net and alignment with ACA-Net, is input into the downstream prediction network (such as a multilayer perceptron MLP). The model ultimately outputs the risk probability value P (0≤P≤1) of the patient developing pneumonia.

[0028] Risk grading and early warning push: Risk grading: Set thresholds based on probability P, for example: low risk: P < 0.3; medium risk: 0.3 ≤ P < 0.7; high risk: P ≥ 0.7; Warning Triggering and Push Notifications: Warnings are automatically triggered when the risk level is medium or high. Warning information is pushed to patients, along with key indicators and regional descriptions relevant to the model, enhancing information credibility. Simultaneously, the warnings are pushed to the doctor's system, allowing doctors to view detailed risk breakdown diagrams and model justifications to support decision-making.

[0029] Clinical support and continuous optimization, decision support: The system provides patients with historical data comparison and risk trend charts, and can recommend antiviral or antibacterial regimens based on etiological and drug sensitivity results; feedback loop: The system records the diagnosis and treatment path and actual outcome after the warning; the data will be used as incremental training samples and input into the aforementioned fusion framework for model iterative optimization, forming the ability to self-evolve.

[0030] This embodiment solves the problem of semantic alignment and collaborative prediction of heterogeneous medical data, including clinical, laboratory, and imaging data, through a cross-modal fusion mechanism. It can be extended to related fields such as infectious disease risk prediction, early warning of chronic disease complications, health management, and auxiliary diagnosis and treatment.

[0031] Example 2: As Figure 3 As shown, based on Example 1, the process of forming a standardized multimodal feature dataset provided in this embodiment of the invention includes the following steps: S101: Start the data acquisition terminal, establish a communication connection between the data acquisition terminal and the data information system, acquire multi-source data related to influenza through the communication connection, including clinical diagnosis and treatment records, laboratory test results and medical imaging files, and collect all the acquired data to construct an influenza raw dataset; S102: Perform data classification processing on the original influenza dataset, dividing the original dataset into structured data and unstructured image data based on data format and content; the structured data includes patient demographic information, symptom descriptions, past medical history, laboratory test values ​​and classification results, etc., while the unstructured image data includes chest image files, etc. S103: Perform data cleaning on the structured data portion, remove records with a large number of missing or obvious errors, fill missing values ​​in the remaining records with the statistical values ​​of the group to which the feature belongs, identify and correct outliers; then standardize the numerical features in the cleaned structured data to make their numerical range uniform, encode and convert categorical features to convert text categories into numerical labels, and obtain the processed set of structured feature vectors. S104: Perform image preprocessing operations on each image file in the unstructured image data portion, including image size normalization, pixel value range adjustment, and data augmentation, to obtain a preprocessed image data set; input the preprocessed image data set into a self-supervised contrastive learning network that has been pre-trained on a large-scale unlabeled image dataset, learn the deep representation ability of the image through the contrastive learning mechanism, map the input image data into high-dimensional feature vectors, thereby extracting the deep feature vector corresponding to each image, forming an image deep feature vector set; S105: The processed set of structured feature vectors and the set of image depth feature vectors are paired and combined according to the patient identifier, so that each patient corresponds to a multimodal data pair containing structured feature vectors and image depth feature vectors. All patients' multimodal data pairs together constitute a standardized multimodal feature dataset.

[0032] The working principle and beneficial effects of the above technical solution are as follows: This embodiment removes erroneous and missing data through data cleaning, ensuring the quality and reliability of the dataset. Simultaneously, statistical values ​​are used to fill missing values ​​and correct outliers, further enhancing the completeness and accuracy of the data. Numerical features of structured data are standardized to unify the data value range, while categorical features are converted into numerical identifiers through encoding, helping to eliminate the impact of different data scales and improve data consistency. A self-supervised contrastive learning network is used to extract deep feature vectors from image data, not only extracting useful features but also reducing data dimensionality, facilitating data analysis and model training. Structured feature vectors and image depth feature vectors are paired and combined according to patient identifiers to achieve effective fusion of multimodal data, enabling a more comprehensive understanding and analysis of influenza from different perspectives. Through the above steps, this embodiment constructs a standardized multimodal feature dataset, which can better serve subsequent machine learning tasks, such as influenza diagnosis and prognostic analysis, improving the training effect and application value of the model. By integrating multi-source data and performing cleaning, standardization, and feature extraction steps, a high-quality, standardized dataset suitable for multimodal analysis is formed, providing strong data support for influenza-related research and applications.

[0033] Example 3: Based on Example 2, the process of constructing a self-supervised contrastive learning network trained on a large-scale unlabeled image dataset provided in this embodiment of the invention includes the following steps: S1041: Select a large number of unlabeled chest images from the image database, and perform two different random transformation operations on each original chest image, including random cropping, random rotation, color jitter, etc., to generate two derivative images with different appearances but the same original chest image. Collect all the derivative image pairs corresponding to the original chest images to form a training sample pair set. S1042: Constructing the initial feature extraction network. A multi-layer convolutional neural network structure is adopted. The network input layer receives a chest image that has been normalized in size and is processed sequentially into multiple concatenated convolutional blocks. Each convolutional block consists of a convolutional layer, an activation function layer, and a pooling layer. The convolutional layer performs sliding computation on a local region of the image using a learnable convolutional kernel to generate response values ​​for that local region and combines them into a feature map. The activation function layer performs nonlinear mapping on each element in the feature map to enhance the representational complexity. The pooling layer performs downsampling on the feature map, aggregating the feature values ​​of adjacent regions into a representative value, reducing the spatial size of the feature map while retaining the main response information. After passing through multiple convolutional blocks, the original pixel information of the chest image is progressively abstracted into a high-dimensional feature map. The high-dimensional feature map is input to a fully connected layer, which connects each neuron of the high-dimensional feature map to all elements of the high-dimensional feature map. Through weighted summation and nonlinear activation, a fixed-length feature vector is finally generated as the initial feature representation of the chest image. Two derived images from the training sample pair are input into the initial feature extraction network to obtain two corresponding initial feature vectors, forming an initial feature vector pair. S1043: Training the feature extraction network; the training objective is set as follows: for feature vector pairs generated from the same original chest image, the distance in the feature space should be as close as possible; for feature vector pairs generated from different original chest images, the distance in the feature space should be as far apart as possible. To achieve this objective, a contrastive loss function is introduced. The contrastive loss function takes the initial feature vector pair as input, calculates the Euclidean distance between the two feature vectors, and outputs the corresponding loss value based on whether they originate from the same original chest image: if they originate from the same original chest image, the loss value is positively correlated with the distance value, causing the distance to decrease; if they originate from different original chest images, the loss value is negatively correlated with the distance value, causing the distance to increase. All training... The total loss is obtained by summing the loss values ​​of the sample pairs. Using a stochastic gradient descent optimization algorithm, the partial derivatives of the total loss with respect to all weight parameters in the network are first calculated, representing the direction and magnitude of each weight's influence on the total loss, thus obtaining the gradient. Then, according to a preset learning rate, each weight is subtracted from the product of the learning rate and the corresponding gradient, completing one weight update. Through repeated iterative input of a large number of training sample pairs, the total loss is calculated and the weights are updated after each batch of input, gradually adjusting the network parameters to a stable state that meets the training objective. During training, the network's discriminative ability to extract features is periodically evaluated using a validation set. When the loss on the validation set no longer decreases or the performance indicators tend to stabilize, training is stopped, resulting in a successfully trained feature extraction network.

[0034] The working principle and beneficial effects of the above technical solution are as follows: This embodiment realizes the feature extraction of unlabeled image data, the optimized learning of the feature space, and the effective evaluation and adjustment of network performance through the process of constructing a self-supervised contrastive learning network. Ultimately, it achieves the effect of constructing a stable and robust feature extraction network that can distinguish different chest image features.

[0035] Example 4: Figure 4 As shown, based on Embodiment 1, the process of forming an aligned unified feature representation provided by this embodiment of the invention includes the following steps: S201: The standardized multimodal feature dataset is input into a bidirectional modality-aware attention network. Intramodal self-attention enhancement processing is performed on the clinical feature vectors and image depth feature vectors of the structured data in the multimodal feature dataset, respectively. The association weights between each element are calculated through the self-attention mechanism. The original clinical features are then weighted and recombined according to the weights to obtain the enhanced clinical semantic features. The image depth feature sequence is used as input. The association weights between each element are calculated through the self-attention mechanism. The original image features are then weighted and recombined according to the weights to obtain the enhanced image semantic features. S202: The enhanced clinical semantic features and the enhanced image semantic features are input into the bidirectional modality perception attention network. The enhanced clinical semantic features are used as query signals and the enhanced image semantic features are used as key-value pairs. The clinical attention weight to the image is calculated by the degree of matching between the query and the key. The image semantic features are weighted and recombined according to the attention weight to obtain the clinically guided image features. S203: Then, using the enhanced image semantic features as the query signal and the enhanced clinical semantic features as the key-value pair, the attention weight of the image to the clinical is calculated by the degree of matching between the query and the key. Based on the attention weight, the clinical semantic features are weighted and recombined to obtain the image-guided clinical features. The clinical-guided image features are spliced ​​and fused with the image-guided clinical features to form a fused feature representation after bidirectional interaction. The working principle and beneficial effects of the above technical solution are as follows: The self-supervised contrastive learning network construction process trained on a large-scale unlabeled image dataset in this embodiment achieves the following technical effects: By performing random transformation operations such as cropping, rotation, and color jittering, data augmentation is performed on chest images, improving data diversity and providing rich variations for training sample pairs, which helps the network learn more robust feature representations; Through a multi-layer convolutional neural network structure, a step-by-step abstraction from raw pixel information to high-dimensional feature maps is achieved, and then a fixed-length feature vector is output through a fully connected layer, completing the initial feature representation of chest images, which helps the network capture key information in the images; Through the contrastive loss function, the network learns that feature vector pairs generated from the same original image should be as close as possible in the feature space, while feature vector pairs generated from different original images should be as far apart as possible in the feature space, which helps the network distinguish between similar and dissimilar image samples; Through the stochastic gradient descent optimization algorithm, combined with the gradient calculated from the loss function, the weight parameters in the network are adjusted, so that the network parameters are gradually adjusted to a stable state that meets the training objective, improving the network's generalization ability. During training, a validation set is used to evaluate the network's ability to extract features. Training is stopped when the loss on the validation set no longer decreases or the performance metrics stabilize. This helps avoid overfitting and ensures the effectiveness of the network in practical applications.

[0036] In summary, this embodiment achieves feature extraction from unlabeled image data, optimized learning of the feature space, and effective evaluation and adjustment of network performance through the construction process of a self-supervised contrastive learning network. Ultimately, it achieves the technical effect of constructing a stable and robust feature extraction network that can distinguish different chest image features.

[0037] This embodiment realizes deep fusion of multimodal features and model prediction, aiming to solve the fundamental defects of existing fusion methods; (1) Innovative fusion mechanism: Bidirectional Modality-AwareAttention Network (BMA-Net); overcomes the shortcomings of simple splicing or unidirectional attention, and realizes dynamic, bidirectional, and semantic-level interaction between clinical text features and image features; intramodal self-attention enhancement: self-attention modules are used on clinical features after standardization and encoding and image depth features respectively to extract high-level semantic representations within each modality.

[0038] Cross-modal bidirectional cross-attention: Clinical features are used as the query, and imaging features as the key / value pair. This allows key clinical indicators, such as high fever and high CRP, to actively guide the model to focus on related areas in imaging, such as lobar consolidation and ground-glass opacities. Conversely, using imaging features as the query and clinical features as the key and value pair, allows suspected lesions found in imaging to be retrieved in reverse, reinforcing the importance of related clinical indicators. Interpretable output: Automatically generates an attention weight map, visually displaying the key clinical indicators and imaging regions that contribute most to the final prediction, greatly enhancing the model's clinical interpretability.

[0039] (2) The Adversarial Cross-modal Alignment Network (ACA-Net) addresses the problem of significant differences in the distribution and semantic misalignment between clinical data or tables and imaging data or high-dimensional vectors in the original feature space, thereby improving the model's generalization ability to heterogeneous data across institutions. It extracts features through two modality-specific encoders and introduces a modality discriminator to attempt to determine whether the features originate from the clinical or imaging modality. Through adversarial training, the encoder continuously learns to generate feature representations that can deceive the discriminator, forcing the two types of features to converge in a shared semantic space, thus achieving deep alignment.

[0040] (3) Special loss function design: Multi-task Contrastive Alignment Loss (MCAL); Unified optimization of multiple objectives such as classification accuracy, modality alignment, feature consistency and interpretability to address the challenges of training on heterogeneous data; Specific format: ; In the formula, This represents the binary cross-entropy classification loss, ensuring prediction accuracy. The alignment loss includes the adversarial loss of ACA-Net and the maximum mean difference (MMD) loss, which forces the distribution alignment between modalities. This represents contrast loss (such as InfoNCE), which brings the multimodal features of the same patient closer together and pushes the features of different patients further apart, thereby enhancing the consistency of representation; This represents the interpretability constraint loss, encouraging attention weights to be highly correlated with known clinically important indicators (such as the SOFA score); weights , , , It can be dynamically adjusted according to the training stage.

[0041] The essential difference from existing fusion methods Comparison Dimensions Existing technologies (early, late, and simple attention fusion) Technical solution of the present invention Fusion depth Feature-level concatenation or decision-level averaging lacks intermediate semantic interaction. Deep semantic fusion, achieving dynamic bidirectional interaction at the feature level through BMA-Net. Alignment capability The distribution differences between different modes were not shown. Heterogeneous features are mapped to a unified semantic space through adversarial training with ACA-Net. Optimization Objective Typically, only the single objective of optimizing classification accuracy is considered. Multi-objective collaborative optimization, using MCAL to simultaneously optimize classification, alignment, contrast, and interpretability. Explainability Most are "black box" models, with opaque decision-making criteria. Endogenous interpretability outputs visualized decision-making basis (key indicators and image regions) through attention mechanisms. Generalization Sensitive to training data distribution, performance degrades significantly across institutions. Its strong generalization ability, thanks to feature alignment and multi-task learning, enables it to better adapt to new scenario data. Example 5: Based on Example 4, the process of forming a fused feature representation after bidirectional interaction provided in this embodiment of the invention includes the following steps: S2021: Obtain the clinically guided image feature vector, denoted as the first feature vector; obtain the image-guided clinical feature vector, denoted as the second feature vector; read the number of dimensions of the first feature vector and the number of dimensions of the second feature vector respectively; S2022: Allocate a contiguous storage space in memory, the length of which is set to the sum of the dimensions of the first feature vector and the second feature vector; write all the dimension components of the first feature vector into the starting position of the storage space in their original order. S2023: Write all the dimensional components of the second feature vector into the storage space in the original order, immediately following the first feature vector; after all the data has been written, read the contents of the storage space as a complete feature vector from memory and output it. The feature vector is the fused feature representation after bidirectional interaction.

[0042] The working principle and beneficial effects of the above technical solution are as follows: The bidirectional interactive fusion feature representation process in this embodiment achieves information fusion between the two domains by integrating clinical and imaging features. Specifically, the clinically guided imaging feature vector represents image-related features extracted from clinical information, while the image-guided clinical feature vector represents clinically relevant features extracted from imaging data. By merging these two feature vectors into a contiguous storage space in memory while maintaining their respective order, the fusion process produces the following technical effects: Feature vectors from different sources are integrated to form a unified feature vector, which helps to unify data format and dimensions during subsequent processing. Clinical features and imaging features each contain different information; after merging, they can complement each other, improving the comprehensiveness and expressive power of the features. Combining clinical and imaging data can enhance the model's understanding and predictive ability of diseases, as certain clinical symptoms may be associated with specific imaging features; through this feature fusion, the model can learn from richer data, potentially improving model performance and predictive accuracy. It allows for flexible addition or adjustment of the feature vector dimensions as needed, providing space for further model development and optimization. Using a single feature vector to process the model, rather than processing two feature vectors separately, reduces the complexity of model design. This embodiment can provide richer and more accurate data for machine learning and pattern recognition tasks, thereby improving the performance and reliability of the model.

[0043] Example 6: Based on Example 5, the process of writing all dimensional components of the second feature vector sequentially into the storage space immediately following the first feature vector in the original order, as provided in this embodiment of the invention, includes the following steps: S20231: Obtain the storage space status after the clinically guided image feature vector is written. The image feature vector is generated by the guidance of clinical features on image features and carries the image region information of the clinical indicators. Record the end position of the last dimension component of the image feature vector in the storage space. Determine the adjacent subsequent storage unit of the end position as the starting writing position of the image-guided clinical feature vector. S20232: Starting from the first dimension component of the image-guided clinical feature vector, each dimension component is extracted sequentially. The dimension vector is generated by the guidance of image features on clinical features and carries the clinical indicator information enhanced by the image lesions. Each extracted component is stored in the storage unit corresponding to the current starting write position, and the starting write position is moved one storage unit backward to point to the next write position. S20233: Repeat the above retrieval and storage operations until the last dimension component of the image-guided clinical feature vector is written to the storage space; the storage space records all dimension components of the image-guided clinical feature vector from the first storage unit before the starting writing position to the last storage unit written, reflecting the clinical indicator information associated with image discovery.

[0044] The working principle and beneficial effects of the above technical solution are as follows: This embodiment realizes the orderly storage and information transmission of image feature vectors and clinical feature vectors. Image feature vectors and clinical feature vectors are stored continuously in the same storage space, enabling close integration of imaging and clinical information and providing complete data support. It improves data storage efficiency and accessibility while ensuring the integrity and accuracy of information. Furthermore, it helps reduce the complexity of data management and processing because all relevant data is stored and processed uniformly, thereby simplifying the data retrieval and analysis process.

[0045] Example 7: Figure 5 As shown, based on Example 1, the process of inputting the aligned unified feature representation into the downstream prediction network provided by this embodiment of the invention includes the following steps: S301: The aligned unified feature representation is input into the hidden layer of the downstream prediction network. The hidden layer is composed of multiple fully connected layers connected in sequence. The unified feature representation enters each fully connected layer in turn. Each layer first performs a linear transformation on the input features of the fully connected layer, and then maps the transformation result to a new feature representation through a non-linear activation function. After passing through layer by layer, the output is a decision feature vector that has undergone deep abstraction and fusion. S302: The decision feature vector is input to the output layer of the downstream prediction network. The output layer contains a neuron. The neuron performs a weighted summation of the decision feature vector and then converts the summation result into a value between 0 and 1 through the Sigmoid function. The value is the probability value of the patient's risk of developing influenza complicated with pneumonia. S303: Obtain the preset first risk threshold and second risk threshold, and compare the calculated risk probability value with the first threshold and the second threshold in sequence; if the risk probability value is less than the first threshold, it is determined to be a low risk level; if the risk probability value is greater than or equal to the first threshold and less than the second threshold, it is determined to be a medium risk level; if the risk probability value is greater than or equal to the second threshold, it is determined to be a high risk level; according to the determined risk level, retrieve the corresponding early warning information template from the early warning information database and the corresponding clinical auxiliary suggestion template from the clinical auxiliary suggestion database, fill in the patient's identity identifier, key clinical indicator values ​​and imaging feature descriptions into the corresponding positions of the template, and generate personalized early warning information and clinical auxiliary suggestions for the patient.

[0046] The working principle and beneficial effects of the above technical solution are as follows: This embodiment realizes dynamic monitoring and early warning of pneumonia risk in influenza patients. The data acquisition module, through efficient data acquisition technology, achieves comprehensive collection of influenza-related data and effectively transforms the data into two main categories: structured and unstructured data, providing a foundation for subsequent processing. Cleaning, standardization, and encoding of structured data, as well as image preprocessing and deep feature extraction of unstructured data, significantly improve the quality and usability of the dataset. The alignment processing module utilizes bidirectional modal perception and adversarial cross-modal alignment technology to achieve deep fusion and semantic alignment of multimodal data, enhancing the model's understanding of different types of data and improving the uniformity and accuracy of feature representation. It can more comprehensively capture the complex relationship between influenza and pneumonia, laying the foundation for accurate risk assessment. The early warning output module, based on the aligned feature representation, outputs specific pneumonia risk levels and personalized early warning information through risk probability calculation and threshold judgment. This not only helps doctors and patients understand the development of the disease in a timely manner but also provides targeted clinical auxiliary suggestions, thereby achieving dynamic early warning of pneumonia risk in influenza patients and improving the timeliness and accuracy of clinical decision-making. It improved the accuracy of predicting the risk of pneumonia in influenza patients and promoted the scientific and personalized nature of clinical decision-making.

[0047] The core of this embodiment is an end-to-end deep neural network prediction model or downstream prediction network specifically designed for multimodal medical data. Its construction process is not simply model selection and training, but rather involves systematic architecture design, training strategy formulation, and inference process definition, all centered around three main objectives: deep fusion, semantic alignment, and interpretable generalization. The training process of the downstream prediction network is as follows: Figure 6As shown; the specific software development process is as follows: Figure 7 As shown; Training process and usage process: This involves an end-to-end training process and a separate usage (inference) process.

[0048] Training process description: (1) Model Architecture: A multimodal fusion prediction model with a deep neural network as its core is adopted. The model input layer is divided into two branches or one fusion entry point: ① Branch 1 is the structured data branch: receiving preprocessed and encoded clinical and laboratory feature vectors. ② Branch 2 is the image feature branch: receiving image feature vectors output by the encoder part of a pre-trained image feature extraction network, such as the SimCLR network. ③ Fusion layer: concatenating the features of the two branches or weighting them through an attention mechanism. ④ Hidden layer: the fused features are input to a fully connected layer for deep nonlinear transformation. ⑤ Output layer: a single neuron layer using the Sigmoid activation function, outputting the probability of pneumonia occurrence.

[0049] (2) Input: Training sample set, each sample includes: the patient's fused multimodal feature vector, the corresponding output of step 1 as the input feature (X), and the label (0 or 1) of whether the patient was finally diagnosed with influenza complicated with pneumonia as the true value (Y).

[0050] (3) Feedback mechanism: The error between the model's predicted probability and the true label is calculated using the binary cross-entropy loss function; the network weights are iteratively updated through backpropagation algorithm and optimizer, such as Adam, to minimize the loss function.

[0051] (4) Training data features: such as seven dimensions of clinical data including demographic characteristics, lifestyle, medical history, immunization records, clinical symptoms, laboratory test parameters, and clinical diagnosis conclusions, with 26 features, including 13 numerical features and 13 categorical features.

[0052] (5) Data types: Structured tabular data (numerical, categorical) and medical image data (DICOM format).

[0053] (6) Data content: 27 features and corresponding chest images as listed in Table 1 in the background art.

[0054] (7) Data volume and annotation: A sufficient number of historical case data are required, and each case must have a clear outcome annotation of whether influenza was complicated with pneumonia; it is a supervised learning task.

[0055] Usage process description: (1) Use model architecture: consistent with the model architecture after training, but remove training-specific components (such as the Dropout layer being turned off during inference).

[0056] (2) Input: Multimodal feature vectors of new patients or patients requiring assessment, which have undergone the same preprocessing and feature extraction / fusion process.

[0057] (3) Output: The predicted probability of the patient developing influenza complicated with pneumonia.

[0058] Example 8: Based on Example 7, the process of generating a decision feature vector through deep abstraction and fusion, as provided in this embodiment of the invention, includes the following steps: S3011: The aligned unified feature representation is input into the first fully connected layer in the hidden layer. The unified feature representation is formed by fusing clinical feature vectors and image feature vectors through bidirectional interaction and adversarial alignment, and contains a common representation of clinical indicator information and image region information. The first fully connected layer performs a linear combination operation on each element in the input feature representation, corresponding to specific information of clinical indicators or image regions: assign a learnable weight coefficient to each element, sum all the weighted element values, and then add a learnable bias term to obtain the linear combination result; input the linear combination result into a non-linear activation function for mapping, and output the feature vector transformed by the first fully connected layer. S3012: The feature vector output from the first fully connected layer is input into the intermediate fully connected layer in the hidden layer. The intermediate fully connected layer consists of one or more sequentially connected fully connected layers. The feature vector enters each intermediate fully connected layer in turn. Each layer repeats the linear combination and non-linear activation operation on the elements in the input vector, which is the result of the previous layer's further integration of clinical and imaging information. After weighted summation and biasing of the input features, they are mapped by a non-linear activation function. After being passed layer by layer, the output is the feature vector abstracted by each intermediate layer. S3013: Input the feature vector output from the intermediate fully connected layer into the last fully connected layer in the hidden layer. For the elements in the input vector, perform linear combination and nonlinear activation by integrating the deep correlation information between clinical and imaging. After weighted summation and biasing of each element of the input feature, the result is mapped by a nonlinear activation function and output as a decision feature vector after deep abstraction and fusion of all hidden layers.

[0059] The working principle and beneficial effects of the above technical solution are as follows: This embodiment improves the model's representation ability, accuracy and generalization in decision-making tasks by deeply integrating multimodal information, abstracting feature representation layer by layer and automatically learning the optimal features, which helps to obtain better decision results.

[0060] Example 9: Based on Example 8, the process of linear combination and nonlinear activation of deep correlation information between clinical and imaging data provided in this embodiment of the invention includes the following steps: S30131: Obtain the feature vector output by the intermediate fully connected layer. Each element is a deep association representation formed by the layer-by-layer abstraction of clinical indicator information and image region information. Input the feature vector into each neuron of the last fully connected layer. Each neuron pre-stores a set of weight coefficients. The number of coefficients is the same as the dimension of the input feature vector. Each coefficient corresponds to an element in the input feature vector. For each neuron, multiply its corresponding weight coefficient by the corresponding element in the input feature vector to obtain a set of weighted components for that neuron. S30132: Within each neuron, sum all the weighted components obtained to get the weighted sum of the neuron; then add the weighted sum to a bias term pre-stored within the neuron to get the linear combination output value of the neuron; the linear combination output values ​​of all neurons together form a temporary vector; S30133: Each element in the temporary vector is input into the non-linear activation function. The non-linear activation function maps each element independently and outputs the new value after mapping. All the new values ​​are arranged in the original order to form the final output vector of the last fully connected layer, which is the decision feature vector after deep abstraction and fusion of all hidden layers.

[0061] The working principle and beneficial effects of the above technical solution are as follows: This embodiment achieves deep fusion and feature extraction of these two types of information by comprehensively utilizing clinical indicators and imaging region information and leveraging the abstraction capabilities of deep networks; through layer-by-layer abstraction of intermediate fully connected layers, the original clinical and imaging data can be transformed into higher-level representations, enhancing the expressive power of features; through the linear combination of the weight coefficients within neurons and the input feature vector, a weighted sum of features is achieved, and the weighting process highlights the impact of important features on decision-making; the introduction of bias terms allows the model to adjust the weighted sum of features, providing flexibility; the use of nonlinear activation functions introduces nonlinear features into the model, enabling the capture of complex relationships and patterns in the data, enhancing the model's expressive and generalization capabilities; the final output decision feature vector is the result of deep abstraction and fusion of all hidden layers, more accurately describing the deep correlation between clinical and imaging data, providing strong support for clinical decision-making. The combined linear and nonlinear processing effectively improves the model's ability to understand and process complex data, thereby achieving higher accuracy and reliability in clinical decision-making.

[0062] Example 10: Based on Example 9, the process for obtaining the linear combination output value of neurons provided in this embodiment of the invention includes the following steps: S301321: Obtain the weighted sum formed by summing the weighted components within the current neuron. The weighted sum is the comprehensive response value of clinical indicator information and image region information after being modulated by weight coefficients. At the same time, read the pre-configured bias term from the neuron's internal storage unit. The bias term is the inherent adjustment parameter determined by the neuron during training.

[0063] S301322: The weighted sum and the bias term are fed into the addition operation simultaneously. The two values ​​are combined into a new value through numerical addition. The new value is the result of the weighted sum after the bias term is adjusted. It contains the linear combination output of clinical and imaging information on the neuron. S301323: The new value obtained by the addition operation is passed to the output of the neuron as the linear response of the neuron to the clinical and imaging deep correlation information in the current input feature vector; the output value of the neuron is placed in the temporary vector at the position corresponding to the neuron, and together with the linear combination of the output values ​​of other neurons, it forms a complete temporary vector.

[0064] The working principle and beneficial effects of the above technical solution are as follows: This embodiment achieves effective integration of clinical and imaging information from different sources and outputs a comprehensive result through weighting, bias adjustment and output fusion; it can improve the accuracy of diagnosis or analysis because the model considers multiple information sources at the same time and performs appropriate weighting and adjustment.

[0065] Example 11: Based on Example 10, the process of merging two values ​​into a new value through numerical addition provided in this embodiment of the invention includes the following steps: S3013221: The weighted sum is used as the first operand, and the bias term is used as the second operand. The two operands are sent to the adder. After receiving the two operands, the adder extracts each bit of the weighted sum and the corresponding bit of the bias term. S3013222: The adder starts from the least significant bit, aligns the current bit value of the weighted sum with the current bit value of the bias term, merges the two values ​​of the current bit and produces the result of this bit, and generates a carry signal based on the carry value generated during the merging process and passes it to the next bit to participate in the operation. S3013223: After the adder performs the above alignment, merging and carry processing on all bits in sequence, it combines the bit results generated by all bits in order from the least significant bit to the most significant bit to generate a complete value, which is the new value after adding the weighted sum and the bias term.

[0066] The working principle and beneficial effects of the above technical solution are as follows: This embodiment ensures the accuracy of numerical merging by aligning the values ​​of the weighted sum and bias term bit by bit, avoiding errors caused by misalignment of bits. Processing is performed step-by-step starting from the least significant bit, and the carry mechanism ensures that the operation of each bit is based on the result of the previous bit, improving the efficiency of addition operations. By processing all bits sequentially, it is ensured that the addition operation covers all bits, and the generated complete value contains all the information of the original weighted sum and bias term. It is not limited to specific types of values ​​but can be widely applied to various scenarios requiring numerical merging, demonstrating excellent versatility.

[0067] Example 12: Based on Example 11, the process of combining the bit-specific results generated by all bits in order from least significant bit to most significant bit, provided in this embodiment of the invention, includes the following steps: S30132231: Obtain all the bit results generated by the adder after aligning and merging each bit. Each bit result corresponds to the value of one binary bit, and each bit result carries the bit position information of its corresponding bit. The bit position information is determined by the order of operations in the addition process. S30132232: Based on the position information carried by each bit, the bit corresponding to the least significant bit is determined as the starting bit of the entire value, the bit corresponding to the second least significant bit is determined as the subsequent bit of the starting bit, and so on, arranging all bit results into a sequence in order from the least significant bit to the most significant bit. S30132233: Store all local results in the sequence into a contiguous storage area in the order of arrangement. The first local result in the sequence is stored at the beginning of the storage area, the second local result is stored at the next position immediately following it, and so on until the last local result is stored. The complete data formed in the storage area is the new value after adding the weighted sum and the bias term.

[0068] The working principle and beneficial effects of the above technical solution are as follows: This embodiment realizes the complete process from addition operation to effective organization of binary data, ensuring the accuracy, order and storability of the data.

[0069] Example 13: As Figure 8 As shown, based on Examples 1-12, the dynamic early warning system for the risk of pneumonia in influenza patients provided in this embodiment of the invention includes: The data acquisition module is used to start the data acquisition terminal, establish communication between the data acquisition terminal and the data information system, acquire influenza-related data, and construct an influenza dataset. The influenza dataset is divided into structured data and unstructured image data. The structured data is cleaned, standardized, and encoded to obtain processed structured data. The unstructured image data is preprocessed, and a self-supervised contrastive learning network is used to extract deep features and transform them into high-dimensional feature vectors. A standardized multimodal feature dataset is formed. The alignment processing module is used to input the standardized multimodal feature dataset into the bidirectional modality-aware attention network and the adversarial cross-modality alignment network for deep fusion and semantic alignment processing, forming a unified feature representation after alignment; The early warning output module is used to input the aligned unified feature representation into the downstream prediction network. After risk probability calculation and threshold judgment, it generates risk level and corresponding personalized early warning information and clinical auxiliary suggestions.

[0070] The working principle and beneficial effects of the above technical solution are as follows: The dynamic early warning system for pneumonia risk in influenza patients in this embodiment achieves dynamic monitoring and early warning of pneumonia risk in influenza patients by combining the technical features of each module. The data acquisition module achieves comprehensive collection of influenza-related data through efficient data acquisition technology and effectively transforms the data into two categories: structured and unstructured data, providing a foundation for subsequent processing. The cleaning, standardization, and encoding of structured data, as well as image preprocessing and deep feature extraction of unstructured data, significantly improve the quality and usability of the dataset, promoting the accuracy of subsequent analysis. The alignment processing module utilizes bidirectional modal perception and adversarial cross-modal alignment technology to achieve deep fusion and semantic alignment of multimodal data, enhancing the model's ability to understand different types of data and improving the uniformity and accuracy of feature representation; it can more comprehensively capture the complex relationship between influenza and pneumonia, laying the foundation for accurate risk assessment. The early warning output module, based on the aligned feature representation, calculates risk probability and determines thresholds to output specific pneumonia risk levels and personalized early warning information. This not only helps doctors and patients understand the progression of the disease in a timely manner but also provides targeted clinical support suggestions, thereby achieving dynamic early warning of pneumonia risk in influenza patients and improving the timeliness and accuracy of clinical decision-making. It enhances the accuracy of pneumonia risk prediction for influenza patients and promotes more scientific and personalized clinical decision-making.

[0071] This embodiment achieves significantly improved prediction accuracy and robustness: deep fusion and alignment mechanisms can more fully exploit the complementarity of multimodal information, and are expected to achieve significant improvements in core indicators such as AUC on independent validation sets, for example, by more than 10%, and are more robust to changes in data distribution. It possesses clinically credible interpretability: the model can provide decision-making basis that conforms to clinical logic, such as determining high risk based on a patient's persistent high fever and right lower lung consolidation on imaging, which is easily understood and adopted by doctors. It has strong cross-institutional generalization ability: through adversarial alignment and contrastive learning, the model learns the cross-modal associations of the disease's essence, rather than data biases specific to a particular institution, facilitating deployment in different hospitals. It achieves true early warning and closed-loop intervention: high-precision, interpretable risk assessment combined with real-time push notifications can translate warnings into concrete clinical actions, and the system can be continuously optimized through feedback data. It optimizes medical resource allocation and patient experience: it helps to accurately identify high-risk patients and avoid over-examination; at the same time, it provides patients with transparent and personalized risk management services.

[0072] This invention deeply integrates computer science and medical artificial intelligence, and solves key problems such as multimodal data fusion, semantic alignment and interpretability in the risk prediction of influenza complicated with pneumonia through a series of specific computer technology principles.

[0073] The deep learning and neural network architecture employs multi-layer neural networks for hierarchical feature extraction and abstraction from medical images and clinical data. Convolutional kernels automatically learn local patterns in images, such as lobar consolidation and ground-glass opacities, while fully connected layers non-linearly combine features to simulate complex medical decision-making processes. A self-supervised contrastive learning network is used to pre-train an encoder on unlabeled images to extract high-dimensional image features. The downstream prediction network transforms the fused features layer by layer through multiple fully connected layers, ultimately outputting the risk probability. Attention mechanisms and bidirectional interactive self-attention and cross-attention mechanisms calculate the similarity between the query and key, weighting and recombining the value to dynamically focus on important parts of the input. The bidirectional modality-aware attention network BMA-Net enables mutual guidance between modalities. Intramodal self-attention enhancement is applied to clinical and imaging features respectively, refining their respective high-level semantics. Subsequently, bidirectional cross-attention guides clinical features to focus on relevant regions in the images (e.g., high fever corresponds to a lung lobe), while imaging features conversely enhance clinical indicators (e.g., lesions correspond to laboratory parameters), achieving deep semantic interaction. Generative Adversarial Networks (GANs) are used in the modality discriminator. Adversarial training forces encoders of two modalities to generate features whose origins are difficult to distinguish, thus mapping heterogeneous data to the same semantic space. The Adversarial Cross-Modality Alignment Network (ACA-Net) introduces a modality discriminator that engages in a game with the feature encoder, ultimately aligning clinical and imaging features in a shared space. This addresses the problem of large differences in the original feature distributions and improves the generalization ability across institution data. The Multi-Task Contrastive Alignment Loss (MCAL) is designed to uniformly optimize classification accuracy, modality alignment, feature consistency, and interpretability, allowing the model to consider multiple objectives during training and avoiding biases caused by single-objective optimization. Feature fusion and data processing ensure the orderly fusion of clinically guided imaging features and image-guided clinical features; random transformations are used to augment chest images, improving the effectiveness of self-supervised learning.

[0074] This invention, through deep integration of clinical indicators such as body temperature and CRP, and imaging features such as lung consolidation areas, enables the model to more comprehensively assess patient conditions and output a quantified probability of pneumonia risk. Compared to single-modality models, multimodal fusion significantly improves prediction accuracy, providing a reliable basis for clinical decision-making. An attention mechanism automatically generates a weight map, visually displaying the clinical indicators that contribute most to the prediction, such as high fever and imaging areas like ground-glass opacities. Doctors can intuitively understand the model's decision-making basis, enhancing their trust in the AI ​​system. An adversarial alignment network maps heterogeneous data collected from different hospitals and equipment to a unified semantic space, reducing the impact of domain offset; multi-task loss further constrains feature consistency, ensuring the model maintains high performance in new scenarios and solving the problem of cross-institutional performance degradation in traditional methods. Based on risk probability and preset thresholds, the system automatically generates personalized early warning information and clinical recommendations. For example, high-risk patients may trigger emergency intervention prompts, medium-risk patients may be advised to undergo follow-up examinations, and low-risk patients may receive routine guidance, achieving hierarchical management and resource optimization. Self-supervised contrastive learning utilizes massive amounts of unlabeled images for pre-training, extracts general feature representations, reduces reliance on manual annotation, lowers model development costs, and improves feature robustness.

[0075] This invention systematically applies various computer technologies, including deep learning, attention mechanisms, adversarial training, and multi-task learning, to medical multimodal data analysis, solving the following practical technical problems: achieving deep semantic fusion through bidirectional interaction and adversarial alignment; mapping heterogeneous data to a shared space through standardization, encoding, and adversarial training; providing decision-making support through attention weight visualization; and enhancing cross-scenario adaptability through multi-task learning and data augmentation. This results in improved prediction accuracy, increased model transparency, enhanced cross-institutional generalization ability, and more precise clinical decision support. A computer-readable storage medium according to embodiments of the invention is also described.

[0076] Instructions, such as computer-readable instructions, are stored on a non-transitory computer-readable storage medium. When the computer-readable instructions are executed by a processor, the various methods described above can be performed. The non-transitory computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-transitory non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the non-transitory computer-readable storage medium can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the non-transitory computer-readable storage medium, the various methods described above can be performed. This invention does not relate to the diagnosis or treatment of diseases, but only to the processing of related information and data.

[0077] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of equivalents of this invention, this invention is also intended to include these modifications and variations.

Claims

1. A method for dynamic early warning of pneumonia risk in influenza patients, characterized in that, Includes the following steps: A standardized multimodal feature dataset containing structured and unstructured image data from influenza datasets is input into a bidirectional modality-aware attention network and an adversarial cross-modal alignment network for deep fusion and semantic alignment processing to form an aligned unified feature representation. The aligned unified feature representation is input into the downstream prediction network. After risk probability calculation and threshold judgment, a risk level and corresponding personalized early warning information and clinical auxiliary suggestions are formed.

2. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 1, characterized in that, The process of forming a unified feature representation after alignment includes the following steps: A standardized multimodal feature dataset is input into a bidirectional modality-aware attention network. Intramodal self-attention enhancement processing is performed on the clinical feature vectors and image depth feature vectors of the structured data in the multimodal feature dataset. The association weights between each element are calculated through the self-attention mechanism, and the original clinical features are weighted and recombined according to the weights to obtain the enhanced clinical semantic features. The image depth feature sequence is used as input, and the association weights between each element are calculated through the self-attention mechanism. The original image features are weighted and recombined according to the weights to obtain the enhanced image semantic features. The enhanced clinical semantic features and enhanced image semantic features are input into a bidirectional modality-aware attention network. The enhanced clinical semantic features are used as query signals and the enhanced image semantic features are used as key-value pairs. The clinical attention weight to the image is calculated by the degree of matching between the query and the key. The image semantic features are then weighted and recombined based on the attention weight to obtain clinically guided image features. Then, using the enhanced image semantic features as the query signal and the enhanced clinical semantic features as the key-value pair, the attention weight of the image to the clinical context is calculated by the degree of matching between the query and the key. Based on the attention weight, the clinical semantic features are weighted and recombined to obtain image-guided clinical features. The clinical-guided image features and the image-guided clinical features are then spliced ​​and fused to form a fused feature representation after bidirectional interaction.

3. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 1, characterized in that, The process of inputting the aligned, unified feature representation into the downstream prediction network includes the following steps: The aligned unified feature representation is input into the hidden layer of the downstream prediction network. The hidden layer is composed of multiple fully connected layers connected sequentially. The unified feature representation enters each fully connected layer in turn. Each layer first performs a linear transformation on the input features, and then maps the transformation result to a new feature representation through a non-linear activation function. After passing through layer by layer, the output is a decision feature vector that has undergone deep abstraction and fusion. The decision feature vector is input into the output layer of the downstream prediction network. The output layer contains a neuron. The neuron performs a weighted summation of the decision feature vector and then converts the summation result into a value between 0 and 1 through the Sigmoid function. The value is the probability value of the patient developing influenza complicated with pneumonia. Obtain the preset first risk threshold and second risk threshold, and compare the calculated risk probability value with the first threshold and the second threshold in sequence; If the risk probability value is less than the first threshold, it is judged as a low-risk level; If the risk probability value is greater than or equal to the first threshold and less than the second threshold, it is determined to be a medium risk level. If the risk probability value is greater than or equal to the second threshold, it is judged as a high-risk level; Based on the determined risk level, the corresponding early warning information template is retrieved from the early warning information database, and the corresponding clinical auxiliary suggestion template is retrieved from the clinical auxiliary suggestion database. The patient's identity, key clinical indicator values, and imaging feature descriptions are filled into the corresponding positions in the templates to generate personalized early warning information and clinical auxiliary suggestions for the patient.

4. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 3, characterized in that, The process of outputting the decision feature vector after deep abstraction and fusion includes the following steps: The aligned unified feature representation is input into the first fully connected layer in the hidden layer. The unified feature representation is formed by fusing clinical feature vectors and image feature vectors through bidirectional interaction and adversarial alignment, and contains a common representation of clinical indicator information and image region information. The first fully connected layer performs a linear combination operation on each element in the input feature representation, corresponding to the information of clinical indicators or image regions respectively: assigning a learnable weight coefficient to each element, summing all the weighted element values, and adding a learnable bias term to obtain the linear combination result; the linear combination result is input into a non-linear activation function for mapping, and the output is the feature vector transformed by the first fully connected layer. The feature vector output from the first fully connected layer is input into the intermediate fully connected layers in the hidden layer. The intermediate fully connected layers are composed of one or more sequentially connected fully connected layers. The feature vector enters each intermediate fully connected layer in turn. Each layer repeats the linear combination and non-linear activation operation on the elements in the input vector, which is the result of the previous layer's further integration of clinical and imaging information. After weighted summation and biasing of the input features, they are mapped by a non-linear activation function. After being passed layer by layer, the output is the feature vector abstracted by each intermediate layer. The feature vector output from the intermediate fully connected layers is input into the last fully connected layer in the hidden layers. The elements in the input vector are linearly combined and nonlinearly activated by integrating the deep correlation information between clinical and imaging. The weighted summation and bias of each element of the input feature is then mapped by a nonlinear activation function, and the output is a decision feature vector after deep abstraction and fusion of all hidden layers.

5. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 4, characterized in that, The process of integrating deep correlation information from clinical and imaging data through linear combination and nonlinear activation includes the following steps: The feature vector output from the intermediate fully connected layer is obtained. Each element is a deep association representation formed by the layer-by-layer abstraction of clinical indicator information and image region information. The feature vector is input to each neuron of the last fully connected layer. Each neuron stores a set of weight coefficients in advance. The number of coefficients is the same as the dimension of the input feature vector. Each coefficient corresponds to an element in the input feature vector. For each neuron, its corresponding weight coefficient is multiplied by the corresponding element in the input feature vector to obtain a set of weighted components for that neuron. Within each neuron, all weighted components are summed to obtain the neuron's weighted sum; this weighted sum is then added to a pre-stored bias term within the neuron to obtain the neuron's linear combination output value; the linear combination output values ​​of all neurons together form a temporary vector. Each element in the temporary vector is input into a non-linear activation function, which maps each element independently and outputs a new value after mapping. All the new values ​​are arranged in the original order to form the final output vector of the last fully connected layer, which is the decision feature vector after deep abstraction and fusion of all hidden layers.

6. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 5, characterized in that, The process of obtaining the linear combination output value of neurons includes the following steps: The system obtains the weighted sum formed by summing the weighted components within the current neuron. The weighted sum is the comprehensive response value of clinical indicator information and image region information after being modulated by weight coefficients. At the same time, it reads the pre-configured bias terms from the internal storage unit of the neuron. The bias terms are the inherent adjustment parameters determined by the neuron during training. The weighted sum and the bias term are fed into the addition operation simultaneously. The two values ​​are combined into a new value through numerical addition. The new value is the result of the weighted sum after the bias term is adjusted. It contains the linear combination output of clinical and imaging information on the neuron. The new value obtained by the addition operation is passed to the output of the neuron as the linear response of the neuron to the clinical and imaging deep correlation information in the current input feature vector; the output value of the neuron is placed in the temporary vector at the position corresponding to the neuron, and together with the linear combination of the output values ​​of other neurons, it forms a complete temporary vector.

7. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 6, characterized in that, The process of combining two values ​​into a new value using numerical addition includes the following steps: The weighted sum is used as the first operand, and the bias term is used as the second operand. The two operands are sent to the adder. After receiving the two operands, the adder extracts each bit of the weighted sum and the corresponding bit of the bias term. The adder starts from the least significant bit, aligns the current bit value of the weighted sum with the current bit value of the bias term, merges the two values ​​of the current bit and produces the result of this bit, and generates a carry signal based on the carry value generated during the merging process and passes it to the next bit to participate in the operation. After performing alignment, merging, and carry processing on all bits in sequence, the adder combines the bit-specific results of all bits in order from least significant bit to most significant bit to generate a complete value. This value is the new value obtained by adding the weighted sum and the bias term.

8. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 7, characterized in that, The process of combining the bit-specific results of all bits in order from least significant bit to most significant bit includes the following steps: Get all the bit results generated by the adder after aligning and merging each bit. Each bit result corresponds to the value of one binary bit, and each bit result carries the bit position information of its own, which is determined by the order of operations in the addition process. Based on the position information carried by each bit, the bit corresponding to the least significant bit is determined as the starting bit of the entire value, the bit corresponding to the second least significant bit is determined as the subsequent bit of the starting bit, and so on, so that all bit results are arranged into a sequence in order from the least significant bit to the most significant bit. All local results in the sequence are stored in a contiguous storage area in the order of arrangement. The first local result in the sequence is stored at the beginning of the storage area, the second local result is stored at the next position immediately following it, and so on until the last local result is stored. The complete data formed in the storage area is the new value after adding the weighted sum and the bias term.

9. The method for dynamic early warning of pneumonia risk in influenza patients as described in claim 1, characterized in that, It also includes starting the data acquisition terminal, establishing communication between the data acquisition terminal and the data information system, acquiring influenza-related data, and constructing an influenza dataset; dividing the influenza dataset into structured data and unstructured image data; cleaning, standardizing, and encoding the structured data to obtain processed structured data; preprocessing the unstructured image data; using a self-supervised contrastive learning network to extract deep features and convert them into high-dimensional feature vectors; and forming a standardized multimodal feature dataset.

10. A dynamic early warning system for the risk of pneumonia in influenza patients, used to implement the dynamic early warning method for the risk of pneumonia in influenza patients as described in any one of claims 1-9, characterized in that, Include: The data acquisition module is used to start the data acquisition terminal, establish communication between the data acquisition terminal and the data information system, acquire influenza-related data, and construct an influenza dataset. The influenza dataset is divided into structured data and unstructured image data. The structured data is cleaned, standardized, and encoded to obtain processed structured data. The unstructured image data is preprocessed, and a self-supervised contrastive learning network is used to extract deep features and transform them into high-dimensional feature vectors. A standardized multimodal feature dataset is formed. The alignment processing module is used to input the standardized multimodal feature dataset into the bidirectional modality-aware attention network and the adversarial cross-modality alignment network for deep fusion and semantic alignment processing, forming a unified feature representation after alignment; The early warning output module is used to input the aligned unified feature representation into the downstream prediction network. After risk probability calculation and threshold judgment, it generates risk level and corresponding personalized early warning information and clinical auxiliary suggestions.

Images