Method and apparatus for predicting first-time bleeding of esophageal varices

Abstract

The application discloses an esophageal varices rupture first bleeding prediction method and device, relates to the medical technical field, and the method comprises the following steps: S1, an initial prediction model is constructed; S2, a training sample set is acquired; S3, the initial prediction model is trained and optimized, and a final prediction model is acquired; S4, data to be analyzed is acquired; S5, the final prediction model is used for prediction analysis on the data to be analyzed, and a risk prediction result is acquired; the self-attention mechanism of ViT is used to establish the semantic association of the global anatomical structure; meanwhile, a reinforcement learning mechanism is introduced, feature extraction is modeled as a single-step decision task; the defect that the traditional convolution operation can only focus on the local is broken. The reinforcement learning agent can dynamically filter out the feature area with high discriminability in the feature space, and automatically locate the microvessel structure of the lower esophagus and the gastric fundus. The method effectively suppresses the complex background noise interference in the CT image, and greatly improves the detection sensitivity and classification accuracy of the model for high-risk and micro-focal lesions in a complex background.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to a method and device for predicting the first bleeding from ruptured esophageal varices. Background Technology

[0002] Liver cirrhosis has become one of the major public health problems threatening human health worldwide. According to data from the Global Burden of Disease Study (GBD 2021), liver cirrhosis causes more than one million deaths globally each year, and its age-standardized incidence rate is rising steadily at a rate of 0.35% per year. During the course of liver cirrhosis, hepatic blood flow resistance increases significantly, inducing portal hypertension. To compensate for venous return, the circulatory system spontaneously establishes collateral circulation, causing tiny blood vessels that could only withstand low flow to passively endure the impact of high-pressure blood flow, resulting in pathological dilation and tortuosity, i.e., varices. Once these vessels rupture, it will cause fatal esophageal variceal bleeding (EVB). Clinically, the course of liver cirrhosis is divided into compensated and decompensated stages. Approximately 50% of liver cirrhosis patients have esophageal varices, with the highest proportion reaching 85% in decompensated patients. The first esophageal variceal bleeding (FVH) is the landmark event marking the transition from the compensated to the decompensated stage. Studies have shown that the six-week mortality rate for patients experiencing their first bleeding episode is as high as 40%, and the one-year survival rate is only 30%. Therefore, the Baveno International Consensus on Portal Hypertension clearly states that the focus of clinical management must be shifted forward. For compensated patients with significant portal hypertension, preventing the occurrence of free portal hypertension (FVH) is the core strategy for reducing the overall risk of death.

[0003] To assess esophageal varices and their bleeding risk, clinical practice currently relies primarily on two traditional methods, both of which have significant limitations: Endoscopy, while the current gold standard for diagnosis, is an invasive procedure with generally low patient tolerance and inherent risks of bleeding or perforation. This leads to decreased compliance among high-risk patients due to resistance, resulting in missed opportunities for primary prevention. Furthermore, endoscopic varicose vein grading essentially depends on the physician's visual assessment, which is highly subjective. Studies show that even among physicians of different experience levels, the diagnostic consistency coefficient is often below 0.6, and this subjective bias directly impacts subsequent treatment decisions. With the widespread adoption of CT technology, enhanced abdominal CT has become an important non-invasive assessment tool. However, given the complex vascular anatomy and massive amounts of tomographic images, diagnostic accuracy is highly dependent on the radiologist's personal experience. Research indicates that under high workloads, physicians experience impaired visual accommodation and subjective fatigue, leading to decreased sensitivity in lesion detection later in the workday. This results in a significant decrease in the diagnostic AUC (area under the curve), increasing the risk of missed and misdiagnosed cases, making it difficult to meet the needs of large-scale, accurate screening.

[0004] In recent years, to address the shortcomings of manual diagnosis, researchers have attempted to introduce machine learning and deep learning technologies for assisted diagnosis. Existing publicly available research mainly falls into the following categories:

[0005] Limitation 1: Deep learning models based on endoscopic images still cannot overcome the "invasiveness" bottleneck. Existing research has developed automated bleeding risk rating systems based on deep convolutional neural networks or incorporating attention mechanisms. These technologies, by learning lesion characteristics under endoscopic images (such as redness, varicose vein diameter, etc.), have achieved accuracy superior to some endoscopists. However, the prerequisite for such technologies remains that patients must undergo gastroscopy. As long as the initial data acquisition process remains accompanied by high pain and low compliance, meaningful non-invasive early screening cannot be achieved in the compensated, asymptomatic patient population.

[0006] Defect 2: Traditional CT-based radiomics models suffer from superficial feature extraction and weak generalization ability. Some studies utilize automated measurements of specific organ volumes (such as the spleen) in CT scans and combine them with hematological indicators to construct assessment indices; others extract texture and statistical features from CT images and use traditional algorithms such as Support Vector Machines (SVM) to construct radiomics models; still others extract multiple features to construct organ fusion models for prediction. However, these methods heavily rely on expert-designed anatomical priors for feature engineering (such as specifying parameters for spleen and liver lobes). In the feature classification stage, most employ shallow machine learning algorithms such as Support Vector Machines (SVM) or logistic regression. These shallow algorithms struggle to fully exploit the complex nonlinear mapping relationships between high-dimensional CT images and multidimensional clinical indicators, resulting in a low upper limit for prediction and insufficient generalization ability when faced with complex clinical samples.

[0007] Deficiency 3: The non-invasive prediction of first-time esophageal variceal bleeding (FVH) lacks a deep multimodal fusion mechanism. Current non-invasive prediction methods for FVH often extract specific quantitative imaging parameters (such as the ratio of local liver lobe to total liver volume) as independent predictors, or combine virtual computational fluid dynamics indicators to construct traditional clinical prediction nomograms. These existing FVH-specific prediction methods mostly employ a simple "variable splicing" logic, failing to perform low-level semantic alignment and deep dynamic fusion of unstructured imaging data (CT features) with structured clinical physiological and biochemical indicators (such as platelet count, INR, etc.). This results in clinical indicators failing to provide adaptive guidance and calibration for the extraction of imaging features.

[0008] Defect 4: The fundamental deep learning architecture suffers from a dual contradiction between "local receptive field" and "computing burden." Traditional convolutional neural networks (CNNs, such as the ResNet series) struggle to capture long-distance spatial dependencies between esophageal varices and portal hypertension sites when processing enhanced abdominal CT scans due to the "local receptive field" of the convolutional kernels. On the other hand, directly adopting a large Transformer architecture with a global receptive field results in excessively high computational density and memory consumption due to its massive parameter count. In grassroots medical institutions lacking high-performance GPU computing resources, such large models consume significant time for inference, hindering practical deployment. Summary of the Invention

[0009] The purpose of this invention is to design a method and device for predicting the first bleeding from ruptured esophageal varices in order to solve the above-mentioned problems.

[0010] The present invention achieves the above objectives through the following technical solutions:

[0011] Methods for predicting the first bleeding from a ruptured esophageal variceal vein include:

[0012] S1. Construct the initial prediction model, which includes a preprocessing layer, a teacher network, and a student network. The output of the preprocessing layer serves as the input to the teacher and student networks, respectively. The input data includes structured clinical physiological and biochemical index data and CT image data. The preprocessing layer preprocesses the structured clinical physiological and biochemical index data to obtain clinical semantic features, and both the clinical semantic features and CT image data are used as the output of the preprocessing layer. The teacher network, from input to output, includes a visual feature extraction module, a gating branch, a bias correction branch, and a multimodal dynamic fusion module. The visual feature extraction module extracts the visual representation vectors from the CT image data. The gating branch converts the clinical semantic features into a number of classification categories. Corresponding weight vector The bias correction branch is used to generate bias terms based on clinical semantic features. The multimodal dynamic fusion module is used to fuse weight vectors. Bias terms The teacher network outputs visual feature vectors and visual representation vectors; the student network includes a visual encoder, a feature extraction branch, a multimodal fusion module, and a classification layer. The visual encoder encodes CT image data to obtain visual feature vectors; the feature extraction branch extracts clinical feature vectors from clinical semantic features; the multimodal dynamic fusion module fuses visual feature vectors and clinical feature vectors to obtain fused features; and the classification layer maps the fused features to risk prediction results.

[0013] S2. Obtain the training sample set, which includes multiple training samples. Each training sample includes structured clinical physiological and biochemical index data and CT image data.

[0014] S3. Use the training sample set to train and optimize the initial prediction model. After the training and optimization are completed, the final prediction model is obtained. The final prediction model includes the optimized preprocessing layer and the student network from input to output.

[0015] S4. Obtain the data to be analyzed, which includes structured clinical physiological and biochemical indicators and CT image data;

[0016] S5. Use the final prediction model to perform predictive analysis on the data to be analyzed and obtain risk prediction results.

[0017] A predictive device for first-time bleeding from ruptured esophageal varices includes:

[0018] Storage; storage is used to store computer programs;

[0019] Actuator; the actuator is used to execute a computer program stored in a memory, which, when executed, implements the method for predicting the first bleeding from ruptured esophageal varices as described above.

[0020] The beneficial effects of this invention are as follows: it utilizes the self-attention mechanism of ViT (Vision Transformer) to establish semantic associations of global anatomical structures; simultaneously, it introduces a reinforcement learning mechanism to model feature extraction as a single-step decision-making task; and it overcomes the limitation of traditional convolutional operations that can only focus on local areas. The reinforcement learning agent can dynamically filter out feature regions with high discriminative power in the feature space, automatically locating the microvascular structures of the lower esophagus and gastric fundus. This method effectively suppresses complex background noise interference in CT images, significantly improving the model's detection sensitivity and classification accuracy for high-risk, microscopic lesions in complex backgrounds. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the initial prediction model of the prediction method for the first bleeding from ruptured esophageal varices of the present invention.

[0022] Figure 2 This is a schematic diagram of the teacher network in the method for predicting the first bleeding from esophageal variceal rupture in this invention;

[0023] Figure 3 This is a schematic diagram of the visual feature extraction module in the prediction method for the first bleeding from ruptured esophageal varices of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0028] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0029] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] like Figure 1 , Figure 2 , Figure 3 As shown, methods for predicting the first bleeding from ruptured esophageal varices include:

[0032] S1. Construct the initial prediction model, which includes a preprocessing layer, a teacher network, and a student network. The output of the preprocessing layer serves as the input to the teacher network and the student network, respectively. The input data includes structured clinical physiological and biochemical indicator data and CT image data. The preprocessing layer is used to preprocess the structured clinical physiological and biochemical indicator data of the input data to obtain clinical semantic features. and clinical semantic features Both CT image data and other data are used as outputs of the preprocessing layer; the teacher network, from input to output, includes a visual feature extraction module, a gating branch and a bias correction branch, and a multimodal dynamic fusion module; the visual feature extraction module is used to extract the visual representation vectors of the CT image data. Gated branches are used to integrate clinical semantic features. Convert to the number of categories Corresponding weight vector Bias correction branches are used to adjust based on clinical semantic features. Generate bias terms The multimodal dynamic fusion module is used to fuse weight vectors. Bias terms and visual representation vector Obtain teacher network output The student network consists of a visual encoder, a feature extraction branch, a multimodal fusion module, and a classification layer. The visual encoder encodes CT image data to obtain visual feature vectors; the feature extraction branch extracts clinical semantic features. The system generates clinical feature vectors; the multimodal dynamic fusion module is used to fuse visual feature vectors and clinical feature vectors to obtain fused features; the classification layer is used to map the fused features to risk prediction results.

[0033] The visual feature extraction module, from input to output, includes a block embedding layer, a multi-layer Transformer encoder, an agent, an environment module, and a classifier. The block embedding layer segments the CT image data into multiple image blocks. The multi-layer Transformer encoder incorporates multi-head self-attention units, which calculate the attention weight matrix between each image block, concatenate the output features of each attention head, and combine them with a feedforward network to extract global image features layer by layer. The global features are input to the environment module, and through reinforced interactive learning between the agent and the environment module, the final mask feature vector is obtained. The classifier classifies based on the final mask feature vector to obtain the visual representation vector. The multi-layer Transformer encoder is implemented using the ViT architecture; the agent is implemented using the Actor-Critic network architecture, which includes an action network and an evaluation network. Global features are learned through reinforcement interaction between the agent and the environment module to obtain the final mask feature vector, which specifically includes:

[0034] (1) Global features are input into the environment module as the initial environment state, and the environment module passes the environment state to the action network;

[0035] (2) The action network learns based on the environment state and outputs the current interaction action to the environment module;

[0036] (3) The environment module performs a masking operation on the global features based on the current interaction action to generate the current mask features;

[0037] (4) The environment module calculates the environment reward based on the current mask features and inputs the environment reward and the current mask features into the evaluation network;

[0038] (5) The evaluation network scores the interactive actions based on environmental rewards and current mask features to obtain the scoring results;

[0039] (6) The evaluation network and action network optimize and update the current interactive action based on the scoring results;

[0040] (7) Repeat (1)-(6) until the stopping condition is met and then enter (8); when the cumulative sum of the scores during the training process reaches its maximum, the stopping condition is met;

[0041] (8) Perform masking operation on the global features based on the optimized and updated interactive actions to generate the final mask feature vector.

[0042] The global features output by ViT are defined as the environment state by the environment module and input into the Actor network composed of deep multilayer perceptrons. The Actor network, based on the current state and the current interaction action A, acts as a feature selector and adaptively fuses with the original features to obtain the current mask features. Simultaneously, a Critic network is used for value evaluation, combined with a PPO-Clip truncation mechanism and a reward function based on accuracy offset. This co-evolution of the perceptron and decision modules enables RLE-ViT to accurately identify high-risk clinical features through policy iteration while learning from images; combining ViT with reinforcement learning forms a complete structure from global image understanding to dynamic focusing decision-making. In CT images, core pathological features are often only located in key areas, while large background areas are filled with irrelevant noise for ViT. Although multi-head self-attention units provide the model with a global receptive field, as the network depth increases, the model's attention to key lesions is spread across the vast background, potentially causing high-level semantic features to become smoothed, reducing the accuracy of risk assessment. To overcome this bottleneck and achieve key focusing on global features, this paper introduces a reinforcement learning mechanism to constrain the model's search area. An Actor-Critic network architecture is adopted to model the classification process as a single-step decision-making task. Through the dynamic interaction between the agent and the feature space, the model can not only focus more on the key regions that play a decisive role in prediction when processing high-dimensional image data, but also effectively suppress irrelevant background noise and improve the model's representation learning ability.

[0043] Clinical semantic features are sequentially processed through gating branches and Tanh activation functions, and then superimposed with a baseline constant of 1 before being output as a weight vector. The gated branch, from input to output, sequentially includes a first linear layer, layer normalization, ReLU activation function, dropout layer, and a second linear layer; the multimodal dynamic fusion module uses weight vectors... Element-wise nonlinear modulation is applied to the visual representation vector, and then modulated with the bias term. The teacher network output is obtained through fusion; weight vector Represented as: ;

[0044] To provide a sufficient representation space for feature interactions and capture the potential nonlinear relationships between different indicators, the dimensions of clinical semantic features are increased. Then, layer normalization is applied to these increased dimensions. Since medical data may contain extreme outliers, layer normalization can mitigate the impact of outliers on feature distribution, avoid gradient anomalies, and improve numerical stability during training. Finally, the ReLU nonlinear activation function is used to further enhance the expressive power of the features. Here, tanh constrains the output value within the range... Within, the superimposed reference constant 1 makes the final gating coefficient Mapped to Within the range. When clinical indicators do not indicate significant abnormalities, The value is approximately 1.0, and the model will use the original visual modality for judgment; when high-risk indicators such as a significantly decreased platelet count and an abnormally high INR value are present, A value greater than 1.0 indicates an increased risk signal for bleeding; when physiological indicators show that the patient's liver function is well compensated, If the value is less than 1.0, the model will actively suppress possible false positive noise;

[0045] The bias correction branch comprehensively captures risk factors independent of imaging manifestations in similar clinical data; the bias correction branch generates learnable bias terms through linear transformation. , is represented as: ,in, and These are all weight parameters of the bias correction branch; the bias term β directly acts on the decision boundary of the classification layer, achieving fine-tuning of the model output through translation. This design enables the model to simulate atypical imaging manifestations and clinically strong indications of high-risk clinical scenarios, thereby more completely integrating multi-source information.

[0046] Unlike common methods that perform early fusion at the feature embedding layer, the multimodal dynamic fusion module performs fusion at the classification decision layer. The module aims to preserve the feature independence and internal structure of both the visual and clinical modalities to the greatest extent possible, outputting the teacher network output. , is represented as: Among them, This represents the Hadamard product, which is the element-wise multiplication of corresponding elements.

[0047] The multimodal dynamic fusion module constructs a dynamic decision-making framework by integrating multiplicative calibration and additive correction. Its core advantage lies in the flexibility of the final classification boundary, which adapts to the patient's clinical background information. It can make strong predictions when image and clinical information align, and it can also handle situations where the two are ambiguous or conflicting. When visual features are at the classification boundary or clinical indicators strongly suggest high risk, the modulatory mechanism enhances the response of the relevant category, capturing hidden risks. If clinical indicators show that the patient's physiological state is stable, the model will suppress potential visual noise, pulling possible false positive predictions back to a safe range.

[0048] In the joint optimization of multimodal models, the model faces the problem of significant differences in the properties of parameters across different modules. The RLE-ViT visual backbone network already possesses a certain ability to represent image features, with a relatively stable parameter distribution and a high degree of structure. However, the parameters of the newly introduced clinical feature projection layer and gating network are in a random initial state, requiring significant adjustments in the early stages of training to learn feature mappings. Using a uniform learning rate for training, the drastic gradient fluctuations generated by the clinical branch in the initial stage will affect the visual backbone network through backpropagation, causing the learned effective features to degenerate or be forgotten.

[0049] To address the aforementioned issues, a differential learning rate strategy was employed during model training and optimization, assigning different learning rates to different modules. For gating networks, clinical multilayer perceptrons, and classification heads that require rapid adaptation to new tasks, a higher learning rate was used to enable them to quickly establish the association between clinical features and target risks. For pre-trained visual backbone networks, a lower learning rate was used, allowing only minor adjustments to parameters to maintain existing visual representation capabilities while adapting to new tasks.

[0050] use Asymmetric update step size can isolate gradient interference during optimization of different modalities to a certain extent. Differential learning rate configuration retains the original visual perception capability of RLE-ViT while ensuring smooth convergence during multimodal joint training, mitigating the catastrophic forgetting phenomenon that may occur in the early stages of introducing new modalities; among them, These represent the parameters of the pre-trained visual backbone RLE-ViT network; Parameters representing newly added networks; The learning rate; For loss function For parameters The gradient is calculated by taking the derivative, where L is the cross-entropy loss function commonly used in classification neural networks.

[0051] The student network consists of a visual encoder, a feature extraction branch, a multimodal fusion module, and a classification layer. The visual encoder encodes CT image data to obtain visual feature vectors; the feature extraction branch extracts clinical semantic features. The system uses clinical feature vectors; a multimodal dynamic fusion module is used to fuse visual feature vectors and clinical feature vectors to obtain fused features; a classification layer is used to map the fused features to risk prediction results; the visual encoder uses the MobileNetV3 visual encoder.

[0052] The specific processing procedure for student networks is as follows:

[0053] For the input CT image data After removing the original classifier, the final feature map is flattened, outputting a visual feature vector with a dimension of 960. , represented as: ,in, This represents the flattening operation of the feature map. For MobileNetV3 visual encoder;

[0054] To handle structured clinical indicators (such as platelet count and spleen thickness), the student network employs a lightweight two-layer nonlinear mapping network. The feature extraction branch first maps the original feature vector of dimension 1 to 64, followed by batch normalization and ReLU activation to accelerate training and improve stability. To mitigate overfitting, a Dropout layer is introduced, and a second linear transformation compresses the feature dimension to 32, outputting a compact clinical semantic feature vector. , represented as: ,in, Indicates a linear operation. and This represents the weight parameters of the first and second layers of a two-layer network. Used to represent vector dimension For batch normalization operations, Clinical semantic features;

[0055] The multimodal fusion module employs a strategy of feature concatenation and deep nonlinear mapping. First, the extracted visual features and clinical features are concatenated along the channel dimension to form a joint feature vector. , represented as: ;in, This involves splicing features along the channel dimension.

[0056] To enhance the robustness and generalization ability of the model in a lightweight design, this method introduces a Dropout mechanism with a dropout rate of 0.5 after the fusion layer. This design forces the model to avoid over-reliance on any single modality or a few feature channels during training, thus not only simulating the incomplete data situations that may be encountered in clinical practice, but also strengthening the learning of intrinsic correlations across modalities.

[0057] Finally, the classification layer maps the fused features to risk prediction results through a fully connected layer containing non-linear activation. ;

[0058] The student network consists of a depthwise separable convolutional layer, a state perceptron, a feature fusion layer, and an output head. The depthwise separable convolutional layer extracts visual features from preprocessed CT image data, while the state perceptron extracts clinical features from preprocessed structured clinical physiological and biochemical indicator data. The feature fusion layer fuses visual and clinical features to obtain a fusion result. The output head analyzes the fusion result to obtain a risk prediction result. The outputs of the depthwise separable convolutional layer and the state perceptron serve as inputs to the feature fusion layer, and the output of the feature fusion layer serves as input to the output head.

[0059] For high-dimensional CT image data, the preprocessing layer performs standardization operations such as spatial normalization, voxel resampling, and random cropping to ensure the consistency of image distribution in spatial resolution and anatomical location, and enhance the model's translation invariance to scale and location. For structured clinical physiological and biochemical index data, the system performs Z-score standardization on continuous numerical variables (such as platelet count) and label encoding on discrete categorical variables (such as ascites degree). This preprocessing mechanism effectively alleviates the bias in network gradient calculation caused by the difference in absolute values ​​of different physiological and biochemical indicators, and lays a stable and reliable data distribution foundation for subsequent low-level alignment and deep fusion of multimodal information in the same high-dimensional feature space.

[0060] The CT imaging data consisted of enhanced abdominal computed tomography (CT) scans. Structured clinical physiological and biochemical indicators comprehensively covered the patient's liver function and coagulation status, including core features such as platelet count, international normalized ratio (INR), total bilirubin, albumin, ascites level, and etiology of cirrhosis. S2: Obtain the training sample set, which includes multiple training samples, each containing structured clinical physiological and biochemical indicators and CT imaging data.

[0061] S3. Train and optimize the initial prediction model using the training sample set. After training and optimization, the final prediction model is obtained. The final prediction model, from input to output, includes an optimized preprocessing layer and a student network; specifically, it includes:

[0062] S301. Training samples are input into the preprocessing layer. The preprocessing layer preprocesses the structured clinical physiological and biochemical index data to obtain clinical semantic features. Both the clinical semantic features and CT image data are used as the output of the preprocessing layer.

[0063] S302. The teacher network analyzes CT image data and clinical semantic features to obtain teacher network output.

[0064] S303. Calculate the first loss based on the teacher network output and the corresponding true labels of the training samples. The first loss is the cross-entropy loss.

[0065] S304. Using a hierarchical differential learning rate control strategy, update the model parameters of the teacher network according to the first loss to obtain the trained teacher network; the updated model parameters of the teacher network are expressed as: in, These are the network parameters for the current teacher network. The network parameters for the optimized teacher network; Parameters representing newly added networks; The learning rate; For the first loss function For parameters The gradient calculated by differentiation is used as the first loss, which is the cross-entropy loss.

[0066] S305. Determine whether to terminate the first round of training. If yes, the trained teacher network is used as the optimized teacher network and proceeds to S307; otherwise, proceed to S306. When the performance of the trained teacher network reaches the preset performance threshold, terminate the first round of training.

[0067] S306, Let t1 = t1 + 1, and return to S301, where t1 is the current first training round;

[0068] S307. Training samples are input into the preprocessing layer. The preprocessing layer preprocesses the structured clinical physiological and biochemical index data to obtain clinical semantic features. Both the clinical semantic features and CT image data are used as the output of the preprocessing layer.

[0069] S308. The optimized teacher network analyzes CT image data and clinical semantic features to obtain teacher network output; the current student network performs predictive analysis on CT image data and clinical semantic features to obtain risk prediction results.

[0070] S309. Calculate the second loss based on the teacher's network output, risk prediction results, and the corresponding true labels of the training samples. , represented as: For KL divergence loss; and These are the temperature-scaled soft probability distributions of the student network and the teacher network, respectively. This is the first loss; These are the true labels of the training samples; These are the risk prediction results output by students online. For distillation weight, Gradient compensation factor; second loss The design mainly focuses on the gradient compensation factor. and distillation weight According to the chain rule for differentiation, when calculating the softmax gradient of Logits scaled by temperature T, its magnitude will be reduced by approximately 1 / Multiply by 10000 before the KL divergence term. This can compensate for the gradient decay, ensuring that the hidden knowledge from the teacher model has sufficient strength to drive the parameter updates of the student model during backpropagation. Considering the small number of parameters in the student model, overfitting may occur on limited medical data, α=0.95 in this method. This means that during training, 95% of the gradient signal comes from the soft label distribution that makes the output distribution of the student network approximate the distribution of the teacher network, while only 5% comes from the supervision of the real hard labels, allowing the student network to focus on fitting the probability distribution of the teacher network that generalizes better.

[0071] S310. Use the second loss to train the current student network to obtain the trained student network;

[0072] S311. Determine whether to terminate the second round of training. If yes, the trained student network becomes the optimized student network and proceeds to S4; otherwise, proceed to S312. When the performance of the trained student network reaches the preset performance threshold, terminate the second round of training.

[0073] S312, Let t2 = t2 + 1, and return to S307, where t2 is the current second training round.

[0074] To effectively address the industry pain point of high-precision multimodal large models being difficult to deploy on ordinary primary healthcare equipment due to their large number of parameters and high computational consumption, this method introduces a knowledge distillation compression technique based on high-temperature coefficient soft labels in the second training phase. This technique trains a convergent teacher network that not only guides the student network to fit clear disease diagnosis results (i.e., hard labels) but also introduces a temperature coefficient scaling function to soften the probability distribution generated by the teacher network at the output layer. By calculating the KL divergence loss of the teacher and student networks, the lightweight student network learns not only the classification results but also the uncertainty distribution and inter-class logical correlations of the teacher network in the decision-making process. This technique successfully inherits the complex feature discrimination capabilities of large models with a relatively low overhead of only about 3.1 million parameters. After training optimization, the large teacher network is stripped away, and in actual medical clinical inference scenarios, only the lightweight student network with significantly compressed parameters needs to be run independently. Even with limited hardware computing power, the student network can still inherit the excellent discrimination ability of the large model, thus quickly and non-invasively outputting the risk probability and detailed classification warning results of the target patient's first esophageal variceal bleeding (FVH).

[0075] S4. Obtain the data to be analyzed, which includes structured clinical physiological and biochemical indicators and CT image data;

[0076] S5. Use the final prediction model to perform predictive analysis on the data to be analyzed and obtain risk prediction results.

[0077] A predictive device for first-time bleeding from ruptured esophageal varices includes:

[0078] Storage; storage is used to store computer programs;

[0079] Actuator; the actuator is used to execute a computer program stored in a memory, which, when executed, implements the method for predicting the first bleeding from ruptured esophageal varices as described above.

[0080] Example

[0081] Dataset and Experimental Setup: The dataset primarily comes from two tertiary hospitals in Southwest China. This study included 179 patients from 2014 to 2022. Of these, 104 patients who experienced their first varicose vein hemorrhage (FVH) during follow-up were assigned to the FVH group, and 75 patients who did not experience FVH were assigned to the non-FVH group. The dataset was divided into a training set containing 125 patients (72 with hemorrhage, 53 without) and a test set containing 54 patients (32 with hemorrhage, 22 without). In terms of image data size, the training set contained 84,040 CT images, and the test set contained 28,649 CT images.

[0082] To enhance the model's generalization ability and discriminative power in complex scenarios, a composite data augmentation module was constructed during the training phase. This module uses artificially synthesized virtual samples to compensate for the limitations of the original data's limited scale and singular distribution. The module primarily employs data augmentation operations such as scaling, horizontal flipping, random rotation, noise injection, random erasure, and brightness adjustment.

[0083] In terms of experimental environment configuration, the hardware computing equipment is equipped with an Intel Xeon Gold5218R central processing unit with a main frequency of 2.10GHz, an NVIDIA GeForce RTX 3080 Ti graphics processor with 12GB of video memory, and 251GB of system memory. The software environment runs on the Ubuntu 22.04.5 operating system, and the core algorithms are developed based on the Python programming language and the PyTorch deep learning framework, relying on CUDA 12.5 for GPU-based acceleration of model training and inference.

[0084] The comparative experimental results with existing technical solutions are shown in Table 1. Table 1 shows that in the comparative experiments with existing mainstream deep learning models, the prediction accuracy of traditional classic convolutional neural networks (such as ResNet series, DenseNet, Vgg16, etc.) generally remained between 85% and 91%, with a maximum AUC value of only 0.941, and they generally suffered from a bottleneck in achieving a balance between sensitivity and specificity. In contrast, the teacher network proposed in this method achieved significant improvements in all core evaluation indicators. Its overall accuracy reached 98.14%, with an AUC value of 0.990, while simultaneously raising the sensitivity to 96.87% and maintaining a specificity of 100.00%. This significant data leap indicates that, thanks to the enhanced visual feature extraction and dynamic clinical gating mechanism of reinforcement learning, this model can not only identify hidden signs of bleeding risk, thus reducing missed diagnoses, but also reduce false positive interference.

[0085] The results of multimodal model distillation are shown in Table 2. The lightweight architecture of this method demonstrates its value for engineering implementation and practical deployment. Experimental data show that although the teacher network has high diagnostic performance, its network parameters are as high as nearly 89.5 million, and the number of floating-point operations (FLOPs) reaches 16.867G. The huge computational burden limits its operation on ordinary primary healthcare equipment. In contrast, the student network generated after knowledge distillation successfully compresses the parameter size to only about 3.1 million, with a compression rate of 96.5%, and the computational complexity is reduced to 0.232G.

[0086] Table 1

[0087]

[0088] Table 2

[0089]

[0090] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for predicting the first bleeding from ruptured esophageal varices, characterized in that, include: S1. Construct the initial prediction model, which includes a preprocessing layer, a teacher network, and a student network. The output of the preprocessing layer serves as the input to the teacher and student networks, respectively. The input data includes structured clinical physiological and biochemical index data and CT image data. The preprocessing layer is used to preprocess the structured clinical physiological and biochemical index data of the input data to obtain clinical semantic features, and both the clinical semantic features and CT image data are used as the output of the preprocessing layer. The teacher network, from input to output, includes a visual feature extraction module, a gating branch, a bias correction branch, and a multimodal dynamic fusion module. The visual feature extraction module is used to extract the visual representation vector of the CT image data. The gating branch is used to convert the clinical semantic features into a number of classification categories. Corresponding weight vector ; The bias correction branch is used to generate bias terms based on clinical semantic features. The multimodal dynamic fusion module is used to fuse weight vectors. Bias terms The teacher network outputs visual feature vectors and visual representation vectors; the student network includes a visual encoder, a feature extraction branch, a multimodal fusion module, and a classification layer. The visual encoder encodes CT image data to obtain visual feature vectors; the feature extraction branch extracts clinical feature vectors of clinical semantic features; and the multimodal dynamic fusion module fuses visual feature vectors and clinical feature vectors to obtain fused features. The classification layer is used to map the fused features to risk prediction results; S2. Obtain the training sample set, which includes multiple training samples. Each training sample includes structured clinical physiological and biochemical index data and CT image data. S3. Use the training sample set to train and optimize the initial prediction model. After the training and optimization are completed, the final prediction model is obtained. The final prediction model includes the optimized preprocessing layer and the student network from input to output. S4. Obtain the data to be analyzed, which includes structured clinical physiological and biochemical indicators and CT image data; S5. Use the final prediction model to perform predictive analysis on the data to be analyzed and obtain risk prediction results.

2. The method for predicting the first bleeding from ruptured esophageal varices according to claim 1, characterized in that, The visual feature extraction module includes a block embedding layer, a multi-layer Transformer encoder, an agent, an environment module, and a classifier from input to output. The block embedding layer is used to divide the CT image data into multiple image blocks. The multi-layer Transformer encoder has a built-in multi-head self-attention unit. The multi-head self-attention unit calculates the attention weight matrix between each image block, splices the output features of each attention head, and combines the feedforward network to extract global image features layer by layer. Global The feature input environment module obtains the final mask feature vector through enhanced interactive learning between the agent and the environment module. The classifier classifies the data based on the final mask feature vector to obtain the visual representation vector.

3. The method for predicting the first bleeding from ruptured esophageal varices according to claim 1, characterized in that, The agent comprises an action network and an evaluation network. Global features are learned through reinforcement learning between the agent and the environment module, resulting in the final mask feature vector, which specifically includes: (1) Global features are input into the environment module as the initial environment state, and the environment module passes the environment state to the action network; (2) The action network learns based on the environment state and outputs the current interaction action to the environment module; (3) The environment module performs a masking operation on the global features based on the current interaction action to generate the current mask features; (4) The environment module calculates the environment reward based on the current mask features and inputs the environment reward and the current mask features into the evaluation network; (5) The evaluation network scores the interactive actions based on environmental rewards and current mask features to obtain the scoring results; (6) The evaluation network and action network optimize and update the current interactive action based on the scoring results; (7) Repeat (1)-(6) until the stopping condition is met and then proceed to (8); (8) Perform masking operation on the global features based on the optimized and updated interactive actions to generate the final mask feature vector.

4. The method for predicting the first bleeding from ruptured esophageal varices according to claim 1, characterized in that, Clinical semantic features are sequentially processed through gating branches and Tanh activation functions, and then superimposed with a baseline constant of 1 before being output as a weight vector. The gated branches, from input to output, sequentially include a linear layer, layer normalization, ReLU activation function, and dropout layer; the multimodal dynamic fusion module uses weight vectors... Element-wise nonlinear modulation is applied to the visual representation vector, and then modulated with the bias term. The integration process yields teacher network outputs.

5. The method for predicting the first bleeding from ruptured esophageal varices according to claim 1, characterized in that, S3 includes: S301. Training samples are input into the preprocessing layer. The preprocessing layer preprocesses the structured clinical physiological and biochemical index data to obtain clinical semantic features. Both the clinical semantic features and CT image data are used as the output of the preprocessing layer. S302. The teacher network analyzes CT image data and clinical semantic features to obtain teacher network output. S303. Calculate the first loss based on the teacher's network output and the corresponding true labels of the training samples; S304. Use a hierarchical differential learning rate control strategy to update the model parameters of the teacher network according to the first loss to obtain the trained teacher network. S305. Determine whether to terminate the first round of training. If yes, the trained teacher network becomes the optimized teacher network and proceeds to S307; otherwise, proceed to S306. S306, Let t1 = t1 + 1, and return to S301, where t1 is the current first training round; S307. Training samples are input into the preprocessing layer. The preprocessing layer preprocesses the structured clinical physiological and biochemical index data to obtain clinical semantic features. Both the clinical semantic features and CT image data are used as the output of the preprocessing layer. S308. The optimized teacher network analyzes CT image data and clinical semantic features to obtain teacher network output; the current student network performs predictive analysis on CT image data and clinical semantic features to obtain risk prediction results. S309. Calculate the second loss based on the teacher's network output, risk prediction results, and the corresponding true labels of the training samples. S310. Use the second loss to train the current student network to obtain the trained student network; S311. Determine whether to terminate the second round of training. If yes, the trained student network becomes the optimized student network and proceeds to S4; otherwise, proceed to S312. S312, let t2 = t2 + 1, and return to S307, where t2 is the current second training round.

6. The method for predicting the first bleeding from ruptured esophageal varices according to claim 5, characterized in that, The model parameter update for the teacher network is represented as follows: ;in, These are the network parameters for the current teacher network. The network parameters for the optimized teacher network; Parameters representing newly added networks; The learning rate; For the first loss function For parameters The gradient is calculated by differentiation, and the first loss is the cross-entropy loss.

7. The method for predicting the first bleeding from ruptured esophageal varices according to claim 5, characterized in that, Second loss Represented as: , For KL divergence loss; and These are the temperature-scaled soft probability distributions of the student network and the teacher network, respectively. This is the first loss; These are the true labels of the training samples; These are the risk prediction results output by students online. For distillation weight, This is the gradient compensation factor.

8. A device for predicting the first bleeding from a ruptured esophageal variceal vein, characterized in that, include: Storage; Storage is used to store computer programs; Actuator; The actuator is used to execute a computer program stored in a memory, which, when executed, implements the method for predicting the first bleeding from ruptured esophageal varices as described in any one of claims 1-7.

Images