Multi-period outcome prediction method based on improved ResNet encoder

Abstract

The invention relates to the technical field of artificial intelligence and medical image processing, in particular to a multi-stage outcome prediction method based on an improved ResNet encoder, and the method comprises the following steps: collecting a chest CT image of an acute-stage lung disease; an improved ResNet encoder module used for extracting high-dimensional image features in the chest CT image is established through an input convolution layer, four residual blocks, an average pooling layer and a linear transformation layer; and outputting short-term, medium-term and long-term outcome probabilities based on the high-dimensional image features output by the improved ResNet encoder module by using a pre-established multi-stage outcome prediction model. According to the method, a plurality of classification heads are designed to be fused in a cascade mode to realize guidance of time sequence information, an improved cross-period fusion loss function based on Focal Loss is introduced, continuous constraint among prediction results in different periods is realized, and a multi-classification-head cascade structure and a feature extraction encoder structure of a loss cross-period fusion combination are combined, so that the prediction accuracy is improved. And therefore, the performance of the multi-period prediction model can be played to the optimum.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical image processing technology, specifically to a multi-phase outcome prediction method based on an improved ResNet encoder. Background Technology

[0002] In the clinical management of lung diseases (such as severe pneumonia and pulmonary fibrosis), physicians often rely on imaging information to determine the disease progression and develop follow-up plans. Traditional prediction methods are usually based on assessments at a single point in time and lack in-depth modeling of the changing trends of the patient's condition across time scales, making it difficult to meet the needs of personalized follow-up and intervention.

[0003] In recent years, deep learning has been widely used in medical image diagnosis to predict patient outcomes using artificial intelligence. In constructing deep learning models for outcome prediction, a multi-classifier head cascade structure can be used to shift the model's focus from single-phase tasks to simultaneous prediction of multiple phases. Furthermore, by introducing a cross-phase fusion loss constrained by temporal continuity and logical consistency, the model leverages prediction results from earlier and later phases to guide the current prediction, thereby improving the model's multi-phase outcome performance. The high performance of this model structure in multi-phase outcome prediction using medical images relies on the effectiveness of image feature extraction, i.e., the design of the encoder structure. Currently, there is a lack of feature extraction encoder structures specifically designed for the combination of multi-classifier head cascade structures and cross-phase fusion losses. This makes it impossible to guarantee that the multi-phase prediction model formed by these combinations will achieve optimal performance, resulting in insufficient ability to predict patient outcomes across multiple phases. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-stage regression prediction method based on an improved ResNet encoder, in order to solve the technical problem in the prior art of lacking a feature extraction encoder structure for multi-class head cascade structure and loss cross-stage fusion combination, which affects the performance of multi-stage prediction models.

[0005] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: A multi-stage outcome prediction method based on an improved ResNet encoder includes the following steps: Chest CT images of patients diagnosed with lung disease during the acute phase were collected, and the chest CT images were standardized by resampling, cropping, and intensity normalization. An improved ResNet encoder module for extracting high-dimensional image features from chest CT images is built using an input convolutional layer, four residual blocks, an average pooling layer, and a linear transform layer. Using a pre-established multi-period outcome prediction model, short-term, medium-term, and long-term outcome probabilities are output based on the high-dimensional image features output by the improved ResNet encoder module.

[0006] In a preferred embodiment of the present invention, the improved ResNet encoder module takes a chest CT image as input and outputs high-dimensional image features. ; The structural expression of the improved ResNet encoder module is as follows: ; In the formula, Features of high-dimensional images Chest CT image, This is an improved ResNet encoder module; The input convolutional layer includes convolutional layer 1, wherein the input of convolutional layer 1 is... The kernel size of convolutional layer 1 is 7*7*7, the stride is 1, and the number of channels is 31. The first residual block includes convolutional layer 2, convolutional layer 3, convolutional layer 4, and convolutional layer 5. The output of convolutional layer 2, the input and output of convolutional layer 3, the input and output of convolutional layer 4, and the input and output of convolutional layer 5 are sequentially connected. The input of convolutional layer 2 is connected to the output of convolutional layer 1. The input of convolutional layer 2 is connected to the input of convolutional layer 4. The input of convolutional layer 4 is connected to the output of convolutional layer 5. The kernel size of convolutional layer 2 is 3*3*3, the stride is 2, and the number of channels is 32. The kernel size of convolutional layers 3, 4, and 5 is 3*3*3, the stride is 1, and the number of channels is 32. The second residual block includes convolutional layers 6, 7, 8, 9, and 10. The output of convolutional layer 6, the input and output of convolutional layer 7, the input and output of convolutional layer 9, and the input and output of convolutional layer 10 are sequentially connected. The input of convolutional layer 6 is connected to the output of convolutional layer 5. The input of convolutional layer 6 and the input of convolutional layer 9 are connected by a skip connection through convolutional layer 8. The input of convolutional layer 9 and the output of convolutional layer 10 are connected by a skip connection. The kernel size of convolutional layer 6 is 3*3*3, the stride is 2, and the number of channels is 64. The kernel size of convolutional layers 7, 9, and 10 is 3*3*3, the stride is 1, and the number of channels is 64. The kernel size of convolutional layer 8 is 1*1*1, the stride is 2, and the number of channels is 64. The third residual block includes convolutional layers 11, 12, 13, 14, and 15. The output of convolutional layer 11, the input and output of convolutional layer 12, the input and output of convolutional layer 14, and the input and output of convolutional layer 15 are sequentially connected. The input of convolutional layer 11 is connected to the output of convolutional layer 10, and the input of convolutional layer 11 is connected to the input of convolutional layer 14 via... Convolutional layer 13 forms a skip connection, the input of convolutional layer 14 forms a skip connection with the output of convolutional layer 15, the kernel size of convolutional layer 11 is 3*3*3, the stride is 2, and the number of channels is 128, the kernel size of convolutional layers 12, 14 and 15 is 3*3*3, the stride is 1, and the number of channels is 128, and the kernel size of convolutional layer 13 is 1*1*1, the stride is 2, and the number of channels is 128; The fourth residual block includes convolutional layers 16, 17, 18, 19, and 20. The output of convolutional layer 16, the input and output of convolutional layer 17, the input and output of convolutional layer 19, and the input and output of convolutional layer 20 are sequentially connected. The input of convolutional layer 16 is connected to the output of convolutional layer 15, and the input of convolutional layer 16 is connected to the input of convolutional layer 19 via... Convolutional layer 18 forms a skip connection, the input of convolutional layer 19 forms a skip connection with the output of convolutional layer 20, the kernel size of convolutional layer 16 is 3*3*3, the stride is 2, and the number of channels is 256, the kernel size of convolutional layers 17, 19 and 20 is 3*3*3, the stride is 1, and the number of channels is 256, and the kernel size of convolutional layer 18 is 1*1*1, the stride is 2, and the number of channels is 256; The average pooling layer has a convolutional kernel size of 3*3*3 and a stride of 1. The linear transform layer has 12 channels.

[0007] As a preferred embodiment of the present invention, the multi-period outcome prediction model includes three cascaded classification heads, namely a first classification head, a second classification head, and a third classification head, wherein each classification head consists of two linear layers and a softmax activation layer.

[0008] As a preferred embodiment of the present invention, the first classification head is used to classify high-dimensional image features. Given the input, output the short-term outcome probability; The structural expression of the first classification head is: ; ; In the formula, The short-term regression probability is the output of the first classifier. , These are the first and second linear layers in the classification head, respectively. High-dimensional image features Intermediate classification features generated through the first linear layer of the first classification head. This is the softmax activation layer.

[0009] As a preferred embodiment of the present invention, the second classification head is used to classify high-dimensional image features. and the intermediate classification features Compositional fusion characteristics Given the input, output the intermediate outcome probability; The structural expression for the second classification head is: ; ; In the formula, This represents the intermediate regression probability output by the second classification head. , These are the first and second linear layers in the classification head, respectively. For fusion features Intermediate classification features generated through the first linear layer of the second classification head. This is a softmax activation layer, and Concat is the feature concatenation operator.

[0010] As a preferred embodiment of the present invention, the third classification head is used to classify high-dimensional image features. The intermediate classification features and the intermediate classification features Compositional fusion characteristics Given the input, output the long-term outcome probability; The structural expression of the third classification head is: ; In the formula, This represents the long-term regress probability output by the third classification head. , These are the first and second linear layers in the classification head, respectively. This is a softmax activation layer, and Concat is the feature concatenation operator.

[0011] As a preferred embodiment of the present invention, the loss function for training the multi-period outcome prediction model is: ; In the formula, The total loss of the multi-period outcome prediction model. For short-term sub-fusion loss, For intermediate sub-fusion loss, For long-term sub-fusion loss; ; In the formula, i represents the corresponding number for short-term, medium-term, and long-term. , where j is the corresponding number of the classification head used for short-term, medium-term, and long-term outcome probability prediction. Focal Loss is the prediction result of the gold standard for the i-th period on the j-th class head. ; In the formula, For the gold standard of the i-th period, Let be the positive probability output by the j-th classification header. Let be the negative probability output by the j-th classification header. An adjustment factor to balance the contribution of difficult and easy samples to the loss; Among them, when This indicates that the positive result has not turned into a positive outcome. This indicates that the negative result has been confirmed. when hour, middle Used to constrain the loss of positive samples predicted by the current classification head. Used to constrain the loss of negative samples predicted by the current classification head; when hour, The gold standard used to constrain the positive probability output by the previous classification head using the current period's positive samples; when hour, The negative probability of the classification head output is used to constrain the negative sample output in the later period using the gold standard of the current period's negative samples.

[0012] As a preferred embodiment of the present invention, the Adam optimizer is used in training the multi-period outcome prediction model to dynamically calculate the total loss in the short, medium, and long term during the training process. Then, backpropagation is performed until the multi-period outcome prediction model achieves the best accuracy on the validation set, thus obtaining the optimal multi-period outcome prediction model.

[0013] Compared with the prior art, the present invention has the following advantages: This invention designs three classification heads corresponding to short-term (1 month), medium-term (3 months), and long-term (6 months) prediction tasks. These heads are cascaded and integrated with intermediate features from the previous period as auxiliary inputs for the next period, guiding the temporal information. Furthermore, an improved inter-period fusion loss function based on Focal Loss is introduced to constrain the continuity between prediction results from different periods. This allows the constructed multi-period outcome prediction model to output high-precision outcome probabilities for three time points using a single acute-phase CT image. The feature extraction encoder structure is tailored to the cascaded multi-classification head structure and the inter-period fusion loss function, ensuring that the multi-period prediction model formed by the cascaded multi-classification head structure and the inter-period fusion loss function achieves optimal performance. Attached Figure Description

[0014] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0015] Figure 1 A flowchart of a multi-phase regression prediction method based on an improved ResNet encoder provided in an embodiment of the present invention; Figure 2 This is a structural diagram of the multi-period outcome prediction model provided in an embodiment of the present invention; Figure 3 This is a structural diagram of the improved ResNet encoder provided in an embodiment of the present invention; Figure 4 Performance evaluation diagram of the improved ResNet encoder provided in the embodiments of the present invention; Figure 5 This is a graph evaluating the effectiveness of the multi-period outcome prediction model provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1 As shown, this invention provides a multi-period outcome prediction method based on an improved ResNet encoder, comprising the following steps: Acquire chest CT images of acute lung diseases and perform standardized resampling, cropping and intensity normalization on the chest CT images; An improved ResNet encoder module for extracting high-dimensional image features from chest CT images is constructed using one input convolutional layer, four residual blocks, one average pooling layer, and one linear transform layer. Figure 3 As shown; Using a pre-established multi-period outcome prediction model, short-term, medium-term, and long-term outcome probabilities are output based on the high-dimensional image features output by the improved ResNet encoder module.

[0018] The improved ResNet encoder module takes chest CT images as input and outputs high-dimensional image features. ; The structural expression of the improved ResNet encoder module is as follows: ; In the formula, Features of high-dimensional images Chest CT image, For an improved ResNet encoder; The input convolutional layer includes convolutional layer 1, where the input to convolutional layer 1 is... The kernel size of convolutional layer 1 is 7*7*7, the stride is 1, and the number of channels is 31. The first residual block includes convolutional layer 2, convolutional layer 3, convolutional layer 4, and convolutional layer 5. The output of convolutional layer 2, the input and output of convolutional layer 3, the input and output of convolutional layer 4, and the input and output of convolutional layer 5 are sequentially connected. The input of convolutional layer 2 is connected to the output of convolutional layer 1. The input of convolutional layer 2 is connected to the input of convolutional layer 4. The input of convolutional layer 4 is connected to the output of convolutional layer 5. The kernel size of convolutional layer 2 is 3*3*3, the stride is 2, and the number of channels is 32. The kernel size of convolutional layers 3, 4, and 5 is 3*3*3, the stride is 1, and the number of channels is 32. The second residual block includes convolutional layers 6, 7, 8, 9, and 10. The output of convolutional layer 6, the input and output of convolutional layer 7, the input and output of convolutional layer 9, and the input and output of convolutional layer 10 are sequentially connected. The input of convolutional layer 6 is connected to the output of convolutional layer 5. The input of convolutional layer 6 and the input of convolutional layer 9 are connected by a skip connection through convolutional layer 8. The input of convolutional layer 9 and the output of convolutional layer 10 are connected by a skip connection. The kernel size of convolutional layer 6 is 3*3*3, the stride is 2, and the number of channels is 64. The kernel size of convolutional layers 7, 9, and 10 is 3*3*3, the stride is 1, and the number of channels is 64. The kernel size of convolutional layer 8 is 1*1*1, the stride is 2, and the number of channels is 64. The third residual block includes convolutional layers 11, 12, 13, 14, and 15. The output of convolutional layer 11, the input and output of convolutional layer 12, the input and output of convolutional layer 14, and the input and output of convolutional layer 15 are sequentially connected. The input of convolutional layer 11 is connected to the output of convolutional layer 10, and the input of convolutional layer 11 is connected to the input of convolutional layer 14 via... Convolutional layer 13 forms a skip connection, the input of convolutional layer 14 forms a skip connection with the output of convolutional layer 15, the kernel size of convolutional layer 11 is 3*3*3, the stride is 2, and the number of channels is 128, the kernel size of convolutional layers 12, 14 and 15 is 3*3*3, the stride is 1, and the number of channels is 128, and the kernel size of convolutional layer 13 is 1*1*1, the stride is 2, and the number of channels is 128; The fourth residual block includes convolutional layers 16, 17, 18, 19, and 20. The output of convolutional layer 16, the input and output of convolutional layer 17, the input and output of convolutional layer 19, and the input and output of convolutional layer 20 are sequentially connected. The input of convolutional layer 16 is connected to the output of convolutional layer 15, and the input of convolutional layer 16 is connected to the input of convolutional layer 19 via... Convolutional layer 18 forms a skip connection, the input of convolutional layer 19 forms a skip connection with the output of convolutional layer 20, the kernel size of convolutional layer 16 is 3*3*3, the stride is 2, and the number of channels is 256, the kernel size of convolutional layers 17, 19 and 20 is 3*3*3, the stride is 1, and the number of channels is 256, and the kernel size of convolutional layer 18 is 1*1*1, the stride is 2, and the number of channels is 256; The average pooling layer has a convolutional kernel size of 3*3*3 and a stride of 1. The linear transform layer has 12 channels.

[0019] This invention enables cross-layer information transmission during the feature extraction process of CT images through an improved ResNet encoder, avoiding gradient vanishing and obtaining more layered image features, which helps improve the performance of subsequent multi-stage regression models.

[0020] The multi-period outcome prediction model includes three cascaded classification heads: the first classification head, the second classification head, and the third classification head. Each classification head consists of two linear layers and a softmax activation layer.

[0021] The first classification head is used to classify high-dimensional image features. Given the input, output the short-term outcome probability; The structural expression for the first classification head is: ; ; In the formula, The short-term regression probability is the output of the first classifier. , These are the first and second linear layers in the classification head, respectively. High-dimensional image features Intermediate classification features generated through the first linear layer of the first classification head. This is the softmax activation layer.

[0022] The second classification head is used to classify high-dimensional image features. and intermediate classification features Compositional fusion characteristics Given the input, output the intermediate outcome probability; The structural expression for the second classification head is: ; ; In the formula, This represents the intermediate regression probability output by the second classification head. , These are the first and second linear layers in the classification head, respectively. For fusion features Intermediate classification features generated through the first linear layer of the second classification head. This is a softmax activation layer, and Concat is the feature concatenation operator.

[0023] The third classification head is used to classify high-dimensional image features. Intermediate classification features and intermediate classification features Compositional fusion characteristics Given the input, output the long-term outcome probability; The structural expression for the third classification head is: ; In the formula, This represents the long-term regress probability output by the third classification head. , These are the first and second linear layers in the classification head, respectively. This is a softmax activation layer, and Concat is the feature concatenation operator.

[0024] In constructing a multi-stage outcome prediction model for predicting the short-term, medium-term, and long-term probabilities of patient outcomes, this invention selects three classification heads corresponding to short-term (1 month), medium-term (3 months), and long-term (6 months) prediction tasks during the model structure design stage. These classification heads are then cascaded and integrated with intermediate features from the previous period as auxiliary inputs for the next period. This approach guides the time-series information, enables the synchronous execution of multi-stage outcome prediction tasks, and improves outcome prediction efficiency.

[0025] In this invention, the first classification head is based solely on high-dimensional image features. The output transition probability is used. The second and third classifiers introduce the intermediate features of the first and the first two classifiers respectively and then combine them as auxiliary information to participate in the prediction. Through the additional design of the cascaded information transmission path between the three classifiers, the classifiers in later periods (medium and long term) can incorporate the information of the earlier periods when making predictions.

[0026] Compared to the first classification head, the second and third classification heads, corresponding to the March and June periods respectively, not only include the image features extracted by the encoder but also incorporate the intermediate classification features from their respective preceding classification heads. Taking the third classification head corresponding to the June period as an example, its input is a concatenation of image features and the intermediate classification features from the first and second classification heads. This cascading classification feature fusion strategy indirectly introduces short-, medium-, and long-term temporal prior information into the classification head's learning target mapping, thus allowing the longer-term classification head to refer to the shorter-term classification response when making decisions. This also aligns with the gradual progression of short-, medium-, and long-term outcomes in clinical practice.

[0027] In constructing a multi-period outcome prediction model for predicting the short-term, medium-term, and long-term probabilities of patient outcomes, this invention introduces an improved inter-period fusion loss function based on Focal Loss during the model structure training phase. This achieves continuity constraints between prediction results at different periods, thereby improving the accuracy of outcome prediction.

[0028] This invention employs Focal Loss as a constraint for each classification head. This loss enhances the model's focus on difficult samples by increasing the weight of those samples, thereby mitigating the problem of imbalanced samples. Furthermore, each classification head includes an additional inter-period loss term. This term incorporates the current period's gold standard label into the supervision of classification heads in other periods, constraining the temporal consistency of prediction results. This additionally constructed inter-period consistency loss term achieves continuity constraints between prediction results from different periods.

[0029] The continuity constraint ensures that the predictions of a multi-phase prediction model are as consistent as possible in time across the three phases. For example, if the intermediate-term prediction leans towards a positive outcome (higher probability of negative), then the long-term prediction for that patient should also lean towards a positive outcome (higher probability of negative). This avoids the shortcomings of other existing methods that do not consider the temporal consistency of patient outcomes when making predictions across different phases. Specifically, it may result in the same patient being predicted to have a positive outcome in the intermediate phase, but then being predicted not to have a positive outcome in the long term, which does not reflect the actual prognostic trend.

[0030] This invention designs a cross-period fusion loss based on Focal Loss to constrain the consistency and continuity of predictions made by the network across different time periods. This cross-period fusion total loss... Composed of three sub-loss items The components correspond to the three monthly periods, 1, 3, and 6, as shown in the following formula. Each sub-fusion loss term includes not only the Focal Loss constraint for the current period, but also constraints on the positive predictions for the preceding months and the negative predictions for the following months.

[0031] Focal Loss is a loss function specifically designed to address class imbalance. By dynamically adjusting the weights of easily classified and difficult-to-classify samples, it makes the model pay more attention to difficult-to-classify samples, thereby improving the model's performance in object detection and rare class identification.

[0032] The loss function for training a multi-period outcome prediction model is: ; In the formula, The total loss of the multi-period outcome prediction model. For short-term sub-fusion loss, For intermediate sub-fusion loss, For long-term sub-fusion loss; ; In the formula, i represents the corresponding number for short-term, medium-term, and long-term. , where j is the corresponding number of the classification head used for short-term, medium-term, and long-term outcome probability prediction. Focal Loss is the prediction result of the gold standard for the i-th period on the j-th class head. ; In the formula, For the gold standard of the i-th period, Let be the positive probability output by the j-th classification header. Let be the negative probability output by the j-th classification header. An adjustment factor to balance the contribution of difficult and easy samples to the loss; Among them, when This indicates that the positive result has not turned into a positive outcome. This indicates that the negative result has been confirmed. when hour, middle Used to constrain the loss of positive samples predicted by the current classification head. Used to constrain the loss of negative samples predicted by the current classification head; when hour, The gold standard used to constrain the positive probability output by the previous classification head using the current period's positive samples; when hour, The negative probability of the classification head output is used to constrain the negative sample output in the later period using the gold standard of the current period's negative samples.

[0033] like Figure 2 As shown, the constraints corresponding to the three loss terms are illustrated (dashed lines of different colors). Taking the constraint of the loss term corresponding to the mid-term classification head as an example (the constraint consists of two parts: the constraint on the current period and the constraints on the short-term and long-term): First, the mid-term gold standard of the current sample will supervise the output of the current mid-term prediction (that is, if the gold standard is positive, it will constrain the current mid-term classification head to make a positive prediction as much as possible, and vice versa). In addition, through the additional design of the loss term ( Item 1 and item 3 Constraints are applied to both the short-term and long-term ends, specifically: a) When the interim gold standard for the sample is positive, proceed through item 1 (corresponding to...) Figure 2 The intermediate loss term points to the red dotted line of the short-term classification head, which can additionally supervise the short-term classification head to make it as likely as possible to make a short-term positive judgment (in clinical practice, if a patient's intermediate loss is positive, then their short-term loss can also be inferred to be positive). b) When the interim gold standard for the sample is negative, proceed through item 3 (corresponding to...). Figure 2The intermediate-term loss term points to the red dashed line of the long-term classification head, which can additionally supervise the long-term classification head to make a long-term negative judgment as much as possible (similarly, if a patient is negative in the intermediate stage, their long-term should also remain negative).

[0034] This invention introduces temporal modeling and consistency constraints from both the network structure and loss function mechanism aspects, which significantly improves the performance and usability of multi-period prediction models.

[0035] In training a multi-phase regression prediction model, the Adam optimizer was used with an initial learning rate of 1e-4 and a batch size of 4. The total loss for the short, medium, and long term was dynamically calculated during training. Then, backpropagation is performed until the multi-period outcome prediction model achieves the best accuracy on the validation set, thus obtaining the optimal multi-period outcome prediction model.

[0036] This invention performs ablation analysis on an improved ResNet encoder structure. The prediction performance of the improved ResNet encoder is compared with commonly used DenseNet and ViTNet encoders. In the comparison, the network architecture remains unchanged except for the encoder components. The prediction results of the three compared encoders are shown below. Figure 4 As shown, the ResNet encoder provided by this invention achieved the best prediction performance across all follow-up periods. The ViTNet encoder had the lowest performance metrics. It can be observed that ResNet can achieve more reasonable localization. Therefore, the effectiveness of ResNet for the current problem is confirmed.

[0037] This invention compares the performance of the proposed multi-stage outcome prediction model (Ours) constructed with an improved ResNet encoder structure with four state-of-the-art methods to evaluate the effectiveness of the proposed multi-stage prediction model in predicting the probability of multi-stage outcomes from acute-phase CT images. The four methods include: 1) Mikhail et al. developed a U-Net-based multitask model for pneumonia classification, in which pneumonia segmentation is used as an auxiliary task to enhance feature capture (from CT-Based COVID-19 triage: Deep multitask learning improves joint identification and severity quantification.).

[0038] 2) Simon et al. input randomly selected 2D CT slices from different slices into the Inception-ResNet-v2 model to predict the prognosis of progressive fibrotic lung disease (from Deep Learning-based Outcome Prediction in Progressive Fibrotic Lung Disease Using High-Resolution Computed Tomography).

[0039] 3) Yun et al. used CNN to extract depth features from multiple CT slices of chronic obstructive pulmonary disease to predict 3-year and 5-year survival rates (derived from Deep radiomics-based survival prediction inpatients with chronic obstructive pulmonary disease).

[0040] 4) Wang et al. developed an attention alignment model to capture high-risk lesion changes from mammograms to predict the occurrence of breast cancer within 1 to 5 years (in International Conference on Medical Image Computing and Computer-Assisted Intervention.).

[0041] To ensure fair comparison, all methods were trained on the same settings on the dataset of this invention: input was restricted to acute-phase lung CT scans, and output was reformulated to predict short-, medium-, and long-term outcomes using a classification head. The performance of the four comparison methods and the method proposed in this invention was evaluated on the test set of this invention, as detailed below. Figure 5 The proposed method achieved best predictive performance across all time periods, with an average AUC of 0.832 and an accuracy of 0.779. The methods of Simon et al. and Yun et al. significantly underperformed other methods, possibly because 2D slice input cannot provide a global representation of pneumonia lesions. Mikhail et al.'s multi-task framework achieved comparable results in short-term predictions, possibly attributed to improved lesion feature extraction capabilities through a segmentation-assisted task. Similarly, the attention alignment mechanism designed by Wang et al., designed to focus on changes in high-risk lesions, demonstrated strong performance in short-term predictions. However, the method of this invention outperformed the comparative methods in most short, medium, and long-term periods. Therefore, the effectiveness of the cascaded multi-classifier head architecture with loss-crossing fusion was further confirmed, explicitly modeling the temporal relationships between different follow-up stages.

[0042] This invention designs three classification heads corresponding to short-term (1 month), medium-term (3 months), and long-term (6 months) prediction tasks. These heads are cascaded and integrated with intermediate features from the previous period as auxiliary inputs for the next period, guiding the temporal information. Furthermore, an improved inter-period fusion loss function based on Focal Loss is introduced to constrain the continuity between prediction results from different periods. This allows the constructed multi-period outcome prediction model to output high-precision outcome probabilities for three time points using a single acute-phase CT image. The feature extraction encoder structure is tailored to the cascaded multi-classification head structure and the inter-period fusion loss function, ensuring that the multi-period prediction model formed by the cascaded multi-classification head structure and the inter-period fusion loss function achieves optimal performance.

[0043] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A multi-stage outcome prediction method based on an improved ResNet encoder, characterized in that, Includes the following steps: Acquire chest CT images of acute lung diseases and perform standardized resampling, cropping and intensity normalization on the chest CT images; An improved ResNet encoder module for extracting high-dimensional image features from chest CT images is built using an input convolutional layer, four residual blocks, an average pooling layer, and a linear transform layer. Using a pre-established multi-period outcome prediction model, short-term, medium-term, and long-term outcome probabilities are output based on the high-dimensional image features output by the improved ResNet encoder module.

2. The multi-stage outcome prediction method based on an improved ResNet encoder module according to claim 1, characterized in that: The improved ResNet encoder module takes chest CT images as input and outputs high-dimensional image features. ; The structural expression of the improved ResNet encoder module is as follows: ； In the formula, Features of high-dimensional images Chest CT image, This is an improved ResNet encoder module; The input convolutional layer includes convolutional layer 1, wherein the input of convolutional layer 1 is... The kernel size of convolutional layer 1 is 7*7*7, the stride is 1, and the number of channels is 31. The first residual block includes convolutional layer 2, convolutional layer 3, convolutional layer 4, and convolutional layer 5. The output of convolutional layer 2, the input and output of convolutional layer 3, the input and output of convolutional layer 4, and the input and output of convolutional layer 5 are sequentially connected. The input of convolutional layer 2 is connected to the output of convolutional layer 1. The input of convolutional layer 2 is connected to the input of convolutional layer 4. The input of convolutional layer 4 is connected to the output of convolutional layer 5. The kernel size of convolutional layer 2 is 3*3*3, the stride is 2, and the number of channels is 32. The kernel size of convolutional layers 3, 4, and 5 is 3*3*3, the stride is 1, and the number of channels is 32. The second residual block includes convolutional layers 6, 7, 8, 9, and 10. The output of convolutional layer 6, the input and output of convolutional layer 7, the input and output of convolutional layer 9, and the input and output of convolutional layer 10 are sequentially connected. The input of convolutional layer 6 is connected to the output of convolutional layer 5. The input of convolutional layer 6 and the input of convolutional layer 9 are connected by a skip connection through convolutional layer 8. The input of convolutional layer 9 and the output of convolutional layer 10 are connected by a skip connection. The kernel size of convolutional layer 6 is 3*3*3, the stride is 2, and the number of channels is 64. The kernel size of convolutional layers 7, 9, and 10 is 3*3*3, the stride is 1, and the number of channels is 64. The kernel size of convolutional layer 8 is 1*1*1, the stride is 2, and the number of channels is 64. The third residual block includes convolutional layers 11, 12, 13, 14, and 15. The output of convolutional layer 11, the input and output of convolutional layer 12, the input and output of convolutional layer 14, and the input and output of convolutional layer 15 are sequentially connected. The input of convolutional layer 11 is connected to the output of convolutional layer 10, and the input of convolutional layer 11 is connected to the input of convolutional layer 14 via... Convolutional layer 13 forms a skip connection, the input of convolutional layer 14 forms a skip connection with the output of convolutional layer 15, the kernel size of convolutional layer 11 is 3*3*3, the stride is 2, and the number of channels is 128, the kernel size of convolutional layers 12, 14 and 15 is 3*3*3, the stride is 1, and the number of channels is 128, and the kernel size of convolutional layer 13 is 1*1*1, the stride is 2, and the number of channels is 128; The fourth residual block includes convolutional layers 16, 17, 18, 19, and 20. The output of convolutional layer 16, the input and output of convolutional layer 17, the input and output of convolutional layer 19, and the input and output of convolutional layer 20 are sequentially connected. The input of convolutional layer 16 is connected to the output of convolutional layer 15, and the input of convolutional layer 16 is connected to the input of convolutional layer 19 via... Convolutional layer 18 forms a skip connection, the input of convolutional layer 19 forms a skip connection with the output of convolutional layer 20, the kernel size of convolutional layer 16 is 3*3*3, the stride is 2, and the number of channels is 256, the kernel size of convolutional layers 17, 19 and 20 is 3*3*3, the stride is 1, and the number of channels is 256, and the kernel size of convolutional layer 18 is 1*1*1, the stride is 2, and the number of channels is 256; The average pooling layer has a convolutional kernel size of 3*3*3 and a stride of 1. The linear transform layer has 12 channels.

3. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 2, characterized in that: The multi-period outcome prediction model includes three cascaded classification heads: a first classification head, a second classification head, and a third classification head. Each classification head consists of two linear layers and a softmax activation layer.

4. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 3, characterized in that: The first classification head is used to classify high-dimensional image features. Given the input, output the short-term outcome probability; The structural expression of the first classification head is: ； ； In the formula, The short-term regression probability is the output of the first classifier. , These are the first and second linear layers in the classification head, respectively. High-dimensional image features Intermediate classification features generated through the first linear layer of the first classification head. This is the softmax activation layer.

5. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 4, characterized in that: The second classification head is used to classify high-dimensional image features. and the intermediate classification features Compositional fusion characteristics Given the input, output the intermediate outcome probability; The structural expression for the second classification head is: ； ； In the formula, This represents the intermediate regression probability output by the second classification head. , These are the first and second linear layers in the classification head, respectively. For fusion features Intermediate classification features generated through the first linear layer of the second classification head. This is a softmax activation layer, and Concat is the feature concatenation operator.

6. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 5, characterized in that: The third classification head is used to classify high-dimensional image features. The intermediate classification features and the intermediate classification features Compositional fusion characteristics Given the input, output the long-term outcome probability; The structural expression of the third classification head is: ； In the formula, This represents the long-term regress probability output by the third classification head. , These are the first and second linear layers in the classification head, respectively. This is a softmax activation layer, and Concat is the feature concatenation operator.

7. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 6, characterized in that: The loss function for training the multi-period outcome prediction model is: ； In the formula, The total loss of the multi-period outcome prediction model. For short-term sub-fusion loss, For intermediate sub-fusion loss, For long-term sub-fusion loss; ； In the formula, i represents the corresponding number for short-term, medium-term, and long-term. , where j is the corresponding number of the classification head used for short-term, medium-term, and long-term outcome probability prediction. FocalLoss is the prediction result of the gold standard for the i-th period on the j-th class head. ； In the formula, For the gold standard of the i-th period, Let be the positive probability output by the j-th classification header. Let be the negative probability output by the j-th classification header. An adjustment factor to balance the contribution of difficult and easy samples to the loss; Among them, when This indicates that the positive result has not turned into a positive outcome. This indicates that the negative result has been confirmed. when hour, middle Used to constrain the loss of positive samples predicted by the current classification head. Used to constrain the loss of negative samples predicted by the current classification head; when hour, The gold standard used to constrain the positive probability output by the previous classification head using the current period's positive samples; when hour, The negative probability of the classification head output is used to constrain the negative sample output in the later period using the gold standard of the current period's negative samples.

8. The multi-phase outcome prediction method based on an improved ResNet encoder according to claim 7, characterized in that: The Adam optimizer is used in training a multi-phase outcome prediction model to dynamically calculate the total loss in the short, medium, and long term during training. Then backpropagation is performed until the multi-period outcome prediction model achieves the best accuracy on the validation set, thus obtaining the optimal multi-period outcome prediction model.

Images