Training method, risk prediction method and system of radioactive pneumonia risk prediction model

Abstract

The present disclosure provides a training method, a risk prediction method and a system of a radiation pneumonia risk prediction model, wherein the training method comprises: obtaining a sample training set, each group of samples in the sample training set comprising sample multi-modal data and corresponding sample radiation pneumonia result information; the sample multi-modal data comprises sample image data, sample dose distribution data, sample anatomical structure data and sample blood data; feature fusion is performed based on the sample image data, the sample dose distribution data, the sample anatomical structure data and the sample blood data to form sample feature fusion data; an initial preset model is trained based on the sample feature fusion data and the sample radiation pneumonia result information to obtain a radiation pneumonia risk prediction model; quantitative prediction of severe radiation pneumonia risk is realized, and risk probability and risk stratification results are output to provide auxiliary decision support for individualized radiotherapy.

Description

Technical Field

[0001] This disclosure relates to the field of medical data processing, and in particular to a training method, risk prediction method, and system for a radiation pneumonia risk prediction model. Background Technology

[0002] Radiation pneumonitis is a common complication of radiotherapy for thoracic malignancies, especially esophageal cancer. Severe radiation pneumonitis can significantly impact a patient's treatment process, quality of life, and even survival. Current risk assessment methods for radiation pneumonitis typically rely on dose-volume statistics in the treatment planning system, empirical thresholds, or traditional statistical models constructed based on a limited number of manually generated features. While these methods can describe the relationship between radiation-induced lung injury and radiation dose to some extent, they still have the following limitations:

[0003] First, traditional methods typically compress three-dimensional dose information into a small number of statistics, resulting in the loss of important spatial information such as dose spatial heterogeneity, dose gradient changes, and dose landing points, making it difficult to fully reflect the actual tissue irradiation situation. Second, lung tissue and adjacent organs are not homogeneous structures, and the same dose applied to different anatomical locations may cause different biological effects, while existing methods usually lack explicit modeling of anatomical spatial information. Third, patients' sensitivity to radiation damage varies significantly among individuals, and biological factors such as hematological inflammation and immune status can affect the probability of developing radiation pneumonitis, and it is difficult to characterize this difference by simply relying on dose or imaging features. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is to overcome the shortcomings of existing technologies that mainly rely on dose-volume statistical indicators or manual experience thresholds for radiation pneumonitis risk assessment, which are difficult to fully represent the differences between dose space, anatomical structure and individual biology. This disclosure provides a training method, risk prediction method and system for a radiation pneumonitis risk prediction model.

[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0006] According to a first aspect of this disclosure, a training method for a radiation pneumonia risk prediction model is provided, the training method comprising:

[0007] A sample training set is obtained, wherein each group of samples in the sample training set includes sample multimodal data and corresponding sample radiation pneumonia result information; the sample multimodal data includes: sample image data, sample dose distribution data, sample anatomical structure data and sample blood data.

[0008] Feature fusion is performed based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data to form sample feature fusion data;

[0009] The initial preset model is trained based on the sample feature fusion data and the sample radiation pneumonia result information to obtain the radiation pneumonia risk prediction model used to predict the actual radiation pneumonia result information under actual multimodal data.

[0010] Optionally, the sample image data, sample dose distribution data, and sample anatomical structure data are all three-dimensional image data;

[0011] The blood sample data is text data;

[0012] Prior to the step of feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data, the training method further includes:

[0013] The sample image data, sample dose distribution data, and sample anatomical structure data are formatted uniformly and spatially registered based on the patient's positioning coordinate system.

[0014] Optionally, the step of spatial registration based on the patient's location coordinate system includes:

[0015] Based on the sample image data, rigid registration or equivalent spatial alignment processing is performed on the sample dose distribution data and the sample anatomical structure data.

[0016] Optionally, before the step of feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data, the training method further includes:

[0017] The sample image data is subjected to grayscale phase and normalization processing;

[0018] And / or,

[0019] The sample dose distribution data were numerically normalized.

[0020] And / or,

[0021] The anatomical data of the sample is encoded as multi-channel mask data;

[0022] And / or,

[0023] The blood sample data is screened, discretized, encoded, and sequenced.

[0024] Optionally, the sample anatomical data includes at least the target area structure and the structure of the organ at risk;

[0025] The organ structures at risk include at least one or more of the following: the two lungs, the heart, the spinal cord, and the body contour.

[0026] And / or,

[0027] The sample dose distribution data were obtained using a treatment planning system.

[0028] Optionally, before the step of training the initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information, the training method further includes:

[0029] Construct the initial preset model;

[0030] The initial preset model includes: a visual encoding module, a text encoding module, a cross-modal fusion module, and a prediction module;

[0031] The visual encoding module is used to perform image encoding on the sample image data, the sample dose distribution data, and the sample anatomical structure data.

[0032] The text encoding module is used to encode the sample blood data into text.

[0033] The cross-modal fusion module is used to fuse the encoded image data, the sample dose distribution data, the sample anatomical structure data, and the sample blood data to obtain the sample feature fusion data.

[0034] The prediction module is used to output the sample radiation pneumonia result information corresponding to the sample feature fusion data.

[0035] Optionally, the visual encoding module includes multiple independent 3D ResNet network branches;

[0036] And / or,

[0037] The text encoding module includes a Transformer encoder;

[0038] And / or,

[0039] The cross-modal fusion module includes a fusion structure based on an attention mechanism;

[0040] And / or,

[0041] The prediction module uses the Focal Loss function.

[0042] According to a second aspect of this disclosure, a method for predicting the risk of radiation pneumonitis is provided, the method comprising:

[0043] Obtain the actual multimodal data corresponding to the target patient;

[0044] The actual multimodal data is input into the radiation pneumonia risk prediction model obtained by the training method of the radiation pneumonia risk prediction model described in the first aspect of this disclosure, and the actual radiation pneumonia result information corresponding to the actual multimodal data is obtained.

[0045] According to a third aspect of this disclosure, a training system for a radiation pneumonia risk prediction model is provided, the training system comprising:

[0046] The sample data acquisition module is used to acquire a sample training set. Each group of samples in the sample training set includes multimodal data of the sample and corresponding sample radiation pneumonia result information. The multimodal data of the sample includes: sample image data, sample dose distribution data, sample anatomical structure data and sample blood data.

[0047] The feature fusion module is used to perform feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data and sample blood data to form sample feature fusion data;

[0048] The model training module is used to train an initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information to obtain the radiation pneumonia risk prediction model used to predict the actual radiation pneumonia result information under actual multimodal data.

[0049] Optionally, the sample image data, sample dose distribution data, and sample anatomical structure data are all three-dimensional image data;

[0050] The blood sample data is text data;

[0051] The training system also includes a preprocessing module, which is used to perform feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data before:

[0052] The sample image data, sample dose distribution data, and sample anatomical structure data are formatted uniformly and spatially registered based on the patient's positioning coordinate system.

[0053] Optionally, the preprocessing module is further configured to:

[0054] Based on the sample image data, rigid registration or equivalent spatial alignment processing is performed on the sample dose distribution data and the sample anatomical structure data.

[0055] Optionally, the preprocessing module is further configured to:

[0056] The sample image data is subjected to grayscale phase and normalization processing;

[0057] The sample dose distribution data were numerically normalized.

[0058] The anatomical data of the sample is encoded as multi-channel mask data;

[0059] The blood sample data is screened, discretized, encoded, and sequenced.

[0060] Optionally, the sample anatomical data includes at least the target area structure and the structure of the organ at risk;

[0061] The organ structures at risk include at least one or more of the following: the two lungs, the heart, the spinal cord, and the body contour.

[0062] And / or,

[0063] The sample dose distribution data were obtained using a treatment planning system.

[0064] Optionally, the training system further includes a model building module, which is used to build the initial preset model before training the initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information;

[0065] The initial preset model includes: a visual encoding module, a text encoding module, a cross-modal fusion module, and a prediction module;

[0066] The visual encoding module is used to perform image encoding on the sample image data, the sample dose distribution data, and the sample anatomical structure data.

[0067] The text encoding module is used to encode the sample blood data into text.

[0068] The cross-modal fusion module is used to fuse the encoded image data, the sample dose distribution data, the sample anatomical structure data, and the sample blood data to obtain the sample feature fusion data.

[0069] The prediction module is used to output the sample radiation pneumonia result information corresponding to the sample feature fusion data.

[0070] Optionally, the visual encoding module includes multiple independent 3D ResNet network branches;

[0071] And / or,

[0072] The text encoding module includes a Transformer encoder;

[0073] And / or,

[0074] The cross-modal fusion module includes a fusion structure based on an attention mechanism;

[0075] And / or,

[0076] The prediction module uses the Focal Loss function.

[0077] According to a fourth aspect of this disclosure, a prediction system for the risk of radiation pneumonitis is provided, the prediction system comprising:

[0078] The actual data acquisition module is used to acquire the actual multimodal data corresponding to the target patient;

[0079] The actual result acquisition module is used to input the actual multimodal data into the radiation pneumonia risk prediction model obtained by the training system of the radiation pneumonia risk prediction model described in the third aspect of this disclosure, and to obtain the actual radiation pneumonia result information corresponding to the actual multimodal data.

[0080] According to a fifth aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement a training method for a radiation pneumonitis risk prediction model according to a first aspect of this disclosure, and / or a radiation pneumonitis risk prediction method according to a second aspect of this disclosure.

[0081] According to a sixth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a training method for a radiation pneumonia risk prediction model as described in the first aspect of this disclosure, and / or a radiation pneumonia risk prediction method as described in the second aspect of this disclosure.

[0082] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a training method for a radiation pneumonitis risk prediction model as described in a first aspect of this disclosure, and / or a radiation pneumonitis risk prediction method as described in a second aspect of this disclosure.

[0083] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0084] The positive and progressive effects of this disclosure are as follows:

[0085] The training method for the radiation pneumonitis risk prediction model provided in this disclosure uses three-dimensional dose distribution as the core driving variable, and combines CT images, anatomical structures, and hematological / inflammatory indicators to construct multimodal fusion feature data, which can more fully characterize the dose space, spatial anatomical structure, and individual differences in the occurrence of radiation pneumonitis. Furthermore, this disclosure improves the alignment accuracy and input consistency between different modalities by performing unified format conversion, spatial registration, structural encoding, resolution integration, and feature standardization processing on the multimodal data, which is conducive to improving the model training stability and generalization ability. Furthermore, this disclosure achieves quantitative prediction of radiation pneumonitis risk by jointly modeling information from different sources through visual encoding, text encoding, and cross-modal fusion, and can output risk probability and risk stratification results. It can also be applied to risk assessment and scheme optimization before radiotherapy planning, providing auxiliary decision support for individualized radiotherapy. Attached Figure Description

[0086] Figure 1 This is a flowchart illustrating the training method for the radiation pneumonia risk prediction model provided in Embodiment 1 of this disclosure;

[0087] Figure 2 This is a schematic diagram of the framework structure of the initial preset model provided in Embodiment 1 of this disclosure;

[0088] Figure 3 This is a schematic diagram of the encoder provided in Embodiment 1 of this disclosure;

[0089] Figure 4 This is a schematic diagram of the structure of the visual encoding module provided in Embodiment 1 of this disclosure;

[0090] Figure 5 This is a schematic diagram of the structure of the text encoding module provided in Embodiment 1 of this disclosure;

[0091] Figure 6 This is a schematic diagram of the cross-modal fusion module provided in Embodiment 1 of this disclosure;

[0092] Figure 7 The ROC (receiver operating characteristic curve) results for predicting severe radiation pneumonitis using different data provided in Embodiment 1 of this disclosure are shown in Figure 1.

[0093] Figure 8 This is a flowchart illustrating the radiation pneumonia risk prediction system provided in Embodiment 2 of this disclosure;

[0094] Figure 9 This is a schematic diagram of the training system for the radiation pneumonia risk prediction model provided in Embodiment 3 of this disclosure;

[0095] Figure 10This is a schematic diagram of the structure of the radiation pneumonia risk prediction system provided in Embodiment 4 of this disclosure;

[0096] Figure 11 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of this disclosure. Detailed Implementation

[0097] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0098] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0099] In this embodiment of the disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good morals.

[0100] Example 1

[0101] like Figure 1 As shown in the figure, this embodiment provides a training method for a radiation pneumonia risk prediction model. The training method includes:

[0102] S11: Obtain the sample training set. Each group of samples in the sample training set includes multimodal data and corresponding radiation pneumonitis results. The multimodal data includes: sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data.

[0103] In this embodiment, the sample image data, sample dose distribution data, and sample anatomical structure data are all three-dimensional image data, which are then registered to the same spatial coordinate system to achieve a precise voxel-to-voxel correspondence; the sample blood data is text data.

[0104] Specifically, the sample image data is CT image data;

[0105] The sample dose distribution data are three-dimensional dose distribution data generated by the treatment planning system. The sample anatomical data includes at least the structural information of the planned target volume (PTV) and organs at risk (OARs).

[0106] In this embodiment, when resampling the sample image data, sample dose distribution data, and sample anatomical structure data, trilinear interpolation is used for the sample image data and sample dose distribution data, and nearest neighbor interpolation is used for the sample anatomical structure data. The unified spatial resolution after resampling is 1×1×1mm³.

[0107] The blood sample data are baseline hematological data collected within a pre-defined time window before the radiotherapy plan is implemented.

[0108] The blood sample data consisted of hematological / inflammatory marker data, specifically including white blood cell count, hemoglobin, neutrophil count, lymphocyte count, monocyte count, platelet count, eosinophil count, basophil count, and inflammatory indices calculated based on these indicators. Inflammatory indices included at least the neutrophil-lymphocyte ratio, lymphocyte-monocyte ratio, platelet-lymphocyte ratio, and systemic immune inflammatory indices.

[0109] The blood sample data in this embodiment can be expanded to include more data related to inflammation, biochemistry, or immune indicators.

[0110] S12: Feature fusion is performed based on sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data to form sample feature fusion data;

[0111] S13: The initial preset model is trained based on the sample feature fusion data and the sample radiation pneumonia result information to obtain a radiation pneumonia risk prediction model for predicting the actual radiation pneumonia result information under actual multimodal data.

[0112] The training method of the radiation pneumonitis risk prediction model provided in this disclosure uses three-dimensional dose distribution as the core driving variable, and combines CT images, anatomical structures and hematological / inflammatory indicators to construct a multimodal feature space. This can more fully characterize the dose-effect relationship, spatial heterogeneity and individual differences in the occurrence of radiation pneumonitis, and realize the quantitative prediction of the risk of severe radiation pneumonitis, outputting risk probability and risk stratification results. It can be applied to risk assessment and scheme optimization before radiotherapy planning, and provide auxiliary decision support for individualized radiotherapy.

[0113] Before the feature fusion step based on sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data, the training method also includes:

[0114] The sample image data, sample dose distribution data, and sample anatomical structure data were standardized in format and spatially registered based on the patient's positioning coordinate system.

[0115] The steps for spatial registration based on the patient's location coordinate system include:

[0116] Based on the sample image data, rigid registration or equivalent spatial alignment processing is performed on the sample dose distribution data and sample anatomical structure data.

[0117] Specifically, based on the sample image data, the sample dose distribution data and sample anatomical structure data are first converted into their formats, and then spatial registration is performed based on the CT positioning system to ensure voxel-level correspondence. Furthermore, based on the sample image data, the sample dose distribution data and sample anatomical structure data are resampled at a uniform resolution and cropped to obtain standardized model input.

[0118] By registering sample image data, sample dose distribution data, and sample anatomical structure data, the sample image data, sample dose distribution data, and sample anatomical structure data corresponding to the patient can be registered to a standardized coordinate system, thereby solving the differences between individuals and improving the accuracy of the model.

[0119] Before the feature fusion step based on sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data in this embodiment, the training method further includes:

[0120] Perform grayscale and normalization processing on the sample image data;

[0121] Numerical normalization was performed on the sample dose distribution data;

[0122] The anatomical structure data of the samples were encoded into multi-channel mask data;

[0123] The blood sample data was screened, discretized, encoded, and sequenced.

[0124] The grayscale and normalization processing of the sample image data includes: limiting the CT value to the lung window range and linearly mapping it to a preset interval, wherein the lung window range is -1000HU to 200HU, and after normalization, it is mapped to the interval [0, 1].

[0125] The numerical normalization process for the sample dose distribution data includes: linearly normalizing the sample dose distribution data according to the clinical dose range to eliminate dose scale differences between different patients, and mapping the three-dimensional dose distribution data to the interval [0, 1] according to the dose range of 0 Gy to 70 Gy.

[0126] The sample anatomical data in this embodiment includes at least the target area structure and the structure of the organ at risk; the structure of the organ at risk includes at least one or more of the following: lungs, heart, spinal cord, and body contour.

[0127] The anatomical structure data of the samples are encoded as multi-channel mask data. Specifically, a multi-channel binary mask encoding method is used, with each anatomical structure corresponding to an independent channel.

[0128] The process of screening, discretizing, encoding, and constructing sequences for blood sample data includes: discretizing the data based on the corresponding clinical reference intervals to obtain categories of data that are below the normal range, within the normal range, and above the normal range, with the discretization encoding corresponding to the encoding values ​​0, 1, and 2, respectively.

[0129] In this embodiment, before the step of training the initial preset model based on sample feature fusion data and sample radiation pneumonia result information, the training method further includes:

[0130] Construct the initial preset model;

[0131] The initial preset model includes: a visual encoding module, a text encoding module, a cross-modal fusion module, and a prediction module;

[0132] The visual encoding module is used to perform image encoding on sample image data, sample dose distribution data, and sample anatomical structure data.

[0133] The text encoding module is used to encode the sample blood data into text.

[0134] The cross-modal fusion module is used to fuse encoded local image data, sample dose distribution data, sample anatomical structure data, and sample blood data to obtain sample feature fusion data. Each input modality uses an independent encoding branch, and encoders for different modalities can be flexibly enabled or removed based on data availability.

[0135] The prediction module is used to output the sample radiation pneumonia result information corresponding to the sample feature fusion data.

[0136] The visual encoding module in this embodiment includes multiple independent 3D ResNet network branches, which are used to extract CT image features, 3D dose distribution features, and 3D anatomical structure features, respectively. The network parameters of each branch are not shared to adapt to the differences in statistical distribution and semantic attributes of different modalities. The 3D convolutional neural network branch is a 3D ResNet-34 network.

[0137] The visual encoding module in this embodiment is not limited to the 3D ResNet network branch, but can also use other 3D convolutional networks, 3D DenseNet, 3D Swing Transformer or hybrid structures.

[0138] The text encoding module in this embodiment includes a Transformer encoder to capture potential dependencies between different hematological indicators; specifically, it is used to embed hematological / inflammatory indicator sequences and model dependencies. The Transformer encoding network includes an embedding layer, a multi-head self-attention sub-layer, and a feedforward network sub-layer.

[0139] The text encoding module in this embodiment is not limited to the Transformer encoding network, but can also use recurrent neural networks, MLP-Mixer or other sequence modeling networks.

[0140] The cross-modal fusion module in this embodiment includes a fusion structure based on an attention mechanism. Specifically, the cross-modal fusion module is used to map visual features and text features to a unified feature space and perform interactive fusion through a self-attention mechanism to obtain a fused feature representation. The cross-modal fusion module adopts a multi-layer attention fusion structure to perform layer-by-layer interactive modeling of visual tokens and text tokens. The multi-layer attention fusion structure has three layers.

[0141] The attention mechanism is a token-level attention fusion mechanism. In this embodiment, the cross-modal fusion module is not limited to the token-level attention fusion mechanism, but may also employ gating fusion, tensor fusion, dual-limitation fusion or cross-attention fusion.

[0142] In this embodiment, the prediction module employs the Focal Loss loss function to address the class imbalance problem. It also combines the AdamW optimizer, learning rate warm-up and cosine annealing strategies, data augmentation, gradient pruning, and early stopping mechanisms to improve model training stability and generalization ability. Furthermore, the prediction module integrates features into the classification head and outputs the probability of severe radiation pneumonitis or the corresponding binary classification result.

[0143] The model training process also includes using the AdamW optimizer, learning rate preheating and cosine annealing scheduling strategies, as well as an early stopping strategy to select the optimal model parameters.

[0144] like Figure 2 As shown, the architecture of the initial preset model in this embodiment is a ResNet-Transformer Fusion Network (ResTrans-Fusion) multimodal deep learning architecture used for risk prediction of RP (radiation pneumonia). This model belongs to a hybrid three-dimensional vision-language fusion network.

[0145] The ResTrans-Fusion model consists of three core modules: (1) a modality-specific 3D visual encoder, which is used to extract the spatial semantic features of CT images, 3D physical dose distribution and anatomical structures respectively; (2) a Transformer-based text encoder, which is used to perform sequence modeling of text information HSIP (hematological and systemic inflammatory parameters); and (3) a cross-modal fusion and risk prediction module, which realizes deep interaction between visual and text features through a multi-head self-attention mechanism and finally outputs the probability of SRP (severe radiation pneumonitis).

[0146] The visual information includes three types of volumetric data: CT images, dose distribution, and anatomical structures.

[0147] The data input for this architecture is in the following formats: CT image branch: input is single-channel 3D CT volume data; dose distribution branch: input is single-channel 3D dose volume data; anatomical structure branch: input is multi-channel binary mask volume data, with each structure corresponding to an independent channel.

[0148] In this architecture, the three visual modalities are encoded by independent 3D ResNet-34 networks. The branches maintain a consistent structure but do not share parameters.

[0149] The 3D ResNet-34 encoder disclosed herein is adapted from the standard 2D ResNet-34 network structure and has been systematically extended and adjusted for the spatial characteristics of medical 3D volumetric data.

[0150] Specifically, the network integrates the convolution, pooling and residual units in the two-dimensional ResNet with the corresponding three-dimensional operators (such as expanding 3×3 convolution into 3×3×3 convolution).

[0151] Overall network such as Figure 3As shown, the initial convolutional layer of the network uses a 3×3×3 convolutional kernel (stride 1, padding=1) with 64 output channels. A 3×3×3 max-pooling layer (stride 2, padding=1) is then introduced to achieve initial downsampling. The main body of the network consists of four residual stages with output channels of 64, 128, 256, and 512 respectively, corresponding to 3, 4, 6, and 3 residual blocks. Each residual block consists of two 3×3×3 convolutional layers (stride 1, padding=1), and residual learning is achieved through residual connections. When the number of input and output channels of a residual block is inconsistent or spatial downsampling occurs, the residual connection path uses a 1×1×1 convolution for channel and scale alignment; otherwise, an identity mapping is used. At the end of feature extraction, the network compresses the spatial dimension through global average pooling to obtain a 512-dimensional channel feature vector; then a linear projection layer is introduced to map it to a 256-dimensional latent feature space as a high-level semantic representation of the corresponding visual modality, which is used for subsequent cross-modal fusion and SRP risk prediction tasks.

[0152] In this embodiment, 256-dimensional features from three visual branches—CT, dose, and anatomical structure—are concatenated along the channel dimension to form a joint visual feature vector with a dimension of 768 (256×3). To reduce feature redundancy and enhance discriminative ability, this joint feature is further input into a feedforward network (FFN). This network consists of linear layers, layer normalization, and ReLU activation functions. 1×1 convolutions are used to achieve feature fusion and suppress redundant information, compressing the features to 512 dimensions.

[0153] Subsequently, the image modality alignment module remaps the 512-dimensional features to a 768-dimensional unified visual embedding space, providing a representation dimension consistent with the text features for subsequent cross-modal fusion.

[0154] like Figure 4 As shown, the 20 HSIP-related tokens obtained in the preprocessing stage are encoded using a Transformer encoder structure based on a multi-head self-attention mechanism. The embedding layer first maps each token to a 128-dimensional learnable embedding vector and transforms discrete labels into continuous semantic features. The encoder layer consists of a multi-head self-attention sublayer and an FFN sublayer, where the multi-head self-attention sublayer has 8 attention heads; the FFN sublayer consists of two fully connected layers.

[0155] After processing by the Transformer encoder, a 128-dimensional hematological feature representation is obtained. This feature is further processed by the text modality alignment module, and converted into a 768-dimensional text embedding vector through linear mapping and feature copying operations.

[0156] After completing modal alignment, the joint features (CT, dose, and anatomical structure) of the visual branch are compressed into a 768-dimensional global visual token, and the HSIP features of the text branch are compressed into a 768-dimensional global text token. The two are then stacked to form a cross-modal token sequence with a dimension of 2×768, which serves as the input to the subsequent multi-layer attention fusion module.

[0157] The cross-modal token sequence is sequentially input into the three-layer fusion module. The structure of each fusion module is as follows: Figure 5 As shown, it is composed of Figure 6 The diagram shows the multi-head self-attention sublayer and the FFN sublayer. The multi-head self-attention sublayer models the attention weights between visual and text tokens, adaptively learning the importance of different modal features for the prediction task; subsequently, the FFN performs a nonlinear transformation on the sub-positional features.

[0158] The fused cross-modal features are averaged along the token dimension to obtain a 768-dimensional global feature representation. This feature is input to the classification head, which outputs 2-dimensional logits, corresponding to the SRP and non-SRP classes respectively. After normalization by the softmax function, the occurrence probability of SRP is obtained (value range [0,1]).

[0159] This architecture uses FocalLoss as the training loss function to reduce the weight of easily classified samples and focus the learning on difficult-to-classify SRP samples. Its definition is as follows:

[0160]

[0161] in, This represents the model's predicted probability of the true class. This is the category balancing factor, used to adjust the relative weights of different categories in the loss function; This is a focusing parameter used to control the degree of weight reduction for easily classified samples.

[0162] The model uses the AdamW optimizer (initial learning rate 5×10). -5The model employs a warm-up and cosine annealing strategy to improve training stability and convergence performance. Training is accelerated using PyTorch and GPU, and regularization techniques such as weight decay, dropout, data augmentation (random flipping and pruning), and gradient pruning are introduced. During training, the validation set AUC is used as the monitoring metric, an early stopping strategy is implemented, and the best-performing model is retained. The data used for model training consists of esophageal cancer patients undergoing radical radiotherapy, requiring complete pre-treatment CT images, three-dimensional dose data, and baseline hematological data, with at least 6 months of follow-up and a clear RP outcome; patients with prior chest radiotherapy, infections or autoimmune diseases, severe underlying lung diseases, or incomplete data are excluded.

[0163] A total of 444 cases were ultimately included, divided into a training set (n = 323), a validation set (n = 67), and a test set (n = 54). RP was assessed according to the CTCAE 5.0 criteria, and RP occurring within 6 months post-radiotherapy was defined as SRP.

[0164] In this embodiment, AUC is used as the core discrimination metric. Simultaneously, the key performance indicators of the model in the classification task are calculated as follows: accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

[0165] ROC (Receiver Operating Characteristic) curve and AUC (Area Under the ROC Curve; it is a numerical value ranging from 0.5 to 1).

[0166] The overall discriminative performance of the model under different classification thresholds was evaluated by plotting ROC curves. AUC was used as the core discriminative metric.

[0167] Classification performance indicators

[0168] Accuracy

[0169]

[0170] Sensitivity / Recall

[0171]

[0172] Specificity

[0173]

[0174] PPV

[0175]

[0176] NPV

[0177]

[0178] In the above formula, TP (True Positive), TN (True Negative), FP (False Positive), and FN (False Negative) are used.

[0179] The accuracy of the radiation pneumonia risk prediction model was verified using the methods described above.

[0180] In this embodiment, four sets of comparative experiments were also designed, with the following specific configurations: (1) Basic model: CT+Dose (Exp1); (2) Incremental model 1: CT+Dose+HSIP (Exp2); (3) Incremental model 2: CT+Dose+Structure (Exp3); (4) Full-modal model: CT+Dose+Structure+HSIP (Exp4). As shown in Table 1 (performance comparison data of different multimodal models in SRP prediction), adding HSIP (model 2) to the baseline model (1) has limited effect on improving model performance, while adding three-dimensional anatomical structures (model 3) has a significant contribution to model performance, proving that anatomical structures are the core information for improving SRP prediction. The full-modal fusion model (model 4: CT+Dose+Structure+HSIP) brings the greatest comprehensive benefit, reflecting the advantages of multimodal synergy.

[0181] Figure 7 This is a ROC (Receiver Operating Characteristic curve) result plot, used to evaluate the discriminative ability of different combination models. The horizontal axis (X-axis) represents the false positive rate (FPR). The vertical axis (Y-axis) represents the sensitivity, i.e., the true positive rate (TPR).

[0182] Each curve represents the ROC curve of a corresponding combined model. The figure shows the area under the curve (AUC) and the 95% confidence interval (95% CI) for each model. The closer the curve is to the upper left corner (higher sensitivity, lower false positive rate), the better the model performance. The ROC results also show that the baseline model has limited discriminative ability, and only a slight improvement after adding HSIP; the model performance is significantly enhanced after introducing anatomical structural information. The full-modal model performed best, with the highest AUC, demonstrating the significant improvement of SRP prediction ability of multimodal fusion.

[0183] Table 1

[0184]

[0185] Example 2

[0186] like Figure 8 As shown, a method for predicting the risk of radiation pneumonitis is provided. The prediction method includes:

[0187] S71: Obtain the actual multimodal data corresponding to the target patient;

[0188] S72: Input the actual multimodal data into the radiation pneumonia risk prediction model obtained by the training method of the radiation pneumonia risk prediction model, and obtain the actual radiation pneumonia result information corresponding to the actual multimodal data.

[0189] In one specific implementation, the actual multimodal data of the target patient is first acquired. The actual multimodal data includes at least actual CT image data, actual three-dimensional dose distribution data, actual anatomical structure data, and actual hematological / inflammatory marker data.

[0190] Subsequently, preprocessing operations corresponding to the training phase are performed on the actual multimodal data, including but not limited to format unification, spatial registration, normalization, structural encoding, discretization encoding, resolution unification, and region clipping.

[0191] The preprocessed actual multimodal data is input into the severe radiation pneumonia risk prediction model trained in Implementation Method 1 to obtain the risk probability of the target patient developing severe radiation pneumonia.

[0192] The risk prediction result is output based on the stated risk probability.

[0193] In some embodiments, the risk probability can be divided into one or more risk levels among low risk, medium risk, and high risk according to a preset threshold; in some embodiments, the risk prediction results can be used for risk assessment and / or optimization of radiotherapy plans before radiotherapy planning.

[0194] The method for predicting the risk of radiation pneumonitis provided in this disclosure enables quantitative prediction of the risk of severe radiation pneumonitis and can output risk probability and risk stratification results; it can be applied to risk assessment and scheme optimization before radiotherapy planning and provides auxiliary decision support for individualized radiotherapy.

[0195] Example 3

[0196] like Figure 9 As shown, a training system for a radiation pneumonia risk prediction model is provided. The training system includes:

[0197] The sample data acquisition module 101 is used to acquire the sample training set. Each group of samples in the sample training set includes sample multimodal data and corresponding sample radiation pneumonia result information. The sample multimodal data includes: sample image data, sample dose distribution data, sample anatomical structure data and sample blood data.

[0198] The feature fusion module 102 is used to perform feature fusion based on sample image data, sample dose distribution data, sample anatomical structure data and sample blood data to form sample feature fusion data;

[0199] The model training module 103 is used to train the initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information, so as to obtain a radiation pneumonia risk prediction model for predicting the actual radiation pneumonia result information under actual multimodal data.

[0200] In this embodiment, the sample image data, sample dose distribution data, and sample anatomical structure data are all three-dimensional image data;

[0201] The sample blood data is text data;

[0202] The training system also includes a preprocessing module 104, which is used to perform feature fusion based on sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data before:

[0203] The sample image data, sample dose distribution data, and sample anatomical structure data were spatially registered based on the patient's location coordinate system.

[0204] In this embodiment, the preprocessing module 104 is also used for:

[0205] Based on the sample image data, rigid registration or equivalent spatial alignment processing is performed on the sample dose distribution data and sample anatomical structure data.

[0206] The preprocessing module in this embodiment is also used for:

[0207] Perform grayscale and normalization processing on the sample image data;

[0208] Numerical normalization was performed on the sample dose distribution data;

[0209] The anatomical structure data of the samples were encoded into multi-channel mask data;

[0210] The blood sample data was screened, discretized, encoded, and sequenced.

[0211] The sample anatomical data in this embodiment includes at least the target area structure and the structure of the organs at risk;

[0212] The organs at risk include at least one or more of the following: the lungs, heart, spinal cord, and body contour.

[0213] The training system in this embodiment also includes a model building module 105, which is used to build an initial preset model before training the initial preset model based on sample feature fusion data and sample radioactive pneumonia result information.

[0214] The initial preset model includes: a visual encoding module, a text encoding module, a cross-modal fusion module, and a prediction module;

[0215] The visual encoding module is used to perform image encoding on sample image data, sample dose distribution data, and sample anatomical structure data.

[0216] The text encoding module is used to encode the sample blood data into text.

[0217] The cross-modal fusion module is used to fuse the encoded local image data, sample dose distribution data, sample anatomical structure data, and sample blood data to obtain sample feature fusion data;

[0218] The prediction module is used to output the sample radiation pneumonia result information corresponding to the sample feature fusion data.

[0219] The visual encoding module in this embodiment includes multiple independent 3D ResNet network branches;

[0220] And / or,

[0221] The text encoding module includes a Transformer encoder;

[0222] And / or,

[0223] The cross-modal fusion module includes an attention-based fusion structure;

[0224] And / or,

[0225] The prediction module uses the Focal Loss function.

[0226] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0227] The training system of the radiation pneumonitis risk prediction model provided in this disclosure uses three-dimensional dose distribution as the core driving variable, and combines CT images, anatomical structures, and hematological / inflammatory indicators to construct a multimodal feature space, which can more fully characterize the dose-effect relationship, spatial heterogeneity, and individual differences in the occurrence of radiation pneumonitis. Furthermore, this disclosure improves the alignment accuracy and input consistency between different modalities by performing unified format conversion, spatial registration, structural encoding, resolution integration, and feature standardization on multimodal data, which is conducive to improving the model training stability and generalization ability. Furthermore, this disclosure achieves quantitative prediction of the risk of severe radiation pneumonitis by jointly modeling information from different sources through visual encoding, text encoding, and cross-modal fusion, and can output risk probability and risk stratification results. This can be applied to risk assessment and scheme optimization before radiotherapy planning, providing auxiliary decision support for individualized radiotherapy.

[0228] Example 4

[0229] like Figure 10 As shown, a prediction system for the risk of radiation pneumonitis is provided. The prediction system includes:

[0230] The actual data acquisition module 201 is used to acquire the actual multimodal data corresponding to the target patient;

[0231] The actual result acquisition module 202 is used to input the actual multimodal data into the radiation pneumonia risk prediction model obtained by the training system of the radiation pneumonia risk prediction model in the third aspect of this disclosure, and obtain the actual radiation pneumonia result information corresponding to the actual multimodal data.

[0232] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0233] The radiation pneumonitis risk prediction system provided in this disclosure enables quantitative prediction of the risk of severe radiation pneumonitis and can output risk probability and risk stratification results; it can be applied to risk assessment and scheme optimization before radiotherapy planning, and provide auxiliary decision support for individualized radiotherapy.

[0234] Example 5

[0235] Figure 11This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the training method of the radiation pneumonia risk prediction model provided in any of the above embodiments 1, and / or the radiation pneumonia risk prediction method provided in embodiment 2. Figure 11 The electronic device 90 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0236] like Figure 11 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0237] Bus 93 includes a data bus, an address bus, and a control bus.

[0238] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0239] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, such program module 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0240] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the training method for the radiation pneumonia risk prediction model provided in any of the above embodiments 1, and / or the radiation pneumonia risk prediction method provided in embodiment 2.

[0241] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0242] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0243] Example 6

[0244] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the radiation pneumonia risk prediction model provided in any of the above embodiments 1, and / or the radiation pneumonia risk prediction method provided in embodiment 2.

[0245] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0246] Example 7

[0247] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the training method for the radiation pneumonia risk prediction model provided in any of the above embodiments 1, and / or the radiation pneumonia risk prediction method provided in embodiment 2.

[0248] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0249] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A training method for a radiation pneumonia risk prediction model, characterized in that, The training method includes: A sample training set is obtained, wherein each group of samples in the sample training set includes sample multimodal data and corresponding sample radiation pneumonia result information; the sample multimodal data includes: sample image data, sample dose distribution data, sample anatomical structure data and sample blood data. Feature fusion is performed based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data to form sample feature fusion data; The initial preset model is trained based on the sample feature fusion data and the sample radiation pneumonia result information to obtain the radiation pneumonia risk prediction model used to predict the actual radiation pneumonia result information under actual multimodal data.

2. The training method for the radiation pneumonia risk prediction model according to claim 1, characterized in that, The sample image data, sample dose distribution data, and sample anatomical structure data are all three-dimensional image data. Prior to the step of feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data, the training method further includes: The sample image data, sample dose distribution data, and sample anatomical structure data are formatted uniformly and spatially registered based on the patient's positioning coordinate system.

3. The training method for the radiation pneumonia risk prediction model according to claim 2, characterized in that, The steps for spatial registration based on the patient's location coordinate system include: Based on the sample image data, rigid registration or equivalent spatial alignment processing is performed on the sample dose distribution data and the sample anatomical structure data.

4. The training method for the radiation pneumonia risk prediction model according to claim 2, characterized in that, Prior to the step of feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data, and sample blood data, the training method further includes: The sample image data is subjected to grayscale phase and normalization processing; And / or, The sample dose distribution data were numerically normalized. And / or, The anatomical data of the sample is encoded as multi-channel mask data; And / or, The blood sample data is text data; The training method also includes: The blood sample data is screened, discretized, encoded, and sequenced.

5. The training method for the radiation pneumonia risk prediction model according to claim 4, characterized in that, The anatomical data of the sample includes at least the target area structure and the structure of the organs at risk; The organ structures at risk include at least one or more of the following: the two lungs, the heart, the spinal cord, and the body contour. And / or, The sample dose distribution data were obtained using a treatment planning system.

6. The training method for the radiation pneumonia risk prediction model according to claim 1, characterized in that, Before the step of training the initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information, the training method further includes: Construct the initial preset model; The initial preset model includes: a visual encoding module, a text encoding module, a cross-modal fusion module, and a prediction module; The visual encoding module is used to perform image encoding on the sample image data, the sample dose distribution data, and the sample anatomical structure data. The text encoding module is used to encode the sample blood data into text. The cross-modal fusion module is used to fuse the encoded image data, the sample dose distribution data, the sample anatomical structure data, and the sample blood data to obtain the sample feature fusion data. The prediction module is used to output the sample radiation pneumonia result information corresponding to the sample feature fusion data.

7. The training method for the radiation pneumonia risk prediction model according to claim 6, characterized in that, The visual encoding module includes multiple independent 3D ResNet network branches; And / or, The text encoding module includes a Transformer encoder; And / or, The cross-modal fusion module includes a fusion structure based on an attention mechanism; And / or, The prediction module uses the Focal Loss function.

8. A method for predicting the risk of radiation-induced pneumonia, characterized in that, The prediction method includes: Obtain the actual multimodal data corresponding to the target patient; The actual multimodal data is input into the radiation pneumonia risk prediction model obtained by the training method of the radiation pneumonia risk prediction model according to any one of claims 1-7, and the actual radiation pneumonia result information corresponding to the actual multimodal data is obtained.

9. A training system for a radiation pneumonia risk prediction model, characterized in that, The training system includes: The sample data acquisition module is used to acquire a sample training set. Each group of samples in the sample training set includes multimodal data of the sample and corresponding sample radiation pneumonia result information. The multimodal data of the sample includes: sample image data, sample dose distribution data, sample anatomical structure data and sample blood data. The feature fusion module is used to perform feature fusion based on the sample image data, sample dose distribution data, sample anatomical structure data and sample blood data to form sample feature fusion data; The model training module is used to train an initial preset model based on the sample feature fusion data and the sample radiation pneumonia result information to obtain the radiation pneumonia risk prediction model used to predict the actual radiation pneumonia result information under actual multimodal data.

10. A system for predicting the risk of radiation-induced pneumonia, characterized in that, The prediction system includes: The actual data acquisition module is used to acquire the actual multimodal data corresponding to the target patient; The actual result acquisition module is used to input the actual multimodal data into the radiation pneumonia risk prediction model obtained by the training system of the radiation pneumonia risk prediction model described in claim 9, and obtain the actual radiation pneumonia result information corresponding to the actual multimodal data.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the radiation pneumonia risk prediction model according to any one of claims 1 to 7, and / or the radiation pneumonia risk prediction method according to claim 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the radiation pneumonia risk prediction model according to any one of claims 1 to 7, and / or the radiation pneumonia risk prediction method according to claim 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the radiation pneumonia risk prediction model as described in any one of claims 1 to 7, and / or the radiation pneumonia risk prediction method as described in claim 8.

Images