A method and system for predicting the risk of radiation pneumonitis

Abstract

The application provides a radiation pneumonia risk prediction method and system, which comprises the following steps: acquiring a CT image and a radiotherapy dose image, and performing rigid and non-rigid registration; adopting a piecewise linear normalization method to pretreat the registered radiotherapy dose image; adopting a CT feature extraction network to perform first feature extraction on the CT image, and simultaneously adopting an improved 3D convolution network to perform second feature extraction on the pretreated radiotherapy dose image; according to a multi-head cross attention mechanism and a position adaptive dynamic weighted fusion strategy, the first feature and the second feature are interactively modeled to obtain an interactive feature; the interactive feature is respectively delivered to a main task branch for predicting an RP risk score and an auxiliary task branch for generating a sensitivity heat map, and the RP risk score is output through the main task branch, so that more accurate and personalized RP risk assessment is realized.

Description

Technical Field

[0001] This invention belongs to the field of radiation pneumonia risk prediction technology, and specifically relates to a method and system for radiation pneumonia risk prediction. Background Technology

[0002] Radiation pneumonitis (RP) is a common and serious pulmonary complication in patients with thoracic tumors who have undergone radiotherapy. Its clinical manifestations mainly include dry cough, fever, and dyspnea; in severe cases, it can lead to respiratory failure and even endanger the patient's life.

[0003] In recent years, with the rapid development of deep learning technology, using artificial intelligence to build RP risk prediction models has become a research hotspot. In particular, by combining dose distribution maps and patient CT images (Computed Tomography, CT), multimodal feature extraction and fusion can be performed to achieve accurate assessment of RP risk.

[0004] Existing methods typically employ a strategy of "separate modeling followed by simple fusion," that is, encoding dose maps and CT images independently before performing feature exchange for prediction. This approach suffers from the following two prominent problems:

[0005] (1) Insufficient spatial and semantic correlation: Existing methods have failed to effectively explore the spatial and semantic correlation between the two modalities (CT images and dose images), resulting in the model being unable to fully capture the interaction information between lung anatomy and local dose distribution, thus affecting the prediction accuracy of RP risk.

[0006] (2) Ignoring regional dose response heterogeneity: Although studies have shown that different anatomical regions of the lungs exhibit significant differences in their biological responses to the same dose of radiation, i.e., significant regional dose response heterogeneity, existing methods have not fully considered this important factor. Therefore, ignoring the impact of regional dose on the risk of radiation pneumonitis further limits the predictive ability of the model. Summary of the Invention

[0007] Based on this, the present invention provides a method and system for predicting the risk of radiation pneumonitis, aiming to achieve a more accurate and personalized RP risk assessment.

[0008] A first aspect of this invention provides a method for predicting the risk of radiation pneumonitis, the method comprising:

[0009] Acquire CT images and radiotherapy dose images, and perform rigid and non-rigid registration to ensure spatial correspondence between the two types of images;

[0010] Piecewise linear normalization was used to preprocess the registered radiotherapy dose images;

[0011] A CT feature extraction network is used to extract the first feature from the CT image. The CT feature extraction network adopts an improved HRNetV2p structure. At the same time, an improved 3D convolutional network is used to extract the second feature from the preprocessed radiotherapy dose image.

[0012] Based on the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy, the first feature and the second feature are interactively modeled to obtain interactive features;

[0013] The interaction features are fed to the main task branch for predicting the RP risk score and the auxiliary task branch for generating the sensitivity heatmap, respectively. The RP risk score is output through the main task branch. In the auxiliary task branch, a sensitivity response map is extracted based on the interaction features using a 1×1×1 convolution. A pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on CT images. The lung ventilation map is used as a pseudo-label for the sensitivity response map.

[0014] Furthermore, in the step of preprocessing the registered radiotherapy dose image using a piecewise linear normalization method, the expression for the piecewise linear normalization method is:

[0015]

[0016] in, The original dose at coordinates (x, y, z) is... D is a unit of dosage. max For the maximum dose, This is the preprocessed image of the radiotherapy dose at coordinates (x, y, z).

[0017] Furthermore, in the step of using a CT feature extraction network to perform the first feature extraction on the CT image, the initial input CT image is denoted as... The initial input CT image, after passing through the CT feature extraction network, enters four parallel scale branches, with the resolutions of the four parallel scale branches being sequentially... ;

[0018] Record No. In the first stage The feature maps of each scale branch are Within each scale branch, features are updated via stacked 3D convolutional units, as shown below:

[0019] ;

[0020] in, This represents the l-th stage of the CT feature extraction network. Indicates the first One scale branch, These are the intermediate features output by the 3D convolutional unit in the l-th stage;

[0021] The fusion process, in which branches at different scales exchange information in a two-way manner, is represented as follows:

[0022] ;

[0023] in, Indicates the first The features of the first scale branch are transformed to the same level as the first scale branch through upsampling or downsampling. Operations with the same resolution across all scale branches;

[0024] In addition, a non-local attention module is added at the end of each scale branch to capture the connections between distant anatomical regions through global attention, for any location within the scale branch. The response of the non-local attention module is:

[0025] ;

[0026] in, and These represent the input feature maps at different positions. and location The eigenvector at that location; To measure position and Similarity function of relevance To position The feature mapping function that maps features to a high-dimensional embedding space. This is the normalization factor after summing over all positions; For non-local attention modules at location The updated feature representation is output at that point.

[0027] Furthermore, in the step of extracting the second feature from the preprocessed radiotherapy dose image using an improved 3D convolutional network, the initial input preprocessed radiotherapy dose image is denoted as... The first convolutional layer uses a large kernel. With a step size of 2, the expression is:

[0028] ;

[0029] Subsequently, both convolutional layers used... Convolution, and introduce dynamic ReLU activation:

[0030] ;

[0031] in For input, For learnable gating coefficients;

[0032] After feature extraction, the data is fed into a multi-scale pooling module for further processing. Four receptive fields are downsampled, and the pooling results are concatenated along the channel dimension and then restored to the original resolution using trilinear interpolation. Deformable convolution is introduced into the convolutional layer after multi-scale pooling, expressed as:

[0033] ;

[0034] in, For output position, For standard convolution sampling positions, For learnable offsets, These are the convolution weights.

[0035] Furthermore, the step of performing interaction modeling on the first feature and the second feature based on the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy to obtain the interaction feature includes:

[0036] The first feature and the second feature are mapped to a unified embedding space, through shared... The convolution yields the query vector Q, key vector K, and value vector V used for cross-attention, denoted as:

[0037] ;

[0038] The channel dimension of the query vector, key vector, and value vector is divided into: There are n subchannel groups, each subchannel group corresponding to an attention head. Let the nth subchannel group be... The subspace features corresponding to each attention head are represented as follows:

[0039] ;

[0040] Perform scaled dot product attention independently in each subspace to obtain the first... Output of each attention head:

[0041] ;

[0042] in, Key vector The dimension;

[0043] Concatenate the outputs of all attention heads along the channel dimension, and then map the results using the output mapping matrix. Linear transformation yields the final interaction features:

[0044] ;

[0045] in, For the final interactive features; For the number of attention heads; The dimension of the key vector in each attention head; Input the number of channels. This represents the number of output channels after mapping. The first Each attention head corresponds to a query, key, and value.

[0046] Furthermore, in the main task branch, the interaction features are first compressed into a one-dimensional vector through 3D global average pooling, then connected to a Dropout layer to prevent overfitting, and mapped to a single output node through two fully connected linear layers. The output value is then activated by a Sigmoid activation function to obtain the final RP risk score. The main task branch primarily optimizes for binary classification tasks, using focus loss or binary cross-entropy as the loss function.

[0047] A second aspect of this invention provides a radiation pneumonitis risk prediction system for implementing the radiation pneumonitis risk prediction method provided in the first aspect, the system comprising:

[0048] The registration module is used to acquire CT images and radiotherapy dose images, and to perform rigid and non-rigid registration to ensure spatial correspondence between the two images;

[0049] The preprocessing module is used to preprocess the registered radiotherapy dose image using a piecewise linear normalization method;

[0050] The feature extraction module is used to perform first feature extraction on CT images using a CT feature extraction network, wherein the CT feature extraction network adopts an improved HRNetV2p structure, and simultaneously uses an improved 3D convolutional network to perform second feature extraction on the preprocessed radiotherapy dose image;

[0051] The fusion module is used to perform interactive modeling on the first feature and the second feature according to the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy to obtain interactive features;

[0052] The delivery module is used to deliver the interaction features to the main task branch for predicting the RP risk score and the auxiliary task branch for generating the sensitivity heatmap, respectively. The RP risk score is output through the main task branch. In the auxiliary task branch, a sensitivity response map is extracted based on the interaction features using a 1×1×1 convolution. A pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on CT images. The lung ventilation map is used as a pseudo-label for the sensitivity response map.

[0053] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the radiation pneumonia risk prediction method provided in the first aspect.

[0054] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the radiation pneumonia risk prediction method provided in the first aspect.

[0055] The radiation pneumonia risk prediction method and system provided in this invention have the following beneficial effects:

[0056] 1. Improve prediction accuracy: Through dynamic weighted fusion mechanism and regional dose sensitivity modeling, the interaction between CT images and radiotherapy dose images can be fully explored, and the dose sensitivity of different lung regions can be finely modeled, which makes the model show a significant improvement in RP risk prediction.

[0057] 2. Personalized treatment assessment: By taking into account regional dose response heterogeneity, it can provide personalized RP risk assessment for different patients, optimize treatment plans, and accurately model the differences in dose response in different lung regions, so that the treatment plan can be adjusted according to each patient's lung characteristics and dose distribution.

[0058] 3. Enhanced model interpretability: The auxiliary task branch provides an intuitive visualization tool by generating sensitivity heatmaps, enabling users to better understand the model's decision-making basis. By overlaying the sensitivity heatmaps onto CT images or radiotherapy dose images, users can clearly identify high-risk areas, thereby providing a basis for the development of personalized radiotherapy plans and enhancing the clinical usability and interpretability of the model. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the implementation of a radiation pneumonia risk prediction method provided in Embodiment 1 of the present invention.

[0060] Figure 2 A flowchart illustrating the main task branch and auxiliary task branches;

[0061] Figure 3 A structural block diagram of a radiation pneumonia risk prediction system provided in Embodiment 2 of the present invention;

[0062] Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0063] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0064] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0066] Example 1

[0067] Please see Figure 1 and Figure 2 , Figure 1 The following is a flowchart illustrating the implementation of a radiation pneumonia risk prediction method provided in Embodiment 1 of the present invention. Figure 2 The flowchart of the main task branch and auxiliary task branch is shown. The radiation pneumonia risk prediction method specifically includes steps S01 to S05.

[0068] Step S01: Acquire CT images and radiotherapy dose images, and perform rigid and non-rigid registration to ensure spatial correspondence between the two images.

[0069] Specifically, using DICOM format data, open-source tools (such as Elastix or MIM software 8) are used to achieve accurate registration, ensuring the consistency of the two images in spatial coordinates. The registered images should meet the requirement that the spatial registration accuracy error does not exceed 1 mm, so as to ensure the accuracy and reliability of subsequent feature extraction.

[0070] After registration, the image is cropped into a ROI (Region of Interest), typically a volumetric data of 256×256×128, to reduce the data dimensionality while retaining key lung information.

[0071] Step S02: The registered radiotherapy dose image is preprocessed using a piecewise linear normalization method.

[0072] It should be noted that the expression for the piecewise linear normalization method is:

[0073]

[0074] in, The original dose at coordinates (x, y, z) is... D is a unit of dosage. max For the maximum dose, This is the preprocessed radiation dose image at coordinates (x, y, z). Preprocessing maps different dose levels to a uniform scale while maintaining high dynamic resolution in key dose ranges, thus better reflecting the relationship between dose and lung tissue response.

[0075] Step S03: The CT feature extraction network is used to extract the first feature from the CT image. The CT feature extraction network adopts an improved HRNetV2p structure. At the same time, an improved 3D convolutional network is used to extract the second feature from the preprocessed radiotherapy dose image.

[0076] It should be noted that the CT feature extraction network adopts an improved HRNetV2p structure. This network extracts image features at different resolutions in parallel through multi-scale branches, maintaining high-resolution feature representation while gradually fusing low-resolution contextual semantics. The initial input CT image is denoted as... The initial input CT image, after passing through the CT feature extraction network, enters four parallel scale branches, with the resolutions of the four parallel scale branches being sequentially... ;

[0077] Record No. In the first stage The feature maps of each scale branch are Within each scale branch, features are updated via stacked 3D convolutional units, as shown below:

[0078]

[0079] in, This represents the l-th stage of the CT feature extraction network. Indicates the first One scale branch, These are the intermediate features output by the 3D convolutional unit in the l-th stage;

[0080] The fusion process, in which branches at different scales exchange information in a two-way manner, is represented as follows:

[0081]

[0082] in, Indicates the first The features of the first scale branch are transformed to the same level as the first scale branch through upsampling or downsampling. Operations with the same resolution across all scale branches;

[0083] In addition, a non-local attention module is added at the end of each scale branch to capture the connections between distant anatomical regions through global attention, for any location within the scale branch. The response of the non-local attention module is:

[0084]

[0085] in, and These represent the input feature maps at different positions. and location The eigenvector at that location; To measure position and Similarity function of relevance To position The feature mapping function that maps features to a high-dimensional embedding space. This is the normalization factor after summing over all positions; For non-local attention modules at location The updated feature representation is output at that point.

[0086] The radiotherapy dose image branch employs a customized 3D convolutional network structure. The initial input, preprocessed radiotherapy dose image is denoted as... The first convolutional layer uses a large kernel. With a step size of 2, the expression is:

[0087] ;

[0088] Subsequently, both convolutional layers used... Convolution, and introduce dynamic ReLU activation:

[0089] ;

[0090] in For input, For learnable gating coefficients;

[0091] After feature extraction, the data is fed into a multi-scale pooling module for further processing. Four receptive fields are downsampled, and the pooling results are concatenated along the channel dimension and then restored to the original resolution using trilinear interpolation. Deformable convolution is introduced into the convolutional layer after multi-scale pooling, expressed as:

[0092] ;

[0093] in, For output position, For standard convolution sampling positions, For learnable offsets, These are the convolution weights.

[0094] Step S04: Based on the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy, the first feature and the second feature are interactively modeled to obtain interactive features.

[0095] To effectively capture the complementary information and spatial interaction between CT images and radiotherapy dose images, this invention designs a fusion strategy based on a multi-head cross-attention mechanism and a position-adaptive dynamic weighted fusion module to improve the model's sensitivity to changes in local structure and dose field.

[0096] Specifically, the first feature and the second feature are mapped to a unified embedding space, through shared... The convolution yields the query vector Q, key vector K, and value vector V used for cross-attention, denoted as:

[0097] ;

[0098] The channel dimension of the query vector, key vector, and value vector is divided into: There are n subchannel groups, each subchannel group corresponding to an attention head. Let the nth subchannel group be... The subspace features corresponding to each attention head are represented as follows:

[0099] ;

[0100] Perform scaled dot product attention independently in each subspace to obtain the first... Output of each attention head:

[0101] ;

[0102] in, Key vector The dimension;

[0103] Concatenate the outputs of all attention heads along the channel dimension, and then map the results using the output mapping matrix. Linear transformation yields the final interaction features:

[0104] ;

[0105] in, For the final interactive features; For the number of attention heads; The dimension of the key vector in each attention head; Input the number of channels. This represents the number of output channels after mapping. The first Each attention head corresponds to a query, key, and value. Interaction feature F CA It can express the complex nonlinear interaction between CT structure and local dose;

[0106] A dynamic weighting mechanism is introduced to adaptively adjust the spatial distribution of the modality fusion process. Specifically, a lightweight network consisting of 1×1×1 convolutions and a sigmoid function generates weight maps for the first feature, the second feature, and the interaction feature, respectively, while satisfying a normalization constraint. The interaction feature is obtained by weighted summation. The expression for the normalization constraint is:

[0107] ;

[0108] The expression for the interaction feature is:

[0109] ;

[0110] This fusion mechanism allows the network to dynamically adjust the proportion of modal information based on the importance of each spatial location, thereby enhancing the model's ability to focus on key lung regions and improving the discriminative power of predictions.

[0111] Step S05: The interaction features are fed to the main task branch for predicting the RP risk score and the auxiliary task branch for generating the sensitivity heatmap, respectively. The RP risk score is output through the main task branch. In the auxiliary task branch, the sensitivity response map is extracted using a 1×1×1 convolution based on the interaction features. A pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on the CT image. The lung ventilation map is used as a pseudo-label of the sensitivity response map.

[0112] Understandably, the main task branch and the auxiliary task branch adopt a parallel architecture, starting from the fused shared feature map (i.e., interactive features) and extracting the target features required by their respective tasks.

[0113] In the main task branch, the interactive features are first processed through 3D Global Average Pooling (GAP) to obtain a global semantic representation, compressing the spatial dimension. A Dropout layer is then connected to prevent overfitting. After being mapped to a single output node via a fully connected layer, the output value is activated by a Sigmoid activation function to obtain the final radiation pneumonitis risk probability score. This score is based on supervised learning of follow-up annotations after radiotherapy, where the follow-up results indicate whether the patient developed Grade ≥ 2 radiation pneumonitis (RP) after the treatment course. Since this pathway is a binary classification task, the loss function can be either Binary Cross Entropy or Focal Loss, and the model is trained in conjunction with the annotated RP occurrence information.

[0114] In the auxiliary task branch, to maintain the spatial resolution of the fused features and extract local fine-grained response features, a pre-trained 3D-UNet structure is used to decode the fused features. The 3D-UNet contains a symmetrical encoder-decoder structure. The encoder gradually extracts deeper spatial semantic information through 3D convolutional layers and downsampling operations; the decoder recovers the spatial resolution step by step through upsampling and uses a skip connection structure to fuse the shallow local texture features of the encoder with the deep semantic features, thereby improving the preservation of local structural boundaries and regional details. In this invention, the input of the 3D-UNet is the fused feature map, and the output after decoding is a probability map of lung function regions with voxel-level resolution consistent with CT images, used to predict dose-sensitive local lung function change regions during radiotherapy. This branch's task belongs to voxel-level regression or binary classification tasks, and the mean squared error (MSE) loss or IoU-based loss can be selected for supervised learning depending on the specific annotation format.

[0115] Through this parallel structure with task decoupling, the model can improve the interpretability and visualization of auxiliary tasks while maintaining the discriminative ability of the main task, avoiding information competition and gradient interference, thereby achieving more robust joint optimization.

[0116] In summary, the radiation pneumonitis risk prediction method in the above embodiments of the present invention acquires CT images and radiotherapy dose images and performs rigid and non-rigid registration; preprocesses the registered radiotherapy dose images using a piecewise linear normalization method; extracts first features from the CT images using an improved HRNetV2p backbone network, and simultaneously extracts second features from the preprocessed radiotherapy dose images using an improved 3D convolutional network; interactively models the first and second features based on a multi-head cross-attention mechanism and a position-adaptive dynamic weighted fusion strategy to obtain interactive features; and feeds the interactive features to the main task branch for predicting RP risk scores and the auxiliary task branch for generating sensitivity heatmaps, respectively, and outputs the RP risk score through the main task branch, thereby achieving a more accurate and personalized RP risk assessment.

[0117] Example 2

[0118] Please see Figure 3 , Figure 3 This is a structural block diagram of a radiation pneumonia risk prediction system provided in Embodiment 2 of the present invention. The radiation pneumonia risk prediction system 200 includes: a registration module 21, a preprocessing module 22, a feature extraction module 23, a fusion module 24, and a delivery module 25, wherein:

[0119] The registration module 21 is used to acquire CT images and radiotherapy dose images, and perform rigid and non-rigid registration to ensure spatial correspondence between the two images;

[0120] Preprocessing module 22 is used to preprocess the registered radiotherapy dose image using a piecewise linear normalization method. The expression for the piecewise linear normalization method is as follows:

[0121]

[0122] in, The original dose at coordinates (x, y, z) is... D is a unit of dosage. max For the maximum dose, This is the preprocessed image of the radiotherapy dose at coordinates (x, y, z);

[0123] Feature extraction module 23 is used to perform first feature extraction on CT images using a CT feature extraction network, wherein the CT feature extraction network adopts an improved HRNetV2p structure, and simultaneously uses an improved 3D convolutional network to perform second feature extraction on the preprocessed radiotherapy dose image. The initial input CT image is denoted as... The initial input CT image, after passing through the CT feature extraction network, enters four parallel scale branches, with the resolutions of the four parallel scale branches being sequentially... ;

[0124] Record No. In the first stage The feature maps of each scale branch are Within each scale branch, features are updated via stacked 3D convolutional units, as shown below:

[0125]

[0126] in, This represents the l-th stage of the CT feature extraction network. Indicates the first One scale branch, These are the intermediate features output by the 3D convolutional unit in the l-th stage;

[0127] The fusion process, in which branches at different scales exchange information in a two-way manner, is represented as follows:

[0128]

[0129] in, Indicates the first The features of the first scale branch are transformed to the same level as the first scale branch through upsampling or downsampling. Operations with the same resolution across all scale branches;

[0130] In addition, a non-local attention module is added at the end of each scale branch to capture the connections between distant anatomical regions through global attention, for any location within the scale branch. The response of the non-local attention module is:

[0131]

[0132] in, and These represent the input feature maps at different positions. and location The eigenvector at that location; To measure position and Similarity function of relevance To position The feature mapping function that maps features to a high-dimensional embedding space. This is the normalization factor after summing over all positions; For non-local attention modules at location The updated feature representation output at the location;

[0133] Additionally, the preprocessed radiotherapy dose image of the initial input is denoted as... The first convolutional layer uses a large kernel. With a step size of 2, the expression is:

[0134] ;

[0135] Subsequently, both convolutional layers used... Convolution, and introduce dynamic ReLU activation:

[0136] ;

[0137] in For input, For learnable gating coefficients;

[0138] After feature extraction, the data is fed into a multi-scale pooling module for further processing. Four receptive fields are downsampled, and the pooling results are concatenated along the channel dimension and then restored to the original resolution using trilinear interpolation. Deformable convolution is introduced into the convolutional layer after multi-scale pooling, expressed as:

[0139] ;

[0140] in, For output position, For standard convolution sampling positions, For learnable offsets, These are the convolution weights;

[0141] The fusion module 24 is used to perform interactive modeling on the first feature and the second feature according to the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy to obtain interactive features;

[0142] The delivery module 25 is used to deliver the interaction features to the main task branch for predicting the RP risk score and the auxiliary task branch for generating the sensitivity heatmap, respectively. The main task branch outputs the RP risk score. In the auxiliary task branch, a sensitivity response map is extracted based on the interaction features using a 1×1×1 convolution, and a pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on CT images. The lung ventilation map is used as a pseudo-label for the sensitivity response map. In the main task branch, the interaction features are first compressed into a one-dimensional vector through a 3D global average pooling operation, then connected to a Dropout layer to prevent overfitting, and mapped to a single output node through two linear fully connected layers. The output value is then processed by a Sigmoid activation function to obtain the final RP risk score. The main task branch primarily optimizes for a binary classification task, using focus loss or binary cross-entropy as the loss function.

[0143] Furthermore, in some other embodiments of the present invention, the fusion module 24 includes:

[0144] The mapping unit is used to map the first feature and the second feature to a unified embedding space, through a shared... The convolution yields the query vector Q, key vector K, and value vector V used for cross-attention, denoted as:

[0145] ;

[0146] The partitioning unit is used to divide the query vector, key vector, and value vector into channel dimensions. There are n subchannel groups, each subchannel group corresponding to an attention head. Let the nth subchannel group be... The subspace features corresponding to each attention head are represented as follows:

[0147] ;

[0148] The execution unit is used to independently perform scaled dot product attention in each subspace to obtain the first... Output of each attention head:

[0149] ;

[0150] in, Key vector The dimension;

[0151] The splicing unit is used to splice the outputs of all attention heads along the channel dimension and then map them through the output mapping matrix. Linear transformation yields the final interaction features:

[0152] ;

[0153] in, For the final interactive features; For the number of attention heads; The dimension of the key vector in each attention head; Input the number of channels. This represents the number of output channels after mapping. The first Each attention head corresponds to a query, key, and value.

[0154] Example 3

[0155] In another aspect, the present invention also proposes an electronic device, please refer to [link to relevant documentation]. Figure 4 The electronic device shown is an embodiment of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, it implements the radiation pneumonia risk prediction method as described above.

[0156] In some embodiments, the processor 10 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 20 or process data, such as executing access restriction programs.

[0157] The memory 20 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of an electronic device, such as the hard disk of the electronic device. In other embodiments, the memory 20 can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 20 can include both internal and external storage units of the electronic device. The memory 20 can be used not only to store application software and various types of data of the electronic device, but also to temporarily store data that has been output or will be output.

[0158] It should be pointed out that, Figure 4 The structure shown does not constitute a limitation on the electronic device. In other embodiments, the electronic device may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0159] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the radiation pneumonia risk prediction method described above.

[0160] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0161] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0162] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0163] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0164] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method of predicting the risk of radiation pneumonitis, characterized by, The method includes: Acquire CT images and radiotherapy dose images, and perform rigid and non-rigid registration to ensure spatial correspondence between the two types of images; Piecewise linear normalization was used to preprocess the registered radiotherapy dose images; A CT feature extraction network is used to extract the first feature from the CT image. The CT feature extraction network adopts an improved HRNetV2p structure. At the same time, an improved 3D convolutional network is used to extract the second feature from the preprocessed radiotherapy dose image. Based on the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy, the first feature and the second feature are interactively modeled to obtain interactive features; The interactive features are fed to the main task branch for predicting RP risk scores and the auxiliary task branch for generating sensitivity heatmaps, respectively. The RP risk score is output through the main task branch. In the auxiliary task branch, a sensitivity response map is extracted based on the interactive features using a 1×1×1 convolution. A pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on CT images. The lung ventilation map is used as a pseudo-label for the sensitivity response map. In the step of preprocessing the registered radiotherapy dose image using a piecewise linear normalization method, the expression for the piecewise linear normalization method is: in, The original dose at coordinates (x, y, z) is... D is a unit of dosage. max For the maximum dose, This is the preprocessed image of the radiotherapy dose at coordinates (x, y, z); In the step of performing the first feature extraction on the CT image using a CT feature extraction network, the initial input CT image is denoted as... The initial input CT image, after passing through the CT feature extraction network, enters four parallel scale branches, with the resolutions of the four parallel scale branches being sequentially... ; Record No. In the first stage The feature maps of each scale branch are Within each scale branch, features are updated via stacked 3D convolutional units, as shown below: in, This represents the l-th stage of the CT feature extraction network. Indicates the first One scale branch, These are the intermediate features output by the 3D convolutional unit in the l-th stage; The fusion process, in which branches at different scales exchange information in a two-way manner, is represented as follows: in, Indicates the first The features of the first scale branch are transformed to the same level as the first scale branch through upsampling or downsampling. Operations with the same resolution across all scale branches; In addition, a nonlocal attention module is added at the end of each scale branch to capture the connections between distant anatomical regions through global attention. For any position r in the scale branch, the response of the nonlocal attention module is: in, and These represent the feature vectors of the input feature map at positions r and q, respectively. To measure the similarity function between positions r and q, This is a mapping function that maps features at position q to a high-dimensional embedding space. This is the normalization factor after summing over all positions; This represents the updated feature representation output by the nonlocal attention module at position r; The step of performing interaction modeling on the first feature and the second feature based on the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy to obtain the interaction feature includes: The first feature and the second feature are mapped to a unified embedding space, through shared... The convolution yields the query vector Q, key vector K, and value vector V used for cross-attention, denoted as: ； The query vector, key vector, and value vector are divided into H′ sub-channel groups based on their channel dimensions. Each sub-channel group corresponds to an attention head. Let the H′ be the first sub-channel group. The subspace features corresponding to each attention head are represented as follows: ； Perform scaled dot product attention independently in each subspace to obtain the first... Output of each attention head: ； in, Key vector The dimension; Concatenate the outputs of all attention heads along the channel dimension, and then map the results using the output mapping matrix. Linear transformation yields the final interaction features: ； in, The final interaction feature; H′ represents the number of attention heads; Input the number of channels. This represents the number of output channels after mapping. The first Each attention head corresponds to a query, key, and value.

2. The method for predicting the risk of radiation-induced pneumonia according to claim 1, characterized in that, In the step of extracting the second feature from the preprocessed radiotherapy dose image using an improved 3D convolutional network, the initial input preprocessed radiotherapy dose image is denoted as... The first convolutional layer uses a large kernel. With a step size of 2, the expression is: ； Subsequently, both convolutional layers used... Convolution, and introduce dynamic ReLU activation: ； in For input, For learnable gating coefficients; After feature extraction, the data is fed into a multi-scale pooling module for further processing. Four receptive fields are downsampled, and the pooling results are concatenated along the channel dimension and then restored to the original resolution using trilinear interpolation. Deformable convolution is introduced into the convolutional layer after multi-scale pooling, expressed as: ； in, For output position, For standard convolution sampling positions, For learnable offsets, These are the convolution weights.

3. The method for predicting the risk of radiation-induced pneumonia according to claim 2, characterized in that, In the main task branch, the interaction features are first compressed into a one-dimensional vector through 3D global average pooling, then connected to a Dropout layer to prevent overfitting, and mapped to a single output node through two fully connected linear layers. The output value is then activated by a Sigmoid activation function to obtain the final RP risk score. The main task branch primarily optimizes for binary classification tasks, using focus loss or binary cross-entropy as the loss function.

4. A radiation pneumonia risk prediction system, characterized in that, The system for implementing the radiation pneumonia risk prediction method according to any one of claims 1-3, the system comprising: The registration module is used to acquire CT images and radiotherapy dose images, and to perform rigid and non-rigid registration to ensure spatial correspondence between the two images; The preprocessing module is used to preprocess the registered radiotherapy dose image using a piecewise linear normalization method; The feature extraction module is used to perform first feature extraction on CT images using a CT feature extraction network, wherein the CT feature extraction network adopts an improved HRNetV2p structure, and simultaneously uses an improved 3D convolutional network to perform second feature extraction on the preprocessed radiotherapy dose image; The fusion module is used to perform interactive modeling on the first feature and the second feature according to the multi-head cross-attention mechanism and the position-adaptive dynamic weighted fusion strategy to obtain interactive features; The delivery module is used to deliver the interaction features to the main task branch for predicting the RP risk score and the auxiliary task branch for generating the sensitivity heatmap, respectively. The RP risk score is output through the main task branch. In the auxiliary task branch, a sensitivity response map is extracted based on the interaction features using a 1×1×1 convolution. A pre-trained 3D-Unet is used to generate a lung ventilation map representing lung function based on CT images. The lung ventilation map is used as a pseudo-label for the sensitivity response map.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the radiation pneumonia risk prediction method as described in any one of claims 1-3.

6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the program, implements the radiation pneumonia risk prediction method as described in any one of claims 1-3.

Images