Respiration-related displacement prediction method based on attention embedded dilated convolution

Abstract

This invention belongs to the field of medical temporal signal processing technology, specifically a method for predicting respiratory-related displacement based on attention-embedded dilated convolution. The method involves acquiring respiratory-related temporal data of the target object over continuous time; inputting this data into a trained respiratory-related displacement temporal prediction network model, which outputs the predicted respiratory displacement. The network model comprises multiple cascaded dilated convolutional residual blocks and a prediction layer. Each dilated convolutional residual block includes a causal dilated convolutional unit, a self-attention mechanism unit, and a residual connection unit. The output of the last dilated convolutional residual block is input into the prediction layer, which outputs the respiratory-related displacement of the target object at future time points. This invention dynamically weights key time steps during each layer's temporal feature extraction process, thereby improving the predictive ability for respiratory-related displacement and tumor motion.

Description

Technical Field

[0001] This invention belongs to the field of medical time-series signal processing technology, specifically relating to a method for predicting respiratory-related displacement based on attention-embedded dilated convolution. Background Technology

[0002] The human chest and abdomen region contains numerous organs such as the lungs, liver, heart, diaphragm, and gastrointestinal tract, as well as various lesions, target tissues, and physiological marker sites. The spatial position and morphological contours of these structures undergo periodic, nonlinear three-dimensional displacement and deformation in conjunction with the body's voluntary breathing movements. Furthermore, the movement patterns exhibit significant individual differences and rhythmic variability. Even within the same subject, the amplitude, frequency, and phase of movement fluctuate considerably depending on different respiratory states, body positions, and physiological conditions. This respiratory-induced displacement effect permeates multiple core medical scenarios, including medical imaging acquisition, clinical diagnostic and treatment procedures, physiological function monitoring, and rehabilitation assessment and intervention, becoming a key common issue restricting the precision, accuracy, and safety of various medical procedures and treatments.

[0003] For example, during radiotherapy for thoracic and abdominal tumors, the patient's respiratory movements can cause periodic or non-periodic displacement of the tumor target area, resulting in a deviation between the target location in the treatment plan and the actual tumor location at the time of irradiation. To reduce the impact of system delays introduced during image acquisition, data processing, signal transmission, and control execution on irradiation accuracy, it is usually necessary to predict the motion state in the short term based on historical respiratory-related displacement data, thereby achieving motion compensation and real-time guidance.

[0004] Among existing methods for predicting respiratory-related displacement, recurrent neural networks and their variants, such as RNN, LSTM, and GRU, can describe a certain degree of temporal dependence, but they usually rely on time-by-time recursive operations, making it difficult to achieve efficient parallel computing. When dealing with longer sequences, they are also prone to problems such as low training efficiency, gradient decay, and insufficient modeling of long-term dependencies.

[0005] Dilated convolution can expand the temporal receptive field by increasing the dilation rate, extracting dependency information over a longer time range without significantly increasing the number of parameters. However, it often ignores the effects of dynamic changes such as phase drift, inspiratory-exhalation transition, and local fluctuation anomalies in respiratory sequences, resulting in poor prediction stability.

[0006] Furthermore, existing time series models that incorporate attention mechanisms have failed to effectively utilize these mechanisms to continuously enhance key time step features based on signals such as respiratory-related displacements, which exhibit quasi-periodicity, phase drift, frequency variation, and local irregular fluctuations. This makes it difficult to simultaneously meet the requirements of long-term dependency modeling, local dynamic capture, and real-time causal prediction.

[0007] Therefore, there is an urgent need for a time-series modeling method for predicting respiratory-related displacements to solve the above problems. Summary of the Invention

[0008] The purpose of this invention is to provide a method for predicting respiratory-related displacement based on attention-embedded dilated convolution, which improves the ability to predict respiratory-related displacement by embedding attention enhancement units inside each dilated convolution residual block.

[0009] To achieve the above objectives, the first aspect of the present invention provides a method for predicting respiratory-related displacement based on attention-embedded dilated convolution, comprising the following steps: S1. Obtain the respiratory-related time-series data of the target object over continuous time; S2. Input the respiratory-related time series data into the trained respiratory-related displacement time series prediction network model, and output the respiratory-related displacement prediction results. The respiratory-related displacement time-series prediction network model includes multiple cascaded dilated convolutional residual blocks and prediction layers; Each dilated convolutional residual block includes a causal dilated convolutional unit, a self-attention mechanism unit, and a residual connection unit. The operations within each dilated convolutional residual block are performed in a fixed order: after the dilated convolution output, Q / K / V mapping is performed, then the temporal attention weights are calculated, and then feature fusion is performed. The temporal attention weights are embedded in the residual structure and stacked in multiple layers. The causal dilated convolution unit is used to perform dilated convolution operations on the respiratory-related time-series data to extract time-series features that meet the causal requirements; The self-attention layer is used to map the temporal features that meet the causality requirement into query vector Q, key vector K, and value vector V, respectively. Based on the query vector Q and key vector K, the temporal attention weight is calculated, and the value vector V is weighted and fused using the temporal attention weight to obtain attention-enhanced features. The temporal attention weight is used to enhance the feature representation of key phase time steps in the respiratory-related displacement sequence. The residual connection unit is used to fuse the attention enhancement feature with the residual branch feature to obtain the output of the current dilated convolutional residual block, and then input it into the next dilated convolutional residual block; The output of the last dilated convolution residual block is input into the prediction layer, and the breathing-related displacement of the target object at a future time is output.

[0010] Furthermore, the training method for the respiratory-related displacement temporal prediction network model includes the following steps: S21. Collect respiratory-related time-series data of the target object over continuous time and the corresponding respiratory-related displacement time-series information, and establish a unified spatial coordinate system; S22. Divide the respiratory-related time series data into n time periods with a total duration of T1, and divide the respiratory-related displacement time series information into n time periods with a total duration of T2. Establish the time correspondence between the respiratory-related time series data and the respiratory-related displacement time series information to obtain n training samples; wherein the time corresponding to the respiratory-related displacement time series information of each training sample is the future time of the time corresponding to the respiratory-related time series data. S23. Using the respiratory-related time-series data as input and the corresponding respiratory-related displacement time-series information as output, train the respiratory-related displacement time-series prediction network model to obtain the trained respiratory-related displacement time-series prediction network model.

[0011] Furthermore, the sampling step size of the respiratory-related time-series data and the respiratory-related displacement time-series information is the same; both the respiratory-related time-series data and the respiratory-related displacement time-series information are divided into time periods according to sliding windows. In step S2, when generating the three-dimensional motion trajectory based on the output respiratory-related displacement prediction results, the respiratory-related displacement prediction results obtained from the latest prediction window are used as the standard, or the weights are divided according to the order of the prediction windows. The final respiratory-related displacement prediction result at the current moment is obtained by weighted summation of the respiratory-related displacement prediction results at the same moment in each prediction window.

[0012] Furthermore, the expansion rate of each of the expanded convolutional residual blocks is set in a progressively increasing manner; the training loss function is the mean squared error loss function; and the self-attention layer uses a scaled dot product attention mechanism to calculate the temporal attention weights.

[0013] Furthermore, the respiratory-related time-series data includes one or more of the following: chest and abdominal surface point cloud motion time-series data, respiratory rate time-series data, intrathoracic pressure time-series data, and abdominal pressure time-series data; The respiratory-related displacement prediction results include respiratory displacement values ​​or target anatomical structure displacement values ​​at future time points. The respiratory displacement value includes the displacement value of the thoracic and abdominal organs, and the displacement value of the target anatomical structure includes the displacement value of the thoracic and abdominal tumor; the displacement value of the thoracic and abdominal tumor or the displacement value of the thoracic and abdominal organs are the three-dimensional motion trajectory of the center coordinates of the tumor or organ, or the three-dimensional motion trajectory of the edge contour.

[0014] Furthermore, when using the motion time series data of the chest and abdomen surface point cloud, the surface point cloud is divided into several regions. Based on the standard deviation of the geometric center coordinates of each region, the regions are sorted from largest to smallest, and the top K regions with a standard deviation greater than a preset value are selected. The standard deviation value and / or ranking of each region are used as significance representation information and associated with the corresponding geometric center coordinates for encoding. The spliced ​​motion time series data of the surface point cloud is then used as the input of the respiratory-related displacement time series prediction network model.

[0015] A second aspect of the present invention provides a respiratory-related displacement prediction system based on attention-embedded dilated convolution, comprising: a data acquisition module for acquiring respiratory-related time-series data of a target object over continuous time; Multiple cascaded dilated convolutional residual blocks are used, each of which performs dilated convolution operations on the respiratory-related time-series data to extract time-series features that meet causality requirements. Then, the time-series features that meet causality requirements are mapped to query vector Q, key vector K, and value vector V, respectively. Temporal attention weights are calculated based on query vector Q and key vector K, and the value vector V is weighted and fused using the temporal attention weights to obtain attention-enhanced features. Next, the attention enhancement features are fused with the residual branch features to obtain the output of the current dilated convolutional residual block, which is then input into the next dilated convolutional residual block. The prediction module is used to predict and output the breathing-related displacement of the target object at future time steps based on the fused attention-enhanced features of the output of the last dilated convolutional residual block.

[0016] A third aspect of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for predicting breathing-related displacement based on attention-embedded dilated convolution.

[0017] A fourth aspect of the present invention provides an organ displacement prediction device, comprising a computer-readable storage medium and a processor; The computer-readable storage medium is used to store instructions; The processor is used to read instructions stored in the computer-readable storage medium, execute the breathing-related displacement prediction method based on attention-embedded dilated convolution, and the breathing-related displacement prediction result is a prediction of the movement of thoracic and abdominal organs.

[0018] Furthermore, the respiratory-related displacement prediction results obtained by the respiratory-related displacement prediction method are used for tumor motion management, delayed step size, or real-time tracking control. When used for real-time tracking control, the tracking position for the future T2 time period is initially planned based on the respiratory-related displacement prediction results obtained in the j-th prediction window. Then, the prediction results for that moment are updated based on the respiratory-related displacement prediction results at the same time in the j+1-th prediction window and the j-th prediction window. The updated results are used to correct the tracking position for that moment in the initially planned future T2 time period.

[0019] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages: 1. The respiratory-related displacement prediction method based on attention-embedded dilated convolution provided by this invention predicts respiratory-related displacement based on respiratory-related time-series data. It fully considers the influence of dynamic changes such as phase drift, inspiratory-expiratory transition, and local fluctuation anomalies in the respiratory sequence. The attention enhancement unit is embedded inside each dilated convolution residual block, so that the feature extraction of each layer has dynamic enhancement capability and adaptively adjusts the importance of different time steps. Therefore, it can continuously highlight the end of inspiration, the beginning of expiration, the phase switching segment, and local irregular fluctuation segments in the feature extraction process of each layer. It takes into account the requirements of long-term dependency modeling, local dynamic capture, and real-time prediction of causality, improves the ability to identify dynamic changes in respiratory-related displacement, and thus improves the accuracy and stability of prediction.

[0020] 2. By combining intra-block attention enhancement with incremental expansion rate stacking, the dynamic response capability to phase switching and local anomalous segments is improved while taking into account long-term dependency modeling. By combining intra-block attention enhancement with residual connections and adopting an incremental expansion rate stacking method, multi-scale features ranging from short-term local fluctuations to long-term periodic changes can be covered, improving training stability and comprehensive representation capability.

[0021] 3. By expanding the temporal receptive field through dilated convolution, the computational bottleneck caused by the time-by-time recursion of recurrent neural networks is avoided. While maintaining a low number of parameters and high parallel computing efficiency, dependencies over a long time span are extracted.

[0022] 4. It adopts a causal convolutional architecture to ensure that the current prediction only depends on the history and current input, without leaking future information, making it suitable for online prediction and real-time radiotherapy guidance scenarios.

[0023] 5. This invention can predict future displacement values ​​based on historical respiratory displacement sequences, and is suitable for compensating for system delays caused by image acquisition, signal processing, and control execution, thereby improving the accuracy of tumor motion prediction or other respiratory displacement prediction.

[0024] 6. This invention adopts a sliding window approach and predicts respiratory-related displacements over a future period of time each time. Initial prediction values ​​can be used for preliminary determination, and then the next prediction window can be used for updating. The latest prediction window contains the historical respiratory information closest to the current target time. Therefore, using the latest prediction results can reflect changes in respiratory status more promptly, reduce the lag effect caused by old window information, and help improve the real-time performance and trajectory continuity of online prediction output. Attached Figure Description

[0025] Figure 1 The flowchart shows the overall process of the breathing-related displacement prediction method based on attention-embedded dilated convolution provided in the embodiments of the present invention.

[0026] Figure 2This is a schematic diagram of the overall architecture of a temporal prediction network composed of multiple attention-embedded dilated convolutional residual blocks in an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of the structure of a single attention-embedded dilated convolutional residual block in an embodiment of the present invention.

[0028] Figure 4 This paper compares the results of the attention-embedded dilated convolution-based respiratory displacement prediction method provided by this invention with those of the existing TCN+ATT network prediction method. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0030] Please see Figure 1-3 This invention provides a method for predicting respiratory-related displacement based on attention-embedded dilated convolution, comprising the following steps: S1. Obtain the respiratory-related time-series data of the target object over continuous time; S2. Input the respiratory-related time series data into the trained respiratory-related displacement time series prediction network model, and output the respiratory-related displacement prediction results. The respiratory-related displacement temporal prediction network model includes multiple cascaded dilated convolutional residual blocks and prediction layers. Each dilated convolutional residual block includes a causal dilated convolutional unit, a self-attention mechanism unit, and a residual connection unit. The calculations within each dilated convolutional residual block are performed in a fixed order: after the dilated convolution output, Q / K / V mapping is performed, then the temporal attention weights are calculated, and then feature fusion is performed. The temporal attention weights are embedded in the residual structure and stacked in multiple layers. The causal dilated convolution unit is used to perform dilated convolution operations on the respiratory-related time-series data to extract time-series features that meet the causal requirements; The self-attention layer is used to map the temporal features that meet the causality requirement into query vector Q, key vector K, and value vector V, respectively. Based on the query vector Q and key vector K, the temporal attention weight is calculated, and the value vector V is weighted and fused using the temporal attention weight to obtain attention-enhanced features. The temporal attention weight is used to enhance the feature representation of key phase time steps in the respiratory-related displacement sequence. The residual connection unit is used to fuse the attention enhancement feature with the residual branch feature to obtain the output of the current dilated convolutional residual block, and then input it into the next dilated convolutional residual block; The output of the last dilated convolution residual block is input into the prediction layer, and the breathing-related displacement of the target object at a future time is output.

[0031] By embedding the attention enhancement unit inside each dilated convolution residual block and fixing the operation order of "causal dilated convolution—Q / K / V mapping—temporal dimension attention weight calculation—value vector weighted fusion—residual fusion", the present invention enables the network to dynamically weight key time steps during the temporal feature extraction process of each layer, thereby improving the prediction ability of respiratory-related displacement and tumor motion. This solves the problem that although dilated convolutional networks can expand the temporal receptive field in the prior art, they lack the ability to dynamically enhance key phase change time steps in respiratory-related displacement sequences layer by layer, resulting in insufficient prediction accuracy in scenarios with phase drift, irregular local fluctuations, and system delay compensation requirements.

[0032] Specifically, the core of this invention lies in constructing an attention-embedded dilated convolutional residual block. Unlike schemes that only add attention layers outside the network, this invention sets up attention enhancement units inside each dilated convolutional residual block, enabling attention computation to participate in the layer-by-layer construction process of temporal features at each layer.

[0033] The order of operations within each dilated convolution residual block is fixed as follows: causal dilated convolution → Q / K / V mapping → temporal attention weight calculation → value vector weighted fusion → residual fusion.

[0034] in: Causal dilated convolutional units are used to ensure that the output at any given time depends only on the current and historical time step inputs, and not on information from future time steps, thereby meeting the causal requirements of online prediction. Causal convolution is used to avoid the leakage of future information.

[0035] Dilated convolution operation: used to expand the receptive field without significantly increasing the number of parameters, extracting long-term dependent features from respiratory-related displacement sequences. Let the input feature sequence be... If the kernel size is k and the dilation rate is d, then the dilated convolution output at time step t can be expressed as: .

[0036] in, Let i be the parameters of the i-th convolutional kernel. For convolution, sampling is performed at intervals in the time dimension. Here, b represents the output feature of the convolution, and b is the bias. is the activation function. By adjusting the expansion rate d, a larger receptive field can be obtained without significantly increasing the number of parameters, thus capturing dependencies over a longer time range.

[0037] Q / K / V mapping: converting dilated convolution output features The query vector Q, key vector K, and value vector V are obtained respectively through linear mapping: , , ,in, , , This is a trainable parameter matrix.

[0038] Attention weight calculation in the time dimension: The present invention preferably calculates attention weight in the time dimension to enhance the characterization of key phase time steps in the respiratory sequence.

[0039] Preferably, a scaled dot product attention mechanism is used: .

[0040] in, The dimension of the key vector. This represents the attention weight matrix between different time steps. This is the query vector at time step t. and These are the key vectors at time steps s and s', respectively.

[0041] Value vector weighted fusion: The value vectors are weighted and summed using the time-dimensional attention weights to obtain the enhanced temporal feature representation. .

[0042] in, For time step td i The value vector.

[0043] This processing allows the network to retain the advantage of the large receptive field of dilated convolution while giving higher weights to the end of inspiration, the beginning of expiration, the phase switching region, and the region with abnormal local fluctuations.

[0044] Residual fusion: Attention-enhanced features are additively fused or equivalently fused with the output of residual branches to alleviate the gradient decay problem when deep layers are stacked and improve the stability of network training.

[0045] In some implementations, multiple dilated convolutional residual blocks are cascaded along the network depth direction, with the dilation rate of adjacent residual blocks increasing to form a multi-scale temporal receptive field. Preferably, the dilation rate adopts an exponential increasing form, such as 1, 2, 4, 8 or 2, 4, 8, etc. Shallow residual blocks are used to extract short-term local fluctuation features, while deep residual blocks are used to extract long-term periodic trend features, thereby achieving multi-scale modeling from local dynamics to global patterns.

[0046] A prediction layer is set at the end of the network to map the output of the last residual block to a target displacement value at a future time. The target displacement value can be a univariate respiratory displacement value, a directional component of tumor motion, or multiple components in three-dimensional tumor coordinates.

[0047] Furthermore, the training method for the respiratory-related displacement temporal prediction network model includes the following steps: S21. Collect respiratory-related time-series data of the target object over continuous time and the corresponding respiratory-related displacement time-series information, and establish a unified spatial coordinate system; S22. Divide the respiratory-related time series data into n time periods with a total duration of T1, and divide the respiratory-related displacement time series information into n time periods with a total duration of T2. Establish the time correspondence between the respiratory-related time series data and the respiratory-related displacement time series information to obtain n training samples; wherein the time corresponding to the respiratory-related displacement time series information of each training sample is the future time of the time corresponding to the respiratory-related time series data. S23. Using the respiratory-related time-series data as input and the corresponding respiratory-related displacement time-series information as output, train the respiratory-related displacement time-series prediction network model to obtain a trained respiratory-related displacement time-series prediction network model. The training loss function is the mean squared error loss function.

[0048] During the training phase, historical time windows are input into the network to predict future target values, and supervised training is performed using real labeled values. The total training time, T1, can be set according to the sampling frequency, the range of the main breathing cycle, the system delay compensation requirements, and the prediction lead. The preferred loss function is the mean squared error loss function. .

[0049] in, For the true value, is the predicted value, and N is the number of samples.

[0050] In some implementations, mean absolute error, smoothing regularization terms, frequency domain consistency constraints, or combinations thereof can be used to improve prediction smoothness and generalization ability. The optimizer can be Adam, SGD, or RMSProp, and learning rate decay, normalization layers, Dropout, or early stopping mechanisms can be configured during training.

[0051] The sampling step size of the respiratory-related time-series data and the respiratory-related displacement time-series information is the same. Both the respiratory-related time-series data and the respiratory-related displacement time-series information are divided into time periods using sliding windows, and the sliding step size of two adjacent samples is the same. In step S2, when generating the three-dimensional motion trajectory based on the output respiratory-related displacement prediction results, the respiratory-related displacement prediction result obtained from the latest prediction window is used as the standard, or the weights are divided according to the order of the prediction windows. The final respiratory-related displacement prediction result for the current moment is obtained by weighted summing of the respiratory-related displacement prediction results at the same time in each prediction window. For example, if the total time of each prediction window is T2 and is 3 seconds, and the step size is 1 second, then if the first prediction window predicts 5-7 seconds, the second and third prediction windows predict 6-8 seconds and 7-9 seconds respectively, resulting in three prediction results for the 7th second. The final prediction result for the 7th second from the third prediction window can be used as the standard, or the weights can be weighted and summed sequentially forward, with the latest prediction window having the largest weight (i.e., the third prediction window having the largest weight).

[0052] Furthermore, the self-attention layer employs a scaled dot product attention mechanism to calculate the temporal dimension attention weights.

[0053] Furthermore, the respiratory-related time-series data includes one or more of the following: chest and abdominal surface point cloud motion time-series data, respiratory rate time-series data, intrathoracic pressure time-series data, and abdominal pressure time-series data; the respiratory-related displacement prediction results include chest and abdominal tumor motion prediction or chest and abdominal organ motion prediction (e.g., diaphragmatic displacement, cardiac displacement, airway wall deformation, etc.). Tumor motion prediction helps improve the accuracy of radiotherapy target areas, enhance treatment efficacy and safety, while chest and abdominal organ motion prediction helps remove motion-induced artifacts, improve the clarity of medical images, and thus improve the accuracy of lesion identification; it can also improve accuracy during interventional treatment. The actual respiratory-related displacement values ​​are obtained through real-time tracking of medical images, for example, by obtaining the three-dimensional position information of the tumor target area at the corresponding time phase through 4D-CT, and by obtaining the three-dimensional position information of the corresponding organs at the corresponding time phase through 4D-CT / 4D-MRI.

[0054] Specifically, the respiratory-related time-series data is a one-dimensional respiratory displacement sequence; in other embodiments, the input sequence is a multi-dimensional respiratory-related representation sequence. The multi-dimensional respiratory-related representation sequence may be derived from a body surface motion acquisition device, infrared marker trajectory, surface-guided radiotherapy system signal, physiological signals (intrathoracic pressure, abdominal pressure, etc.), or time-series features formed by body surface point clouds after dimensionality reduction, local feature extraction, or pseudo-voxenization.

[0055] In summary, the present invention provides a medical temporal modeling structure that is suitable for multi-layer, multi-scale stacking, has good training stability, and meets the requirements of online prediction. It can be used to provide more accurate future displacement prediction results for delay compensation or real-time tracking in radiotherapy motion management.

[0056] Furthermore, when using the motion time series data of the chest and abdomen surface point cloud, the surface point cloud is divided into several regions. Based on the standard deviation of the geometric center coordinates of each region, the regions are sorted from largest to smallest, and the top K regions with a standard deviation greater than a preset value are selected. The standard deviation value and / or ranking of each region are used as significance representation information and associated with the corresponding geometric center coordinates for encoding. The spliced ​​motion time series data of the surface point cloud is then used as the input of the respiratory-related displacement time series prediction network model.

[0057] Furthermore, the predicted movement of thoracic and abdominal tumors or organs is a three-dimensional motion trajectory of the center coordinates of the tumor or organ, or a three-dimensional motion trajectory of the edge contour.

[0058] A second aspect of the present invention provides a respiratory-related displacement prediction system based on attention-embedded dilated convolution, comprising: a data acquisition module for acquiring respiratory-related time-series data of a target object over continuous time; Multiple cascaded dilated convolutional residual blocks are used, each of which performs dilated convolution operations on the respiratory-related time-series data to extract time-series features that meet causality requirements. Then, the time-series features that meet causality requirements are mapped to query vector Q, key vector K, and value vector V, respectively. Temporal attention weights are calculated based on query vector Q and key vector K, and the value vector V is weighted and fused using the temporal attention weights to obtain attention-enhanced features. Next, the attention enhancement features are fused with the residual branch features to obtain the output of the current dilated convolutional residual block, which is then input into the next dilated convolutional residual block. The prediction module is used to predict and output the breathing-related displacement of the target object at future time steps based on the fused attention-enhanced features of the output of the last dilated convolutional residual block.

[0059] A third aspect of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for predicting breathing-related displacement based on attention-embedded dilated convolution.

[0060] A fourth aspect of the present invention provides an organ displacement prediction device, comprising a computer-readable storage medium and a processor; The computer-readable storage medium is used to store instructions; The processor is used to read instructions stored in the computer-readable storage medium, execute the breathing-related displacement prediction method based on attention-embedded dilated convolution, and the breathing-related displacement prediction result is a prediction of the movement of thoracic and abdominal organs.

[0061] The respiratory-related displacement prediction method described above provides prediction results for tumor motion management, delayed step size, or real-time tracking control. When used for real-time tracking control, the tracking position for the future time period T2 is initially planned based on the respiratory-related displacement prediction results obtained in the j-th prediction window. Then, the prediction result for that moment is updated based on the respiratory-related displacement prediction results at the same time in the (j+1)-th prediction window and the j-th prediction window. The updated result is then used to correct the initially planned tracking position for that moment in the future time period T2. This operation improves the efficiency and accuracy of tracking control.

[0062] Example 1: Respiratory Displacement Prediction Example In this embodiment, the object to be predicted is the respiratory displacement sequence of a patient or phantom. First, respiratory-related time-series data of the target object are continuously collected along the time axis, and input samples are constructed using a fixed-length historical window, with the respiratory displacement at the next future time step as the prediction target. The input samples are then fed into a time-series prediction network composed of multiple attention-embedded dilated convolutional residual blocks. Each residual block executes the following steps in a fixed order: causal dilated convolution, Q / K / V mapping, temporal attention calculation, weighted fusion, and residual fusion. After network training is complete, the online prediction phase outputs predicted respiratory displacement values ​​for future time steps based on the latest historical respiratory sequence.

[0063] In this embodiment, since respiratory signals are usually quasi-periodic and also have phase drift, frequency changes and local irregular fluctuations, intra-block attention enhancement can continuously highlight key time segments in each layer, which is beneficial to improving the short-term prediction accuracy of respiratory displacement.

[0064] Example 2: Tumor Motility Prediction Example In this embodiment, the input sequence is a respiratory-related representation sequence. This respiratory-related representation sequence can be a body surface displacement sequence, a respiratory monitoring sequence output by a surface-guided radiotherapy system, or a low-dimensional temporal feature formed by local feature extraction from a body surface point cloud. This input sequence is fed into a temporal prediction network composed of multiple attention-embedded dilated convolutional residual blocks, executed in a fixed order of "causal dilated convolution, Q / K / V mapping, temporal attention calculation, weighted fusion, and residual fusion," outputting the tumor displacement component or three-dimensional tumor position coordinates at future time points. The prediction results can be used for real-time compensation or motion management during radiotherapy.

[0065] Example 3: Comparison of Intra-block Attention and External Attention This embodiment illustrates the advantages of the "intra-block embedded attention" method compared to "external network attention" in this invention. The input sequences are all point clouds of the chest and abdomen surface, and the output displacement prediction is the tumor displacement prediction value.

[0066] Option A: A single independent attention layer (ATT) is uniformly set after multiple dilated convolutional residual blocks (TCNs), corresponding to... Figure 4 TCN+ATT in the middle.

[0067] Option B: An attention enhancement unit is set inside each dilated convolution residual block, and the processing is performed in the fixed order described in Example 2 (i.e., the respiratory-related displacement temporal prediction network model of the present invention).

[0068] like Figure 4 In the task of predicting respiratory-related displacement, Scheme B can continuously enhance the key time step features during the feature extraction process of each layer. Therefore, it can achieve more stable prediction results in scenarios with obvious phase switching, enhanced local fluctuations, or delay compensation requirements.

[0069] In summary, this invention can adaptively address nonlinear changes in the respiratory waveform (such as phase drift and frequency shift). In comparative experiments on the Lübeck dataset, RTA-Net (the network model of this invention) achieved a RMSE 13.2% lower than TCN and a RMSE 15.3% lower than GRU. Compared to LSTM, this structure is based on convolutional operations, supports parallel computation, has fewer parameters, and faster inference speed.

[0070] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting respiratory-related displacement based on attention-embedded dilated convolution, characterized in that, Includes the following steps: S1. Obtain the respiratory-related time-series data of the target object over continuous time; S2. Input the respiratory-related time series data into the trained respiratory-related displacement time series prediction network model, and output the respiratory-related displacement prediction results. The respiratory-related displacement time-series prediction network model includes multiple cascaded dilated convolutional residual blocks and prediction layers; Each of the dilated convolutional residual blocks includes a causal dilated convolutional unit, a self-attention mechanism unit, and a residual connection unit. The operations within each dilated convolutional residual block are performed in a fixed order: after the dilated convolution output, Q / K / V mapping is performed, then the temporal attention weights are calculated, and then feature fusion is performed. The temporal attention weights are embedded in the residual structure and stacked in multiple layers. The causal dilated convolution unit is used to perform dilated convolution operations on the respiratory-related time-series data to extract time-series features that meet the causal requirements; The self-attention layer is used to map the temporal features that meet the causality requirements into query vector Q, key vector K, and value vector V, respectively. Based on the query vector Q and key vector K, the temporal attention weight is calculated, and the value vector V is weighted and fused using the temporal attention weight to obtain attention-enhanced features. The temporal attention weight is used to enhance the feature representation of key phase time steps in the respiratory-related displacement sequence. The residual connection unit is used to fuse the attention enhancement feature with the residual branch feature to obtain the output of the current dilated convolutional residual block, and then input it into the next dilated convolutional residual block; The output of the last dilated convolution residual block is input into the prediction layer, and the breathing-related displacement of the target object at a future time is output.

2. The method for predicting respiratory-related displacement based on attention-embedded dilated convolution according to claim 1, characterized in that, The training method for the respiratory-related displacement temporal prediction network model includes the following steps: S21. Collect respiratory-related time-series data of the target object over continuous time and the corresponding respiratory-related displacement time-series information; S22. Divide the respiratory-related time series data into n time periods with a total duration of T1, and divide the respiratory-related displacement time series information into n time periods with a total duration of T2. Establish the time correspondence between the respiratory-related time series data and the respiratory-related displacement time series information to obtain n training samples; wherein the time corresponding to the respiratory-related displacement time series information of each training sample is the future time of the time corresponding to the respiratory-related time series data. S23. Using the respiratory-related time-series data as input and the corresponding respiratory-related displacement time-series information as output, train the respiratory-related displacement time-series prediction network model to obtain the trained respiratory-related displacement time-series prediction network model.

3. The method for predicting respiratory-related displacement based on attention-embedded dilated convolution according to claim 2, characterized in that, The sampling step size of the respiratory-related time-series data and the respiratory-related displacement time-series information is the same; the respiratory-related time-series data and the respiratory-related displacement time-series information are both divided into time periods according to sliding windows; in step S2, when generating the three-dimensional motion trajectory based on the output respiratory-related displacement prediction results, the respiratory-related displacement prediction results obtained from the latest prediction window are used as the standard, or the weights are divided according to the order of the prediction windows, and the final respiratory-related displacement prediction result at the current time is obtained by weighted summation of the respiratory-related displacement prediction results at the same time in each prediction window.

4. The method for predicting respiratory-related displacement based on attention-embedded dilated convolution according to claim 2, characterized in that, The dilation rate of each dilated convolutional residual block is set in a progressively increasing manner; the training loss function is the mean squared error loss function; The self-attention layer uses a scaled dot product attention mechanism to calculate the time-dimensional attention weights.

5. The method for predicting respiratory-related displacement based on attention-embedded dilated convolution according to claim 1, characterized in that, The respiratory-related time-series data includes one or more of the following: chest and abdominal surface point cloud motion time-series data, respiratory rate time-series data, intrathoracic pressure time-series data, and abdominal pressure time-series data. The respiratory-related displacement prediction results include respiratory displacement values ​​or target anatomical structure displacement values ​​at future time points. The respiratory displacement value includes the displacement value of the thoracic and abdominal organs, and the displacement value of the target anatomical structure includes the displacement value of the thoracic and abdominal tumor; the displacement value of the thoracic and abdominal tumor or the displacement value of the thoracic and abdominal organs are the three-dimensional motion trajectory of the center coordinates of the tumor or organ, or the three-dimensional motion trajectory of the edge contour.

6. The method for predicting respiratory-related displacement based on attention-embedded dilated convolution according to claim 5, characterized in that, When using the motion time series data of the chest and abdomen surface point cloud, the surface point cloud is divided into several regions. Based on the standard deviation of the geometric center coordinates of each region, the regions are sorted from largest to smallest, and the top K regions with a standard deviation greater than a preset value are selected. The standard deviation value and / or ranking of each region are used as significance representation information and associated with the corresponding geometric center coordinates for encoding. The spliced ​​surface point cloud motion time series data is then used as the input of the respiratory-related displacement time series prediction network model.

7. A respiratory-related displacement prediction system based on attention-embedded dilated convolution, characterized in that, include: The data acquisition module is used to collect respiratory-related time-series data of the target object over continuous time. Multiple cascaded dilated convolutional residual blocks are used, each of which performs dilated convolution operations to extract temporal features that meet causality requirements. These temporal features that meet causality requirements are then mapped to query vector Q, key vector K, and value vector V, respectively. Temporal attention weights are calculated based on query vector Q and key vector K, and the value vector V is weighted and fused using these temporal attention weights to obtain attention-enhanced features. Next, the attention enhancement features are fused with the residual branch features to obtain the output of the current dilated convolutional residual block, which is then input into the next dilated convolutional residual block. The prediction module is used to predict and output the breathing-related displacement of the target object at future time steps based on the fused attention-enhanced features of the output of the last dilated convolutional residual block.

8. A storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the breathing-related displacement prediction method based on attention-embedded dilated convolution as described in any one of claims 1-6.

9. A device, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store instructions; The processor is configured to read instructions stored in the computer-readable storage medium and execute the breathing-related displacement prediction method based on attention-embedded dilated convolution as described in any one of claims 1-6.

10. The device according to claim 9, characterized in that, The respiratory-related displacement prediction results obtained by the respiratory-related displacement prediction method are used for tumor motion management, delayed step size or real-time tracking control. When used for real-time tracking control, the tracking position for the future T2 time period is initially planned based on the respiratory-related displacement prediction results obtained in the j-th prediction window. Then, the prediction results for that moment are updated based on the respiratory-related displacement prediction results at the same time in the j+1-th prediction window and the j-th prediction window. The updated results are used to correct the tracking position for that moment in the initially planned future T2 time period.

Images