A respiratory motion prediction method and system

Abstract

A respiratory motion prediction method and system, relating to the field of deep learning technology, specifically to the field of respiratory motion prediction technology. The method comprises the following steps: acquiring respiratory motion trajectory data; preprocessing the respiratory motion trajectory data to obtain a preprocessed sample set; dividing the sample set into a training set, a validation set, and a test set; constructing a respiratory motion prediction model, the model sequentially comprising a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module, and a flattening layer; using the training set to train the respiratory motion prediction model; using the validation set to adjust the model parameters; using the test set to evaluate the respiratory motion prediction model, and obtaining a respiratory motion prediction result.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and specifically to the field of respiratory motion prediction technology. Background Art

[0002] At present, respiratory motion prediction technology is developing continuously, and machine learning methods have been widely used in this field. Traditional machine learning methods include non-parametric regression prediction, wavelet decomposition and neural network decomposition, adaptive Kalman filter and support vector machine regression prediction. In recent years, with the development of deep learning, new ideas have been provided for respiratory prediction methods. Deep learning methods mainly include multi-layer perceptron, convolutional neural network and long short-term memory network. These methods have made significant progress in the short-term time series prediction of respiratory motion.

[0003] However, the existing prediction methods still have large errors in the long-term time series prediction of respiratory signals. First, long-term time series prediction is affected by the complexity and nonlinearity of respiratory signals. Respiratory signals have multiple frequency components and nonlinear characteristics, which makes it difficult for prediction methods to accurately capture the dynamic changes of respiratory signals. In addition, respiratory signals are also affected by many factors, such as heart rate, body position, and emotions, which increases the difficulty of predicting longer time series signals. Secondly, the existing prediction methods also have large prediction errors in the prediction of irregular peaks of respiratory signals. The peaks in respiratory signals usually represent the peak period of respiratory movement. However, due to the irregularity of respiratory signals and the presence of noise, the current prediction methods have certain limitations in capturing and predicting irregular peak signals, which may lead to inaccurate and unstable prediction results. These two problems are urgently needed to be solved in the current use of deep learning technology for respiratory movement prediction.

[0004] Therefore, it is particularly important to use an improved deep learning respiratory motion hybrid prediction model method to improve the accuracy of irregular peaks and long-term time series prediction of respiratory signals. Summary of the invention

[0005] The present invention provides a method for predicting respiratory motion, the method comprising the following steps:

[0006] S1, obtaining respiratory motion trajectory data, wherein the respiratory motion trajectory data is eigenmode function components of different frequencies obtained after modal decomposition of the collected human respiratory motion trajectory data;

[0007] S2, preprocessing the respiratory motion trajectory data to obtain a preprocessed sample set;

[0008] S3, dividing the sample set into a training set, a validation set and a test set;

[0009] S4, constructing a respiratory movement prediction model, wherein the respiratory movement prediction model sequentially includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module and a flattening layer;

[0010] In the two-dimensional multi-channel convolutional neural network module, the number of channels is determined by the feature type and size of the input data, the feature type of the input data refers to the type of frequency in the intrinsic mode function components of different frequencies obtained in step S1, one frequency type represents one feature, and convolution kernels of different sizes are used in each two-dimensional convolutional neural network; the two-dimensional multi-channel convolutional neural network module performs feature extraction on the input data, specifically including capturing local features of different time scales of the respiratory signal by designing different convolution kernel sizes, and extracting feature information of the respiratory signal by designing a suitable number of convolution channels according to the type and size of the local features, and sliding the feature information of the respiratory signal in the vertical direction to capture time correlation and learn the features of the respiratory signal at different time scales;

[0011] S5. Use the training set to train the respiratory movement prediction model; use the validation set to adjust the model parameters; use the test set to evaluate the respiratory movement prediction model, and obtain the respiratory movement prediction result.

[0012] The present invention further provides a respiratory motion prediction system, the system comprising:

[0013] Data acquisition module: acquiring respiratory motion trajectory data, wherein the respiratory motion trajectory data is the intrinsic mode function components of different frequencies obtained after modal decomposition of the collected human respiratory motion trajectory data;

[0014] Data preprocessing module: preprocess the respiratory motion trajectory data to obtain a preprocessed sample set;

[0015] Data partitioning module: partitioning the sample set into a training set, a validation set and a test set;

[0016] Model construction module: construct a respiratory motion prediction model, which includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module and a flattening layer in sequence;

[0017] In the two-dimensional multi-channel convolutional neural network module, the number of channels is determined by the feature type and size of the input data, the feature type of the input data refers to the type of frequency in the intrinsic mode function components of different frequencies obtained by the data acquisition module, one frequency type represents one feature, and convolution kernels of different sizes are used in each two-dimensional convolutional neural network; the two-dimensional multi-channel convolutional neural network module performs feature extraction on the input data, specifically including capturing local features of the respiratory signal at different time scales by designing different convolution kernel sizes, and extracting feature information of the respiratory signal by designing a suitable number of convolution channels according to the type and size of the local features, and sliding the feature information of the respiratory signal in the vertical direction to capture time correlation and learn the features of the respiratory signal at different time scales;

[0018] Respiratory movement prediction module: uses the training set to train the respiratory movement prediction model; uses the validation set to adjust the model parameters; uses the test set to evaluate the respiratory movement prediction model and obtains the respiratory movement prediction results.

[0019] The beneficial effects of the present invention are:

[0020] Firstly, the respiratory motion prediction model constructed by the present invention first designs a series of two-dimensional convolution layers to extract multiple frequency features of the respiratory signal decomposed by modal decomposition; these convolution layers use convolution kernels of different sizes, such as (3*1), (3*5), etc., instead of the traditional (1*1), (3*3) regular convolution kernel sizes, and capture the local features of the respiratory signal at different time scales by designing different convolution kernel sizes, and designing a suitable number of convolution channels according to the type and size of the features to fully extract the feature information of the respiratory signal, and make it slide in the vertical direction to better capture the time correlation and effectively learn the respiratory signal features at different time scales.

[0021] Secondly, the respiratory movement prediction model constructed by the present invention adds a lightweight attention mechanism after each two-dimensional neural network convolution channel to optimize the model. When constructing the lightweight attention mechanism, the traditional fully connected layer is not used, but a one-dimensional convolution layer is used for improvement; the local cross-channel interaction strategy without dimensionality reduction is effectively implemented through one-dimensional convolution, and each channel and its neighbors are considered to capture cross-channel interactions, and the weight coefficients of each convolution channel are efficiently learned, capturing the important features of the respiratory signal from local to global. Compared with other attention mechanisms, it has fewer parameters and model complexity, which can help the respiratory movement prediction model improve the prediction performance of long-term time series prediction and irregular peaks;

[0022] In order to cooperate with the constructed lightweight attention mechanism, the model constructed by the present invention adopts multi-channel input at the initial input, adds a 2*2 maximum pooling layer after each channel attention mechanism, and then obtains richer features through weighted aggregation through splicing aggregation layer, thereby realizing parallel multi-scale feature fusion.

[0023] Finally, a temporal convolutional neural network module (Temporal Convolutional Network) is used to predict the respiratory signal. The causal relationship between different frequency features is identified by the causal convolution in the temporal convolutional neural network, and the long-term dependency of the sequence is captured by the dilated convolution layer and residual connection, so as to obtain the future information of the respiratory signal.

[0024] The respiratory movement prediction model constructed by the present invention can alleviate the problem of insufficient feature capture in respiratory signal prediction, and further alleviate the problem of gradient explosion in long-term prediction windows, significantly improving the prediction ability of the respiratory prediction model in long-term time series prediction and irregular peaks of respiratory signals, and providing a new method for improving the accuracy and stability of respiratory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a structural diagram of the respiratory motion prediction model;

[0026] Figure 2 The structure diagram of the lightweight attention mechanism constructed;

[0027] Figure 3 It is the structure diagram of the temporal convolutional neural network;

[0028] Figure 4 It is the variation trend of each error index of the respiratory motion prediction model with the delay time at different delay times;

[0029] Figure 5 The variation trend of the determination coefficient of the respiratory motion prediction model with the delay time at different delay times;

[0030] Figure 6 The mean absolute error (MAE) and root mean square error (RMSE) of the respiratory motion prediction model change with the input sequence;

[0031] Figure 7 This is a comparative experiment of the mean absolute error between the respiratory motion prediction model and the CNN_BiLSTM and DeepEST prediction models;

[0032] Figure 8 This is a comparative experiment of the root mean square error between the respiratory motion prediction model and the CNN_BiLSTM and DeepEST prediction models;

[0033] Fig. 9This is a comparison diagram of the predicted curve and the actual position curve in specific experiments using the respiratory motion prediction model in the present invention and other models. DETAILED DESCRIPTION

[0034] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0035] This embodiment provides a respiratory motion prediction method, the method comprising the following steps:

[0036] S1, obtaining respiratory motion trajectory data;

[0037] S2, preprocessing the respiratory motion trajectory data to obtain a preprocessed sample set;

[0038] S3, dividing the sample set into a training set, a validation set and a test set;

[0039] S4, constructing a respiratory movement prediction model, wherein the model sequentially includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module, and a flattening layer;

[0040] S5. Use the training set to train the respiratory movement prediction model; use the validation set to adjust model parameters; use the test set to test the respiratory movement prediction model, and obtain a respiratory movement prediction result.

[0041] Each step is described in detail below.

[0042] For step S1, obtaining respiratory motion trajectory data: the respiratory motion trajectory data is the intrinsic mode function components of different frequencies obtained after modal decomposition of the collected human respiratory motion trajectory data;

[0043] For step S2, the respiratory motion trajectory data is preprocessed to obtain a preprocessed sample set:

[0044] Preprocessing includes data screening, outlier removal, smoothing and denoising, and normalization. Data screening refers to selecting more than 200 respiratory motion trajectories with more than 80,000 data points from the data set as experimental data; the threshold method is used to remove outliers, that is, the respiratory signal is divided into windows using a window-based anomaly detection algorithm, and then the range of each window is calculated. Whether the value is an outlier is determined based on whether the range exceeds the threshold, and finally it is removed; smoothing and denoising uses SG filters to smooth and denoise, which can effectively maintain the original shape and trend characteristics of the respiratory signal; normalization uses maximum and minimum value normalization to normalize the experimental respiratory signal to (0, 1) to avoid a certain characteristic value from having too much influence on the model; the formula is as follows:

[0045]

[0046] Where: It is the normalized data; It is the original data; and are the maximum and minimum values ​​of the original data respectively.

[0047] For step S3, the sample set is divided into a training set, a validation set and a test set:

[0048] The first 80% of the data was used as the training set, and the other 20% was used as the test set of the model. The training set and the validation set were divided again in a ratio of 8:2, and 10,000 data points were intercepted from the data sample set as experimental data. The respiratory movements of the divided data sets were labeled, and the delay time t was set to 40ms, 100ms, 200ms, 320ms, 400ms, 450ms, 600ms, 700ms, etc. The delay time was 20ms. as the predicted target label.

[0049] For step S4, constructing a respiratory movement prediction model, the model sequentially includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module and a flattening layer;

[0050] The respiratory motion prediction model is as follows: Figure 1 As shown, in Figure 1In the figure, three channels of the two-dimensional multi-channel convolutional neural network layer are reflected. The specific number of channels is determined by the feature type and size of the input data. The feature type of the input data refers to the type of frequency in the intrinsic mode function components of different frequencies obtained in step S1. One frequency type represents one feature. However, it should be noted that if there are too many channels, the network parameters may be too large and the feature repetition rate may be too high when the features are merged, aggregated and spliced, resulting in feature congestion and slow update speed.

[0051] First, the input data is feature extracted through a two-dimensional multi-channel convolutional neural network module, and a lightweight attention mechanism module is constructed. Each lightweight attention mechanism realizes efficient learning of the weight coefficient of each convolution channel, capturing important local to global features of the respiratory signal. The lightweight attention mechanism module realizes channel weight coefficient learning by utilizing the local cross-channel interaction strategy without dimensionality reduction of its one-dimensional convolution. The constructed lightweight attention mechanism is added after the two-dimensional convolutional neural network of each channel.

[0052] The steps to build a lightweight attention mechanism are:

[0053] Step 1: To ensure that the attention mechanism does not reduce the dimension, it is necessary to aggregate the convolution features through the global average pooling layer (GlobalAveragePooling2D) and then adaptively determine the convolution kernel size. , mainly achieved through adaptive functions; specifically, ensuring that the calculated convolution kernel size is an odd number before performing subsequent operations;

[0054] The calculation formula is as follows:

[0055] ;

[0056] ;

[0057] Where: and It is a parameter used to calculate the size of the convolution kernel, C represents the channel dimension, and k determines the coverage of channel interaction. Indicates odd number operation;

[0058] Step 2: Adjust the feature tensor obtained by global average pooling to a specified shape; here, adjusting to a specified shape means making the dimension of the feature tensor the same as the dimension of the subsequent convolution operation. Since the subsequent convolution layer in this method is one-dimensional, this means adjusting the dimension of the feature tensor to one-dimensional.

[0059] Step 3: Perform channel feature learning on the above specified shape feature tensor, calculate the attention of each channel through a one-dimensional convolutional layer (Conv1D), and learn the importance of different channels and the relationship between channels;

[0060] Step 4: After the convolution result passes through the activation layer (using the Sigmoid activation function), the normalized weight value of each channel is obtained;

[0061] Step 5: Multiply the normalized weight value and the original feature map input into the lightweight attention mechanism module to generate a weighted feature map.

[0062] The constructed attention mechanism is as follows Figure 2 shown.

[0063] The maximum pooling layer (MaxPooL2D) performs feature dimensionality reduction processing on the weighted feature map to extract important features and discard irrelevant features. After multiple convolution operations, the model parameters are too large, and some features are useless or repeated. The pooling operation can reduce the number of parameters and capture important features. Each lightweight attention mechanism corresponds to a maximum pooling layer, and the window size in the maximum pooling layer is set to 2*2.

[0064] The concatenation and aggregation layer (Concatenate) concatenates and aggregates the output of the maximum pooling layer and then outputs it;

[0065] The flattening layer expands the output of the splicing and aggregation layer to obtain one-dimensional data and then outputs it;

[0066] The temporal convolutional neural network performs respiratory motion prediction processing based on the output of the flattening layer and outputs the predicted respiratory motion trajectory.

[0067] like Figure 1 As shown in Figure 1, the temporal convolutional neural network is composed of residual blocks connected in sequence. Figure 1 The figure shows three residual blocks connected in sequence. The specific number of residual blocks used is determined by the amount of original data, the computational efficiency and complexity of the model.

[0068] like Figure 3 The figure shows the residual block structure diagram, in which the basic convolution layer includes a causal convolution layer and a dilated convolution layer; the causal convolution layer is used to extract the local features of the sequence, and the dilated convolution layer is used to capture the long-term dependencies of the sequence.

[0069] The residual block is mainly composed of causal convolution, dilated convolution and residual connection. In the residual block, it is necessary to add an activation function ReLu after its two basic convolution layers to introduce nonlinear mapping, and apply weight normalization after each basic convolution layer. Finally, Dropout regularization is added after each basic convolution layer, and the Dropout regularization parameter is set to 0.3.

[0070] The flattening layer flattens the output predicted by the time series convolutional neural network module to obtain the predicted one-dimensional time series respiratory motion trajectory and output it;

[0071] For step S5, the respiratory motion prediction model is trained using the training set, the model parameters are adjusted using the validation set, and the respiratory motion prediction model is evaluated using the test set:

[0072] Repeatedly adjust the respiratory motion prediction parameters according to the validation set, and observe the MSE loss function curve. When the loss function curve tends to converge, obtain the final optimizer, training rounds and other parameters;

[0073] The experiment was conducted under the Tensorflow deep learning framework; the experimental equipment was Intel(R) Core(TM) i5-8300H CPU, graphics card NVIDIA GeForce GTX 105 and laboratory GPU service RTX3090 graphics card;

[0074] Using the test set of the respiratory motion prediction model to predict respiratory motion, obtaining the relevant error value between the predicted curve and the true curve, and completing the evaluation of the prediction model;

[0075] The prediction process has been tested, including the following:

[0076] Experiments on the respiratory motion prediction model at different delay times. The experimental results are as follows Figure 4 and Figure 5 As shown in Figure 2, the error indicators and the coefficient of determination (R-square) change with the delay time. (2) Experiments on the respiratory motion prediction model with different input sequences. The experimental results are shown in Figure 2. Figure 6 As shown in the figure, the mean absolute error (MAE) and root mean square error (RMSE) change with the input sequence. The milliseconds marked in the figure represent the delay time;

[0077] (3) Comparative experiment of respiratory motion prediction model with CNN_BiLSTM, DeepEST and other prediction models; the experimental results are as follows Figure 7 and Figure 8 As shown, Ours represents the respiratory motion prediction model in the present invention;

[0078] The respiratory motion prediction model of the present invention and other models are used to perform specific experiments to compare the predicted curves with the actual position curves. Fig. 9 As shown, the three models all selected data points between 2390 and 2450 from the data set to demonstrate the prediction effect. The dotted box clearly shows the comparison of the prediction effects between different models at the irregular peaks of individual respiratory signals.

[0079] In the above embodiments, the respiratory motion prediction method of the present invention has been extensively experimented with different delay times ranging from 100ms to 600ms, and compared with the two prediction methods of DeepEST and CNN_BiLSTM. The experimental results verify that the present invention effectively improves the prediction performance of respiratory signals in long-term time series prediction and irregular peaks. When the delay time is 450ms, the mean absolute error and root mean square error of the respiratory motion prediction model of the present invention are 0.027mm and 0.038mm, respectively, and the determination coefficient is 97.29%, which are 3.86% and 4.42% higher than those of CNN_BiLSTM and DeepEST, respectively.

Claims

1. A respiratory motion prediction method, characterized in that: The method comprises the following steps: S1, obtaining respiratory motion trajectory data; The respiratory motion trajectory data is the intrinsic mode function components of different frequencies obtained after modal decomposition of the collected human respiratory motion trajectory data; S2, preprocessing the respiratory motion trajectory data to obtain a preprocessed sample set; S3, dividing the sample set into a training set, a validation set and a test set; S4, constructing a respiratory movement prediction model, wherein the respiratory movement prediction model sequentially includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module and a flattening layer; In the two-dimensional multi-channel convolutional neural network module, the number of channels is determined by the feature type and size of the input data, the feature type of the input data refers to the type of frequency in the intrinsic mode function components of different frequencies obtained in step S1, one frequency type represents one feature, and convolution kernels of different sizes are used in each two-dimensional convolutional neural network; the two-dimensional multi-channel convolutional neural network module performs feature extraction on the input data, specifically including capturing local features of different time scales of the respiratory signal by designing different convolution kernel sizes, and extracting feature information of the respiratory signal by designing a suitable number of convolution channels according to the type and size of the local features, and sliding the feature information of the respiratory signal in the vertical direction to capture time correlation and learn the features of the respiratory signal at different time scales; S5. Use the training set to train the respiratory movement prediction model; use the validation set to adjust the model parameters; use the test set to evaluate the respiratory movement prediction model, and obtain the respiratory movement prediction result.

2. The respiratory motion prediction method according to claim 1, characterized in that: The preprocessing includes data screening, outlier removal, smoothing, noise reduction and normalization.

3. The respiratory motion prediction method according to claim 2, characterized in that: The steps to build the lightweight attention mechanism module are: Step 1: First aggregate the convolution features through the global average pooling layer and then adaptively determine the convolution kernel size , the calculation formula is as follows: ； ； Where: and is the parameter used to calculate the convolution kernel size, C represents the channel dimension, Indicates odd number operation; Step 2: Adjust the feature tensor obtained by global average pooling to a specified shape, where adjusting to the specified shape means making the dimension of the feature tensor the same as the dimension of the subsequent convolution operation; Step 3: Perform channel feature learning on the specified shape feature tensor, calculate the attention of each channel through a one-dimensional convolutional layer, and learn the importance of different channels and the relationship between channels; Step 4: After the convolution result passes through the activation layer, the normalized weight value of each channel is obtained; Step 5: multiply the normalized weight value and the original feature map input into the lightweight attention mechanism module to generate a weighted feature map; The lightweight attention mechanism module learns the weight coefficient of each convolution channel to capture the local to global features of the respiratory signal.

4. The respiratory motion prediction method according to claim 3, characterized in that: The maximum pooling layer performs feature dimensionality reduction processing on the weighted feature map, extracts important features and discards irrelevant features; Each lightweight attention mechanism in the lightweight attention mechanism module corresponds to a maximum pooling layer, and the window size in the maximum pooling layer is set to 2*2.

5. The respiratory motion prediction method according to claim 4, characterized in that: The splicing and aggregation layer performs splicing and aggregation processing on the output of the maximum pooling layer and then outputs it.

6. The respiratory motion prediction method according to claim 5, characterized in that: The flattening layer flattens the input data, converts the data into one dimension, and then outputs it.

7. The respiratory motion prediction method according to claim 6, characterized in that: The temporal convolutional neural network module is composed of residual blocks connected in sequence, wherein the number of residual blocks is artificially set according to actual needs, and the residual blocks include causal convolution, dilated convolution and residual connection; the temporal convolutional neural network module predicts the respiratory motion trajectory according to the input data and then outputs it.

8. A respiratory motion prediction system, characterized in that: The system comprises: Data acquisition module: acquiring respiratory motion trajectory data, wherein the respiratory motion trajectory data is the intrinsic mode function components of different frequencies obtained after modal decomposition of the collected human respiratory motion trajectory data; Data preprocessing module: preprocess the respiratory motion trajectory data to obtain a preprocessed sample set; Data partitioning module: partitioning the sample set into a training set, a validation set and a test set; Model construction module: construct a respiratory motion prediction model, which includes a two-dimensional multi-channel convolutional neural network module, a lightweight attention mechanism module, a maximum pooling layer, a splicing aggregation layer, a flattening layer, a temporal convolutional neural network module and a flattening layer in sequence; In the two-dimensional multi-channel convolutional neural network module, the number of channels is determined by the feature type and size of the input data, the feature type of the input data refers to the type of frequency in the intrinsic mode function components of different frequencies obtained by the data acquisition module, one frequency type represents one feature, and convolution kernels of different sizes are used in each two-dimensional convolutional neural network; the two-dimensional multi-channel convolutional neural network module performs feature extraction on the input data, specifically including capturing local features of the respiratory signal at different time scales by designing different convolution kernel sizes, and extracting feature information of the respiratory signal by designing a suitable number of convolution channels according to the type and size of the local features, and sliding the feature information of the respiratory signal in the vertical direction to capture time correlation and learn the features of the respiratory signal at different time scales; Respiratory movement prediction module: uses the training set to train the respiratory movement prediction model; uses the validation set to adjust the model parameters; uses the test set to evaluate the respiratory movement prediction model and obtains the respiratory movement prediction results.

Images