An optimization method for the body surface-tumor association model incorporating quasi-periodic motion constraints

Abstract

The present invention discloses an optimization method for a body surface-tumor association model incorporating quasi-periodic motion constraints, comprising the following steps: using a depth camera to collect chest and abdomen point cloud information, performing point cloud processing on the collected point cloud information to obtain a chest and abdomen voxel model, performing LLE dimensionality reduction processing on the voxel model to obtain a one-dimensional characterization vector, establishing an association model between the one-dimensional characterization vector and the tumor pose, selecting the best-matching one from three respiratory motion models, fitting the parameters of the respiratory motion model using a genetic algorithm, building an LSTM-CNN neural network structure, and obtaining the predicted tumor pose by bringing the test set data into the model after training the model with the training set data. The beneficial effects of the present invention are: it can effectively reduce the influence brought by the respiratory disturbance at the beginning and end of the patient's breathing, improve the association accuracy of the association model at the beginning and end of the breathing, and contribute to improving the association accuracy of the body surface-in vivo respiratory motion association model.

Description

Technical Field

[0001] The present invention relates to the technical field of respiratory tracking technology for precision radiotherapy robots, and specifically to an optimization method for a body surface-tumor association model integrating quasi-periodic motion constraints. Background Art

[0002] During the irradiation treatment of lung tumors by precision radiotherapy robots, an association model between the motion information of body surface marker points and the motion information of in-vivo tumors is established to predict the tumor pose without the aid of X-rays. However, the number of marker points placed on the chest and abdomen is limited and it is difficult to characterize the respiratory characteristics of the chest and abdomen. Therefore, the present invention extracts the respiratory characteristics of the chest and abdomen by collecting body surface point cloud information and establishing a body surface voxel model. Since the human body is prone to respiratory disturbances at the beginning and end of breathing, and the segmented association model established by traditional phase segmentation is prone to large fluctuations in the association results at the peaks and valleys, resulting in a decrease in the association accuracy of the association model in these two stages. Therefore, studying how to improve the association accuracy of the association model at the beginning and end of breathing is the key to improving the radiotherapy accuracy of radiotherapy robots.

[0003] The movement of lung tumors is mainly affected by the free breathing movement of patients. Therefore, the movement characteristics of tumors are largely consistent with the respiratory movement characteristics. The most significant feature of respiratory movement is its quasi-periodicity, that is, respiratory movement can be approximately regarded as a periodic movement based on time series. Therefore, the movement of tumors is affected by respiratory movement and its movement characteristics also have periodic characteristics. When establishing the association model between body surface-in-vivo movement information, the periodic respiratory movement model can be considered to be combined with the traditional association model, and the respective advantages of the two models can be taken for fusion to obtain an optimized association model.

[0004] After obtaining the two groups of models, how to effectively fuse them is also a key step. Commonly used and simple data fusion methods such as the weighted average method perform weighted averaging on the two groups of data to achieve data fusion. Although this method can process data in real time, it cannot formulate the weight allocation of data at different times and scenarios, and is greatly affected by noise. The present invention selects an artificial neural network. By defining suitable data inputs, the neural network can learn the feature associations between data, endowing the computer with the same thinking ability as humans, and achieving a better fusion effect.

[0005] Therefore, studying the effective fusion of the association model and the respiratory movement model is of great significance for improving the radiotherapy accuracy of precision radiotherapy robots. Summary of the Invention

[0006] The object of the present invention is to provide an optimization method for a body surface-tumor association model integrating quasi-periodic motion constraints to solve the problems raised in the above background art.

[0007] The technical solution of the present invention is: a method for optimizing a body surface-tumor association model integrating quasi-periodic motion constraints, comprising the following steps:

[0008] Step 1: Use a depth camera to collect chest and abdomen point cloud information;

[0009] Step 2: Perform point cloud processing on the collected point cloud information, including point cloud registration, segmentation, filtering and smoothing, to obtain a chest and abdomen voxel model;

[0010] Step 3: Perform LLE dimensionality reduction on the voxel model to obtain a one-dimensional feature vector of the body surface respiratory motion;

[0011] Step 4: Acquire the position of the simulated tumor using an electromagnetic tracker, and establish a correlation model with the one-dimensional feature vector of the body surface respiratory motion obtained in Step 3;

[0012] Step 5: construct error functions by combining the three candidate tumor respiratory motion models with the tumor motion information;

[0013] Step 6: Use genetic algorithm to minimize the error functions in step 5 respectively, obtain model parameters and select the model with the minimum error as the respiratory motion model;

[0014] Step 7: Build the LSTM-CNN neural network structure;

[0015] Step 8: After the model is trained using the training set data, the test set data is introduced into the model to obtain the predicted tumor position.

[0016] Further preferably, the point cloud acquisition method in step 1 is to use two Kinect v2 depth cameras to respectively acquire a set of point cloud information to ensure the acquisition accuracy of the point cloud information.

[0017] Further preferably, the point cloud processing in step 2 includes performing ICP registration on the two sets of point cloud information collected in step 1, and then performing point cloud segmentation, filtering, and smoothing processing.

[0018] Further preferably, the LLE dimensionality reduction process in step 3 is to first assign a minimum bounding box to the voxel model of each frame, where the length, width and height of the bounding box are sufficient to accommodate the voxel of the largest frame, and then take out the voxels in the same order and encode them as 0 and 1 to form a one-dimensional column vector x consisting of several sample points i (i=1, 2, ..., n), find the k nearest neighbor points x of each sample point based on the Euclidean distance ij (j=1, 2, ..., k), find the local covariance matrix Z of the sample point and its neighboring points i =(x i -x ij )(x i -xij ) T , and calculate the weight coefficient vector of the sample and its neighbor points. The calculation formula is as follows:

[0019]

[0020] Where Z i is the local covariance matrix between the sample point and its neighbor points, and I k is a column vector with k rows and all elements being 1.

[0021] Merge W i to obtain the weight coefficient matrix W. The calculation formula of the matrix is:

[0022] M = (I - W)(I - W) T ,

[0023] Calculate the first d + 1 eigenvalues of the matrix, and calculate the eigenvectors {ψ1, ψ2,..., ψ d+1}}. Select the eigenvector with the largest correlation with tumor motion as the one-dimensional eigenvector of the surface respiratory motion.

[0024] Further preferably, the three tumor respiratory motion models in step 5 are three classical models representing the information of quasi-periodic motion characteristics. The mathematical expressions of their error functions are respectively:

[0025]

[0026] Where c0 is the position of the tumor at the start of respiration, N is the order, a n and b n represent the amplitudes of the sine function and the cosine function respectively, and nω represents the respiratory period;

[0027]

[0028] Where c0 is the position of the tumor at the start of respiration, c1 reflects the change degree of the tumor position at different times, N is the order, A is the amplitude range, f i represents the respiratory period, and φ i represents the phase shift size;

[0029] z(t) = z0 - bcos 2n (πt / τ - φ),

[0030] Where Z0 is the position of the tumor at the start of respiration, b is the amplitude range, τ is the respiratory period, n is the parameter determining the steepness of the model shape, is the phase shift size.

[0031] Further preferably, the genetic algorithm in step 6 is an optimization method for minimizing errors. By constructing an error function based on the error between the respiratory motion model and the tumor motion and minimizing the error, the parameter values of the motion model are obtained. The algorithm first initializes the parameter values and re-encodes them, and then enters an iterative process of selection, crossover, and mutation of the encoded information. Finally, the parameter values that satisfy the global optimal solution are obtained. The error function is as follows:

[0032]

[0033] where F is the tumor respiratory motion model, y is the tumor motion information, and n is the data length.

[0034] Further preferably, the LSTM-CNN neural network structure in step 7 consists of two long short-term memory network units LSTM, four one-dimensional convolutional kernels Conv1D, one flattening layer Flatten, and two linear fully connected layers Dense.

[0035] Further preferably, the long short-term memory network unit LSTM is used to extract the hidden temporal features in the time series. Compared with a single-layer LSTM, a two-layer LSTM has better feature extraction ability, which helps to further extract the internal and external connections of the time series. The one-dimensional convolutional kernel Conv1D is used to extract the local features in the time series. Although LSTM can capture sequence features well, LSTM tends to capture the global features of the time series. Therefore, we further introduce a one-dimensional CNN structure. The flattening layer Flatten is used to remove the redundant one dimension as a transition layer between the convolutional layer and the fully connected layer. The linear fully connected layer Dense is used to map the feature space to the sample label space through a linear transformation to obtain the final tumor pose correlation value.

[0036] Further preferably, the training set in step 8 includes input data and label data. The input data includes the body surface-tumor association model obtained in step 4, the respiratory motion model obtained in step 6, and the difference between the results of the two models. The label data is the true value of the tumor motion. Among them, the training environment of the model is the open-source machine learning platform TensorFlow.

[0037] Further preferably, the evaluation index selected during the model training process of the training set data in step 8 is the mean absolute error (MAE), which is obtained by calculating the mean absolute error between the true value and the predicted value. The calculation formula is as follows:

[0038]

[0039] The loss function selected during the model training process of the training set data in step 8 is the mean square loss function (MSE), and its calculation formula is as follows:

[0040]

[0041] The advantages of the present invention are as follows: The method for optimizing the body surface-tumor association model based on the LSTM-CNN neural network disclosed by the present invention uses the LSTM-CNN neural network to fuse the association model and the periodic respiratory motion model. Compared with the traditional association model, the association effect of the model at the beginning and end of respiration is optimized, the prediction accuracy of the association model for the tumor pose is improved, the influence brought by the respiratory disturbance of the patient at the beginning and end of respiration can be effectively reduced by combining the front and back association information of respiratory motion, the association accuracy of the association model at the beginning and end of respiration is improved, and it helps to improve the association accuracy of the body surface-in vivo respiratory motion association model. Brief Description of the Drawings

[0042] The present invention will be further described below in conjunction with the drawings and embodiments:

[0043] Figure 1 It is the flowchart of the method for optimizing the body surface-tumor association model disclosed in the specific embodiment of the present invention;

[0044] Figure 2 It is the scene diagram of the information collected by the depth camera disclosed in the specific embodiment of the present invention;

[0045] Figure 3 It is the extraction process of the one-dimensional feature characterization quantity of the chest and abdomen body surface disclosed in the specific embodiment of the present invention;

[0046] Figure 4 It is the schematic diagram of the genetic algorithm disclosed in the specific embodiment of the present invention;

[0047] Figure 5 It is the composition diagram of the network structure disclosed in the specific embodiment of the present invention;

[0048] Figure 6 It is the comparison result diagram before and after the optimization of the association model disclosed in the specific embodiment of the present invention. Detailed Embodiments

[0049] Embodiment:

[0050] The following are specific embodiments of the present invention and in conjunction with the drawings, the technical solutions of the present invention will be further described, but the present invention is not limited to these embodiments.

[0051] As Figure 1 shown, a method for optimizing the body surface-tumor association model by fusing quasi-periodic motion constraints includes the following steps:

[0052] Step 1, Use two kinect v2 depth cameras to collect the point cloud information of the chest and abdomen, as Figure 2 shown;

[0053] Step 2: Perform point cloud processing on the acquired point cloud information and conduct ICP registration, including point cloud registration, segmentation, filtering, and smoothing processing, to obtain a chest and abdomen voxel model;

[0054] Step 3: Perform LLE dimensionality reduction processing on the voxel model to obtain a one-dimensional feature vector of the body surface respiratory movement. First, assign a minimum bounding box to each frame of the voxel model. The length, width, and height of the bounding box are such that they can accommodate the largest frame of voxels. Then, sequentially extract them in the same order and encode them as 0 and 1 to form a one-dimensional column vector x i (i = 1, 2,..., n). Based on the Euclidean distance, find the k nearest neighbor points x ij (j = 1, 2,..., k) of each sample point, and calculate the local covariance matrix Z of the sample point and its nearest neighbor points i =(x i -x ij )(x i -x ij ) T , and calculate the weight coefficient vector of the sample and its nearest neighbor points. The calculation formula is:

[0055]

[0056] where Z i is the local covariance matrix of the sample point and its nearest neighbor points, I k is a column vector with k rows and all elements being 1,

[0057] Merge W i , to obtain the weight coefficient matrix W. The calculation formula of the matrix is:

[0058] M=(I - W)(I - W) T ,

[0059] Calculate the first d + 1 eigenvalues of the matrix and calculate the eigenvectors {ψ1, ψ2,..., ψ d+1}. Select the one with the largest correlation with tumor movement in the eigenvectors as the one-dimensional feature vector of the body surface respiratory movement, as Figure 3 shown, which is the acquisition process of the one-dimensional feature vector;

[0060] Step 4: Obtain the pose of the simulated tumor through an electromagnetic tracker and establish an association model with the one-dimensional feature vector of the body surface respiratory movement obtained in Step 3;

[0061] Step 5: Construct error functions for three candidate tumor respiratory movement models and the tumor movement information respectively. The three tumor respiratory movement models are three classical models that characterize information with quasi-periodic motion characteristics. The mathematical expressions of their error functions are respectively:

[0062]

[0063] Among them, c0 is the position of the tumor at the start of respiration, N is the order, a n and b n represent the amplitudes of the sine function and the cosine function respectively, and nω represents the respiratory cycle;

[0064]

[0065] Among them, c0 is the position of the tumor at the start of respiration, c1 reflects the degree of change in the tumor position at different times, N is the order, A is the amplitude range, f i represents the respiratory cycle, and φ i represents the phase shift magnitude;

[0066] z(t) = z0 - bcos 2n (πt / τ - φ),

[0067] Among them, Z0 is the position of the tumor at the start of respiration, b is the range of the amplitude, τ is the respiratory cycle, n is the parameter determining the steepness of the model shape, is the phase shift magnitude;

[0068] To avoid overfitting of the model, the value of N in the first two of the above three mathematical expressions of the tumor respiratory motion model should be less than or equal to 4, and the value of n in the last mathematical expression should be less than or equal to 2;

[0069] Step 6, Use the genetic algorithm to minimize the error function in Step 5 respectively, obtain the model parameters and select the model with the minimum error as the respiratory motion model. This genetic algorithm is an optimization method for minimizing errors. As Figure 4 shown, it is the schematic diagram of the genetic algorithm. By constructing the error between the respiratory motion model and the tumor motion, an error function is established and the error is minimized to obtain the parameter values of the motion model. The algorithm first initializes the parameter values and re - encodes them, and then enters the iterative process of selection, crossover and mutation of the encoded information, and finally obtains the parameter values that meet the global optimal solution. This error function is:

[0070]

[0071] Among them, F is the tumor respiratory motion model, y is the tumor motion information, and n is the data length;

[0072] Step 7, Build the LSTM - CNN neural network structure. This LSTM - CNN neural network structure includes two long short - term memory network units LSTM, four one - dimensional convolutional kernels Conv1D, one flattening layer Flatten and two linear fully - connected layers Dense. As Figure 5 shown, it is the composition diagram of the neural network structure;

[0073] Step 8: After training the model with the training set data, use the test set data to input into the model to obtain the tumor prediction pose, that is, obtain the final correlation value. As shown in Figure 6 the figure, it is the comparison result diagram before and after the optimization of the correlation model; the evaluation index selected during the model training process of the training set data is the mean absolute error (MAE), which is obtained by calculating the mean absolute error between the true value and the predicted value. Its calculation formula is:

[0074]

[0075] The loss function selected during the model training process of the training set data in Step 8 is the mean square loss function (MSE), and its calculation formula is:

[0076]

[0077] In this application, the long short-term memory network unit LSTM is used to extract the hidden temporal features in the time series. Compared with a single-layer LSTM, a two-layer LSTM has better feature extraction ability, which helps to further extract the internal and external connections of the time series. At the same time, we set a smaller number of hidden layer units for the LSTM to make our model tend to be a narrow structure; the one-dimensional convolutional kernel Conv1D is used to extract the local features in the time series. Although the LSTM can capture the sequence features well, the LSTM tends to capture the global features of the time series. Therefore, we further introduce a one-dimensional CNN structure; the flattening layer Flatten is used to remove the redundant one dimension as the transition layer between the convolutional layer and the fully connected layer, and the linear fully connected layer Dense is used to map the feature space to the sample label space through a linear transformation to obtain the final tumor pose correlation value.

[0078] In this application, in Step 8, the training set includes input data and label data. The input data includes the body surface-tumor correlation model obtained in Step 4, the respiratory motion model obtained in Step 6, and the difference between the results of the two groups of models. The label data is the true value of tumor motion. Among them, the training environment of the model is the open-source machine learning platform TensorFlow. TensorFlow is an end-to-end open-source machine learning platform that has a comprehensive and flexible ecosystem including various tools, libraries, and community resources, which can allow researchers to promote the development of advanced technologies in the field of machine learning and enable developers to easily build and deploy applications supported by machine learning.

[0079] Of course, the above embodiments are only used to illustrate the technical concept and characteristics of the present invention, and their purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. All modifications made according to the spirit of the main technical solution of the present invention should be covered within the protection scope of the present invention.

Claims

1. An optimization method for the body surface-tumor association model incorporating quasi-periodic motion constraints, characterized in that: It includes the following steps: Step 1: Use a depth camera to collect chest and abdomen point cloud information; Step 2: Perform point cloud processing on the collected point cloud information, including point cloud registration, segmentation, filtering, and smoothing, to obtain a chest and abdomen voxel model; Step 3: Perform LLE dimensionality reduction on the voxel model to obtain a one-dimensional feature vector of the body surface respiratory movement. The LLE dimensionality reduction process is as follows: First, assign a minimum bounding box to the voxel model of each frame. The length, width, and height of the bounding box are set to accommodate the largest voxel frame. Then, extract and encode them as 0 and 1 in the same order successively to form a one-dimensional column vector composed of a number of sample points , and calculate the nearest neighbor points of each sample point based on the Euclidean distance , calculate the local covariance matrix between the sample point and its nearest neighbor points , and calculate the weight coefficient vector between the sample and its nearest neighbor points. The calculation formula is as follows: ， Among them, is the local covariance matrix of the sample point and its neighboring points, is a column vector with rows and all elements being 1. Merge , to obtain the weight coefficient matrix . The formula for calculating the matrix is: ， Calculate the first eigenvalues of the matrix, and calculate the eigenvectors , and select the one with the greatest correlation with tumor motion from the eigenvectors as the one-dimensional eigenvector of the surface respiratory motion; Step 4: Obtain the pose of the simulated tumor through an electromagnetic tracker and establish an association model with the one-dimensional feature vector of the surface respiration movement obtained in Step 3; Step 5: Construct error functions for each of the three candidate tumor respiration movement models and the tumor movement information. The three tumor respiration movement models are three classical models that characterize quasi-periodic movement characteristics information. The mathematical expressions of their error functions are respectively: ， Among them, is the position of the tumor at the start of breathing, is the order, and represent the amplitudes of the sine function and the cosine function respectively, represents the breathing cycle; ， Among them, is the position of the tumor at the start of breathing, reflects the degree of change in the tumor position at different times, is the order, is the amplitude range, represents the breathing cycle, represents the magnitude of the phase shift; ， Among them, is the position of the tumor at the start of breathing, is the range of the amplitude, is the breathing cycle, n is the parameter determining the steepness of the model shape, is the phase offset magnitude; Step 6: Use the genetic algorithm to minimize the error functions in Step 5 respectively, obtain the model parameters, and select the model with the smallest error as the respiration movement model; Step 7: Build an LSTM-CNN neural network structure; Step 8: After training the model with the training set data, use the test set data to input into the model to obtain the predicted pose of the tumor.

2. The optimization method of the body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 1, characterized in that: The point cloud acquisition method in Step 1 is to use two Kinect v2 depth cameras to collect a group of point cloud information respectively.

3. The optimization method of the body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 2, wherein: The point cloud processing in Step 2 includes performing ICP registration on the two groups of point cloud information collected in Step 1, and then performing point cloud segmentation, filtering, and smoothing.

4. The optimization method of the body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 1, characterized in that: The genetic algorithm in Step 6 is an optimization method for minimizing errors. An error function is established by constructing the error between the respiration movement model and the tumor movement, and the parameter values of the movement model are obtained by minimizing the error. The algorithm first initializes the parameter values and re-encodes them, and then enters an iterative process of selection, crossover, and mutation of the encoded information. Finally, the parameter values that satisfy the global optimal solution are obtained. The error function is: ， Among them, is a tumor respiratory motion model, is tumor motion information, and n is the data length.

5. The optimization method of the body surface-tumor association model integrating quasi-periodic motion constraints according to claim 1, characterized in that: The LSTM-CNN neural network structure in Step 7 consists of two long short-term memory network units LSTM, four one-dimensional convolutional kernels Conv1D, a flattening layer Flatten, and two linear fully connected layers Dense.

6. The optimization method of the body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 5, characterized in that: The long short-term memory network unit LSTM is used to extract the hidden temporal features in the time series. The one-dimensional convolutional kernel Conv1D is used to extract the local features in the time series. The flattening layer Flatten is used to remove the redundant one dimension as a transition layer between the convolutional layer and the fully connected layer. The linear fully connected layer Dense is used to map the feature space to the sample label space through a linear transformation to obtain the final tumor pose association value.

7. A method for optimizing a body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 1, characterized in that: The training set in Step 8 includes input data and label data. The input data includes the surface-tumor association model obtained in Step 4, the respiration movement model obtained in Step 6, and the difference between the results of the two models. The label data is the true value of the tumor movement.

8. An optimization method for a body surface-tumor association model incorporating quasi-periodic motion constraints according to claim 7, characterized in that: The evaluation index selected during the model training process of the training set data in Step 8 is the mean absolute error (MAE), which is obtained by calculating the mean absolute error between the true value and the predicted value. The calculation formula is: ； The loss function selected during the model training process of the training set data in Step 8 is the mean square loss function (MSE), and its calculation formula is: 。

Images