An intelligent breathing prediction method for spinal surgery

Abstract

The present invention provides an intelligent breathing prediction method for spinal surgery. The undulations of vertebral bodies are recorded by an infrared optical camera within a certain period of time. The six-dimensional motion is simplified by using the PCA (Principal Component Analysis) method, and a new database is created from the newly simplified low-dimensional motion, which is used for the training of the network. The training process is carried out according to the ELBO (Evidence Lower Bound) of the VAE (Variational Autoencoder) network, and at the same time, the training of the LSTM (Long Short-Term Memory) is also completed through this process. By performing low-pass filtering or Kalman filtering on the average value obtained from multiple parallel values output by multiple networks and the values in the sliding window, high-frequency noise is filtered out to obtain continuous and low-noise output values. The prediction convergence time of the present invention is controllable, has patient specificity, has small noise in the prediction output, and processes information relatively concisely.

Description

Technical Field

[0001] The present invention provides an intelligent breathing prediction method for spinal surgery, belonging to the field of biomedical engineering. Background Art

[0002] In actual clinical surgeries, due to micro-movements of intervertebral joints and vertebral body displacements caused by the patient's breathing, etc., the positioning deviation of the robot is an important reason for problems such as inaccurate positioning and extended surgical time in current robot-assisted spinal surgeries. On the other hand, different from human environmental understanding and interaction methods, the surgical robot system operates under the guidance of medical images and navigators and cannot predict respiratory movements. In clinical practice, it is common for the registered spatial information of the robot to change. Therefore, this core technology of respiratory movement prediction is an urgent problem to be solved in the clinical application of existing spinal surgery robot systems.

[0003] The respiratory movement prediction method for spinal surgery depends on the respiratory movement detection method. Usually, the detection methods of respiratory movement include: ventilator air flow detection method, epidermal paste type sensor detection method, X-ray detection method, etc. Among them, one of the most accurate vertebral position detection methods in spinal surgery is the detection method using an optical tracking frame fixed at the spinous process position. This detection method has been successfully used in spinal robot surgeries and belongs to an accurate and practical detection method.

[0004] Application No.: 201910852550.9 provides a respiratory movement prediction modeling method based on time series analysis, belonging to the field of biomedical engineering. The present invention calculates the prediction results of respiratory movement based on the autoregressive integrated moving average ARIMA model, the method based on STL decomposition and exponential smoothing ETS, and the neural network autoregressive NNAR model respectively, and defines model evaluation indexes based on RMSE and information entropy, which can select the prediction model parameters with the characteristics of both "small error" and "high stability", improving the scientific nature of the prediction method. Compared with the existing technology, the time length of respiratory movement prediction has been greatly improved, upgrading the previous short-term prediction model that usually only has a few hundred milliseconds to a long-term prediction model that can predict the amplitude of respiratory movement within dozens of seconds in the future. However, this method has the following disadvantages:

[0005] 1. Low accuracy, the epidermal paste method cannot meet the high-precision requirements;

[0006] 2. Different application scenarios, this patent is for radiotherapy and cannot be used on the spine. This patent is for vertebral body movements generated by respiratory movements, rather than simple respiratory movements

[0007] 3. The network is relatively complex and real-time performance cannot be guaranteed. The network of this patent is relatively lightweight and suitable for integration into robots;

[0008] 4. This patent does not consider the individual differences among patients. In clinical practice, this patent trains the model on-site for each patient during the operation, maintaining a certain degree of personalization.

[0009] CN 113813005 A provides a robot for excising spinal laminae. This robot is used to obtain the static reference plane of the spinal laminae and the spinal dynamic offset generated when a person performs respiratory movements. Based on the static reference plane, the dynamic reference plane of the reference laminae can be determined by combining the spinal offset. Then, based on the dynamic reference plane and the force acting on the manipulator, the target speed of the manipulator is jointly determined. Finally, the robotic arm moves at the target speed to drive the manipulator to mill the spinal laminae. This robot combines the user's force and the dynamic reference plane to jointly output the target speed of the manipulator, thereby enabling precise control of the cutting allowance during milling.

[0010] This patent involves a spinal motion prediction model, but its disadvantages are as follows:

[0011] 1. Different functions. The spinal offset involved in this patent uses methods such as Gaussian approximation functions to fit the position and time. This process can obtain the trend of respiratory movement changes, but cannot obtain the motion model of respiratory movement. This patent uses a neural network to fit the position at the previous moment (time period) and the position at the next moment (time period), and can obtain the motion model of respiratory movement.

[0012] 2. Insufficient accuracy. This spinal regression method uses R2 to calculate the fitting degree, and only selects one cycle of respiratory movement during the fitting process. This patent collects respiratory movement points for 1 - 3 minutes, divides the dataset, test set, and validation set on-site, and can obtain a more general and accurate model.

[0013] 3. Relatively redundant. This patent collects vertebral movements in all directions, but in this patent, the dimensionality reduction method is first used to obtain the main motion axis of vertebral movement. Then, regression is performed on the vertebral movement, changing the multi-variable to a single-variable, which improves the prediction efficiency.

[0014] Clinically, methods such as respiratory gating and Kalman filtering are usually adopted, achieving certain effects. However, these methods are all passive error compensation methods and have their own limitations respectively. In recent years, many scholars have proposed algorithms and mathematical models for respiratory waveform prediction, which can actively predict the future respiratory movement waveform of patients and adjust the robot pose in a timely manner according to the prediction results. A precise respiratory movement prediction model can, on the one hand, reduce the error caused by the patient's respiratory movement, and on the other hand, reduce the delay of the spinal robot itself.

[0015] Disadvantages of the prior art:

[0016] Uncontrollable prediction convergence time: The current solutions using Kalman filtering and extended Kalman filtering belong to adaptive methods. The main idea is to estimate the parameters of the Fourier series based on the current data. This solution relies on the parameter estimation function of Kalman filtering and is essentially an iterative optimization of parameters. The time for the parameters to converge in this estimation method is uncontrollable, and based on technology or theory, the time for the model to stabilize cannot be given.

[0017] Lack of patient specificity: The method using a large - data general - purpose model (pre - trained model) cannot guarantee that it can still stably track and predict vertebral body movement during surgery and does not have specificity for each patient.

[0018] Large noise in the prediction output: When traditional prediction methods using neural networks face raw data with large noise, the output also has large noise, and there are certain problems in training at this time.

[0019] Redundant information processing: Traditional methods use multi - modal and multi - dimensional prediction of vertebral body movement without considering that some directions are not the main components of movement. Therefore, the data processing is redundant. Summary of the Invention

[0020] In view of the above - mentioned technical problems, the present invention provides an intelligent breathing prediction method for spinal surgery, including the following steps:

[0021] Record the undulation of the vertebral body by an infrared optical camera within a certain period of time. The recorded data can be integrated into an initial database in the form of a sliding window - corresponding prediction point pair.

[0022] The six - dimensional movement of the vertebral body is recorded in the initial database. The PCA (Principal Component Analysis) method is used to reduce the dimension of the six - dimensional movement data, simplify it to 1 degree of freedom as the main movement direction, and project the six - dimensional movement onto the main movement axis; a new database is made using the projected one - dimensional movement data, and this database is used for the training of the breathing movement prediction network.

[0023]

[0024] The above formula is the PCA method, where X represents the coordinates of all sampled points, is the average value of the coordinates of all sampled points, n is the number of sampled points, and C ov is the covariance matrix; through SVD (Singular Value Decomposition), it is decomposed into several terms, and several eigenvalues and eigenvectors are found; in the formula, σ k is the eigenvalue, and v k is the eigenvector. For the largest eigenvalue, the corresponding eigenvector is retained as the main movement axis, and the original data is projected onto this axis to obtain the coordinates in the new database.

[0025] The respiratory motion prediction network structure is composed of a cascade of an LSTM (Long Short-Term Memory network), a VAE (Variational Autoencoder), a low-pass filter, or a Kalman filter;

[0026] By constructing an LSTM neural network, a linear network, and an activation network in software, and connecting the neural networks in series according to the sequence of LSTM - linear layer - activation layer - latent variable - linear layer - activation layer - linear layer - Tanh activation layer, the overall network architecture is obtained; the input of this neural network is 10 - 20 consecutive sampling points within a sliding window, and the output is a specific one of the 1 - 4 points after the sliding window;

[0027] The training process is carried out according to the ELBO of the VAE network, and the cost function of the network is as follows:

[0028]

[0029] where x represents the input sampling points, z represents the latent variable, p θ 、 represent random distributions. KL represents the KL divergence, E represents the expectation function, and β is the error backpropagation gain;

[0030] By paralleling multiple LSTM - VAE networks, the parallel values of several predicted points are output, and the standard deviation of the several parallel values is calculated to obtain the confidence region:

[0031]

[0032] where y i represents the predicted value output by the i-th network, represents the average of the sum of the predicted values output by all networks, c represents the number of parallel networks, and σ y represents the standard deviation of the predicted values output by multiple parallel networks, and the confidence region is [-2 * σ y , 2 * σ y .

[0033] By performing low-pass filtering or Kalman filtering on the average value obtained from the multiple parallel values output by multiple networks and all the values in the sliding window, the high-frequency noise in is filtered out to obtain a continuous and low-noise output value, which is the required predicted value.

[0034] The beneficial effects brought by the technical solution of the present invention:

[0035] 1. The prediction convergence time is controllable: After collecting the data, on-site training is carried out. The overall model does not need to be adaptively adjusted, and the confidence interval of the predicted value is given, and the overall prediction is relatively reliable.

[0036] 2. Specific to the patient: Trained separately using small-sample data, it can predict respiratory movement based on the patient's respiratory curve after anesthesia on the same day, stably track and predict vertebral movement, and is specific to each patient.

[0037] 3. Low noise in prediction output: When using the prediction method of neural network to face raw data with high noise, the output noise is eliminated by the filter to ensure the smoothness of the output of the respiratory prediction model.

[0038] 4. Streamlined information processing: Collect multi-dimensional movement data of the vertebral body, and perform projection and data compression of the main movement and main direction, which improves the processing efficiency without losing too much accuracy. Brief Description of the Drawings

[0039] Figure 1 is the flowchart of the present invention;

[0040] Figure 2 is the network structure diagram of the present invention;

[0041] Figure 3 is the overall architecture diagram of the present invention. Detailed Embodiment

[0042] Combine the drawings to illustrate the specific technical solutions of the present invention.

[0043] As Figure 1 shown in the overall process:

[0044] Record the undulation of the vertebral body within a certain period of time through an infrared optical camera, and the recorded data can be saved as a database in the input-output form.

[0045] Since the 6D movement of the vertebral body is recorded in this database, in order to simplify the movement variables, the PCA principal component analysis method is used to simplify the six-dimensional movement to 1-3 degrees of freedom. While obtaining the main direction (1-3 degrees of freedom), project the 6D movement onto the main movement axis. The newly simplified low-dimensional movement can be used to create a new database, which is used for the training of the respiratory movement prediction network;

[0046] The above formula is the PCA principal component analysis method. In the above formula, X represents the coordinates of all sampling points recorded, is the average value of the coordinates of all sampling points, n is the number of sampling points, and C ov is the covariance matrix; through SVD decomposition, it is decomposed into several terms, and several eigenvalues and eigenvectors are found; in the formula, σ k is the eigenvalue, and v k is the eigenvector; for the largest eigenvalue, retain the corresponding eigenvector as the main movement axis, and project the original data onto this axis to obtain the coordinates in the new database.

[0047] The respiratory motion prediction network structure is composed of a cascade of LSTM: Long Short-Term Memory Network, VAE: Variational Autoencoder, a low-pass filter or a Kalman filter;

[0048] Among them, the network structure is as Figure 2 shown.

[0049] By constructing an LSTM neural network, a linear network, and an activation network on software, and connecting the neural networks in series according to LSTM - linear layer - activation layer - latent variable - linear layer - activation layer - linear layer - Tanh activation layer, the overall network architecture is obtained; the input of this neural network is 10 - 20 consecutive sampling points within a sliding window, and the output is a specific 1 point among 1 - 4 points after the sliding window; the specific position can be configured through a program.

[0050] The training process is carried out according to the ELBO of the VAE network, and at the same time, the training of the LSTM is also completed through this process. The cost function of the network is as follows:

[0051]

[0052] Among them, x represents the input sampling points, z represents the latent variable, p θ , represents a random distribution; KL represents the KL divergence, E represents the expectation function, and β is the error backpropagation gain;

[0053] During the respiratory motion detection process, the traditional method cannot provide the confidence of the current prediction point. In the present invention, multiple LSTM-VAE networks are connected in parallel to output the parallel values of several prediction points, and the standard deviation of the several parallel values is obtained:

[0054]

[0055] Among them, y i represents the predicted value output by the i-th network, represents the average of the sum of the predicted values output by all networks, c represents the number of parallel networks, and σ y represents the standard deviation of the predicted values output by multiple parallel networks, and the confidence region is [-2*σ y , 2*σ y ;

[0056] In addition, since VAE or LSTM cannot guarantee the filtering of noise in the training set during the output process, and there will inevitably be relatively large noise in the network during training, these noises will cause the instability of the network output value. By performing low-pass filtering (or Kalman filtering) on the average value obtained from the multiple parallel values output by multiple networks and the values in the sliding window to filter out high-frequency noise, a continuous and low-noise output value can be obtained. The overall architecture is asFigure 3 as shown

Claims

1. An intelligent breathing prediction method for spinal surgery, characterized in that, Including the following steps: Record the vertebral body undulation within a certain period of time through an infrared optical camera, and the recorded data can be integrated into an initial database in the form of point pairs of a sliding window - corresponding prediction points; What is recorded in the initial database is the six - dimensional motion of the vertebral body. The PCA (Principal Component Analysis) method is used to reduce the dimension of the six - dimensional motion data, simplify it to 1 degree of freedom as the main motion direction, and project the six - dimensional motion onto the main motion axis; A new database is made using the projected one - dimensional motion data, and this database is used for the training of the respiratory motion prediction network; The respiratory motion prediction network structure is composed of a cascade of LSTM (Long Short - Term Memory network), VAE (Variational Auto - Encoder), a low - pass filter or a Kalman filter; By constructing an LSTM neural network, a linear network, and an activation network on software, connect the neural networks in series according to LSTM - linear layer - activation layer - latent variable - linear layer - activation layer - linear layer - Tanh activation layer to obtain the overall network architecture; The input of this neural network is 10 - 20 consecutive sampling points within a sliding window, and the output is a specific 1 point among 1 - 4 points after the sliding window; The training process is carried out according to the ELBO of the VAE network, and the cost function of the network is as follows: where x represents the input sampling points, z represents the latent variable, and p θ , represents a random distribution; KL represents the KL divergence, E represents the expectation function, and β is the error backpropagation gain; Through multiple parallel LSTM - VAE networks, output the parallel values of several prediction points, and calculate the standard deviation of the several parallel values to obtain the confidence region: where y i represents the predicted value of the i-th network output, represents the average of the sum of all network output predicted values, c represents the number of parallel networks, σ y represents the standard deviation of the predicted values of the outputs of multiple parallel networks, and the confidence region is [-2*σ y , 2*σ y ; The average value obtained by calculating multiple parallel values output by multiple networks Perform low-pass filtering or Kalman filtering on all values in the sliding window to filter out the high-frequency noise in it to obtain a continuous and low-noise output value, which is the predicted value sought 2. The intelligent respiration prediction method for spinal surgery according to claim 1, wherein The PCA (Principal Component Analysis) method is as follows: In the above formula, X represents the coordinates of all recorded sampling points, is the average value of the coordinates of all sampling points, n is the number of sampling points, and C ov is the covariance matrix; through SVD decomposition, it is decomposed into several terms, and several eigenvalues and eigenvectors are found; in the formula, σ k is the eigenvalue, and v k is the eigenvector; for the largest eigenvalue, the corresponding eigenvector is retained as the main motion axis, and the original data can be projected onto this axis to obtain the coordinates in the new database.

Images