Active motion compensation method and system for flexible brain electrode implantation

Abstract

The application provides a kind of active motion compensation method and system for flexible brain electrode implant, comprising: step S1: collecting flexible brain electrode image;Step S2: estimate the three-dimensional coordinates of flexible brain electrode implant target;Step S3: based on the prediction result of the three-dimensional coordinates and the estimated time delay, further predict the three-dimensional motion of moving target, obtain motion prediction trajectory;Step S4: make sensor coordinate system conversion to robot, and then make robot adjust angle and position according to motion prediction trajectory.The application realizes active motion compensation for flexible brain electrode implant device under physiological motion, and through active compensation, more than 96% of relative motion between target and instrument can be reduced.

Description

Technical Field

[0001] This invention belongs to the field of neuroengineering technology, specifically relating to an active motor compensation method and system for flexible brain electrode implantation. More specifically, it is a system for active motor compensation of tissue physiological movement for flexible brain electrode implantation. Background Technology

[0002] Flexible neural electrodes, with their excellent biocompatibility, are considered a key pathway to achieving long-term stable brain-computer interfaces. Their low modulus properties can significantly reduce the chronic pressure and inflammatory response of traditional rigid electrodes on fragile brain tissue, and significantly extend the functional lifespan of implanted devices. However, brain tissue is not a static environment; respiratory rhythms, heartbeats, muscle contractions, and even daily head movements all cause continuous dynamic displacements of brain tissue at the micrometer to millimeter level.

[0003] This dynamic characteristic places higher demands on the precision and stability of flexible electrode implantation. Failure to adequately consider the physiological movement of brain tissue during surgery can easily lead to electrode implantation failure or even greater brain damage. Traditional surgical methods for dynamic tissues mostly use mechanical fixation to reduce tissue movement and thus simplify the procedure. However, this approach can easily cause greater intraoperative damage, especially when dealing with highly sensitive tissues such as the brain. Forcibly restricting its natural movement may affect cerebral blood flow, potentially inducing irreversible neurological damage.

[0004] With the development of robot-assisted surgery, an increasing number of methods for actively compensating for relative motion between robots and tissues are being proposed to achieve relative stillness between the tissue and the instrument. This method can handle more flexible tissue movements. For the task of implanting flexible brain electrodes, given the physiological movements caused by cerebral pumping after craniotomy, there is an urgent need to develop high-precision and robust active compensation algorithms to achieve precise electrode implantation.

[0005] Patent document CN111067622B discloses a respiratory motion compensation method for percutaneous lung puncture. The disadvantage of this method is that it cannot perform motion compensation based on real-time images, and the compensation accuracy is not high. It does not take into account multiple factors such as time delay and motion prediction.

[0006] Patent document CN118177968A discloses a target calibration method and system for intracranial electrodes, and a surgical navigation method and system. However, this method cannot maintain the stability of movement within the field of view.

[0007] This problem urgently needs to be solved. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide an active motion compensation method and system for flexible brain electrode implantation.

[0009] An active motor compensation method for flexible brain electrode implantation provided by the present invention includes: Step S1: Acquire images of flexible brain electrodes; Step S2: Estimate the three-dimensional coordinates of the flexible brain electrode implantation target; Step S3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Step S4: Transform the sensor coordinate system to the robot, and then let the robot adjust its angle and position according to the motion prediction trajectory.

[0010] Preferably, in step S1, images of the flexible brain electrodes are acquired using a camera, and the time on the camera screen is interpolated with the time on the captured image to obtain the sensor delay. ; and thus through delay Compensation sensor; In step S3, a neural network is used to predict the three-dimensional motion of the moving target to obtain the motion prediction trajectory.

[0011] Preferably, in step S3, neural network prediction is used, including: Step S3.1: Collect observation data from historical windows; Step S3.2: Construct a prediction model based on a neural network; Step S3.3: Train the prediction model using the loss function; The mathematical expression for the observation data of the historical window is:

[0012] in, Historical observation data Indicates the first One observation data, ∈ (1, ), It is a constant. Represents real numbers, superscript Indicates the spatial dimension of motion; The mathematical expression for the prediction model is:

[0013] in, Indicates the future The trajectory of the step. Represents a neural network model; The mathematical expression for the loss function is:

[0014] in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; Represents the predicted future T + h Motion at all times Indicates actual T + h Motion at all times This represents the historical window used for prediction.

[0015] Preferably, the periodic values ​​of physiological movements are collected and then used for the prediction of historical windows; the mathematical expression for the prediction of historical windows is:

[0016] in, Indicates the frequency of physiological movements; Indicates the historical window used for prediction, symbol This indicates rounding up to the nearest integer.

[0017] Preferably, in step S4, the sensor coordinate system is transformed to robot coordinates, and the mathematical expression is:

[0018] in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In step S4, the robot's motion is compensated using a motion compensation algorithm; the mathematical expression for the motion compensation algorithm is:

[0019] in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

[0020] An active motion compensation system for flexible brain electrode implantation provided by the present invention includes: Module M1: Acquires images from flexible brain electrodes; Module M2: Estimates the three-dimensional coordinates of the target site for flexible brain electrode implantation; Module M3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Module M4: Transforms the sensor coordinate system to the robot, enabling the robot to adjust its angle and position based on the predicted motion trajectory.

[0021] Preferably, in module M1, images of the flexible brain electrodes are acquired by a camera, and the time on the camera screen is interpolated with the time on the captured image to obtain the sensor delay. ; and thus through delay Compensation sensor; In module M3, a neural network is used to predict the three-dimensional motion of the moving target to obtain the motion prediction trajectory.

[0022] Preferably, in module M3, neural network prediction is employed, including: Module M3.1: Collects observation data from historical windows; Module M3.2: Constructing a prediction model based on neural networks; Module M3.3: Trains the prediction model using a loss function; The mathematical expression for the observation data of the historical window is:

[0023] in, Historical observation data Indicates the first One observation data, ∈ (1, ), It is a constant. Represents real numbers, superscript Indicates the spatial dimension of motion; The mathematical expression for the prediction model is:

[0024] in, Indicates the future The trajectory of the step. Represents a neural network model; The mathematical expression for the loss function is:

[0025] in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; Represents the predicted future T + h Motion at all times Indicates actual T + h Motion at all times This represents the historical window used for prediction.

[0026] Preferably, the periodic values ​​of physiological movements are collected and then used for the prediction of historical windows; the mathematical expression for the prediction of historical windows is:

[0027] in, Indicates the frequency of physiological movements; Indicates the historical window used for prediction, symbol This indicates rounding up to the nearest integer.

[0028] Preferably, in module M4, the sensor coordinate system is transformed to robot coordinates, and the mathematical expression is:

[0029] in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In module M4, a motion compensation algorithm is used to compensate for the robot's motion; the mathematical expression of the motion compensation algorithm is:

[0030] in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

[0031] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention comprehensively considers multiple issues related to motion compensation for dynamic tissues during flexible brain electrode implantation surgery, including motion estimation, delay measurement, motion prediction, and coordinate system registration.

[0032] 2. This invention enables active motion compensation of flexible brain electrode implantation devices under physiological movement.

[0033] 3. This invention can reduce the relative motion between the target and the instrument by more than 96% through active compensation. Attached Figure Description

[0034] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic diagram of three-dimensional motion estimation provided by the present invention; Figure 2 This is a schematic diagram of the process provided by the present invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0036] This invention proposes an active motion compensation algorithm for flexible brain electrode implantation. This method measures the real-time physiological motion of the patient's implantation target, analyzes the motion patterns using intelligent algorithms, calculates the compensation amount, and dynamically adjusts the position and angle of the implantation device. This invention can significantly improve the placement accuracy of the implantation device.

[0037] Specifically, the physiological movements include cardiac movements and respiratory movements.

[0038] An active motor compensation method for flexible brain electrode implantation provided by the present invention includes: Step S1: Acquire images of flexible brain electrodes; Step S2: Estimate the three-dimensional coordinates of the flexible brain electrode implantation target; Step S3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Step S4: Transform the sensor coordinate system to the robot, and then let the robot adjust its angle and position according to the motion prediction trajectory.

[0039] Specifically, in step S1, images of the flexible brain electrodes are acquired using a camera, and the time on the camera screen is interpolated with the time on the captured image to obtain the sensor delay. ; and thus through delay Compensation sensor; In step S3, a neural network is used to predict the three-dimensional motion of the moving target to obtain the motion prediction trajectory.

[0040] Specifically, in step S3, neural network prediction is employed, including: Step S3.1: Collect observation data from historical windows; The mathematical expression for the observation data of the historical window is:

[0041] in, Historical observation data Indicates the first One observation data, ∈ (1, ), It is a constant. Represents real numbers, superscript Indicates the spatial dimension of motion; Step S3.2: Construct a prediction model based on a neural network; The mathematical expression for the prediction model is:

[0042] in, Indicates the future The trajectory of the step. Represents a neural network model; Step S3.3: Train the prediction model using the loss function; The mathematical expression for the loss function is:

[0043] in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; Represents the predicted future T + h Motion at all times Indicates actual T + h Motion at all times This represents the historical window used for prediction.

[0044] Specifically, the periodic values ​​of physiological movements are collected and then used for predictive historical windows; the mathematical expression for the predicted historical window is:

[0045] in, Indicates the frequency of physiological movements; Indicates the historical window used for prediction, symbol This indicates rounding up to the nearest integer.

[0046] Specifically, in step S4, the sensor coordinate system is transformed to robot coordinates, and the mathematical expression is:

[0047] in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In step S4, the robot's motion is compensated using a motion compensation algorithm; the mathematical expression for the motion compensation algorithm is:

[0048] in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

[0049] The present invention also provides an active motor compensation system for flexible brain electrode implantation. The active motor compensation system for flexible brain electrode implantation can be implemented by executing the process steps of the active motor compensation method for flexible brain electrode implantation. That is, those skilled in the art can understand the active motor compensation method for flexible brain electrode implantation as a preferred embodiment of the active motor compensation system for flexible brain electrode implantation.

[0050] An active motion compensation system for flexible brain electrode implantation provided by the present invention includes: Module M1: Acquires images from flexible brain electrodes; Module M2: Estimates the three-dimensional coordinates of the target site for flexible brain electrode implantation; Module M3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Module M4: Transforms the sensor coordinate system to the robot, enabling the robot to adjust its angle and position based on the predicted motion trajectory.

[0051] In module M3, neural network prediction is employed, including: Module M3.1: Collects observation data from historical windows; Module M3.2: Constructing a prediction model based on neural networks; Module M3.3: Trains the prediction model using a loss function; In module M4, the sensor coordinate system is transformed to robot coordinates, and the mathematical expression is:

[0052] in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In module M4, a motion compensation algorithm is used to compensate for the robot's motion; the mathematical expression of the motion compensation algorithm is:

[0053] in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

[0054] Physiological Motion Measurement: Flexible brain electrode implantation, as an invasive procedure, requires opening the skull and dura mater during surgery to precisely implant the electrodes into specific brain regions. Studies have shown that after the skull is opened, the brain experiences significant pulsation or displacement due to cerebral pulse and blood circulation, which may affect the accuracy and stability of electrode implantation. In other surgeries, various high-precision sensors are typically used to accurately measure this physiological motion, such as tissue-embedded ultrasound crystals, ultrasound images, and endoscopic images. However, for high-precision surgeries like flexible brain electrode implantation, especially within the complex and dynamic physiological environment of the brain, utilizing endoscopy as a tool for measuring tissue motion is particularly important.

[0055] The camera mounted on the surgical instrument in the endoscope can acquire real-time images of the surgical area from different angles, avoiding occlusion issues. Through image matching and depth calculation, the endoscope generates three-dimensional spatial information, thereby locating the target tissue and assessing its movement. This method provides high-resolution dynamic imaging, which is an important technical support for precise brain electrode implantation surgery. Specifically, the three-dimensional position of the implantation site (POI) is calculated by a triangulation algorithm using binocular stereo vision or by calculating disparity using existing binocular matching algorithms, and then obtained based on binocular calibration parameters.

[0056] Furthermore, since the measured physiological motion data usually has a certain time delay, especially when it involves rapidly changing physiological processes, the delay can affect the accuracy and real-time performance of electrode implantation.

[0057] Therefore, to achieve accurate compensation for physiological movements, a detailed analysis of the characteristics of these movements is necessary to identify their patterns and influencing factors. By combining this with time-delay analysis, the temporal relationship between measured data and actual movement can be clarified, allowing predictive algorithms to estimate future physiological movements. These algorithms process historical movement data and real-time feedback to build a movement model and predict the physiological movement trajectory over a future period.

[0058] Considering the advantages of neural networks in modeling nonlinear mappings and time-series signals, this invention employs neural networks for physiological motion prediction. Specifically, given a period of historical motion observation data...

[0059] Where d is the spatial dimension of motion, typically 3D, and the predictive model aims to estimate the future. The trajectory of the step;

[0060] in, This represents a neural network model. The model is trained by minimizing the error between the predicted trajectory and the true trajectory. The commonly used loss function is the mean squared error, or MSE for short.

[0061] in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; This refers to the predicted future motion, i.e., the predicted future. T + h Motion in constant motion; Indicates actual future motion, i.e., actual T + h Motion in constant motion.

[0062] In other words, because the physiological movements of tissues possess certain amplitude-frequency characteristics, a recurrent neural network is used to learn these characteristics from historical data. Furthermore, a historical window for prediction is provided based on the periodic values ​​of the physiological movements. This is to ensure the effective use of information and the efficiency of computation.

[0063] The core of motion compensation is maintaining a constant relative distance between the control device and the tissue. By combining predicted physiological motion values ​​with the coordinate system transformation relationship between the sensor and the robot, the motion compensation algorithm obtains the robot's desired position and controls the robot to track the target trajectory, thus achieving relative stillness between the robot and the tissue.

[0064] Time delay analysis: Since various sensors have a certain delay during data acquisition, in order to align the acquired data in motion prediction, it is necessary to estimate all delays in acquisition and communication in advance and align the acquired data on the time axis for subsequent analysis.

[0065] Delay analysis is performed by interpolating the time displayed on the screen with the time shown in the captured image using a camera, thus obtaining the current sensor delay. Specifically, the delay is used to determine the prediction window length h for predicting future motion. The delay represents the actual time of the current observation. -l, then the prediction length window for future motion is h = -l+1, which predicts the next moment from the current moment. This is used to obtain the control value for the next moment in the prediction model described above.

[0066] Prediction Algorithm: Physiological motion prediction algorithm refers to the use of mathematical models and computer algorithms to predict possible motion changes in human tissues during surgery by analyzing and processing real-time physiological data.

[0067] Physiological motion prediction algorithms are typically based on the patient's physiological parameters and the tissue's historical motion information during the surgical procedure. By establishing a mathematical model of physiological motion and combining technologies such as machine learning and deep learning, the prediction results are updated in real time.

[0068] The physiological parameters include heart rate, respiratory rate, and electromyography.

[0069] Predictive algorithms can simulate a patient's physiological responses during surgery, identify movement patterns, and provide predictions of movement trends. Existing physiological movement prediction algorithms include: A. Regression Model: This model uses regression analysis of historical data to predict how certain physiological parameters will change during surgery. It is often used to predict simple linear movement patterns.

[0070] B. Time series analysis: Based on past physiological data, time series methods, such as the ARIMA model, are used to predict the movement status in the future, which is suitable for dynamic changes.

[0071] C. Neural Networks and Deep Learning: Utilizing neural networks, especially deep learning models such as Long Short-Term Memory (LSTM), to process large-scale physiological data with complex patterns, thereby achieving high-precision prediction of complex motion patterns.

[0072] D. Kalman filtering: Commonly used in real-time systems to smooth noisy physiological data and predict the future state of the system. It is suitable for real-time correction during surgery.

[0073] Sensor-robot coordinate system transformation matrix calibration: Since it is necessary to transform the predicted physiological motion information into the robot's coordinate system to obtain the robot's control commands, hand-eye calibration is required before surgery to obtain the transformation relationship between the camera and robot coordinate systems.

[0074] Based on the transformation relationship, the robot can automatically transform the predicted physiological movements from the camera's coordinate system to the robot's control coordinate system, thereby controlling the robot to actively track. The hand-eye calibration process usually requires the use of a calibration board, and the transformation relationship between the coordinate systems is obtained by optimizing the positional relationship between multiple sets of images and the corresponding robots.

[0075] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0076] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An active motor compensation method for flexible brain electrode implantation, characterized in that, include: Step S1: Acquire images of flexible brain electrodes; Step S2: Estimate the three-dimensional coordinates of the flexible brain electrode implantation target; Step S3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Step S4: Transform the sensor coordinate system to the robot's coordinate system, and then allow the robot to adjust its angle and position according to the predicted motion trajectory; In step S1, images of the flexible brain electrodes are acquired using a camera, and the time on the camera screen is interpolated with the time on the captured image to obtain the sensor delay. ; and thus through delay Compensation sensor; In step S3, a neural network is used to predict the three-dimensional motion of the moving target to obtain the motion prediction trajectory; In step S3, neural network prediction is used, including: Step S3.1: Collect observation data from historical windows; Step S3.2: Construct a prediction model based on a neural network; Step S3.3: Train the prediction model using the loss function; The mathematical expression for the observation data of the historical window is: in, Historical observation data Indicates the first One observation data, , It is a constant. Represents real numbers, superscript Indicates the spatial dimension of motion; The mathematical expression for the prediction model is: in, Indicates the future The trajectory of the step. Represents a neural network model; The mathematical expression for the loss function is: in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; Represents the predicted future T + h Motion at all times Indicates actual T + h Motion at all times This represents the historical window used for prediction.

2. The active motion compensation method for flexible brain electrode implantation according to claim 1, characterized in that, The periodic values ​​of physiological movements are collected and then used for predictive historical windows; the mathematical expression for the predicted historical window is: in, Indicates the frequency of physiological movements; Indicates the historical window used for prediction, symbol This indicates rounding up to the nearest integer.

3. The active motor compensation method for flexible brain electrode implantation according to claim 1, characterized in that, In step S4, the sensor coordinate system is transformed to the robot coordinate system, and the mathematical expression is: in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In step S4, the robot's motion is compensated using a motion compensation algorithm; the mathematical expression for the motion compensation algorithm is: in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

4. An active motor compensation system for flexible brain electrode implantation, characterized in that, include: Module M1: Acquires images from flexible brain electrodes; Module M2: Estimates the three-dimensional coordinates of the target site for flexible brain electrode implantation; Module M3: Based on the prediction results of the three-dimensional coordinates and the estimated time delay, the three-dimensional motion of the moving target is predicted to obtain the motion prediction trajectory; Module M4: Transforms the sensor coordinate system to the robot's coordinate system, enabling the robot to adjust its angle and position based on the predicted motion trajectory; In module M1, images of the flexible brain electrodes are acquired via a camera, and the time on the camera screen is interpolated with the time on the captured image to obtain the sensor delay. ; and thus through delay Compensation sensor; In module M3, a neural network is used to predict the three-dimensional motion of the moving target to obtain the motion prediction trajectory; In module M3, neural network prediction is employed, including: Module M3.1: Collects observation data from historical windows; Module M3.2: Constructing a prediction model based on neural networks; Module M3.3: Trains the prediction model using a loss function; The mathematical expression for the observation data of the historical window is: in, Historical observation data Indicates the first One observation data, , It is a constant. Represents real numbers, superscript Indicates the spatial dimension of motion; The mathematical expression for the prediction model is: in, Indicates the future The trajectory of the step. Represents a neural network model; The mathematical expression for the loss function is: in, This represents the mean squared error as a loss function; Indicates the number of steps; Indicates the ordinal number of the current step; Represents the predicted future T + h Motion at all times Indicates actual T + h Motion at all times This represents the historical window used for prediction.

5. The active motor compensation system for flexible brain electrode implantation according to claim 4, characterized in that, The periodic values ​​of physiological movements are collected and then used for predictive historical windows; the mathematical expression for the predicted historical window is: in, Indicates the frequency of physiological movements; Indicates the historical window used for prediction, symbol This indicates rounding up to the nearest integer.

6. The active motion compensation system for flexible brain electrode implantation according to claim 4, characterized in that, In module M4, the sensor coordinate system is transformed to robot coordinates, and the mathematical expression is: in, This indicates the target's position in the sensor coordinate system. This indicates the position of the target in the robot's coordinate system. and This represents the rotation and translation transformation matrix obtained through hand-eye calibration between two coordinate systems; In module M4, a motion compensation algorithm is used to compensate for the robot's motion; the mathematical expression of the motion compensation algorithm is: in, Indicates the target position at time zero. Indicates in The target location at any given time Indicates in The target location of the robot at any given time.

Images