METHOD FOR CALIBRATION AND DETERMINATION OF POSITION DATA OF AN INERTITAL MEASURING UNIT, TRAINING SYSTEM AND MEDICAL INSTRUMENT WITH AN INERTITAL MEASURING UNIT

Description

Technical field

[0001] The present disclosure relates to a method for training and calibrating and for predicting or determining at least one position or position data of an inertial measurement unit. Furthermore, the present disclosure relates to a training system comprising an inertial measurement unit and a computer-readable storage medium according to the preambles of the dependent claims.

[0002] During endoscopic surgery, navigation within a patient's body presents a significant challenge. Optical navigation systems with optical tracking, for example, have limited visibility and lose their ability to determine position or orientation if the line of sight to the tracked object is interrupted. Furthermore, limitations regarding adequate ergonomics exist with the instruments used, such as those caused by bulky trackers. Consequently, optical navigation systems are poorly suited for use in endoscopic surgery.

[0003] Efforts are underway to utilize inertial measurement units (IMUs) for navigation in medical technology. Inertial measurement units offer the crucial advantage of not causing visual obstructions and can be integrated into various systems or products without significantly impacting their ergonomics.

[0004] Inertial measurement units (IMUs) still exhibit significant drawbacks that currently prevent or at least severely complicate their use in surgical environments. While IMUs allow for the measurement of acceleration and orientation or rotation rates, they do not provide global, absolute position information. According to mathematical theory, the relative position of the IMU relative to an initial / starting position can be calculated from the acceleration data measured by the IMU by integrating twice. However, calculating the position from acceleration data through integration is highly sensitive to measurement errors, as these errors propagate and even amplify over time. A small error at the beginning of the measurement is amplified along the IMU's path.It has therefore been shown that the double integration of acceleration data leads to significant errors in position determination over time. Furthermore, orientation data is also dependent on sensor calibration and other interfering factors. These factors make achieving high accuracy in orientation measurements difficult.

[0005] In medical technology, cost and size play a crucial role. A large and relatively precise inertial measurement unit, such as those used in aviation, is unsuitable for surgical procedures. Semiconductor sensors are particularly attractive as inertial measurement units because they are small and inexpensive. However, acceleration-based position and velocity estimates from inexpensive sensors are generally very prone to error. This is partly because the sensor orientation must be determined very precisely to distinguish the effects of gravity from the actual physical acceleration of the sensors. Even small errors in the orientation estimation lead to large errors in the measured acceleration. These errors, through double integration, result in even larger errors in the position estimation.

[0006] Without additional external sensors, it is often impossible to make an accurate position estimate. The accelerometers measure both physical acceleration and the influence of normal forces caused by gravity. Therefore, the motion data inevitably includes the influence of gravity. Furthermore, the integration is performed using a finite time step, which also requires a precisely determined time measurement.

[0007] Since determining position solely by integrating motion data from an IMU does not provide sufficient accuracy, the state of the art suggests supplementing the IMU's motion data with additional data.

[0008] It is known from the prior art to use algorithms as filters to calibrate or smooth data. For example, current research from a different field has shown that a vehicle can be equipped with an IMU (Integrated Measurement Unit). The IMU records motion data while the vehicle is driving. Simultaneously, the vehicle's (global) position is tracked via GPS, resulting in redundant data acquisition. It is also known from the prior art to use Kalman filters for estimating and smoothing IMU data.

[0009] From "Calibration of IMUs using Neural Networks and Adaptive Techniques" by Claesson et al., a calibration method for an IMU using a neural network is described. For a training dataset, acceleration data from the IMU is measured by an accelerometer. A robot is used to precisely align the IMU. This precise alignment allows the influence of gravity to be factored out of the measured values, thus determining the "true" acceleration. The training dataset therefore consists of the measured acceleration as input and the true acceleration as output.

[0010] However, there is currently no known way to individually calibrate an IMU and to calculate, determine, or predict a precise position of an IMU in space. Summary of Revelation

[0011] The purpose of this disclosure is to avoid or at least reduce the disadvantages of the prior art and, in particular, to provide a method, a training system, and a computer-readable storage medium for determining at least the position of an IMU in space as precisely as possible based on measured motion data. A further purpose is to determine, without specifying mathematical formulas, a link between measured motion data, especially measured acceleration and measured orientation, and at least position data of the IMU.

[0012] These problems of the present disclosure are solved according to the invention with respect to a method by the features of claim 1, according to the invention with respect to a training system by the features of claim 9, and according to the invention with respect to a computer-readable storage medium by the features of claim 10.

[0013] A fundamental idea behind this disclosure is to use an AI system (Artificial Intelligence system) to establish a link between the measured (raw) motion data from the inertial measurement unit (IMU), particularly the measured acceleration data and the measured rotation rates or orientation, and at least position data. The AI ​​system is trained accordingly for this purpose. The trained AI system is then able to act as a kind of converter between input and output, transforming the measured motion data as input into (at least) a precise position indication, or into precise position data from the IMU as output, and thus at least predicting the position. This achieves a kind of calibration of the IMU, which can be individually configured and does not require any specific predefined mathematical formulas.

[0014] The problems of the present disclosure are thus solved with regard to a method for calibrating and predicting at least position data of an inertial measurement unit (IMU) as follows. The method specifically comprises the following steps. The IMU is actively moved in space along a trajectory, in particular a predefined trajectory, by a motorized system or actuator system. Thus, at least the position of the IMU changes, in particular its position, orientation, and further preferably its acceleration, along the trajectory. In particular, the trajectory can be defined at the beginning of the movement to follow a target trajectory. The predefined trajectory has a changing velocity and a changing acceleration. During this movement, measured motion data is acquired by the IMU and provided to a control unit.The IMU is used to read the internally measured (raw) motion data along the trajectory and provide it to the control unit. Additionally, an optical tracking system with one or more cameras captures the position and / or orientation (i.e., position) of the IMU as it moves along the trajectory. A further step of the process preferably involves linking or assigning the measured motion data to the captured positions, particularly the positions and orientations, to obtain a training dataset. An AI system (artificial intelligence system) is trained with this dataset to create a trained AI system and thus a calibrated IMU. The trained AI system is then ready for use with a single input.Finally, in this process, the measured motion data from the IMU is recorded as input for the trained AI system, and the trained AI system outputs a position and / or orientation, in particular a location, of the IMU based on the input motion data. Specifically, the position and / or orientation, in particular the location, of the IMU is provided to a navigation system, which can then output the position and / or orientation, for example, via an output device.

[0015] The IMU is attached to or mounted on the motorized system. The motorized system moves along a trajectory, specifically a predetermined / predefined trajectory, through space. The IMU is thus mounted on the motorized system in such a way that it can be moved along the trajectory through space by the motorized system and can change its position, particularly its position and orientation. The IMU records the measured motion data during the movement of the motorized system. This motion data preferably consists of acceleration in three directions (a_x, a_y, a_z) and orientation of the IMU in three directions, particularly in the form of rotation rates or orientations around three axes (e.g., around the X, Y, Z axes). Specifically, six individual values ​​are recorded as motion data: three acceleration values ​​and three rotation rate values.Preferably, a magnetic field can also be detected by the IMU.

[0016] While the IMU executes the movement along the trajectory via the motorized system, its position and / or orientation in space is simultaneously recorded by a tracking system. This tracking system provides the recorded absolute position and / or orientation (also referred to as position data) in space. The tracking system is an optical tracking system. As a result of acquiring the measured motion data and the position data, two time-dependent datasets are available. The motion data and the position data are linked in such a way that the corresponding position and / or orientation are assigned to the motion data at specific times or time intervals. This creates a linked (training) dataset that correlates the motion data and the corresponding position data.

[0017] The linked dataset is used to train the AI ​​system. Therefore, the linked dataset serves as the training dataset for the AI ​​system. The AI ​​system is preferably an artificial neural network. The motion data is the input, and the position and / or orientation is the output for the AI ​​system. During the training phase, the individual weights of the neural network are adjusted to establish a relationship or correlation between the motion data and the position data, without requiring any explicit input. This completes the training phase of the AI ​​system. The trained AI system is essentially an IMU calibration.This means that the raw, captured motion data is individually corrected by the trained AI system and linked with corresponding positional data. For example, the influence of gravity and other disturbances can be removed from the raw data. In a subsequent step, the trained AI system is then used to predict the position and / or orientation of the IMU in space based on the measured motion data. Thus, the present method, or rather the trained AI system, allows the position and / or orientation of the IMU to be determined very precisely based on the measured motion data. In particular, predictions for the IMU's position data are generated for one or more new trajectories that have not yet been captured by the tracking system.

[0018] In summary, the core of the revelation lies in capturing motion data for trajectories measured by the IMU, while simultaneously recording the position and / or orientation using the tracking system. The AI ​​system is trained using this captured data. The trained AI system can then predict / output the position and / or orientation along an unknown trajectory based on measured motion data, without requiring position tracking from the IMU.

[0019] The disclosed method offers the following advantages. As explained above, calculating position and / or orientation from acceleration data using double integration leads to significant computational errors. The influence of gravity introduces additional noise or disturbances into the acquired acceleration data. These problems are solved by the disclosed method and the trained AI system. The trained AI system can generate the most accurate possible prediction of the IMU's position and / or orientation from the acquired motion data. Through the learning process or training of the AI ​​system, disturbances caused by gravity and systematic calibration errors can be eliminated. Furthermore, the AI ​​system can be individually tailored to each IMU. Trained AI systems can also be provided for a specific class or set of IMUs.By eliminating systematic measurement errors and accurately predicting the position and / or orientation of the IMU, small and cost-effective sensors can be used while still achieving sufficiently high accuracy. This reduces costs and optimizes the use of available space on medical instruments equipped with the IMU. An additional advantage is the ease, efficiency, and speed with which sufficient training data can be generated for the AI ​​system.

[0020] The problem described in this disclosure is further solved by a training system for calibrating (and predicting) at least the positional data of the inertial measurement unit (IMU). The training system includes the IMU to be calibrated, which acquires measured motion data and provides it to a control unit. Furthermore, the training system includes a motorized system adapted to move the IMU to be calibrated along a predefined trajectory in space, wherein the trajectory has a changing velocity and acceleration. The training system also includes an optical tracking system with one or more cameras adapted to acquire the position and / or orientation of the IMU and provide it to the control unit. The control unit is adapted to combine the measured motion data with the acquired position and / or orientation to obtain a training dataset.The training system ultimately includes an AI system that trains the control unit with the training data set to obtain a trained AI system for IMU calibration.

[0021] The IMU is moved along the trajectory by the motorized system. The IMU records the motion data measured during this movement. In the controlled environment, the tracking system also very precisely records the IMU's position and / or orientation during movement. The respective recorded data is linked by the control unit to generate the linked training dataset. The AI ​​system is trained using this generated training dataset. Through this training system, the AI ​​system is trained to calibrate the IMU's motion data and to make the best possible prediction of the IMU's position and / or orientation based on the recorded motion data. Therefore, the training system allows an AI system to be specifically trained on an individual IMU or a set of (nearly) identical IMUs.This trained AI system, which is stored in particular on a computer-readable storage medium, can then be used together with the IMU for an application in a medical field, especially in surgical navigation.

[0022] Furthermore, the task of disclosure is accomplished by means of a computer-readable storage medium and a computer program, each comprising instructions which, when executed by a computer, cause it to carry out the procedural steps of the disclosed process.

[0023] The term "position" refers to a geometric position in three-dimensional space, which is specified in particular by means of coordinates of a Cartesian coordinate system. Specifically, the position can be specified by the three coordinates X, Y, and Z.

[0024] The term "orientation" refers to a direction (such as position) in space. One can also say that orientation specifies a direction or rotation in three-dimensional space. In particular, orientation can be specified using three angles.

[0025] The term "location" encompasses both position and orientation. Specifically, location can be specified using six coordinates: three positional coordinates X, Y, and Z, and three angular coordinates for orientation.

[0026] The position or orientation is considered here, in particular, relative to a coordinate system, preferably with reference to a defined coordinate system. For example, the tracking system can record the position and orientation of the inertial measurement unit at a start time and then define this start data as the (start) coordinate system (i.e., position zero, orientation zero) and then (continuously) track the position and orientation within this defined coordinate system as the IMU moves. Alternatively, a local coordinate system of the tracking system can be defined as the coordinate system, the start position and orientation of the IMU can be determined at a start time, and then the position and orientation can be tracked as the IMU moves relative to the start position and orientation.

[0027] The term "rotation rate" here refers, as is generally the case with inertial measurement units, to a rotational speed or three angular velocities that can be measured. In particular, starting from a first orientation (such as that of the IMU) and a rotation rate measured (e.g., continuously over time), it is possible to infer the orientation at a later point in time, so that rotation rate and orientation can be linked by a functional relationship.

[0028] Advantageous further developments of the present disclosure are the subject of the dependent claims and are explained in particular below.

[0029] Preferably, in the step of acquiring the measured motion data, at least an acceleration on the one hand and an orientation or rotation rate on the other are recorded. Furthermore, an orientation relative to a magnetic field can preferably be recorded. The acceleration is recorded by one or more accelerometers. Preferably, the IMU has three orthogonal accelerometers to record three accelerations in three different directions (a_x, a_y, a_z). The orientations or rotation rates about the axes, in particular about the X, Y, and Z axes, are recorded by a gyroscope, in particular by three orthogonal gyroscopic sensors. Thus, the IMU preferably has six sensors: three accelerometers and three gyroscopic sensors. The IMU can also have only one accelerometer and one sensor for orientation, such as a gyroscopic sensor.It is only necessary that the IMU can detect acceleration in three directions and orientation in or around three directions. By capturing this motion data, the movement of the IMU can be adequately recorded. The captured, measured motion data is part of the training dataset.

[0030] Preferably, the IMU is attached to a medical instrument between the steps of training an AI system with the training dataset and capturing measured motion data from the IMU as input for the trained AI system. That is, the IMU is removed from the motorized system and transferred to the medical instrument. Likewise, the trained AI system is transferred to the medical instrument, or at least a data connection is established to the medical instrument so that the medical instrument can access the trained AI system. When the medical instrument is moved, the IMU captures the motion data of this movement.While it was previously possible to track the position and / or orientation of the IMU during movement by the motorized system, tracking the position and / or orientation of the IMU while the medical instrument is moving is no longer desired by the tracking system to prevent visual interference. Therefore, the trained AI system calculates / estimates / predicts the position and / or orientation of the IMU from the captured movement data of the IMU.

[0031] According to the invention, in the step of acquiring the position and / or orientation of the IMU by the tracking system, the position and / or orientation of the IMU is acquired by an optical tracking system. The optical tracking system acquires the position and / or orientation of the IMU using one or more cameras. The tracking system enables sufficiently accurate acquisition of the (global) position and / or orientation of the IMU. In particular, this step takes place in a controlled environment in which, for example, there is always good line of sight between a camera of the optical tracking system and the tracked IMU, and where, for example, additional rigid trackers (such as IR markers) are attached to enable precise measurement. Once the AI ​​system is trained, these trackers are no longer needed, and the IMU can be used in challenging environments.

[0032] Preferably, in the linking step, the measured motion data at a first time point, in particular at a plurality of discrete time points with preferably defined time intervals, are assigned to the recorded position and / or orientation at that first time point. Both the recorded motion data and the recorded position and / or orientation are time-dependent. This time dependency allows a position and / or orientation to be assigned to a movement at any given time point. This enables the generation of the linked training dataset. In particular, a first position and orientation, i.e., position at a first time point, and a second position, together with a measured acceleration and rotation rate of the IMU at the first time point, can be linked as a dataset.Through training, the AI ​​system then finds a connection independently, so to speak, in order to infer the second position from the first position based on the measured movement data and to imitate the tracking system.

[0033] Preferably, the moving step involves a movement of the IMU by a robot.

[0034] Preferably, the steps of acquiring the measured motion data, recording the position and / or orientation of the IMU, and linking the data are performed at discrete time points. The values ​​are recorded at specific time intervals. This facilitates the linking of the acquired data, since although continuous measurement is theoretically possible, in practice discrete time measurements are always used, also for the corresponding determination.

[0035] Preferably, the measured motion data includes three acceleration values ​​in three directions and three orientation values ​​about three axes, and the AI ​​system has at least six nodes in its input layer. When acquiring the measured motion data from the IMU, six values ​​are determined: the acceleration in three directions and the orientation in three directions. Since these six values ​​serve as input for the AI ​​system, the AI ​​system preferably has six nodes in its input layer. This allows the motion data to be input directly into the AI ​​system without prior conversion.

[0036] Preferably, the AI ​​system uses an artificial neural network, in particular a recurrent deep neural network, most preferably a Long Short Term Memory Network (LSTM network) and / or a convolutional recurrent network. This allows, in particular, the depth of the neural network to be adjusted and training to be carried out until a sufficiently accurate weighting is achieved. The acquired data is time-dependent. The aforementioned artificial neural networks are specifically designed to be trained with time-dependent data.

[0037] According to the invention, the step of moving along at least one trajectory, here a predefined trajectory, preferably at least two predefined trajectories, involves a changing speed and / or changing acceleration. To collect sufficient training data, the trajectory or trajectories are traversed at different speeds, accelerations, and / or orientations.

[0038] Preferably, in the output step, in addition to the position and / or orientation, a (corrected) velocity and / or a corrected acceleration from the IMU is output. The trained AI system is essentially a calibration of the IMU to determine and output at least a position and / or orientation, a corrected acceleration (approximating the actual acceleration), and a (corrected) velocity from measured motion data. The calibration or filtering of the raw motion values ​​corrects the raw motion values. Specifically, the trained AI system outputs such corrected motion data, which is free of the effects of gravity or sensor calibration. The corrected motion values ​​can also be output by the AI ​​system. This provides the control unit with corrected values ​​that can be used as needed.The IMU velocity is output by the trained AI system without the errors that would occur when integrating it from raw acceleration data. In particular, the AI ​​system can be trained not only on the relationship between motion data (with acceleration and rotation rate) and position and / or orientation, but also between motion data (with acceleration and rotation rate or rotation rate) and acceleration and orientation or rotation rate.

[0039] Preferably, the output position and / or orientation includes three position coordinates and / or three orientation coordinates or angular coordinates. To know the exact position of the IMU in space, the AI ​​system must output at least three position coordinates. The same applies to the orientation. To output these values, the AI ​​system should have at least three nodes in its output layer. This allows the position and / or orientation to be output directly without any conversion of values. It is, of course, conceivable that the AI ​​system has six nodes in its output layer. In this case, the AI ​​system could output three orientations and three positions.

[0040] Preferably, a movement is performed along a second trajectory (different from the first). To collect as much training data as possible, preferably more than one trajectory is followed, and the position and / or orientation during the multiple trajectories is recorded by the tracking system.

[0041] Preferably, the motorized system can be a robot, in particular a robotic arm, which guides the IMU. The robot can move the IMU along the trajectory.

[0042] According to the invention, the tracking system has an optical tracking system.

[0043] Preferably, the IMU records the measured motion data, at least an acceleration and an orientation, and preferably an alignment to a magnetic field.

[0044] Preferably, the IMU is attached to the motorized system and moves along the trajectory with it. During this movement, the IMU records the motion data.

[0045] Preferably, the IMU captures the measured motion data, preferably three acceleration values ​​in three directions and three orientation values ​​around three axes, and the AI ​​system has at least six nodes in the input layer.

[0046] Preferably, the measured motion data is assigned by the control unit to the recorded position and / or orientation at a first time point. This assignment generates the linked training data set.

[0047] Preferably, the AI ​​system of the training system is an artificial neural network, in particular a recurrent deep neural network, most preferably a Long Short Term Memory Network (LSTM network) and / or a convolutional recurrent network. These networks are particularly suitable for processing time-dependent data.

[0048] Preferably, the AI ​​system has three or six nodes in its output layer.

[0049] Preferably, the medical instrument can be an endoscope, preferably a rigid endoscope, which incorporates the IMU, particularly in a handle and / or at its distal tip. The IMU captures motion data during the movement of the medical instrument. This captured motion data serves as input for the trained AI system. When the endoscope is inserted into a patient's body orifice and moved, the IMU captures the motion data.

[0050] Preferably, the medical instrument is a gait analysis system comprising at least one IMU, in particular at a knee and / or hip and / or foot of a subject. The IMU is attached to the gait analysis system in such a way that it can record the movement of the patient during a procedure (e.g., actively moved by a physician) or of a subject while walking. During walking, the IMU records the movement data.

[0051] Preferably, the medical instrument is a movable surgical robot that has the IMU at its distal tip, in particular at a distal tip of a medical end effector. Brief description of the characters

[0052] The disclosure is further explained below with reference to preferred embodiments and the accompanying figures. These show: Fig. 1 a schematic front view of a training system according to a preferred embodiment of the present disclosure, which moves an IMU provided at a distal tip from a first position along a trajectory to a second position; Fig. 2 a schematic view of a training system according to the invention with an optical tracking system and an actively moved IMU; Fig. 3 a schematic perspective view of a medical instrument according to a preferred embodiment of the present disclosure in the form of a rigid endoscope; Fig. 4 a flowchart of a method of a preferred embodiment of the present disclosure; Fig. 5a an exemplary time-dependent course of an input value for an AI system; and Fig. 5b shows an exemplary time-dependent course of an output value of the AI ​​system based on the input value from Fig. 5a .

[0053] The figures are schematic and intended only to aid in understanding the revelation. Identical elements are marked with the same reference symbols. The features of the different versions are interchangeable. Detailed description of the figures

[0054] Fig. 1Figure 1 shows a schematic representation of a training system 1 according to a preferred embodiment of the present disclosure. The training system 1 comprises an inertial measurement unit (IMU; hereinafter referred to as IMU) 2, for which IMU calibration and the assignment of motion data to a position and orientation are to be performed. For this purpose, the training system 1 comprises a motorized system 4 in the form of a robot, which is adapted to actively move the IMU 2. The IMU 2 is a sensor set that acquires motion data, preferably acceleration and orientation. In this embodiment, the IMU 2 comprises acceleration and gyroscopic sensors (not shown) to acquire both (three-dimensional) acceleration (in all spatial directions) and rotation rates about three axes, and thus the orientation of the IMU 2 in space, as measured motion data.

[0055] Specifically, the IMU 2 is attached to a distal tip of the motorized system 4. The motorized system 4 moves the IMU 2 along a predefined trajectory T through space. During its movement by the motorized system 4, the IMU 2 acquires motion data along the predefined trajectory T. Fig. 1 Two different positions of the robot at two different times are shown with different hatching for better understanding.

[0056] In an example not belonging to the claimed invention, the training system 1 exhibits in Fig. 1A tracking system 6 in the form of a mechanically-kinematically based tracking system 6 is implemented. The motorized system 2 has actuators, in this case servomotors, which very precisely adjust the position of the IMU and, with appropriate knowledge of the kinematic model, can also read it out. Specifically, the motorized system 4 is a robot with a movable robot arm connected to a fixed robot base 12. This robot arm has several mutually movable robot arm segments 14, and the joint angle between the individual robot arm segments 14 can be adjusted. In this way, the position and orientation, i.e., the position, of the IMU 2 can be measured by the robot's internal tracking system 6.

[0057] Furthermore, the training system comprises a control unit 8 and an AI system 10. Both the measured motion data and the position and orientation recorded by the tracking system 6 are provided to the control unit 8. The control unit 8 combines the motion data recorded by the IMU 2 with the position and / or orientation of the IMU 2 recorded by the tracking system 6 into a linked training dataset. This training dataset is used to train the AI ​​system 10 to obtain IMU calibration data for position and orientation. In this embodiment, the AI ​​system 10 of the training system 1 is an artificial neural network, specifically a recurrent deep neural network, in particular a long short-term memory network and / or a convolutional recurrent network. This recurrent deep neural network is particularly well-suited for time-dependent data, such as that recorded by the IMU in this case.

[0058] Training system 1 trains the AI ​​system by having the control unit direct the robot to move the IMU along several (predefined) trajectories with varying accelerations (and thus changing velocities), different translational movements in various directions, and rotations or orientations. These variations in acceleration, direction, and orientation allow for the generation of particularly detailed training datasets, which can then be used to train the AI ​​system.In particular, the IMU2 is detachably attached to the distal tip of the motorized system 2, one or more trajectories are followed until a sufficiently detailed training dataset for the AI ​​system to be trained has been created, and then the trained AI system is stored, in particular on a computer-readable storage medium on the IMU, to establish a unique IMU calibration for the individual IMU 2. Finally, the IMU 2 is removed from the training system 1 and can be installed in a medical instrument (see ). Fig. 3 ) are used.

[0059] In this way, several IMUs 2 can be individually calibrated one after the other. These IMUs 2, together with the associated AI system 10, then provide a particularly precise individual mapping of motion data to position and orientation, which is used in surgical applications.

[0060] Training system 1 enables IMU calibration and the mapping of motion data to position and orientation, independent of the subsequent operating environment. Once the AI ​​system is trained to IMU 2, IMU 2 can be used together with its associated AI system 10. This provides a two-stage system. In the first stage, training system 1 trains the AI ​​system and thus calibrates the IMU. In the second, separate stage, the IMU is used with a sufficiently precise position and orientation determined based on the measured motion data to perform navigation even without an external (global) tracking system. The trained AI system 10 therefore emulates a tracking system. This makes it possible to perform precise navigation entirely without an external tracking system.

[0061] In particular, the trained AI system 10 and the IMU 2 can of course also be integrated into a navigation system with a tracking system. In addition to the position and orientation determined by the tracking system, the trained AI system 10 and the IMU 2 can also determine the position and orientation. This allows, for example, in the case of an optical tracking system, if the line of sight is interrupted, the trained AI system 10 and the IMU 2 to continue determining the position and orientation, and once the line of sight is re-established, the optical tracking system to resume its determination.

[0062] Fig. 2 Figure 1 shows a schematic view of a training system 1 according to the invention.

[0063] Unlike training system 1 from Fig. 1The training system 1, as motorized system 4 and tracking system 6, does not include a robot, but rather a general motorized system (not shown here) and an optical tracking system 6. This optical tracking system 6 has several spaced-apart cameras 16 to enable three-dimensional spatial acquisition by means of image analysis by the tracking system 6, in particular by the control unit 8, especially by triangulation. The cameras 16 are arranged such that they track the IMU 2 along the trajectory T in space and (via appropriate determination by the tracking system 6) thereby acquire the position and orientation of the IMU 2 in space. This position and orientation, acquired during movement, is then provided to the control unit 8.

[0064] The control unit 8 links the measured motion data acquired by the IMU 2 and the position and / or orientation of the IMU 2, acquired by the tracking system 6, to form a linked training dataset. The AI ​​system 10 is trained using this training dataset. In this embodiment, the AI ​​system 10 is again an artificial neural network in the form of a recurrent deep neural network, specifically a long short-term memory network and / or a convolutional recurrent network. The AI ​​system 10 to be trained has six input nodes for input of three acceleration values ​​measured by the IMU 2 and three measured rotation rates or angular velocities. Furthermore, the AI ​​system has six output nodes, with three output nodes having three position coordinates and three output nodes having three orientation coordinates (here, angular coordinates).In this embodiment, the control unit 8 and the AI ​​system 10 to be trained are connected to each other via a wireless data connection, such as WLAN or Bluetooth, so that the IMU 2 and the AI ​​system 10 form a single module which can later be used together in isolation in a medical instrument.

[0065] Fig. 3 Figure 1 shows a perspective view of a medical instrument 18 according to a preferred embodiment of the present disclosure. The medical instrument 18 is designed as a rigid endoscope and has the IMU 2 at its distal tip and the control unit 8 and the trained AI system 10 in a handle.

[0066] When the medical instrument 18, in the form of an endoscope, is moved within a patient's body (not shown), the (global) position of its distal tip 20 cannot be determined by an optical tracking system of a navigation system. Therefore, the position of the medical instrument is determined via the IMU 2, which is attached to the distal tip of the medical instrument 18. The position of the IMU 2 is determined based on the motion data it measures. During the movement of the medical instrument 18, the IMU 2 records this motion data. The recorded motion data is provided to the control unit 8, which uses it as input for the AI ​​system 10 trained on the IMU 2. Based on the recorded motion data, the trained AI system 10 makes a determination / prediction.a very accurate estimate for the position and / or orientation of the IMU 2 and thus of the distal tip 20 of the medical instrument 18. In particular, the medical instrument can have an IMU 2 with associated trained AI system 10, as provided by the training system 1 of the present disclosure.

[0067] In Fig. 3 The medical instrument 18 is designed as an endoscope. However, it is also conceivable that the medical instrument 18 is designed as a system for gait analysis or as a movable surgical robot with an end effector (neither of which are shown).

[0068] Fig. 4 shows a flowchart of a process according to a preferred embodiment of the present disclosure.

[0069] In a first step S1, the IMU 2 is moved by the motorized system 4 along the predefined trajectory T.

[0070] During the movement of the IMU 2, the motion data, here the acceleration and orientation, are recorded by the IMU 2 in one step S2.

[0071] Simultaneously, in step S3, the position and orientation of the IMU 2 are recorded by the (external) tracking system 6. The recorded movement data and the recorded position and orientation of the IMU 2 are provided to the control unit 8.

[0072] In step S4, the control unit 8 links the recorded movement data with the recorded position and / or orientation. Each position and orientation is assigned its corresponding movement data over time. This generates a training data set.

[0073] In a further step, S5, the generated training dataset is used to train the AI ​​system 10. The trained AI system 10 is essentially an IMU calibration for the recorded motion data, enabling the correlation of motion data with position and orientation. With step S5, the training of the AI ​​system is either provisionally or definitively completed.

[0074] In particular, the IMU 2 is then connected to a medical instrument 18 in an intermediate step (see, for example, Fig. 3 ), such as a rigid endoscope. A data connection between the medical instrument or control unit and the trained AI system is also provided. Preferably, the trained system is integrated into the medical instrument.

[0075] In a further step S6, the IMU 2 then acquires motion data, preferably belonging to a different trajectory than the (especially predefined) trajectory T followed during training. This acquired motion data serves as input for the trained AI system 10. In a final step S7, the trained AI system 10 outputs the position and orientation of the IMU 2 based on the acquired motion data, thus mimicking the tracking system.

[0076] The Figures 5a and 5b serve to further explain a relationship between the input and output values ​​of the AI ​​system 10. Fig. 5aThis shows a time course of a measurement from IMU 2. In reality, three orientations and three accelerations are recorded by IMU 2 as motion data and used as input for AI system 10. Based on these inputs, the trained AI system 10 outputs values. Such an example of an output value is shown in Fig. 5b shown. For the sake of simplicity, only one output value is displayed. In this case, in Fig. 5a for example, an acceleration in the X direction (a_x) may be shown, and in Fig. 5b correspondingly a position coordinate in the X direction (r_x). Alternatively, Fig. 5a an orientation around an axis and Fig. 5bA corrected orientation around the axis is shown. In reality, three positions or three orientations are output. The trends of the input and output values ​​are time-dependent. During the movement of IMU 2, new data is input into the AI ​​system 10 in real time, and the trained AI system outputs corresponding values ​​based on this input. Since the two acquired data sets are time-dependent, it is therefore possible to assign a position at precisely a specific time to a set of movement data at a specific time.

[0077] With the help of the present disclosure, an (arbitrary) inertial measurement unit can be used, which is calibrated before actual use by means of training an AI system, so that later the trained AI system, as a kind of translator or imitator of a tracking system, can determine at least a position and / or orientation with sufficient accuracy based on internal measurements of the IMU (motion data) and, in particular, eliminates global errors and deviations of the IMU 2, e.g., due to production, in advance. Reference symbol list

[0078] 1 Training system 2, 2 Inertial measurement unit (IMU) 4 Motorized system 6 Tracking system 8 Control unit 10 AI system 12 Robot base 14 Link / Robot arm segment 16 Camera 18 Medical instrument 20 Distal tip Trajectory Step 1: Move the IMU along the trajectory. Step 2: Capture measured motion data. Step 3: Capture position and orientation using a tracking system. Step 4: Link the data. Step 5: Train the AI ​​system with the training dataset. Step 6: Capture motion data as input for the AI ​​system. Step 7: Determine and output position and orientation using the AI ​​system.

Claims

1. A method for calibrating and determining at least one position and / or orientation of an inertial measurement unit (2) of a medical instrument (18), characterized by the steps of: - moving the inertial measurement unit (2) by a motorized system (4) along a predefined trajectory (T) in space, the trajectory (T) having a changing speed and a changing acceleration; - capturing, during the movement, measured movement data by the inertial measurement unit (2) and providing the movement data to a control unit (8); - capturing, during the movement, a position and / or orientation of the inertial measurement unit (2) by an optical tracking system (6) including one or more cameras (16); - linking the measured movement data to the captured position and / or orientation in order to obtain a training data set; - training an AI system (10) of the medical instrument (18) with the training data set to obtain an IMU calibration of movement data to position and / or orientation; - capturing measured movement data of the inertial measurement unit (2) as input to the trained AI system (10) of the medical instrument (18), and - determining and outputting, based on the input movement data, a position and / or orientation of the inertial measurement unit (2) by the trained AI system (10).

2. The method according to claim 1, characterized in that in the step of capturing the measured movement data, at least one acceleration and one orientation, in particular at least one acceleration and at least one rotation rate, and preferably also an orientation with respect to a magnetic field, are captured.

3. The method according to any of the preceding claims, characterized in that, in the step of linking, the measured movement data, in particular the acceleration and rotation rate, are assigned to a first point in time to the captured position and / or orientation at the first point in time.

4. The method according to any of the preceding claims, characterized in that the step of moving comprises a movement of the inertial measurement unit (2) by a robot.

5. The method according to any of the preceding claims, characterized in that the measured movement data includes three acceleration values in three directions and three rotation rate values or three orientation values about three axes and the AI system (10) comprises at least six nodes in an input layer.

6. The method according to any of the preceding claims, characterized in that an artificial neural network, in particular a recurrent deep neural network, particularly preferably a long short-term memory network and / or a convolutional recurrent network, is used as the AI system (10).

7. The method according to any of the preceding claims, characterized in that the step of moving takes place along a predefined trajectory (T), preferably at least two predefined trajectories, and has a changing speed or orientation.

8. The method according to any of the preceding claims, characterized in that in the step of determining and outputting, in addition to the position and / or orientation, a corrected acceleration and / or a corrected speed of the inertial measurement unit (2) is output.

9. A training system (1) for calibrating and determining at least one position and / or orientation of an inertial measurement unit (2), comprising: - an inertial measurement unit (2) to be calibrated, which captures measured movement data and provides it to a control unit (8); - a motorized system (4) which is adapted to move the inertial measurement unit (2) to be calibrated along a predefined trajectory (T) in space, the trajectory (T) having a changing speed and a changing acceleration; - an optical tracking system (6) including one or more cameras (16), which is adapted to capture a position and / or orientation of the inertial measurement unit (2) and to provide it to the control unit (8); - a control unit (8) linking the measured movement data with the captured position and / or orientation to obtain a training data set; and - an AI system (10) which is trained by the control unit (8) with the training data set to obtain a trained AI system with an IMU calibration of movement data on position and / or orientation.

10. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the method steps of the method according to any of claims 1 to 8.