Camera-Based Tracking System

Abstract

To provide accurate tracking at an earlier stage, a control device for the operation of a camera-based tracking system is provided. The control device has a controller and a control signal output. The controller generates an operation signal for the tracking system with a calibration compensation instruction during a start-up phase of the tracking system. The calibration compensation instruction has instructions for at least one of i) actively adjusting a temperature of the camera-based tracking system, and ii) adapting a calibration-related camera output of the camera-based tracking system used in the tracking calculation. The control signal output provides the operation signal to the tracking system. In one approach, pre-heating is provided to achieve a (steady-state) calibration state by providing a control signal to the camera for operation in a high thermal energy generation mode. In another approach, calibration is provided as a dynamic calibration model that allows calibration for system conditions outside the steady state.

Description

[Technical Field]

[0001] The present invention relates to a controller for the operation of a camera-based tracking system, a camera-based tracking system, and a method for operating a camera-based tracking system. [Background technology]

[0002] Navigation systems are used in medical interventions to track instruments. For example, the instruments are provided with markers that are detected by the tracking system. One example of tracking markers is the use of an optical tracking system, in which a camera tracks markers in the image data provided by the camera. As an example, as the surgeon moves the instrument, the position of the tracked instrument relative to the patient's anatomical structures is indicated on an image of the patient. Thus, the surgeon uses the system to "navigate" the position of the instrument. The instrument position feedback provided by the system can be useful in situations where the surgeon cannot actually see the tip of the instrument, such as minimally invasive surgery. Tracking devices are also provided in computer-assisted surgery (CAS) systems, which are increasingly used for preoperative planning and for instrument navigation during surgical procedures.

[0003] As an example, using a surgical navigation system, a surgeon uses special instruments that are tracked by the navigation system.

[0004] As a further example, U.S. Patent Application Publication No. 2018 / 235715(A1) relates to bone and tool tracking in computer-assisted orthopedic surgery and robotic computer-assisted surgery. To this end, the tracking device may include a 3D camera for performing range imaging. Such range tracking requires that the 3D camera be calibrated to achieve appropriate accuracy and precision of tracking, for example, using a calibration pattern.

[0005] Tracking or navigation systems may be moved between operating rooms. Clinical use and schedules may require the system to be easy to set up, quickly start up, and ready to use within minutes. However, it has been shown that optical tracking systems used for surgical navigation can take as long as 30 minutes or even an hour to warm up and stabilize after a cold start. Because of this start-up behavior, the system's ability to track markers attached to the patient and surgical instruments can be temporarily impaired, potentially resulting in inaccurate results. Summary of the Invention [Problem to be solved by the invention]

[0006] Thus, for example, improved and accurate tracking at an earlier stage may be required. [Means for solving the problem]

[0007] The object of the present invention is solved by the subject matter of the independent claims, and further embodiments are incorporated in the dependent claims. It is noted that the below-described aspects of the present invention also apply to an apparatus for operating a camera-based tracking system, a camera-based tracking system, and a method for operating a camera-based tracking system.

[0008] According to the present invention, there is provided a control device for operation of a camera-based tracking system. The control device includes a controller and a control signal output unit. The controller is configured to generate, at least during a start-up phase of the tracking system, at least one operating signal for the tracking system having at least one calibration compensation instruction. The at least one calibration compensation instruction includes instructions for at least one of the group: i) actively adjusting a temperature of the camera-based tracking system, and ii) adapting a calibration-related camera output of the camera-based tracking system used in tracking calculations. The control signal output unit is configured to provide the at least one operating signal to the tracking system.

[0009] Effectively, compensation instructions are provided to at least partially compensate for discrepancies between the current state of the camera-based tracking system and predetermined conditions of the system assumed for system calibration, which improves the accuracy of the tracking system.

[0010] According to one example, the calibration compensation instructions include at least one control signal for temporarily operating the camera-based tracking system in an increased heat generation mode.

[0011] According to one example, to temporarily operate the camera-based tracking system in a high heat generation mode, the calibration compensation instructions include at least one control signal for temporarily operating at least one of the at least two cameras in a heating mode to generate heat by at least one of the at least two cameras. The at least one control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode, in which higher energy consumption by the camera occurs when the camera is operated for tracking than in a normal tracking mode. The control signal output unit is configured to provide at least one operating signal to the tracking system to operate at least one of the at least two cameras in the high energy consumption mode, thereby heating at least the camera for higher energy consumption compared to the normal tracking mode.

[0012] According to one example, for a high heat mode, the calibration compensation instructions include at least one control signal for temporarily activating a heating element of a camera-based tracking system.

[0013] In one example, markers are used for tracking, while in another example, markerless tracking is provided.

[0014] This addresses the desirability of reducing warm-up time and minimizing temporary system accuracy degradation during cold starts in the case of optical tracking systems. A further advantage is that the camera can be configured with an external computer. This is based on the fact that the camera's power consumption is directly related to the generation of heat by the camera and the subsequent dissipation of this heat to the parts of the system thermally connected to the camera (heat sink). More heat generated in the camera results in a more rapid temperature rise in the camera and thermally connected components. Note that much of the thermal stabilization process occurs internally to the camera. This is addressed by the present solution without the need for major modifications to the camera, such as expanding the camera housing and moving the sensor location. The warm-up of the camera to the level required for stable system operation can be further accelerated by using a set of camera parameters that requires more power consumption than is required for actual normal operation.

[0015] For example, tracking of markers on the patient and devices or instruments is achieved by determining their precise spatial position relative to the tracking system. Knowledge of the spatial position of the tracking system within a local reference grid or spatial reference system in the operating room allows linking and embedding the markers to the local reference grid. Determining the spatial position of the markers is essentially achieved by triangulation. For example, the distances to the markers are measured from different sources, i.e., different cameras. By triangulation, the spatial position relative to the cameras can be determined. Triangulation makes it possible to spatially align the images provided by the cameras. However, this requires precise knowledge of the actual (spatial) positions of the cameras and their (spatial) orientation. This precise geometric knowledge is provided by the calibration of the camera system. To implement a camera system, several cameras are provided, which must be fixed to a reference. For this reason, the cameras are physically mounted to a common support structure. Since the support structure is constantly subject to thermal expansion, depending on the material, design, and size, there are always geometric changes that affect the position and orientation of the camera, and consequently the accuracy of detecting the marker. The thermal expansion is mainly caused by the internal heat generated by the operation of the camera. The heat causes thermal expansion of the camera itself (optical system relative to the image detection element), the mount by which the camera is fixed to the support structure, and the support structure itself. Consequently, a calibration of the system is provided that takes into account conditions that are assumed to be stable. Such conditions are taken for the situation during normal operation of the tracking system, i.e., the situation when the system is in use.

[0016] The present invention addresses this by providing a means to match the current state with the state assumed for calibration: by providing additional heat, the state of the system better matches the assumed state applied for calibration.

[0017] According to one example, the controller is configured to take measured external conditions into account for the control signal when generating the operating signal, the external conditions including at least one of the group of ambient temperature, lighting conditions, and indoor airflow.

[0018] According to one example, the controller is configured to provide a dynamic calibration model for the camera-based tracking system. The dynamic calibration model includes a plurality of calibration coefficients depending on a temperature-related state of the system. For a current temperature-related state of the camera-based tracking system, the controller is also configured to determine a matching calibration coefficient from the plurality of calibration coefficients. Furthermore, the control signal output is configured to provide the matching calibration coefficient for an application during operation of the camera-based tracking system.

[0019] In one example, the controller is configured to provide calibration compensation instructions including operational instructions for operating at least one of the at least two cameras in a high energy consumption mode. The controller is also configured to provide a dynamic calibration model for the camera-based tracking system and to determine and provide matching calibration coefficients. Furthermore, the controller is further configured to operate the at least two cameras in an imaging mode while operating the cameras in the high energy consumption mode. The controller is further configured to apply the matching calibration coefficients to generated images.

[0020] According to the present invention, there is also provided a camera-based tracking system, the system having a plurality of at least two cameras and a control device according to one of the previous examples, wherein at least one operating signal is provided for operating at least one of the at least two cameras.

[0021] Optionally, the camera-based tracking system is a camera-based medical tracking system for use in a hospital or other medical related setting.

[0022] According to one example, the high energy consumption mode includes image acquisition with energy consumption at least 50% higher than the energy consumption during the normal tracking mode of the system.

[0023] Optionally, the high energy consumption mode includes image acquisition with at least one of the following groups: frame rate, exposure time, bit depth, bit packing, region of interest, clocking frequency, color format, image compression, and on-board image pre-processing that is at least a 50% increase compared to the system's normal tracking mode.

[0024] According to one example, a set of markers that can be tracked by the system is provided, the markers being attachable to at least one of the following groups: a subject, an object, and a device.

[0025] In another example, markerless tracking is provided.

[0026] According to the present invention, there is also provided a method for operating a camera-based tracking system, the method comprising the following steps: generating, at least during a start-up phase of the tracking system, at least one operating signal for the tracking system, the at least one calibration compensation instruction comprising instructions for at least one of the group: i) actively adjusting a temperature of the camera-based tracking system, and ii) adapting a calibration-related camera output of the camera-based tracking system used in the tracking calculations. · Providing at least one motion signal to a tracking system. · Operating the camera-based tracking system based on the operating signal including at least one calibration compensation command.

[0027] According to one aspect, tracking errors due to calibration states that have not yet been performed are minimized by providing countermeasures to compensate for the tracking errors.

[0028] According to one approach, preheating is provided to achieve a (steady) calibration state (applied for system calibration) by providing a control signal to the camera for operation in a high heat energy generation mode. Therefore, existing equipment is used to generate heat. Therefore, thermal energy is provided exactly where it will have the greatest effect, i.e., at the camera location. Preheating mimics the heat generated during normal operation, but in a faster boost mode, for example.

[0029] According to another approach, instead of trying to achieve a steady state by preheating, which forms the basis for a static calibration, the calibration is provided as a dynamic calibration model that allows calibration also for states of the system outside the steady state: even if the operating temperature has not yet been reached, a respectively adapted calibration is provided by the dynamic model.

[0030] According to a further approach that combines the above two approaches, a dynamic model is provided in combination with a heating procedure with a special operating signal for the camera. In combination with an optional temperature sensor, accuracy can be further improved.

[0031] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0032] Exemplary embodiments of the present invention are described below with reference to the following drawings: [Brief explanation of the drawings]

[0033] [Figure 1] 1 illustrates a schematic example of a controller for the operation of a camera-based tracking system. [Figure 2] 1 illustrates an example of a camera-based tracking system in the context of a catheterization lab. [Figure 3a] 1 illustrates an example of a camera-based tracking system within a housing structure. [Figure 3b]3b shows an example of the camera-based tracking system of FIG. 3a without the housing structure. [Figure 4] 1 illustrates steps of an example method for operating a camera-based tracking system. [Figure 5] 10 is a graph showing tracking error versus time after initiation. [Figure 6] 1 is a graph showing the temperature of the system versus time after start-up. [Figure 7] 4 shows a graph illustrating system temperature over time for different operating modes. [Figure 8] 10 shows graphs illustrating example warm-up patterns for system temperature and camera power consumption over time. [Figure 9] 10 shows a graph illustrating another example of a warm-up pattern for system temperature and camera power consumption over time. [Figure 10] 10 shows graphs illustrating further examples of warm-up patterns for system temperature and camera power consumption over time. DETAILED DESCRIPTION OF THE INVENTION

[0034] Specific embodiments will now be described in detail with reference to the accompanying drawings. In the following description, like drawing reference numbers are used for like elements in different drawings. Matters defined herein, such as detailed configurations and elements, are provided to facilitate a comprehensive understanding of the exemplary embodiments. Additionally, well-known functions or configurations will not be described in detail since they would obscure the embodiments in unnecessary detail. Furthermore, phrases such as "at least one of," when preceding a list of elements, modify the entire list of elements, and do not modify individual elements of the list.

[0035] Surgical navigation systems relying on optical cameras can have inherent marker tracking accuracy in the range of 0.5 to 1 mm. The inherent 3D marker tracking accuracy of a system is optimal (minimum) when the system is in the same "state" as when it was calibrated. If the system is in a different state, this results in less accurate marker tracking and therefore larger 3D errors. One example is when the relative geometric position and / or orientation of the optical cameras are changed by mechanical forces (shock, vibration). This may require recalibration of the system. Furthermore, when using optical camera systems, temperature is a significant factor that can introduce changes in the actual image, resulting in additional tracking errors of the same order of magnitude as the desired accuracy. As an example, during startup of the navigation system, tracking accuracy can be temporarily degraded when the hardware is in a "different state" due to temperature-related effects. There are two main effects that cause measurable thermally induced errors. First, the frame suspending the optical cameras undergoes some temperature-related expansion, causing a temporary different geometric relationship between the cameras. Second, thermally related changes in the camera housing and lens system cause measurable differences in the projection of the image on the camera imaging sensor. Both effects contribute to a temporary decrease in accuracy as calibration is performed when the system is in thermal steady state and while the system is in (temporarily) different transient states.

[0036] Generally, the startup of an optical tracking system can be divided into two phases. In the first phase, also called Phase 1, the system warms up. At the start, the camera temperature is close to the ambient temperature of the system. It increases due to heat dissipation from active components, such as the camera processor, imaging chip, etc. The temperature-related tracking error is greatest at the beginning of this phase and least at the end of this phase. In the second phase, also called Phase 2, the steady state of the system is reached. The hardware reaches thermal equilibrium, and the optical tracking accuracy is at the same level as in the calibration state.

[0037] In one example, the thermal behavior of an optical tracking system may be more complex because there may be a certain delay in heat transfer from the camera to other parts of the system that are thermally connected and act as a heat sink. Furthermore, preferably, the warm-up behavior of the navigation system is such that the thermal response is minimal and the time required to reach a steady state after startup is minimal. This solution does not necessarily require smart hardware selection and / or design of the system, thus avoiding additional construction-related costs. Another way to trigger calibration affecting changes can be provided by ambient temperature differences, for example, by moving the camera from a hot storage room to an air-conditioned operating room.

[0038] When an optical tracking system is warming up from its surrounding ambient temperature, its internal components inevitably change size and shape, resulting in start-up behavior. The present solution speeds up the warm-up of the optical tracking system to a steady-state level and reduces temperature-related differences in tracking accuracy, so that tracking can be performed at an optimal level of accuracy sooner (faster) after start-up without adding hardware and / or hardware complexity to the system.

[0039] FIG. 1 schematically illustrates an example of a control device 10 for operating a camera-based tracking system. The control device includes a controller 12 and a control signal output unit 14. The controller 12 is configured to generate at least one operating signal for the tracking system, including at least one calibration compensation instruction, at least during the start-up phase of the tracking system. The at least one calibration compensation instruction includes instructions for at least one of the following group: i) actively adjusting the temperature of the camera-based tracking system; and ii) adapting a calibration-related camera output of the camera-based tracking system used in tracking calculations. The control signal output unit 14 is configured to provide the at least one operating signal to the tracking system to operate the camera-based tracking system.

[0040] 1 by a dashed line, a signal input 16 is provided. The signal input 16 is configured to receive a signal to activate a camera-based tracking system having at least two cameras. Based on the activation signal, the controller is configured to generate at least one operation signal.

[0041] Thus, at least one calibration compensation command is provided at system start-up, allowing for faster use of the system.

[0042] In another option, the signal input 16 is configured to receive a signal for ongoing operation of the camera-based tracking system, and based on the ongoing operation signal, the controller is configured to generate at least one operation signal as a check of calibration.

[0043] Thus, at least one calibration compensation instruction may be provided during system use upon system start-up, for example, to enable improved accuracy during operation in changing environmental conditions.

[0044] A first arrow 18 indicates a signal for activating a camera-based tracking system. A second arrow 20 indicates the output of at least one operating signal to, for example, a camera-based tracking system, as indicated by a dotted frame 22. A further frame 24 indicates that the controller 12, the control signal output 14, and the signal input 16 can be provided within a common structure, such as a housing. However, they can be located separately but data-connected.

[0045] Compensation instructions are provided to at least partially compensate for differences between a current state of the camera-based tracking system and a predetermined state of the system assumed for calibration of the system.

[0046] Active regulation of the temperature of the camera-based tracking system is provided to at least partially compensate for the discrepancy.

[0047] Adaptation of the camera output is provided to at least partially compensate for the difference.

[0048] The control signal output unit 14 is configured to provide at least one operating signal to the tracking system to operate the camera-based tracking system based on the operating signal including at least one calibration compensation instruction, thereby reducing the discrepancy between the current state of the camera-based tracking system and a predetermined state of the system assumed for the calibration of the system.

[0049] Calibration compensation instructions, which may also be referred to as calibration correction instructions, are provided to adjust or modify system settings related to calibration.

[0050] Calibration compensation instructions are provided to minimize the offset between the assumed state on which the calibration is based and the current state of the system.

[0051] As an effect, the system is provided with improved accuracy even in situations where a static calibration state has not yet been achieved, such as during the start-up phase.

[0052] The start-up phase refers to the part of the operation from the start of the system operation until the predetermined conditions of the system that are assumed for the system calibration are reached.

[0053] In a first option, not shown in further detail in FIG. 1, the calibration compensation instructions include at least one control signal for temporarily operating the camera-based tracking system in a high heat-producing mode.

[0054] In an option not shown in further detail in FIG. 1 , the calibration compensation instructions include at least one control signal for temporarily operating at least one of the at least two cameras in a heating mode to generate heat by at least one of the at least two cameras to temporarily operate the camera-based tracking system in a high heat-generation mode. The at least one control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode, in which higher energy consumption by the camera occurs than in a normal tracking mode when the camera is operated to track a marker. The control signal output unit 14 is configured to provide at least one operating signal to the tracking system to operate at least one of the at least two cameras in the high energy consumption mode, thereby heating at least the camera due to higher energy consumption compared to the normal tracking mode.

[0055] The difference may also be referred to as an inconsistency.

[0056] In one option, the heating mode is provided by operating an (already existing) camera in a heating mode to generate thermal energy.

[0057] In another option, the heating modes are provided by operating a separate heating element to generate thermal energy. The separate heating element can be provided with the tracking system. The heating element can be provided with a mount or support structure for the camera, by a separately attached heating element, or by a heating element integrally formed with the camera. The heating element can also be provided by identifying an operating element, such as a circuit within the system, that can be operated in a specified thermal energy generation mode.

[0058] In one example, in the case of a high heat generation mode, the calibration compensation instructions include at least one control signal for temporarily activating a heating element of a camera-based tracking system.

[0059] In one example, the heating element includes at least one integral heating element, for example, a heating element provided within the camera mount, to provide thermal energy at a location where the camera, in an operational mode, transfers the thermal energy to the support structure.

[0060] In another example, an integral heating element is provided within the camera adjacent to the location of the electrical circuitry that generates heat when the camera is in its operational mode.

[0061] An example of such an additional heating element is a resistive stripe attached to the existing structure. Thus, targeted heat generation (and preheating or warm-up) is possible with less energy consumption compared to heating the complete system, for example, in the warm-up area of ​​the operating room or even in a separate warm-up room.

[0062] In one example, higher energy consumption is provided by increased heat generation due to operation, for example, a particular part of an electrical circuit is activated that uses the same energy as other parts during normal operation, but this part or component generates more temperature, i.e., more heat available for dissipation and heating of the system.

[0063] The effect is that heat (to reach the operating conditions on which the calibration is based) is generated exactly where it should be generated. As an advantage in terms of economy and reduced system complexity, this is achieved by using hardware that is already available. Another advantage results from the fact that the functionality of the tracking system is maintained. Even at the start of the heating process, tracking may not be very accurate, and the system is available for other purposes where this may not be an issue, for example, already providing visualization of the area of ​​interest to allow the surgeon to place the interventional device in a convenient location while maintaining proper line of sight, etc.

[0064] The term "activate" relates to the initiation of operation of the tracking system when the system is started, for example converted or transitioned from OFF mode to ON mode. This is also referred to as initiation. Activation or initiation also includes transitioning the system from standby or sleep mode to ON mode. ON mode is the mode in which the system is ready for use, i.e. for normal, specified operation. ON mode can also be referred to as operation mode. Note that the term operation relates to the system in this context and does not necessarily mean that a (medical) operation is performed.

[0065] A "high energy consumption mode" refers to a mode that results in a higher energy consumption of the camera than in the absence of an operating signal for the high energy consumption mode. The high energy consumption mode can also be referred to as a higher energy consumption mode. Energy consumption can also be referred to as (electrical) power usage.

[0066] The "high energy consumption mode" may also be referred to as an artificial or triggered warm-up mode, or a boosted warm-up mode.

[0067] As a result, the warm-up period is reduced by operating in a high energy usage mode.

[0068] In the present context, the term "normal operation" relates to the system settings that are applied when creating normal images with the camera, for example for tracking purposes.

[0069] The term "normal operation" relates to requirements such as camera frame rate, i.e., images per second, etc. As an example, in order to "smooth" surgical navigation tasks, according to current requirements, cameras are adjusted to provide at least 12 frames per second (fps).

[0070] In one example, the term "high energy consumption mode" refers to the camera being operated "harder" during warm-up, for example at 30 frames per second, which significantly reduces the warm-up time.

[0071] In one example, an optical tracking system is provided in which the optical camera in the system is configured to consume more power during startup than during normal operation. This is achieved by setting camera parameters, such as frame rate, acquisition time, color format, image compression, on-board image pre-processing, and camera timing, to values / states that require more on-board image processing and therefore more power consumption. More power consumption results in more heat dissipation, which leads to a more rapid warm-up of the camera and thermally connected components than would be the case with a lower power consumption setting. Once a temperature corresponding to that normally reached at the low power consumption setting is reached, the camera settings are adjusted accordingly. As a result, less time is required to bring the optical system up to operating temperature.

[0072] In one example not shown in detail in FIG. 1 , a control device for operation of a camera-based tracking system is provided. The control device has a signal input unit, a controller, and a control signal output unit. The signal input unit is configured to receive a signal for activating a camera-based tracking system having at least two cameras. Based on the activation signal, the controller is configured to generate at least one operation signal for the tracking system, the operation signal including at least one control signal for temporarily operating at least one of the at least two cameras in a heating mode to generate heat by at least one of the at least two cameras. The at least one control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode, in which higher energy consumption by the camera occurs than in a normal tracking mode when the camera is operated to track a marker. The control signal output unit is configured to provide at least one operation signal to the tracking system to operate at least one of the at least two cameras in the high energy consumption mode, thereby heating at least the camera for higher energy consumption compared to the normal tracking mode.

[0073] In one example, the controller is configured to provide an operating signal to operate the at least one camera in a high energy consumption mode until a predetermined thermal condition is reached.

[0074] Optionally, the controller is configured to provide an operating signal to maintain the high energy consumption mode until at least one of the group of at least one camera and system reaches its normal operating temperature, which is the basis for calibration of the system.

[0075] In another option, the controller is configured to provide an operating signal to provide a high energy consumption mode beyond the point at which the at least one camera has reached its normal operating temperature.

[0076] In one example, the controller is configured to provide a normal operation signal for operating the at least one camera under normal conditions after heating the at least one camera by operating the at least one camera of the at least two cameras in the high energy consumption mode.

[0077] Optionally, the controller is configured to modify settings for normal operation based on the difference between the system temperature at the time of calibration and the expected steady state temperature of the system in a real-time environment.

[0078] 1, the controller is configured to take measured external conditions into account for the control signal when generating the operating signal, the external conditions including at least one of the following group: ambient temperature, lighting conditions, and indoor airflow.

[0079] In one example, external temperature conditions are measured and taken into account. For example, if the ambient temperature is high, the thermal equilibrium temperature of the optical system will also be high. In that case, the threshold temperature for switching from high power to low power consumption should be adjusted accordingly, for example, sooner.

[0080] In one example, lighting conditions are measured and taken into account: the brightness of the lighting can cause a higher (2-3°C) steady-state camera temperature; the brightness can be directly subtracted from the images recorded by the camera and thus serve as an additional input; similarly, if the external lighting cannot be controlled, the brightness of the captured images can still be adjusted via camera parameters (e.g., exposure time).

[0081] In one example, indoor airflow is taken into account, also referred to as airflow, and is provided for the supply of clean air or for cooling or heating purposes, either detected or provided as a known parameter, and the movement of air also affects the camera due to convection effects.

[0082] In one example, a signal input 16 is provided configured to provide a measured external condition.

[0083] In one example not further shown in Figure 1, to generate at least one operating signal for the tracking system, the controller is configured to provide a model of the warm-up of the system and its dynamic thermal behavior and take these into account for at least one control signal, the at least one control signal having a signal sequence with a plurality of varying control sub-signals.

[0084] In one example, the dynamic thermal behavior of the system is taken into account by using a model of the system's warm-up. Rather than simply switching from a high power consumption to a low power consumption setting based on a threshold temperature, a priori knowledge of the system's thermal behavior can be used to fine-tune the exact moment at which camera parameters that affect power consumption change during warm-up.

[0085] In another option not shown in further detail in FIG. 1 , controller 12 is configured to provide a dynamic calibration model for the camera-based tracking system. The dynamic calibration model has a plurality of calibration coefficients depending on a temperature-related state of the system. For a current temperature-related state of the camera-based tracking system, controller 12 is configured to determine a matching calibration coefficient from the plurality of calibration coefficients. Control signal output 14 is configured to provide the matching calibration coefficient for application during operation of the camera-based tracking system.

[0086] In a basic version of this option, the current temperature-related state of the camera-based tracking system is information regarding activation of the system from an OFF state to an ON state. In another version, information regarding the time period since the system was activated is provided. In a further version, one or more temperature measurements are provided, such as temperature readings from internal and / or external sensors.

[0087] In a further option, the controller 12 is configured to provide calibration compensation instructions having operational instructions for operating at least one of the at least two cameras in a high energy consumption mode. The controller 12 is also configured to provide a dynamic calibration model for the camera-based tracking system and to determine and provide matching calibration coefficients. The controller 12 is further configured to operate the at least two cameras in an imaging mode while operating the cameras in the high energy consumption mode. The controller 12 is configured to apply the matching calibration coefficients to generated images.

[0088] In one example, a signal input 16 configured to provide a current temperature-related state of the camera-based tracking system is provided. A controller is configured to provide a dynamic calibration model for the camera-based tracking system. The controller is also configured to determine a matching calibration coefficient from a plurality of calibration coefficients. A control signal output is configured to provide the matching calibration coefficient for application during operation of the camera-based tracking system.

[0089] In one example, also referring to the first and second options above, the controller 12 is configured to provide calibration compensation instructions having operational instructions for operating at least one of the at least two cameras in a high energy consumption mode. The controller 12 is also configured to provide a dynamic calibration model for the camera-based tracking system and to determine and provide matching calibration coefficients. The controller 12 is further configured to operate the at least two cameras in an imaging mode while operating the cameras in the high energy consumption mode. The controller 12 is further configured to apply the matching calibration coefficients to generated images.

[0090] 2 shows an example of a camera-based tracking system 50 in the context of a catheterization lab. The camera-based tracking system 50 has a plurality of at least two cameras 52. An example of a control device 10 according to one of the above and below examples is provided. At least one operating signal is provided to operate at least one of the at least two cameras.

[0091] Optionally, the camera-based tracking system 50 is a camera-based medical tracking system for use in a hospital or other medical related setting.

[0092] In another option, the camera-based tracking system 50 is a camera-based tracking system for use in a non-medical setting.

[0093] In one example, a medical optical tracking system is provided that has a camera-based system capable of imaging a medical device / object and also has a means for directly or indirectly measuring the transient warm-up period.

[0094] 2 shows four cameras 52 mounted on a ring-shaped support structure 54. A data connection 56, shown diagrammatically, connects the four cameras and a controller 10. For example, the controller 10 is connected to a console 58 provided for operation of different equipment within the catheterization lab. As an example, an X-ray imaging system 60 is provided with a C-arm 62 having an X-ray source 64 and an X-ray detector 66 mounted on either end of the C-arm 62. The C-arm 62 is movably mounted to a ceiling rail structure. Additionally, a subject support 68, also referred to as a patient table, is provided for receiving a subject 70. Also shown is a display device 72.

[0095] 3a shows an example of a camera-based tracking system 50 in a housing structure 74 having four cameras 52a, 52b, 52c, and 52d. It should also be noted that fewer cameras, such as two or three, or more cameras, such as five, six, seven, eight, nine, or ten or more cameras, can be provided.

[0096] Figure 3b shows the example of Figure 3a without the housing structure 74. Shown is a ring support 76 to which four cameras 52a, 52b, 52c and 52d are attached. Also provided is a connecting flange 78. Lines 80 below the cameras indicate their overlapping fields of view.

[0097] In one example, the high energy consumption mode comprises image acquisition with energy consumption at least 50% higher than the energy consumption during a normal tracking mode of the system. Optionally, the high energy consumption mode is provided to comprise image acquisition in which at least one of the following group is increased by at least 50% compared to the normal tracking mode of the system: frame rate, exposure time, bit depth, bit packing, region of interest, clocking frequency, color format, image compression, and on-board image pre-processing.

[0098] The exposure time can also be referred to as the acquisition time.

[0099] Image compression may also involve debayering.

[0100] As an example, the increased frame rate is at least 50% higher than the frame rate used in the normal tracking mode of the system.

[0101] In one example, the increased frames are (1 1 / 2) times the normal tracking frame rate.

[0102] In one example, the high energy consumption mode includes image acquisition with an energy consumption that is at least (2+1 / 2) times the energy consumption during the normal tracking mode of the system.

[0103] In one example, the increased frame rate is provided as more than twice the normal tracking frame rate, for example, (2+1 / 2) times the normal tracking frame rate.

[0104] In one example, at least one of the following groups is provided in a double or (2+1 / 2) times amount: exposure time, bit depth, bit packing, region of interest, clocking frequency, color format, image compression, and on-board image pre-processing.

[0105] In one example of the device or system, the control signal is configured to operate at least one camera in a high energy consumption mode until a predetermined thermal condition is reached.

[0106] Optionally, the control signal is configured to maintain the high energy consumption mode until at least one of the group of at least one camera and system reaches a normal operating temperature, which is the basis for the calibration of the system.

[0107] As a further option, the control signal is configured such that the high energy consumption mode is provided once at least one camera has reached its normal operating temperature.

[0108] In one example of the device or system, the control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode to heat the at least one camera, and then provide a normal operation signal to operate the at least one camera under normal conditions.

[0109] Optionally, the control signal is configured such that the settings for normal operation are modified based on the difference between the system temperature at the time of calibration and the expected steady state temperature of the system under real-time conditions.

[0110] In one example of the device or system, the controller for generating at least one operating signal for the tracking system is configured to provide a model of the warm-up of the system and its dynamic thermal behavior and to take these into account for the at least one control signal, optionally provided that the at least one control signal comprises a signal sequence having a plurality of varying control sub-signals.

[0111] In one example, at least one temperature sensor is provided for monitoring the camera temperature, and the camera temperature is taken into account for generating the operating signal. Optionally, the controller is configured to generate an updated operating signal.

[0112] In one example, the camera's built-in temperature sensor is used to monitor the camera temperature, which can be read using the existing computer-camera communication interface without the need for an additional temperature measurement system.

[0113] In one example not shown, the controller 12 is configured to modify the operating signal based on at least one of the group of sensor data and system conditions.

[0114] For example, an internal temperature sensor measures the system temperature and the controller then determines whether additional heating is beneficial.

[0115] For example, if the system is turned off for only a few minutes to move from one operating room to the next and is still close to the steady state temperature assumed for calibration, no further additional heating may be required.

[0116] For example, if the system is stored in an unconditioned storage room during a heat wave, no additional heating may be required.

[0117] In another option, a temperature sensor provides temperature readings throughout normal operation of the system, and the controller compares the readings to a threshold value below which the controller provides for the activation of an intermediate warm-up procedure in which at least one of the at least two cameras is operated in a high energy consumption mode, thereby heating the camera for higher energy consumption compared to a normal tracking mode. This ensures that the system is operating in conditions that are the basis for calibration.

[0118] In one example not shown, a number of sensors 84 (see FIG. 2) are provided that monitor the thermal behavior of the system. The thermal behavior monitored by the sensors is used to provide a more accurate model of the system warm-up.

[0119] Optionally, a model of the system warm-up is provided that is used in combination with multiple temperature readings from the sensor during warm-up to dynamically adjust the system calibration.

[0120] Optionally, the temperature history of the system is provided and the remaining time is predicted and optionally shown before the system reaches the normal operating stage.

[0121] In one example, a model of the thermal behavior of the system combined with temperature readings during warm-up and normal operation is used to dynamically adjust the optical calibration of the system. In one example, an optical calibration look-up table is created by calculating the optical calibration (internal and external camera parameters) at various times during system warm-up under various ambient temperature conditions. Optionally, this look-up table is then used to dynamically adjust the optical calibration of the system based on the thermal state of the system.

[0122] In one example, multiple temperature sensors are used at various locations on an optical tracking system so that a more accurate model can be applied to the system's dynamic thermal behavior. Within the camera, temperatures change most dramatically (typically up to 70°C ambient temperature) and rapidly (as heat is generated within the camera), while the camera housing already exhibits a significantly different temperature range (up to 50°C ambient temperature) and delayed warm-up. Other components of the optical tracking system that are thermally connected to the camera (e.g., camera mount, frame) have delayed warm-up, and their thermal equilibrium temperatures will be lower in the absence of other heat sources. Using multiple sensors to monitor temperatures in various parts of the system helps build a comprehensive model of the system's thermal behavior and more accurately fine-tune warm-up settings.

[0123] In one example, the camera-lens system is thermally modeled. In another example, the complete device, i.e., system, frame, cover, and camera, is thermally modeled. Based on the modeling, it is possible to determine the system's response (transfer function) without having a priori knowledge of the system. In one example, the main inputs to the model are a combination of the signal from the temperature sensor, camera imaging parameters such as frame rate, system power consumption, and image-based parameters such as image brightness.

[0124] In one example, the temperature history is provided as a current temperature history, which may also be referred to as a thermal history.

[0125] In one example, the thermal state of the system based on temperature history is used to predict the time remaining before the system can be used for normal operation. Preferably, that information can be provided as feedback, for example as a countdown timer, progress bar, etc.

[0126] Another option is to provide a modified parameter set after the start-up phase, for example, by operating the system at a significantly lower ambient temperature than during calibration, thus providing for the camera to run, i.e., operate, at a slightly increased frame rate to compensate for the lower ambient temperature.

[0127] In an embodiment not shown, the controller is configured to analyze at least two images from different points in time during warm-up, determine the current displacement speed, and adjust the high energy consumption mode accordingly.

[0128] In one example, a set of markers 82 that can be tracked by the system is provided, as shown as an option in Figure 2. The markers 82 can be attached to at least one of the following groups: a subject, an object, and a device.

[0129] The markers 82 are also referred to as fiducials.

[0130] This solution avoids the high cost of navigation systems. Furthermore, the clinical adoption of surgical navigation systems is also supported, as the use of navigation technology is facilitated and it fits well into clinical routines due to its accelerated warm-up. Such medical, e.g., surgical navigation systems function quickly and easily with simplified procedures, thus not complicating workflows too much. Navigation systems are also suitable for being moved between operating rooms. The systems are easy to set up, quickly start up, and are ready to use within minutes.

[0131] Applying the above operating scheme results in a particular energy consumption pattern, which can be detected, for example, by measuring the power consumption of the cameras in a given optical tracking system, for example using a USB power meter or by measuring voltage and current in the hardware.

[0132] In one example, a temperature measurement system is provided to measure the temperature of the camera and / or other parts of the system.

[0133] In one example, a processor is provided to analyze the at least two images and determine a current displacement rate.

[0134] In one example, a timer is provided to monitor the length of the warm-up period.

[0135] In one example, a processor is provided that is capable of processing transient warm-up period measurement information, configuring the camera, and receiving and processing image data from the camera.

[0136] It is provided that thermal stabilization of the optical tracking system is accelerated by configuring the camera to consume more power during warm-up than during normal operation. Furthermore, once the desired thermal conditions are reached, the camera is configured to provide the desired performance with the lowest possible power consumption.

[0137] Roughly two phases can be distinguished for the warm-up of an optical tracking system. In phase 1, the temperature of the system rises rapidly due to self-heating of the active components within the camera. In phase 2, the system is approaching or reaching thermal equilibrium. Phase 2 may be provided for longer (several hours) than phase 1 (without sustained heating, between 30 minutes and an hour with sustained (boosted) heating being significantly shorter).

[0138] In one example, two fixed sets of image acquisition parameters are used: a first setting associated with high power demands on the camera to heat it, and a second setting associated with normal image acquisition by the camera.

[0139] In one example, during the first stage of camera warm-up, a high power setting is used to speed up the camera warm-up (FIG. 8, curve I) to a temperature level that would have been reached using the normal power setting (FIG. 8, curve II). Once the preset temperature is reached, the system switches to the settings required for normal image acquisition by the camera.

[0140] In one example, a machine vision camera can be controlled by setting image acquisition parameters such as frame rate, exposure time, binning, bit depth, bit packing, region of interest, clocking frequency, and level of on-board image pre-processing, e.g., image compression, debayering. An example is shown in Figure 7. The warm-up time of the camera imaging chip and camera housing is accelerated by approximately 3.5 times and 1.4 times, respectively, by using a higher frame rate for the initial part of startup.

[0141] In one example, similar to the previous example, except that the image acquisition parameters for normal use of the camera do not have fixed values, once phase 2 is reached the parameters are dynamically changed until a temperature close to or equal to the temperature at the moment of calibration is reached. The main advantage of this feature is that the system can to some extent mitigate the effects of external conditions (e.g., ambient temperature, exposure to other heat sources such as surgical lights, etc.).

[0142] In a further example similar to the previous example, except that image acquisition parameters for normal use during stage 2 are dynamically adjusted, external conditions, e.g., ambient temperature, acquisition parameters are varied to maintain a system temperature close to the calibration temperature.

[0143] A further example is provided that is similar to the previous example, except that the system uses a model of the thermal behavior of the system to predict how to adjust image acquisition parameters so that a desired thermal stage / system temperature can be reached and maintained. Preferably, the system is self-learning, adjusting its model based on recorded data from previous warm-up cycles and / or based on input from various sources, e.g., temperature sensors, imaging sensors.

[0144] In another example, similar to the previous example, except that the camera is modeled and controlled according to minimum time control (or optimal control), the system reaches equilibrium with only two control actions (e.g., the fastest possible transient). In such a case, the camera is initially configured to consume as much power as possible, and once a state is reached, the camera is configured to consume as little power as possible until a desired state (steady state) is reached. The camera is then configured for the steady state.

[0145] 4 shows the steps of an example method 100 for the operation of a camera-based tracking system. The method 100 comprises the following steps: In a first step 102, at least one operating signal is generated during a start-up phase of the tracking system, the at least one operating signal comprising at least one calibration compensation instruction during at least the start-up phase of the tracking system, the at least one calibration compensation instruction comprising an instruction for at least one of the group: i) actively adjusting a temperature of the camera-based tracking system, and ii) adapting a calibration-related camera output of the camera-based tracking system used in the tracking calculations. In a second step 104, at least one operating signal is provided to the tracking system. In a third step 106, the camera-based tracking system is operated based on an operating signal having at least one calibration compensation command.

[0146] Actively adjusting a camera-based tracking system involves adjusting the system in its geometric setup. Active adjustment modifies the actual configuration settings. Adapting the camera output involves modifying the image processing with respect to the spatial alignment of the images provided by the camera. Adapting the camera output modifies the virtual setup provided by calibration.

[0147] In one example, shown by the hashed lines in FIG. 3, at an initial step 108, a signal is received to activate a camera-based tracking system.

[0148] In one example, a method option is provided in which generating at least one operating signal for the tracking system includes generating at least one control signal to temporarily operate at least one of the at least two cameras of the system in a heating mode to generate heat by at least one of the at least two cameras. The at least one control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode, in which a higher energy consumption by the camera occurs than in a normal tracking mode when the camera is operated to track a marker. In this example, operating the camera-based tracking system includes operating at least one of the at least two cameras in a high energy consumption mode, thereby heating at least the camera with a higher energy consumption compared to the normal tracking mode.

[0149] A reduction in the discrepancy between the current state of the camera-based tracking system and the predetermined conditions of the system assumed for the calibration of the system is provided by heating of at least one camera with higher energy consumption compared to the normal tracking mode.

[0150] In one example, a method for operating a camera-based tracking system is provided, the method comprising: receiving a signal to activate a camera-based tracking system having at least two cameras; generating, based on the activation signal, at least one operation signal for the tracking system, the operation signal having at least one control signal for temporarily operating at least one of the at least two cameras in a heating mode to cause heat to be generated by at least one of the at least two cameras; the at least one control signal is configured to operate at least one of the at least two cameras in a high energy consumption mode, resulting in higher energy consumption by the camera than in a normal tracking mode in which the camera is operated to track the marker; providing at least one operational signal to the tracking system; operating at least one of the at least two cameras in a high energy consumption mode, thereby heating at least the camera with higher energy consumption compared to a normal tracking mode; It has.

[0151] Thus, a faster warm-up curve is provided to reach a temperature at least essentially similar to the calibration temperature so that errors due to temperature dependence in the optical system are minimized.

[0152] In one example of the method, at least one camera is operated in a high energy consumption mode until a predetermined thermal condition is reached.

[0153] Optionally, the high energy consumption mode is maintained until at least one of the group of at least one camera and system reaches its normal operating temperature, which is the basis for the calibration of the system.

[0154] In another option, the high energy consumption mode is provided beyond the point at which at least one camera has reached its normal operating temperature.

[0155] The normal operating temperature can also be referred to as the calibration temperature of the system.

[0156] In one example, at least one camera is operated in a high energy consumption mode until a predetermined temperature of the camera is reached.

[0157] In another example, at least one camera is operated in a high energy consumption mode for a predetermined period of time.

[0158] In one example, high power consumption is maintained for longer than required to reach the camera's normal operating temperature so that the portion of the system that acts as a heat sink, such as the camera holder or frame, reaches the target temperature more quickly, after which the system switches to a lower power consumption setting.

[0159] In one example, the portion of the system that acts as a heat sink for the camera is designed so that thermal equilibrium of the system is reached as quickly as possible, e.g., providing that the heat flow from the camera to the frame is optimized so that delays in warming up the frame and associated suboptimal system performance are minimized.

[0160] Preferably, the part of the system that acts as a heat sink for the camera is designed so that thermal equilibrium of the system is reached as quickly as possible. One example could be to optimize heat flow from the camera to the frame so that delays in frame warm-up (and associated suboptimal system performance) are minimized.

[0161] In one example of the method, after heating at least one of the at least two cameras by operating the at least one camera in a high energy consumption mode, a normal operation signal is provided and the at least one camera is operated under normal conditions.

[0162] Optionally, the settings for normal operation are modified based on the difference between the system temperature at the time of calibration and the expected steady state temperature of the system under real-time conditions.

[0163] In one example, the settings used for normal operation are not fixed, but are adjusted based on the difference between 1) the system temperature at the time of calibration and 2) the expected steady-state temperature of the system under real-time conditions. For example, if the system temperature during normal operation is higher or lower than the temperature at the time of calibration, the camera parameters are selected so that the camera consumes less or more power, thereby bringing the system closer to the calibrated state.

[0164] In one example, camera settings for normal operation are optimized for minimum power consumption while providing adequate image data. This contributes to the steady-state temperature of the optical system during normal operation, thus reducing the temperature difference between the cold start condition (where the camera is at room temperature) and the temperature at which thermal equilibrium is reached. As a result, not only is the time required to reach normal operating temperature reduced, but the system also experiences fewer temperature fluctuations (and associated performance degradation) during start-up.

[0165] In one example of the method, a model of the warm-up of the system and its dynamic thermal behavior is provided and taken into account for at least one control signal to generate at least one operating signal for the tracking system, the at least one control signal comprising a signal sequence having a plurality of varying control sub-signals.

[0166] An example of the method provides for analyzing at least two images from different points in time during warm-up to determine the current displacement speed and adjust the high energy consumption mode accordingly.

[0167] In one example of this method, multiple sensors are provided that monitor the thermal behavior of the system. The thermal behavior monitored by the sensors is used to provide a more accurate model of the system's warm-up. Optionally, the model of the system's warm-up is used in combination with multiple temperature readings from the sensors during warm-up to dynamically adjust the system's calibration. As a further option, a temperature history of the system is provided and a remaining time is predicted before the system reaches a normal operating phase. Optionally, the remaining time is indicated.

[0168] In an example provided as another option for the method, providing a dynamic calibration model for a camera-based tracking system, the dynamic calibration model having a plurality of calibration coefficients that are dependent on a temperature-related state of the system; providing a current temperature-related state of the camera-based tracking system; determining a matching calibration coefficient from the plurality of calibration coefficients; providing matching calibration coefficients for application during operation of the camera-based tracking system; is provided.

[0169] Therefore, a modified calibration is provided during the start-up phase that provides a matching temperature that is at least essentially similar to the dynamic calibration temperature so that errors due to temperature dependence in the optical system are minimized.

[0170] In one example, the temperature-related conditions include temperature measurements from a sensor. For example, the sensor is provided within the system. Optionally, the sensor is integrated and provides a temperature signal for other system operation aspects as well. Optionally, the sensor is provided exclusively for calibration adjustments. According to one aspect, the temperature-dependent behavior of the optical system is measured and modeled. In addition to knowing the changing characteristics in advance (e.g., based on simulation or empirical data), additional temperature sensors can be provided to monitor the temperature of the camera and supporting hardware in real time.

[0171] Rather than using a static calibration associated with steady state, a dynamic calibration model is provided. In one example, the dynamic calibration model depends on the time elapsed since start-up. In one example, the dynamic calibration depends on the current temperature conditions, for example, using temperature sensor readings. In one example, the dynamic calibration model is used taking into account the time elapsed since start-up and the current temperature conditions.

[0172] In one example, a look-up table is provided in advance.

[0173] In one basic approach, only the time since start-up is used to pick up the correct set of calibration parameters from a look-up table.

[0174] Another basic approach provides measurements from a temperature sensor that are used to pick the correct set of calibration parameters from a look-up table.

[0175] For example, an internal temperature sensor in one of the camera chips is used to select the set of calibration parameters that will likely give the best results.

[0176] In another more advanced approach, the starting temperature of the camera is used in combination with the ambient temperature to estimate the warm-up curve and therefore provide the correct calibration coefficients.

[0177] In one example, the current temperature-related conditions are updated, for example, at constant or varying intervals, to provide two or more updated calibration factors during system warm-up.

[0178] 5 shows a graph with a curve 200 showing tracking error with respect to time after initiation. The vertical axis 202 is the tracking error E TR in millimeters, and the horizontal axis 204 is the time after start-up t AST is shown in minutes. During warm-up of the optical tracking system, the tracking error gradually decreases as the temperature of the system stabilizes until it reaches the system's baseline level 206 (intrinsic accuracy). Preferably, the time A required to reach steady state from the maximum error level 208 is short to allow the user to use the system immediately after start-up. The difference B in tracking error associated with different temperature conditions of the system, e.g., cold start or operating temperature, is a measure of the system's robustness to temperature fluctuations. A vertical line 210 separates a first stage S1, also referred to as stage 1, from a second stage S2, also referred to as stage 2.

[0179] 6 shows a graph with a curve 220 showing the temperature of the system versus time after start-up. The vertical axis 222 shows the temperature T in °C (Celsius), and the horizontal axis 224 shows the time after start-up t AST , in minutes. The graph can be divided into a warm-up phase, e.g., stage S1, and a more steady-state operating phase, e.g., stage S2. The separation is indicated by vertical line 226.

[0180] FIG. 7 shows a graph 230 with different curves showing system temperature over time for different operating modes. The vertical axis 232 shows temperature T in °C, and the horizontal axis 234 shows time t in minutes. In an optical tracking system with an industry-standard machine vision camera, the camera's measured power consumption can be increased by changing image acquisition parameters. In this particular example, increasing the frame rate from 15 frames per second (fps) to 30 frames per second (fps) increases power consumption from 2.6 W to 2.96 W. The warm-up of the camera imaging chip and camera housing was accelerated by 3.5 times and 1.4 times, respectively, by boosting warm-up using a higher frame rate. A first curve 236 shows the camera's internal temperature for 30 fps, and a second curve 238 shows the camera's internal temperature for 15 fps. A third curve 240 shows the external temperature, i.e., the temperature outside the camera, for 30 fps, and a fourth curve 242 shows the external temperature, i.e., the temperature outside the camera, for 15 fps. A further curve 244 shows the air temperature. As can be seen, increasing the frame rate results in a significant increase in temperature during an initial phase where the temperature rises sharply, then levels off towards a steady-state temperature.

[0181] 8 shows graphs illustrating an example of a warm-up pattern of system temperature and camera power consumption over time. In the top graph, the vertical axis 250 represents temperature T in °C, and the horizontal axis 252 represents time t after start-up. AST In the lower graph, the vertical axis 254 represents the power consumption P of the camera in W and the time in minutes after start-up t AST is again shown on the horizontal axis 256. During phase S1, the default high power demand setting D EF The H-PS is used to accelerate the warm-up of the system. When stage S2 is reached, the system warms up according to the temperature curve T S For comparison, a further curve 262 is shown for the default normal power setting, D EFIn the upper graph, a horizontal line 284 indicates the calibration temperature T C As can be seen, application of a higher power demanding camera setting results in an increase in temperature and a faster time to achieve operating temperature.

[0182] 9 shows graphs illustrating another example of a warm-up pattern of system temperature and camera power consumption over time. In the upper graph, vertical axis 270 represents temperature T in °C, and horizontal axis 272 represents time t in minutes. In the lower graph, vertical axis 274 represents camera power consumption P in W, and horizontal axis 276 represents time t in minutes. At the start of stage S2, image acquisition parameters and associated camera power consumption are dynamically adjusted until the calibration temperature is reached. The first portion 278 of the curve represents the warm-up pattern at the default high power setting, D EF The second portion of the curve 280 shows the application of the H-PS, dynamic power setting, D YN The curve 282 in the lower graph shows the application of the camera power consumption CPC. Also in the upper graph, the horizontal line 310 indicates the change in the calibration temperature T C As can be seen, a dynamic adaptation of the power consumption is provided in the transition period at the boundary between the first stage S1 and the second stage S2.

[0183] 10 shows graphs illustrating further examples of warm-up patterns of system temperature and camera power consumption over time. In the upper graph, the vertical axis 290 is the system temperature T S in ° C., and the horizontal axis 292 is the time t AST In the center graph, the vertical axis 294 is the ambient temperature T A in ° C., and the horizontal axis 296 is the time t AST In the lower graph, the vertical axis 298 represents the power consumption P of the camera in W, and the horizontal axis 300 represents the time after start-up t AST During stage S2, image acquisition parameters (and associated camera power consumption) are dynamically adjusted to mitigate the effects of external conditions, e.g., fluctuations in ambient temperature, so as to maintain a system temperature close to that at the time of calibration. The first portion 302 includes a default high power setting, D EFThe second part of the curve 304 shows the application of H-PS, and the second part of the curve 304 shows the dynamic power setting, D YN 3 shows the application of PS. The curve 306 in the middle graph shows a decrease in ambient temperature. The curve 308 in the lower graph shows the changing camera power consumption CPC. As can be seen, even after the initial heating, further adjustments in the system temperature can be accommodated in light of the calibration during operation. As an example, a change in ambient temperature is shown. In response, the camera power consumption is adjusted accordingly.

[0184] The term "subject" may also be referred to as an individual. A "subject" may also be referred to as a patient, although it should be noted that this term does not indicate whether any disease or condition actually exists in the subject.

[0185] In one example, a computer program or program element for controlling an apparatus according to one of the above examples is provided, which program or program element is configured to perform the method steps of one of the above method examples when executed by a processing unit.

[0186] Thus, a computer program element may be stored in a computing unit or distributed across two or more computing units that may be part of an embodiment of the present invention. This computing unit may be configured to perform or direct the execution of the steps of the above-mentioned method. Furthermore, it may be configured to operate the components of the above-mentioned apparatus. The computing unit can be configured to operate automatically and / or to execute a user's order. The computer program may be loaded into the working memory of a data processor. The data processor may thus be configured to perform the method of the present invention.

[0187] Aspects of the present invention may be embodied in a computer program product, which may be a collection of computer program instructions stored on a computer-readable storage device that can be executed by a computer. The instructions of the present invention may be any interpretable or executable code mechanism, including, but not limited to, a script, an interpretable program, a dynamic link library (DLL), or a Java class. The instructions may be provided as a complete executable program, a partial executable program, a modification (e.g., an update) to an existing program, or an extension (e.g., a plug-in) to an existing program. Furthermore, portions of the processing of the present invention may be distributed across multiple computers or processors.

[0188] As described above, a processing unit, e.g., a controller, implements the control method. This controller can be implemented in a variety of ways using software and / or hardware to perform the various functions required. The processor is one example of a controller that uses one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. However, the controller may be implemented with or without a processor, or as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.

[0189] Examples of controller components used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0190] This exemplary embodiment of the present invention encompasses both computer programs that use the present invention from the beginning, and computer programs that convert existing programs into programs that use the present invention by means of an update.

[0191] Furthermore, the computer program element may be capable of providing all the steps necessary to fulfill the procedures of the exemplary embodiments of the methods described above.

[0192] According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented, having stored thereon a computer program element, which computer program element is described by the previous section. The computer program may be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0193] However, the computer program may also be presented over a network such as the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, the computer program element being configured to perform a method according to one of the aforementioned embodiments of the present invention.

[0194] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method-type claims, and other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will understand from the above and below description that, unless otherwise specified, any combination of features belonging to one type of subject matter, as well as any combination between features relating to different subject matters, is disclosed in the present application. However, all features can be combined to provide a synergistic effect that is greater than the simple sum of the features.

[0195] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary or explanatory and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the dependent claims.

[0196] In the claims, the word "comprise" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A controller for operation of a camera-based tracking system, said controller comprising: A controller; a control signal output unit; and the controller is configured to generate, at least during a start-up phase of the tracking system, at least one operation signal for the tracking system having at least one calibration compensation command that at least partially compensates for a difference between a current state of the camera-based tracking system and a predetermined condition of the tracking system assumed for calibration of the tracking system; the at least one calibration compensation instruction comprises instructions for at least one of: i) actively adjusting a temperature of the camera-based tracking system; and ii) correcting image processing with respect to spatial alignment of an image provided by a camera; the control signal output is configured to provide the at least one operating signal to the tracking system. Control device.

2. The controller of claim 1 , wherein the calibration compensation instructions include at least one control signal for temporarily operating the camera-based tracking system in a high heat-producing mode.

3. To temporarily operate the camera-based tracking system in the high heat production mode, the calibration compensation instructions include at least one control signal for temporarily operating at least one of the at least two cameras in a heating mode that generates heat by at least one of the at least two cameras, the at least one control signal being configured to operate the at least one of the at least two cameras in a high energy consumption mode in which higher energy consumption is achieved by the camera than in a normal tracking mode in which the camera is operated for the tracking; the control signal output unit is configured to provide the at least one operating signal to the tracking system to operate the at least one of the at least two cameras in the high energy consumption mode, thereby heating at least the camera with higher energy consumption compared to the normal tracking mode. The control device according to claim 2 .

4. The controller of claim 2 , wherein for the high heat production mode, the calibration compensation instructions include at least one control signal for temporarily activating a heating element of the camera-based tracking system.

5. The controller is configured to take into account measured external conditions relative to the control signal when generating the operating signal, the external conditions comprising: The ambient temperature, The lighting conditions and Indoor airflow and having at least one of the following groups: The control device according to claim 2 .

6. to generate the at least one operating signal for the tracking system, the controller is configured to provide a model of the warm-up of the system and its dynamic thermal behavior and to take these into account for the at least one control signal; the at least one control signal comprises a signal sequence having a plurality of varying control sub-signals; The control device according to claim 2 .

7. the controller is configured to provide a dynamic calibration model for the camera-based tracking system, the dynamic calibration model having a plurality of calibration coefficients responsive to a temperature-related state of the system; for a current temperature-related state of the camera-based tracking system, the controller is configured to determine a matching calibration coefficient from the plurality of calibration coefficients; the control signal output is configured to provide the matching calibration coefficients for application during operation of the camera-based tracking system. The control device according to claim 1 .

8. a plurality of at least two cameras; A control device according to any one of claims 1 to 7; In a camera-based tracking system having the at least one operating signal is provided to operate at least one of the at least two cameras; the camera-based tracking system is a camera-based medical tracking system for use in a hospital or other medical-related setting; Camera-based tracking system.

9. the high energy consumption mode comprises image acquisition with energy consumption at least 50% higher than energy consumption during a normal tracking mode of the system; The high energy consumption mode includes: The frame rate and The exposure time, Bit depth and Bit packing and a region of interest; The clocking frequency, Color format and Image compression and On-board image pre-processing and at least one of the groups has image acquisition that is at least 50% increased compared to a normal tracking mode of the system; The system according to claim 8 dependent on claim 3.

10. a plurality of sensors are provided for monitoring the thermal behavior of the system; the thermal behavior monitored by the sensor is used to provide a more accurate model of the warm-up of the system; the model of the system warm-up is used in combination with multiple temperature readings from sensors during warm-up to dynamically adjust the calibration of the system; A temperature history of the system is provided, and a remaining time before the system reaches a normal operating stage is predicted and the remaining time is indicated. The system of claim 8.

11. 9. The system of claim 8 when dependent on claim 3, wherein the controller is configured to analyze at least two images from different points in time during the warm-up to determine a current displacement rate and adjust the high energy consumption mode accordingly.

12. a set of markers capable of being tracked by the system is provided; the marker is attachable to at least one of the group of an object, an object, and a device; The system of claim 8.

13. 1. A method for operation of a camera-based tracking system, comprising: generating, at least during a start-up phase of the tracking system, at least one operating signal for the tracking system having at least one calibration compensation command that at least partially compensates for a difference between a current state of the camera-based tracking system and a predetermined condition of the tracking system assumed for calibration of the tracking system; the at least one calibration compensation instruction comprises instructions for at least one of i) actively adjusting a temperature of the camera-based tracking system, and ii) correcting image processing with respect to spatial alignment of an image provided by a camera; providing the at least one operating signal to the tracking system; operating the camera-based tracking system based on the operating signal having at least one calibration compensation command; A method having the following.

14. A computer program enabling a processor to carry out the method according to claim 13.

15. A computer readable medium storing the computer program of claim 14.

Images