Coordinate system calibration of an optical sensor system

Abstract

Disclosed herein is a method for calibrating a first coordinate system of a field of view of an optical sensor system (140). The calibration comprises registering the first coordinate system with a second coordinate system of a medical imaging system (100) using a keypoint detection module (412). The keypoint detection module (412) is configured for detecting, as output, a keypoint element (122) of a patient table (120) of the medical imaging system (100) within optical sensor data (414) in response to receiving the optical sensor data (414). The method comprises determining three-dimensional coordinate values describing a position of the detected keypoint element (122) within the first coordinate system, and receiving three-dimensional coordinate values describing a position of the detected keypoint element (122) within the second coordinate system. One or more transformation parameters (420) are determined for a transformation from the first coordinate system to the second coordinate system.

Description

Technical Field

[0001] This invention relates to medical imaging, and more particularly to a method for calibrating a coordinate system of the field of view of an optical sensor system configured for supervising a medical imaging system, a computer program including machine-executable instructions for performing the method, a computing device for performing the method, and a medical imaging system including the computing device. Background Technology

[0002] For monitoring medical imaging systems (such as magnetic resonance imaging, computed tomography, or X-ray imaging systems), optical sensor systems can be used. In order to determine the location of a target structure (such as a patient and / or components of the medical imaging system) within the optical sensor data acquired using such a system, calibration of the coordinate system's coordinate system is necessary. Calibration involves determining the transformation from the coordinate values ​​of the optical sensor system's coordinate system to the coordinate values ​​of the medical imaging system. Based on the calibration, it can be determined which coordinate values ​​in the optical sensor system's coordinate system correspond to which coordinate values ​​in the medical imaging system's coordinate system.

[0003] For this purpose, a current calibration phantom is used, configured to be aligned with a medical imaging system and simultaneously configured to be measurable using an optical sensor system. For alignment, the medical imaging system must typically be in a specific predefined configuration; for example, a specific portion of the medical imaging system must be positioned in a predefined location relative to the rest of the medical imaging system so that the calibration phantom can be aligned with the medical imaging system in the predefined location.

[0004] Calibration phantoms can include checkerboard patterns or other known designs, such as those incorporating asymmetrical elements. The size of the calibration phantom is chosen so that the field of view of the optical sensor system is adequately covered. This typically requires a fairly large phantom, up to human size. The phantom must be stored in the field, or field service personnel must be able to recreate it on-site, for example, by printing. Storage in a hospital setting may not be straightforward due to its size. On the other hand, if individual parts must be joined together using adhesives, recreating the phantom in the field can easily lead to errors, such as incorrect scale in the printed output or misalignment of individual parts. Generally, a trade-off must be made between the handling of the phantom and the calibration accuracy achievable using it. This often results in drawbacks regarding calibration accuracy. Furthermore, field service personnel must be physically present on-site to perform the calibration process using the phantom.

[0005] US11600021B2 and US2022148157A1 disclose examples of using phantoms to correlate the coordinate system of the field of view of an optical sensor system with the coordinate system of a medical imaging system. Summary of the Invention

[0006] This invention provides a method for calibrating a coordinate system for the field of view of an optical sensor system, a computer program including machine-executable instructions for performing such a method, and a computing device for performing such a method. Furthermore, this invention provides a medical imaging system including such a computing device for performing the method.

[0007] In one aspect, the present invention provides a method for calibrating a first coordinate system for the field of view of an optical sensor system. Calibration includes registering the first coordinate system with a second coordinate system of a medical imaging system using a computing device. The computing device includes a keypoint detection module configured to detect keypoint elements of the patient stage of the medical imaging system as output within the optical sensor data in response to receiving optical sensor data acquired using the optical sensor system. The optical sensor system is configured for monitoring the medical imaging system.

[0008] The method includes receiving optical sensor data acquired using an optical sensor system. The received optical sensor data includes optical sensor data from the patient table.

[0009] The key point detection module is used to detect key point elements within optical sensor data.

[0010] Using optical sensor data, the two-dimensional horizontal coordinates of the detected keypoint elements are determined within a first coordinate system. These two-dimensional horizontal coordinates describe the position of the detected keypoint elements in a plane parallel to the sensor plane of the optical sensor system.

[0011] Determine the third vertical coordinate value of the detected keypoint element within the first coordinate system. The third vertical coordinate value describes the distance of the detected keypoint element from the sensor plane of the optical sensor system.

[0012] Receive the three-dimensional coordinates describing the position of the detected keypoint element in the second coordinate system of the medical imaging system.

[0013] One or more transformation parameters are determined using the coordinate values ​​describing the position of the detected keypoint element in the first coordinate system and the coordinate values ​​describing the position of the detected keypoint element in the second coordinate system.

[0014] The example demonstrates calibration without requiring an additional calibration phantom. The patient table is used to perform the calibration, rather than using an additional calibration phantom. Using components of a medical imaging system for calibration can have the advantage of allowing calibration to be performed at any time without the need for additional elements or parts located within the medical imaging system. Furthermore, using components of a medical imaging system for calibration enables remote and / or automated calibration without requiring a physically present field service personnel to perform the calibration.

[0015] For example, one or more individual transformation parameters can be determined for the coordinate values ​​of individual keypoint elements. The determined transformation parameters for the transformation from the first coordinate system to the second coordinate system can, for example, be the average of one or more individual transformation parameters.

[0016] For example, the transformation parameters can be determined for different positions of the patient table, and the transformation parameters obtained for different positions of the patient table are used to determine one or more transformation parameters for the transformation from the first coordinate system to the second coordinate system, for example, as the average value of one or more transformation parameters for different positions of the patient table.

[0017] For example, the keypoint detection module can be configured to determine the position of detected keypoint elements of the patient station within a first coordinate system. The keypoint detection module can be configured to determine the two-dimensional horizontal coordinate values ​​of the detected keypoint elements and / or to determine the third vertical coordinate values ​​of the detected keypoint elements within the first coordinate system.

[0018] Based on calibration, it can be determined which coordinate values ​​in the coordinate system of the optical sensor system correspond to which coordinate values ​​in the coordinate system of the medical imaging system. Using this transformation, it is possible to determine where a target structure (such as a patient and / or a component of the medical imaging system) is located in the physical world relative to the medical imaging system, based on the position of the target structure determined within the optical sensor data. To detect the target structure, i.e., its keypoint elements, a keypoint detection module can be used, configured to detect keypoint elements of the target structure within the optical sensor data as output in response to receiving optical sensor data acquired using the optical sensor system. The keypoint detection module configured for the keypoint elements of the target structure can be the same keypoint detection module configured for the keypoint elements of the patient table or another keypoint detection module. Additionally, the keypoint detection module can be configured to determine the position of the keypoint elements of the target structure within a first coordinate system.

[0019] Optical sensor systems (e.g., incorporating artificial intelligence (AI) algorithms, such as machine learning modules) can be used to guide and automate patient setup in medical imaging systems. Furthermore, optical sensor systems can be used, for example, to detect non-ideal and / or unsafe system use and setup configurations. For instance, the position of the patient on the patient table and / or the position of components of the medical imaging system (such as cables and / or coils) can be checked, particularly relative to the patient and / or the patient table. Moreover, the positioning of the patient table within the medical imaging system can be controlled based on the detected position of the patient, so as to position the patient at a predefined location within the medical imaging system for acquiring medical imaging data.

[0020] When using such AI algorithms, the underlying AI algorithms rely on optical sensor data acquired using an optical sensor system to detect the patient's body and anatomical structures, as well as related equipment components of the medical imaging system. To achieve high detection accuracy, the coordinate system of the optical sensor system (i.e., the coordinate system of the optical sensor system's field of view) needs to be calibrated and registered relative to the coordinate system of the imaging modality (i.e., the second coordinate system of the medical imaging system).

[0021] Such calibration is a prerequisite for identifying the real-world location of elements detected in the optical sensor data (i.e., within the first coordinate system). Here, real-world location refers to the location within the second coordinate system of the medical imaging system.

[0022] Installing a medical imaging system (such as an MRI system) is a time-consuming process, and adding any complexity to that process only increases the challenge. Examples provide methods to facilitate this process.

[0023] The example implements a simple and efficient calibration that can be performed, for example, along with other installation steps of a medical imaging system, can be performed, for example, remotely, and / or can be performed, for example, automatically. For example, a keypoint detection module including a neural network model is used to detect specific keypoint elements of the patient stage of a medical imaging system. Such a keypoint detection module can be implemented as a machine learning module. For example, the keypoint detection module can use a fully convolutional U-Net architecture to detect keypoint elements. The U-Net architecture is described in Olaf Ronneberger, “U-Net: Convolutional Networks for Biomedical Image Segmentation” (https: / / arxiv.org / abs / 1505.04597).

[0024] For example, a keypoint detection module can be configured to determine a confidence level for the detection of keypoint elements within optical sensor data. For example, the keypoint detection module can be configured to assign a confidence level to the detected keypoint elements. The keypoint detection module can be trained, for example, during a training phase to assign confidence levels to keypoint elements detected within training data. These confidence levels determined using the keypoint detection module can, for example, be estimates trained using a loss function that quantifies the deviation of the detected keypoint elements from a baseline true value of the keypoint elements provided by the optical training data. The confidence level can, for example, be a function of the output of the keypoint detection module. The maximum, minimum, average, or integral of the output of the detection module can be examples of quantities that can be used to derive a confidence measure within a training distribution provided by multiple optical training data. Out-of-distribution confidence values ​​can, for example, be derived using the temporal stability of the output signal of the keypoint detection module, the stability of the output signal against perturbations (such as noise), or other spatial and / or temporal properties of the output signal provided by the keypoint detection module as a response to received optical sensor data as input.

[0025] The position of the patient stage relative to the medical imaging system is a calibrated system parameter of the medical imaging system itself; that is, the relative position of the patient stage within the medical imaging system is known, for example, the position of the patient stage relative to the imaging isocenter or a specific reference component of the medical imaging system (e.g., the gantry in the case of an MRI system). Calibration using the patient stage of the medical imaging system has the following advantages: the patient stage can typically be moved in three dimensions, where the positions of all key point elements of the patient stage within the coordinate system of the medical imaging system are always accurately known.

[0026] Patient setup (e.g., preparation) in medical imaging systems is a time-consuming task that typically requires trained personnel. Even for trained and skilled operators, the complexity of this task can distract attention from the patient. Attempting to perform this task efficiently and effectively can lead to increased operator stress levels. Therefore, there are risks that this situation may result in limitations in the consistency of examination quality, potentially impairing the patient and / or operator experience; and the potential for errors, including harm to the patient.

[0027] Optical sensor systems can be used to address these issues by providing assistance in patient settings when needed. Furthermore, they can aid in detecting suboptimal and / or unsafe system use and setup configurations.

[0028] The aid relies on optical sensor data acquired using an optical sensor system to detect relevant parts of the patient's body and anatomical structures, as well as medical imaging systems. Therefore, high-precision and robust detection is required.

[0029] An optical sensor system may include multiple optical sensors. For example, an optical sensor may include one or more RGB sensors, i.e., optical sensors configured to acquire optical sensor data including red (R), green (G), and blue (B) light. An optical sensor may include one or more depth sensors, i.e., optical sensors configured to determine the distance of an object from the sensor, for example, using triangulation. An optical sensor may include one or more thermal sensors, i.e., optical sensors configured to acquire optical sensor data in the infrared (IR) range. For example, a thermal sensor may acquire optical sensor data in the range of 1 μm to 15 μm (e.g., 10 μm) at room temperature.

[0030] To achieve high detection accuracy for key element detection, the first coordinate system of the optical sensor system needs to be calibrated and registered relative to the second coordinate system of the imaging modality (i.e., the optical sensor system). This registration calibration is a field- and system-specific calibration, meaning it depends on the specific implementation of the medical imaging system with the optical sensor system and its field application, and is independent of the manufacturer's inherent calibration of the optical sensor(s)(one or more) of the optical sensor system. Field- and system-specific calibration can be performed, for example, as part of a field service maintenance procedure. Such calibration can be performed, for instance, during setup of the medical imaging system with the optical sensor system and / or during low-routine service intervals.

[0031] Furthermore, vibrations may occur in the medical imaging system, which may necessitate monitoring the accuracy of calibration for preventative maintenance. Such changes can occur, for example, during imaging, as in the case of an MRI or computed tomography (CT) system, or can be caused by door closure. An example could have the beneficial effect that, for such monitoring, a calibration phantom is not required. For monitoring, the calibration method can essentially be repeated, and the results can be compared with previous calibration parameters and test procedures to determine whether the previous calibration parameters remain correct.

[0032] For example, a keypoint detection module is used to detect specific keypoint elements of a patient stage in a medical imaging system. This module may include, for example, a neural network model. The stage dimensions, the positions of the keypoint elements, and the stage orientation and rotation may be known. Furthermore, the patient stage position may be a calibrated system parameter of the medical imaging system itself; that is, the position of the patient stage relative to the medical imaging system (e.g., the gantry and / or imaging isocenter of the medical imaging system) can be detected and known by the medical imaging system. The patient stage position relative to the medical imaging system may be stored, for example, in the system configuration file of the medical imaging system.

[0033] Using information about the position of the patient stage relative to the medical imaging system and information about the position of keypoint elements relative to the patient stage, the position of the keypoint elements relative to the medical imaging system, and therefore their position within the second coordinate system of the medical imaging system, can be determined. For example, this determination is possible as long as the position of the patient stage relative to the medical imaging system is monitored and known by the medical imaging system, regardless of where the stage is located in the lateral and vertical directions.

[0034] Therefore, the patient table can be effectively used for calibration of the first coordinate system with high accuracy because it can move, for example, in three dimensions, where the positions of the key point elements of the patient table in the second coordinate system are always accurately known.

[0035] Using a patient stage for calibration also offers the following advantages: the patient stage has a size corresponding to the size of the patient, which therefore represents the ideal size for calibrating the field of view of the optical sensor system used for patient-related detection. Furthermore, the patient stage defines the position where the patient will later be positioned for a medical imaging procedure performed using the medical imaging system. Therefore, calibration focusing on the field of view of the optical sensor system on the patient stage results in calibration that focuses on the patient's future position within the medical imaging system.

[0036] The keypoint detection module can, for example, be trained to detect keypoint elements of the patient table using optical sensor data acquired from the patient table. The keypoint detection module can also be trained to detect keypoint elements of the patient table using optical sensor data acquired from a 3D physical model of the patient table. Furthermore, the keypoint detection module can be trained to detect keypoint elements of the patient table using a 3D digital model of the patient table (e.g., a scaled CAD model of the patient table), which is typically a stable and accurate design that allows usability assumptions to be made for a specific product range.

[0037] For example, a 3D digital model of the patient table (e.g., a scaled CAD model) is used to generate training data for training a keypoint detection module. Using a 3D digital model offers the following advantages: it provides the model with labels for identifying keypoint elements of the patient table to be detected. Different training data can be generated by scaling the 3D digital model and / or, for example, randomly generating a background for the 3D digital model. Such a background can be generated through random combinations and / or random processing of image data, such as changing colors, altering color distribution, rotating, translating, adding occlusion to the image data, and / or distorting the image data. Furthermore, the position of the 3D digital model can be varied.

[0038] For example, a simulated pipeline is used to generate synthetic training data for training a keypoint detection module.

[0039] For example, a keypoint detection module is configured to detect multiple predefined keypoint elements of the patient table within optical sensor data of the patient table. For example, keypoint elements are elements arranged on the upper side of the patient table. For example, keypoint elements are elements circumferentially distributed around the upper side (e.g., an inclined surface) of the patient table along a contour line of the upper side. The keypoint detection module may, for example, include a neural network trained to detect predefined relevant keypoint elements of the patient table within the optical sensor data of the patient table.

[0040] An optical sensor system may include, for example, one or more optical sensors. The optical sensor system can be positioned relative to a medical imaging system having a patient table, such that the patient table is included within the field of view of the optical sensor system. For example, the optical sensor system can be positioned above the patient table, looking down onto the patient table. Alternatively, the optical sensor system can be positioned above the patient table at its original location.

[0041] In the case where the optical sensor system includes multiple optical sensors with different fields of view, the optical sensors can be arranged relative to the medical imaging system with a patient table, such that the patient table is included in the field of view of the optical sensors among the multiple optical sensors.

[0042] The patient stage of a medical imaging system can be configured, for example, to move relative to the rest of the medical imaging system, for example, in two to three dimensions.

[0043] Depth sensing methods can be used to determine the distance values ​​of detected keypoint elements, which describe the distance of the keypoint elements to the sensor plane of an optical sensor system.

[0044] Depth sensing methods may, for example, include using registered depth images, i.e., optical sensor data that provides a depth map describing the distances of detected keypoint elements from the sensor plane of the optical sensor system.

[0045] Depth sensing methods may, for example, include using the instantaneous apparent lateral (i.e., horizontal) distance between keypoint elements of the patient table and the known apparent lateral distance between keypoint elements at two known vertical distances (e.g., a known maximum distance and a known minimum distance) from the patient table to the sensor plane of the optical sensor system. The position of the patient table with the minimum distance from the sensor plane can, for example, be the original position of the patient table. The instantaneous distances of the patient table and therefore the keypoint elements can be determined using, for example, interpolation.

[0046] When keypoint elements are arranged at different levels in the vertical direction, the instantaneous distance of the keypoint elements can be considered, for example, as the distance of the keypoint elements in the vertical direction relative to the reference level of the patient table. The reference level of the patient table can be, for example, the upper surface of the patient table, such as the inclined surface of the patient table.

[0047] Depth sensing methods may include, for example, using at least two optical sensors arranged at a fixed baseline between optical sensors, and triangulating the distances of key point elements based on optical sensor data of the patient table acquired using the optical sensors.

[0048] Depth sensing methods may, for example, involve moving the patient table from a first position (e.g., from the lowest position at the maximum distance from the sensor plane of the optical sensor system) to a second position (e.g., from the highest position at the minimum distance from the sensor plane of the optical sensor system), thereby generating optical sensor data for the patient table at multiple positions. Based on this optical sensor data of the patient table at the multiple positions, corresponding three-dimensional coordinate values ​​of keypoint elements can be determined. This optical sensor data can, for example, be used in pseudo-stereo methods to determine the distance of keypoint elements from an image to the sensor plane of the optical sensor system. The movement of the patient table can be controlled manually or automatically in a system-controlled manner.

[0049] Calibration can be performed, for example, using automatically stored optical sensor data from a patient station (preferably an empty patient station). For instance, calibration can be triggered autonomously when a malcalibrated error is detected. Calibration can also be a preventative calibration performed to check the transformation parameters by comparing them with those determined by the preventative calibration.

[0050] Transformation parameters for the transformation from the first coordinate system to the second coordinate system are determined using the known and / or measured 3D coordinates of the keypoint elements in both coordinate systems. The determined transformation parameters may include one or more rotations, one or more translations, scaling, and / or one or more distortions.

[0051] A three-dimensional digital model of the patient table, such as a scaled CAD model of the patient table, can be provided to generate synthetic training data with defined keypoint elements and labels identifying these keypoint elements using a simulated pipeline. Using the resulting synthetic training data, a keypoint detection module (e.g., a neural network included in the keypoint detection module) is trained to detect keypoint elements in optical data, particularly in optical sensor data, i.e., in a real scanning environment. For example, a fully convolutional U-net architecture can be used to encode a probabilistic map (e.g., a two-dimensional Gaussian heatmap) of the keypoint elements, from which the coordinates and confidence values ​​of the keypoint elements within the field of view of the optical sensor system can be estimated. However, other neural network architectures may also be suitable.

[0052] Using a keypoint detection module, unique points in three dimensions can be identified from optical sensor data acquired using an optical sensor system. Additionally, depth sensing methods can be used to determine a third coordinate, i.e., the distance of the detected keypoint element from the sensor plane of the optical sensor system.

[0053] The patient table moves, for example, from a first extreme position (e.g., from the lowest position at the maximum distance from the sensor plane of the optical sensor system) to a second extreme position (e.g., from the highest position at the minimum distance from the sensor plane of the optical sensor system). In the first extreme position, the patient table or its upper surface (e.g., an inclined surface) may be positioned, for example, at a first height h0 = table-down. In the second extreme position, the patient table or its upper surface (e.g., an inclined surface) may be positioned, for example, at a second height h1 = table-up. The position of the patient table at height h1 may be, for example, the original position of the patient table. For example, two extreme positions can be selected such that, for a given imaging modality, i.e., a given field of view, the relevant space of the patient area is covered. During the movement of the patient table from one of the two extreme positions to the other, optical sensor data of the patient table can be acquired using an optical sensor system.

[0054] This movement can be driven, for example, by a medical imaging system that can also trigger data acquisition using an optical sensor system, synchronizing the patient table status and camera image acquisition. Therefore, for each data acquisition, the position of the patient table within the medical imaging system can be known.

[0055] Alternatively, the optical data itself can be used to determine the distance between the patient table and the sensor plane of the optical sensor system. The distance between the patient table and the sensor plane of the optical sensor system can be converted into the height h of the patient table. This distance can be determined, for example, by comparing the apparent size s_h of the patient table in the optical sensor data with the apparent sizes s_h0 and s_h1 of the patient table measured at two extreme positions h0 and h1. The apparent size s_h of the patient table in the optical sensor data can be determined, for example, by measuring the distance between detected keypoint elements, for example, arranged on opposite sides of the patient table. To determine the height h, the following formula can be used: h = (s_h – s_h0) / (s_h1 – s_h0) * (h1 – h0) + h0.

[0056] For example, based on the calibration parameters of the medical imaging system itself, the position of the patient stage surface relative to the medical imaging system (e.g., the imaging isocenter of the medical imaging system) is known for the patient stage positioned at a height h1 in its original location. This information can then be used to calculate the three-dimensional position of the keypoint element in the scanner space (i.e., within the second coordinate system) for a given image in the scanner space, as described above. Therefore, three-dimensional coordinate values ​​describing the position of the detected keypoint element within the second coordinate system of the medical imaging system can be provided.

[0057] For the optical sensor space, i.e., the first coordinate system, the two-dimensional position of keypoint elements can be determined using optical sensor data. The determination of the two-dimensional position of keypoint elements can be achieved by using a keypoint detection module to detect portions of the keypoint elements within the optical sensor data.

[0058] The missing third coordinate value can be determined using depth sensing methods. For example, using a registered depth image, i.e., optical sensor data that provides a depth map describing the distance of detected keypoint elements from the sensor plane of the optical sensor system, the depth value of the keypoint element, i.e., the distance, can be simply read out at the two-dimensional keypoint location within the optical sensor data.

[0059] For example, the depth d_kp of a keypoint element can be reconstructed from the apparent size s_h of the patient stage of the keypoint element at d_kp, the corresponding reference depths d_rkp_h0 and d_rkp_h1 of the individual reference keypoint elements, and the height difference D_kp_rkp between the reference keypoints using the following formula: d_kp = (s_h – s_h0) / (s_h1 – s_h0) * (d_rkp_h1 – d_rkp_h0) + d_rkp_ h0 + D_kp_rkp.

[0060] For a given calibration, i.e., a given set of one or more transformation parameters, the original position of the patient station can be transformed from the second coordinate system of the medical imaging system to the first coordinate system of the optical sensor system's field of view. For the transformation from the second coordinate system to the first coordinate system, inverse transformation parameters can be used. A keypoint detection module, applied to the optical sensor data of the patient station positioned in its original position, can periodically compare the obtained transformation estimate with the actual results of keypoint element detection. If a deviation between the transformation estimate and the actual result is detected exceeding a predefined threshold, a notification can be generated and output, for example. For example, additional optical sensor data can be automatically acquired and recorded for new calibration of the first coordinate system. For example, the additional optical sensor data can be used to repeat the method used to calibrate the first coordinate system. The additional optical sensor data can be selected, for example, to include only the optical sensor data of the patient station, where the patient or operator is not within the station area. The additional optical sensor data can be selected, for example, to include a predefined number of detected keypoint elements and / or the following predefined number of detected keypoint elements: the confidence value determined for the detected keypoint elements exceeds a predefined confidence threshold. Therefore, the selection of high-quality optical sensor data for calibration can be ensured.

[0061] For example, if an updated (e.g., accurate) calibration is available, a notification (e.g., output) can be provided, i.e., for one or more updated transformation parameters for the updated transformation from the first coordinate system to the second coordinate system. This notification can be provided to the operator, who can then activate the updated calibration.

[0062] For example, a test can be performed to compare the predicted original position of the patient station with the actual original positions of both the new (i.e., updated) and old (i.e., currently used) calibrations. For this purpose, suitable recent optical sensor data for the patient station (preferably an empty patient station) at its original position is extracted from the stored calibration data, and both calibrations are applied to the optical sensor data. The results can be presented to the user to confirm the improvement. Alternatively, inverse transformation parameters can be determined for both calibrations, and these parameters can be used to predict the positions of keypoint elements within the optical sensor data. These predictions can be compared with the actual detections of keypoint elements within the optical sensor data. For both predictions, the deviation from the determined actual positions can be indicated to confirm the improvement. Finally, the updated calibration can be activated in response to operator approval.

[0063] Alternatively, a notification can be generated to check the calibration. This notification could, for example, be sent to the field service responsible for maintaining the medical imaging system. In yet another example, if the test passes, the updated calibration can be applied automatically.

[0064] This allows the keypoint detection module to be robust to random obstacles, as typical patient tables can be covered with mattresses, padding, blankets, coil interfaces, and other materials. For example, synthetic training data used to train the keypoint detection module can be supplemented with such random obstacles. The resulting robustness allows for periodic verification of the effectiveness of existing calibrations during normal clinical use.

[0065] In another example, the keypoint detection module is trained to determine all three coordinate values ​​of a keypoint element, rather than just the two-dimensional position of the keypoint element projected onto the sensor plane of the optical sensor system (i.e., the imaging plane projected onto the optical sensor data). Therefore, the keypoint detection module can be configured to automatically infer the distance of the keypoint element from the sensor plane of the optical sensor system using the appearance of the patient table in the optical sensor data. Furthermore, the height of the patient table can be determined, for example, from the distance of the keypoint element, and thus the distance of the patient table can be converted into the height of the patient table. In the case of a standard patient table, the examination table is, for example, non-deformable and generally does not rotate relative to the sensor plane of the optical sensor system, such that the apparent size of the patient table is closely related to its distance from the sensor plane of the optical sensor system.

[0066] For example, a transformation described by one or more transformation parameters includes one or more of the following: one or more rotations, one or more translations, one or more distortions, and scaling.

[0067] Since the coordinate system is three-dimensional, there can be up to three rotations about the three vertical axes, for example, extending into the three dimensions, and therefore up to three corresponding rotation parameters. There can also be up to three translations along the three vertical axes, for example, extending into the three dimensions, and therefore up to three corresponding translation parameters. The number of twists and the number of corresponding twist parameters can, for example, be independent of the number of dimensions. For example, there can be a scaling and a corresponding scaling parameter. Where the scaling can vary locally, this variation can, for example, be described by twists.

[0068] For example, the received optical sensor data includes optical sensor data of a patient table positioned relative to the optical sensor system, for which a depth map describing the distance of detected keypoint elements from the sensor plane of the optical sensor system is received. The depth map is used to determine the third vertical coordinate value of the detected keypoint elements.

[0069] The example can have the following beneficial effect: the third vertical coordinate value of the detected keypoint element in the first coordinate system can be read from the depth map.

[0070] For example, the method further includes receiving a first reference value for the horizontal distance between detected keypoint elements of a patient station determined in first optical sensor reference data. The first optical sensor reference data includes optical sensor data of a patient station positioned in a lower first reference position, which has a first vertical reference distance between the detected keypoint elements and the sensor plane of the optical sensor system. A second reference value for the horizontal distance between detected keypoint elements of a patient station determined in second optical sensor reference data is received. The second optical sensor reference data includes optical sensor data of a patient station positioned in an upper second reference position, which has a second vertical reference distance between the detected keypoint elements and the sensor plane of the optical sensor system.

[0071] Determining the third vertical coordinate value of the detected keypoint elements involves using received optical data to determine the horizontal distance between the detected keypoint elements of the patient table. The third vertical coordinate value is estimated using the determined horizontal distance value, a first reference value and a first vertical reference distance received, and a second reference value and a second vertical reference distance received.

[0072] The example can have the following beneficial effect: the horizontal distance between detected keypoint elements can be used to determine a third vertical coordinate value within a first coordinate system, which is then compared to a reference value for the horizontal distance of a known vertical reference distance. This third vertical coordinate value can be determined, for example, using interpolation.

[0073] For example, the lower first reference position is the lower minimum position of the patient table. The upper second reference position is the upper maximum position of the patient table. The estimation of the third vertical coordinate value includes interpolation of the third vertical coordinate value.

[0074] Using the lower minimum and upper maximum positions as reference positions has the following advantages: these positions are well-defined and can be easily reproduced by moving the patient table to the corresponding extreme positions. Furthermore, all other possible positions of the patient table are arranged between these two extreme positions. Therefore, the third vertical coordinate values ​​of the keypoint elements at the corresponding positions can be determined using all other possible positions in the same manner. For example, transformation parameters for these different positions of the patient table can be determined and used to determine transformation parameters for the transformation from the first coordinate system to the second coordinate system, for example, as the average of one or more transformation parameters for the different positions of the patient table.

[0075] For example, the received optical sensor data includes optical sensor data of a patient table in a first horizontal position and optical sensor data of a patient table in a second horizontal position. The first and second horizontal positions of the patient table are spaced a known distance apart from each other. Detecting keypoint elements within the optical sensor data includes detecting keypoint elements within the optical sensor data of the patient table in the first horizontal position and detecting keypoint elements within the optical sensor data of the patient table in the second horizontal position. Determining the third vertical coordinate value of the detected keypoint elements includes performing triangulation using the positions of the keypoint elements detected within the optical sensor data of the patient table in the first horizontal position and the positions of the keypoint elements detected within the optical sensor data of the patient table in the second horizontal position.

[0076] The example can have the following beneficial effect: the third vertical coordinate value of detected keypoint elements can be determined using a pseudo-stereo method based on triangulation. By using optical sensor data from two different horizontal positions of a patient table with a known distance between them, the third vertical coordinate value can be determined using triangulation similar to a stereo method with two optical sensors. Such a pseudo-stereo method is even applicable to a single optical sensor in an optical sensor system.

[0077] Since the patient table and optical sensor system can typically be aligned by several degrees, the small required horizontal travel between the first and second horizontal positions will result in a negligible error in the third vertical coordinate direction.

[0078] For example, the optical sensor system includes at least two optical sensors arranged spaced apart from each other with a known fixed baseline. Received optical sensor data includes first optical sensor data acquired using a first optical sensor of the at least two optical sensors and second optical sensor data acquired using a second optical sensor of the at least two optical sensors. Detecting keypoint elements within the optical sensor data includes detecting keypoint elements within the first optical sensor data and detecting keypoint elements within the second optical sensor data. Determining a third vertical coordinate value for the detected keypoint elements includes performing triangulation using the positions of the keypoint elements detected within the first optical sensor data and the positions of the keypoint elements detected within the second optical sensor data.

[0079] The example can have the following beneficial effects: a triangulation-based stereo method can be used to determine the third perpendicular coordinate value of a detected keypoint element. Using at least two optical sensors arranged spaced apart from each other with a known fixed baseline, the distance to the keypoint element can be determined using an angle at which the at least two optical sensors see the corresponding keypoint element using triangulation. The angle can be determined using the position of the keypoint element detected within the data of the first and second optical sensors. When the third perpendicular coordinate value of the keypoint element is determined, the keypoint element forms the third point of a triangle, which includes at least two optical sensors as a first point and a second point having a known side (i.e., a baseline) and two known angles determined using the optical sensor data.

[0080] For example, in addition to detecting keypoint elements, the keypoint detection module is also configured to output the third vertical coordinate value of the detected keypoint elements in response to receiving optical sensor data. Determining the third vertical coordinate value of the detected keypoint elements includes using the keypoint detection module.

[0081] The example could have the beneficial effect that the keypoint detection module can be configured to directly determine the third vertical coordinate value of the detected keypoint element without requiring additional input beyond the received optical sensor data.

[0082] For example, keypoint elements of a patient table are distributed on the surface of the patient table. For example, keypoint elements can be elements distributed across the surface at grid points of an imaginary mesh laid out on the corresponding surface.

[0083] The surface may be, for example, the upper surface of a patient table. An optical sensor system may be positioned, for example, above the patient table. The optical sensor system may be oriented, for example, to observe the upper surface of the patient table vertically downwards. The upper surface may, for example, be provided with and / or include an inclined surface.

[0084] For example, keypoint elements of the patient table are distributed along the contour line of the patient table. The contour line could be, for example, the contour line of the upper surface of the patient table. Using keypoint elements of a patient table distributed along the contour line can have the advantage that the likelihood of the keypoint elements being covered by, for example, a mattress, padding, coil interface, or other materials arranged on the patient table can be relatively low. Moreover, in this case, the likelihood of the keypoint elements being covered by a patient lying on the upper surface of the patient table (e.g., an inclined surface) can be relatively low.

[0085] For example, calibration is performed in response to receiving a calibration request. For example, calibration is performed automatically in response to receiving an indication that calibration is required. For example, calibration is performed automatically in a preemptive manner. For example, calibration is performed as part of a method for checking the current calibration.

[0086] For example, the method further includes checking the calibration. Checking the calibration includes determining the coordinate values ​​of the positions of keypoint elements detected within optical sensor test data acquired using an optical sensor system in a first coordinate system. The optical sensor test data includes optical sensor data of a patient stage positioned in the test location. For the patient stage positioned in the test location, the coordinate values ​​of the positions of the keypoint elements in a second coordinate system are received. The coordinate values ​​of the first and second coordinate systems are compared using one or more determined transformation parameters for the transformation between the first and second coordinate systems. In response to detecting that the deviation between the compared coordinate values ​​exceeds a predefined threshold, recalibration is applied. The application of recalibration includes initiating another calibration to determine one or more alternative transformation parameters, or using one or more other previously determined transformation parameters as one or more alternative transformation parameters.

[0087] The example can have the following beneficial effects: it allows checking whether the calibration is still valid or has sufficient accuracy. In the event of decalibration (i.e., inadequacy of the current calibration), recalibration can be applied. Such application of recalibration can, for example, involve initiating another calibration to determine one or more alternative transformation parameters. Alternatively, the application of recalibration can, for example, involve using one or more other previously determined transformation parameters as one or more alternative transformation parameters. Thus, the calibration (i.e., the transformation parameters) can remain up-to-date. Decalibration can, for example, be caused by vibration of one or more components of the optical sensor system and / or medical imaging system, which can lead to a misalignment between the optical sensor system and the medical imaging system.

[0088] For example, the method further includes receiving multiple additional optical sensor data acquired using an optical sensor system, in addition to the optical sensor data used for calibration. The additional optical sensor data includes additional optical sensor data from the patient station. A keypoint detection module is used to detect keypoint elements within the additional optical sensor data. The optical sensor data used for calibration is selected from a set of received optical sensor data, which includes additional optical sensor data in addition to the selected data. The selection includes checking whether a confidence value determined for a predefined number of keypoint elements detected within the selected optical sensor data exceeds a predefined threshold.

[0089] An example could have the beneficial effect of selecting optical sensor data for calibration from multiple received optical sensor data sets. When selecting the appropriate optical sensor data, it is ensured that the confidence values ​​determined for a predefined number of keypoint elements detected within the selected optical sensor data exceed a predefined threshold. Therefore, it can be ensured that the optical sensor data used for calibration has sufficient quality to guarantee sufficient quality of the resulting calibration. For example, it might be necessary for the confidence values ​​determined for all keypoint elements detected within the selected optical sensor data to exceed a predefined threshold.

[0090] For example, the method further includes providing a training dataset for training a keypoint detection module. The training dataset includes optical sensor training data and optical sensor training data, wherein keypoint elements of the patient table are labeled within the optical sensor training data. In response to receiving the optical sensor training data of the corresponding training dataset, the keypoint detection module is trained to output the detection of labeled keypoint elements of the patient table within the optical sensor training data of the training dataset.

[0091] The example can have the following beneficial effect: a keypoint detection module can be trained to detect keypoint elements of the patient table within optical sensor data.

[0092] For example, the labels include the two-dimensional horizontal coordinate values ​​of the corresponding keypoint elements of the patient station within a first coordinate system. These two-dimensional horizontal coordinate values ​​can describe the position of the corresponding keypoint element in a plane parallel to the sensor plane of the optical sensor system. The training may further include training a keypoint detection module to determine, in response to receiving optical sensor training data within the corresponding training dataset, the two-dimensional horizontal coordinate values ​​of the labeled keypoint elements of the patient station as output.

[0093] The example can have the following beneficial effect: the keypoint detection module can be additionally trained to determine the two-dimensional horizontal coordinate values ​​of the detected keypoint elements in the first coordinate system.

[0094] For example, the label also includes a third vertical coordinate value of the corresponding keypoint element in a first coordinate system. The third vertical coordinate value describes the distance of the position of the corresponding keypoint element from the sensor plane of the optical sensor system. The training may also include training a keypoint detection module to determine, in response to receiving optical sensor training data of a corresponding training dataset, the third vertical coordinate value of the labeled keypoint element of the patient station within the optical sensor training data of the training dataset as output.

[0095] The example can have the following beneficial effect: the keypoint detection module can be additionally trained to determine the third vertical coordinate value of the detected keypoint element in the first coordinate system.

[0096] For example, a keypoint detection module can be trained to determine confidence levels. These confidence levels can, for example, be estimates trained using a loss function that quantifies the deviation of the detected keypoint elements from the baseline ground truth values ​​of the keypoint elements provided by optical training data. Confidence levels can, for example, be a function of the output of the keypoint detection module. The maximum, minimum, average, or integral of the detection module's output can be examples of quantities that can be used to derive a confidence metric within a training distribution provided by multiple optical training data sets.

[0097] In another aspect, the present invention provides a computer program comprising machine-executable instructions for calibrating a first coordinate system of the field of view of an optical sensor system. The calibration includes registering the first coordinate system with a second coordinate system of a medical imaging system using a keypoint detection module configured to detect keypoint elements of the patient stage of the medical imaging system as output within the optical sensor data in response to receiving optical sensor data acquired using the optical sensor system. The optical sensor system is configured for monitoring the medical imaging system.

[0098] The processor of the computing device executes machine-executable instructions, causing the processor to control the computing device to perform a method for calibrating a first coordinate system for the field of view of an optical sensor system.

[0099] The method includes receiving optical sensor data acquired using an optical sensor system. The received optical sensor data includes optical sensor data from the patient table.

[0100] The key point detection module is used to detect key point elements within optical sensor data.

[0101] Using optical sensor data, the two-dimensional horizontal coordinates of the detected keypoint elements are determined within a first coordinate system. These two-dimensional horizontal coordinates describe the position of the detected keypoint elements in a plane parallel to the sensor plane of the optical sensor system.

[0102] Determine the third vertical coordinate value of the detected keypoint element within the first coordinate system. The third vertical coordinate value describes the distance of the detected keypoint element from the sensor plane of the optical sensor system.

[0103] Receive the three-dimensional coordinates describing the position of the detected keypoint element in the second coordinate system of the medical imaging system.

[0104] One or more transformation parameters are determined using the coordinate values ​​describing the position of the detected keypoint element in the first coordinate system and the coordinate values ​​describing the position of the detected keypoint element in the second coordinate system.

[0105] For example, a computing device can be configured to perform any of the aforementioned examples of a method for calibrating a first coordinate system for the field of view of an optical sensor system.

[0106] In another aspect, the present invention provides a computational device for calibrating a first coordinate system of the field of view of an optical sensor system. Calibration includes registering the first coordinate system with a second coordinate system of a medical imaging system. The computational device includes a keypoint detection module configured to detect keypoint elements of the patient stage of the medical imaging system as output within the optical sensor data in response to receiving optical sensor data acquired using the optical sensor system. The optical sensor system is configured for monitoring the medical imaging system.

[0107] The computing device includes a processor and a memory storing machine-executable instructions therein. The processor of the computing device executes the machine-executable instructions, causing the processor to control the computing device to perform a method for calibrating a first coordinate system for the field of view of an optical sensor system.

[0108] The method includes receiving optical sensor data acquired using an optical sensor system. The received optical sensor data includes optical sensor data from the patient table.

[0109] The key point detection module is used to detect key point elements within optical sensor data.

[0110] Using optical sensor data, the two-dimensional horizontal coordinates of the detected keypoint elements are determined within a first coordinate system. These two-dimensional horizontal coordinates describe the position of the detected keypoint elements in a plane parallel to the sensor plane of the optical sensor system.

[0111] Determine the third vertical coordinate value of the detected keypoint element within the first coordinate system. The third vertical coordinate value describes the distance of the detected keypoint element from the sensor plane of the optical sensor system.

[0112] Receive the three-dimensional coordinates describing the position of the detected keypoint element in the second coordinate system of the medical imaging system.

[0113] One or more transformation parameters are determined using the coordinate values ​​describing the position of the detected keypoint element in the first coordinate system and the coordinate values ​​describing the position of the detected keypoint element in the second coordinate system.

[0114] For example, a computing device can be configured to perform any of the aforementioned examples of a method for calibrating a first coordinate system for the field of view of an optical sensor system.

[0115] In another aspect, the present invention provides a medical imaging system comprising a computing device as described in any of the foregoing examples, an optical sensor system configured to supervise the medical imaging system, and a patient table. The medical imaging system is one of: a magnetic resonance imaging system, a computed tomography system, or an X-ray imaging system.

[0116] The techniques described herein can be applied to the use of optical sensors in a variety of medical applications (i.e., in a variety of medical imaging systems). Medical imaging systems, such as MRI systems, may include radiation therapy (TR) systems. X-ray imaging systems may, for example, be part of image-guided therapy (IGT) systems. X-ray imaging systems may, for example, be diagnostic X-ray (DXR) systems.

[0117] It should be understood that one or more of the foregoing embodiments of the present invention may be combined, as long as the combined embodiments are not mutually exclusive.

[0118] As those skilled in the art will recognize, aspects of the present invention can be implemented as apparatus, method, or computer program product. Accordingly, aspects of the present invention can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects (all of which may be referred to herein as “circuit,” “module,” or “system”). Furthermore, aspects of the present invention can take the form of a computer program product implemented in one or more computer-readable media having computer-executable code implemented thereon.

[0119] Any combination of one or more computer-readable media can be used. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" encompasses any tangible storage medium capable of storing instructions executable by a processor or computing system of a computing device. A computer-readable storage medium may be referred to as a computer-readable non-transient storage medium. A computer-readable storage medium may also be referred to as a tangible computer-readable medium. In some embodiments, a computer-readable storage medium may also be capable of storing data accessible by a computing system of a computing device. Examples of computer-readable storage media include, but are not limited to: floppy disks, magnetic hard disk drives, solid-state drives, flash memory, USB thumb drives, random access memory (RAM), read-only memory (ROM), optical discs, magneto-optical discs, and register files of computing systems. Examples of optical discs include compact discs (CDs) and digital universal discs (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R discs. The term computer-readable storage medium also refers to various types of recording media accessible by a computer device via a network or communication link. For example, data can be retrieved on a modem, the Internet, or a local area network. Computer-executable code implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0120] Computer-readable signal media may include propagated data signals having computer-executable code implemented therein, for example, in baseband or as a carrier wave. Such propagated signals may take any variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and is capable of conveying, propagating, or transmitting a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory that can be directly accessed by a computing system. "Computer storage device" or "storage device" is another example of a computer-readable storage medium. A computer storage device is any non-volatile computer-readable storage medium. In some embodiments, a computer storage device may also be computer memory, or vice versa.

[0122] As used herein, "computing system" encompasses electronic components capable of running programs or machine-executable instructions or computer-executable code. References to computing systems, including examples of "computing systems," should be interpreted as potentially including more than one computing system or processing core. A computing system can, for example, be a multi-core processor. A computing system can also refer to a collection of computing systems within a single computer system or distributed across multiple computer systems. The term computing system should also be interpreted as potentially referring to a collection or network of computing devices, each comprising a processor or computing system. Machine-executable code or instructions can be run by multiple computing systems or processors that may be located within the same computing device or distributed even across multiple computing devices.

[0123] Machine-executable instructions or computer-executable code may include instructions or programs that cause a processor or other computing system to perform aspects of the invention. Computer-executable code for performing operations related to aspects of the invention may be written in any combination of one or more programming languages ​​and compiled into machine-executable instructions, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. In some instances, the computer-executable code may be in the form of a high-level language or in a pre-compiled form and used in conjunction with an interpreter that generates machine-executable instructions at runtime. In other instances, the machine-executable instructions or computer-executable code may be in the form of programming for programmable gate arrays.

[0124] The computer-executable code may run entirely on the user's computer, partially on the user's computer (as a standalone software package), partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may connect to an external computer (e.g., via the Internet provided by an Internet service provider).

[0125] Various aspects of the invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that, where applicable, each block or portion of a flowchart illustration, illustration, and / or block diagram can be implemented by computer program instructions in the form of computer-executable code. It should also be understood that combinations of blocks from different flowchart illustrations, illustrations, and / or block diagrams can be combined when not mutually exclusive. These computer program instructions can be provided to a computing system of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus that produces the machine, such that the instructions, which run via the computer or other programmable data processing apparatus, create units for implementing the functions / actions specified in the flowchart and / or one or more block diagram blocks.

[0126] These machine-executable instructions or computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing that includes instructions that implement the functions / actions specified in flowcharts and / or one or more block diagrams.

[0127] The machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device, thereby producing a computer-implemented process, such that the instructions running on the computer or other programmable apparatus provide a process for the function / action specified in the flowchart and / or one or more block diagram boxes.

[0128] As used herein, a "user interface" is an interface that allows a user or operator to interact with a computer or computer system. A "user interface" can also be referred to as a "human-machine interface device." A user interface can provide information or data to and / or receive information or data from an operator. A user interface enables input from an operator to be received by the computer and output from the computer to the user. In other words, the user interface allows an operator to control or manipulate the computer, and the interface allows the computer to indicate the effects of the operator's control or manipulation. The display of data or information on a monitor or graphical user interface is an example of providing information to an operator. The reception of data via a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, game controller, webcam, headset, pedal, wired gloves, remote control, and accelerometer are all examples of user interface components that implement the reception of information or data from an operator.

[0129] As used herein, "hardware interface" encompasses the interfaces that enable a computer system to interact with and / or control external computing devices and / or devices. A hardware interface allows the computing system to send control signals or instructions to external computing devices and / or devices. It also enables the computing system to exchange data with external computing devices and / or devices. Examples of hardware interfaces include, but are not limited to: Universal Serial Bus (USB), IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, Bluetooth connectivity, wireless LAN connectivity, TCP / IP connectivity, Ethernet connectivity, control voltage interfaces, MIDI interfaces, analog input interfaces, and digital input interfaces.

[0130] As used herein, “display” or “display device” encompasses an output device or user interface suitable for displaying images or data. Displays can output visual, audio, and / or tactile data. Examples of displays include, but are not limited to: computer monitors, television screens, touchscreens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bistable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, projectors, and head-mounted displays.

[0131] The term "machine learning" (ML) refers to computer algorithms used to extract useful information from training datasets by automatically constructing probabilistic models called machine learning modules or models. Machine learning modules can also be called predictive models. Machine learning algorithms build mathematical models based on sample data (called "training data") to make predictions or decisions without being explicitly programmed to perform a specific task. Machine learning modules can be implemented using learning algorithms such as supervised or unsupervised learning. Machine learning modules can be based on various techniques such as clustering, classification, linear regression, reinforcement learning, self-learning, support vector machines, neural networks, etc. Machine learning modules can be, for example, data structures or programs such as neural networks, particularly convolutional neural networks, support vector machines, decision trees, Bayesian networks, etc. Machine learning modules can be adapted (i.e., trained) to predict unmeasured values. Therefore, a trained machine learning module can be made capable of predicting unmeasured values ​​as outputs from other known values ​​as inputs.

[0132] The machine learning module to be trained can be, for example, an untrained machine learning module, a pre-trained machine learning module, or a partially trained machine learning module. The machine learning module being trained can be an untrained machine learning module trained from scratch. Alternatively, the machine learning module being trained can be a pre-trained or partially trained machine learning module. Typically, it may not be necessary to start with an untrained machine learning module, for example, in deep learning. For example, it can start with a pre-trained or partially trained machine learning module. The pre-trained or partially trained machine learning module may have already been pre-trained or partially trained for the same or similar tasks. Using a pre-trained or partially trained machine learning module can, for example, make it possible to train the trained machine learning module to be trained faster, i.e., the training can converge faster. For example, transfer learning can be used to train a pre-trained or partially trained machine learning module. Transfer learning refers to a machine learning process where, when solving different problems, the machine learning process does not start from scratch from previously learned patterns. This allows, for example, prior learning to be utilized, thus avoiding starting from scratch. A pre-trained machine learning module is a machine learning module that has previously been trained, for example, on a large benchmark dataset to solve problems similar to the problem to be solved through additional learning. In the case of a pre-trained machine learning module, the previous learning process has already been successfully completed. A partially trained machine learning module is one that has been partially trained, meaning the training process may not yet be complete. Pre-trained or partially trained machine learning modules can, for example, be imported and trained for the purposes disclosed herein. Attached Figure Description

[0133] Preferred embodiments of the invention will be described below by way of example only and with reference to the accompanying drawings, in which:

[0134] Figure 1 An exemplary method for calibrating a first coordinate system for the field of view of an optical sensor system is illustrated.

[0135] Figure 2 Another exemplary method for calibrating a first coordinate system for the field of view of an optical sensor system is illustrated.

[0136] Figure 3 An exemplary method for estimating a third vertical coordinate value is illustrated;

[0137] Figure 4 An exemplary method for checking calibration is illustrated;

[0138] Figure 5 The illustration shows an exemplary method for selecting optical sensor data to be used for calibration;

[0139] Figure 6 An exemplary method for training a keypoint detection module is illustrated.

[0140] Figure 7 The illustration shows exemplary optical sensor training data used to train a keypoint detection module;

[0141] Figure 8 The illustration shows an example of detection of key point elements in optical sensor data;

[0142] Figure 9 The illustration shows another exemplary detection of key point elements in optical sensor data;

[0143] Figure 10 The illustration shows another exemplary detection of key point elements in optical sensor data;

[0144] Figure 11 An exemplary computing device for calibrating a first coordinate system of the field of view of an optical sensor system is illustrated; and

[0145] Figure 12 An exemplary medical imaging system including a computing device is illustrated. List of reference numerals 100 Medical Imaging System 118 patients 120 patient stations 122 Key Elements 124 Materials placed on the patient table 126 Upper surface 128 Inclined surface 130 Vertical direction 132 Horizontal direction 140 Optical Sensor System 144 Optical Sensors 152 MRI systems 154 Magnets 156 Magnet Chamber 158 Imaging isocenter 160 Magnetic Gradient Coil 161 RF transmitter coil 162 Magnetic field gradient coil power supply 163 transmitter 400 computing devices 402 Calculation Component 404 memory 406 Hardware Interface 408 User Interface 410 Machine-readable instructions 412 Key Point Detection Module 414 Optical Sensor Data 416 Three-dimensional coordinate values 418 training data 420 Transformation Parameters 500 3D Digital Scale Model 504 tags 520 random image elements Detailed Implementation

[0146] Elements with the same number in these figures are equivalent elements or perform the same function. If they are functionally equivalent, elements that have been discussed previously will not necessarily be discussed in the following figures.

[0147] Figure 1 The illustration depicts a method for calibrating a first coordinate system for the field of view of an optical sensor system. Calibration involves registering this first coordinate system with a second coordinate system of a medical imaging system using a computing device. The computing device includes a keypoint detection module configured to detect keypoint elements of the patient stage of the medical imaging system within the optical sensor data as output in response to receiving optical sensor data acquired using the optical sensor system. The optical sensor system is configured for monitoring the medical imaging system.

[0148] In box 200, optical sensor data acquired using an optical sensor system is received. The received optical sensor data includes optical sensor data from the patient station.

[0149] In box 202, the key point detection module is used to detect key point elements within the optical sensor data.

[0150] In box 204, optical sensor data is used to determine the two-dimensional horizontal coordinates of the detected keypoint element within a first coordinate system. These two-dimensional horizontal coordinates describe the position of the detected keypoint element in a plane parallel to the sensor plane of the optical sensor system.

[0151] In box 206, a third vertical coordinate value in the first coordinate system is determined for the detected keypoint element. The third vertical coordinate value describes the distance of the detected keypoint element's position from the sensor plane of the optical sensor system. The third vertical coordinate value can be determined, for example, according to any of the aforementioned examples of depth sensing methods, such as using a registered depth image, using the apparent lateral distance between keypoint elements, using a stereo method with at least two optical sensors, using a pseudo-stereo method with different positions of the patient table, and / or using a keypoint detection module trained to also determine the third vertical coordinate value.

[0152] In box 208, three-dimensional coordinate values ​​describing the position of the detected keypoint element in the second coordinate system of the medical imaging system are received.

[0153] In box 210, one or more transformation parameters are determined for the transformation from the first coordinate system to the second coordinate system. To determine the one or more transformation parameters, coordinate values ​​describing the position of the detected keypoint element in the first coordinate system and coordinate values ​​describing the position of the detected keypoint element in the second coordinate system are used.

[0154] For example, the transformation parameters can be determined for different positions of the patient table, and the transformation parameters obtained for different positions of the patient table are used to determine one or more transformation parameters for the transformation from the first coordinate system to the second coordinate system, for example, as the average value of one or more transformation parameters for different positions of the patient table.

[0155] Figure 2 Another method for calibrating a first coordinate system for the field of view of an optical sensor system is illustrated. Calibration involves registering the first coordinate system with a second coordinate system of the medical imaging system using a computing device. This computing device includes a keypoint detection module configured to detect keypoint elements of the patient stage of the medical imaging system within the optical sensor data as output in response to receiving optical sensor data acquired using the optical sensor system. The optical sensor system is configured for monitoring the medical imaging system.

[0156] In box 300, optical sensor data acquired using an optical sensor system is received. The received optical sensor data includes optical sensor data from the patient station. In box 302, a keypoint detection module is used to detect keypoint elements within the received optical sensor data. In box 304, the two-dimensional horizontal coordinates of the detected keypoint elements in a first coordinate system are determined. The two-dimensional horizontal coordinates describe the position of the detected keypoint element in a plane parallel to the sensor plane of the optical sensor system.

[0157] In box 310, a depth sensing method is applied to determine the third vertical coordinate value of the detected keypoint element within the first coordinate system. In box 312, the third vertical coordinate value is determined. The third vertical coordinate value describes the distance of the detected keypoint element's position from the sensor plane of the optical sensor system. The third vertical coordinate value can be determined, for example, according to any of the aforementioned examples of depth sensing methods, such as using a registered depth image, using the apparent lateral distance between keypoint elements, using a stereo method with at least two optical sensors, using a pseudo-stereo method with different positions of the patient table, and / or using a keypoint detection module trained to also determine the third vertical coordinate value. In box 314, three-dimensional coordinate values, namely the first, second, and third coordinate values, are provided.

[0158] In box 320, the current position of the patient stage is determined. The determined position of the patient stage within the second coordinate system of the medical imaging system is used to determine the three-dimensional coordinate values ​​describing the position of the detected keypoint elements within the second coordinate system of the medical imaging system.

[0159] In box 330, the coordinate values ​​describing the position of the detected keypoint element in the first coordinate system, provided in box 314, and the coordinate values ​​describing the position of the detected keypoint element in the second coordinate system, determined in box 330, are used for calibration. Calibration includes one or more transformation parameters determined for the transformation from the first coordinate system to the second coordinate system.

[0160] For example, boxes 300 to 330 included in box 340 can be repeated for multiple different patient table positions. The resulting transformation parameters from different repetitions of box 340 can be combined with each other (e.g., averaged). In box 340, the resulting transformation parameters are provided, for example, as an output.

[0161] Figure 3 The diagram illustrates a method for determining the third vertical coordinate value of detected keypoint elements. Figure 3 The method can be used, for example, in... Figure 1 In the method box 210 and / or Figure 2 The third vertical coordinate value of the detected key point element is determined in box 312.

[0162] In block 600, a first reference value for the horizontal distance between detected keypoint elements of the patient station, determined in the first optical sensor reference data, is received. The first optical sensor reference data includes optical sensor data of the patient station positioned in a lower first reference position, which has a first vertical reference distance between the detected keypoint elements and the sensor plane of the optical sensor system. In block 602, a second reference value for the horizontal distance between detected keypoint elements of the patient station, determined in the second optical sensor reference data, is received. The second optical sensor reference data includes optical sensor data of the patient station positioned in an upper second reference position, which has a second vertical reference distance between the detected keypoint elements and the sensor plane of the optical sensor system.

[0163] The third vertical coordinate value is determined using estimation (e.g., interpolation). Determining the third vertical coordinate value of the detected keypoint elements includes boxes 604 and 606. In box 604, the horizontal distance between the detected keypoint elements of the patient table is determined using received optical data. In box 606, the third vertical coordinate value is estimated. To estimate the third vertical coordinate value, the determined horizontal distance value, a first reference value for the received horizontal distance, a first vertical reference distance, and a second reference value for the received horizontal distance and a second vertical reference distance are used.

[0164] Figure 4 The illustration depicts a method for calibrating a first coordinate system used to check the field of view of an optical sensor system. In box 620, within the first coordinate system, the coordinate values ​​of the positions of keypoint elements detected within optical sensor test data acquired using the optical sensor system are determined. The optical sensor test data includes optical sensor data of a patient stage positioned at a test location. In box 622, for the patient stage positioned at the test location, the coordinate values ​​of the positions of the keypoint elements within a second coordinate system are received.

[0165] In box 624, the coordinate values ​​of the first and second coordinate systems are compared using one or more determined transformation parameters for the transformation between the first and second coordinate systems. In box 626, it is checked whether the deviation between the compared coordinate values ​​exceeds a predefined threshold. If the deviation between the compared coordinate values ​​is detected to exceed the predefined threshold in box 626, the method continues to apply recalibration in box 628. The application of recalibration includes initiating another calibration to determine one or more alternative transformation parameters, or using one or more other previously determined transformation parameters as one or more alternative transformation parameters. If the deviation between the compared coordinate values ​​is detected to not exceed the predefined threshold in box 626, the method continues to maintain the current calibration in box 630.

[0166] Figure 5 The diagram illustrates the selection of which calibration to use (e.g., based on...). Figure 1 This method involves calibrating optical sensor data. In box 650, multiple optical sensor data acquired using an optical sensor system are received. The multiple optical sensor data includes multiple optical sensor data from a patient station. In box 652, a keypoint detection module detects keypoint elements within the received optical sensor data. In box 654, it is checked whether the confidence value determined for a predefined number of keypoint elements detected within the received multiple optical sensor data exceeds a predefined threshold. If it is determined that the predefined number of keypoint elements detected within the checked optical sensor data exceeds the predefined threshold, the corresponding optical sensor data is selected for calibration in box 656. If it is determined that the predefined number of keypoint elements detected within the checked optical sensor data exceeds the predefined threshold, the method continues to check the next optical sensor data in the multiple optical sensor data in box 654.

[0167] Figure 6The illustration depicts a method for training a keypoint detection module to detect keypoint elements within optical sensor data. In box 670, a keypoint detection module to be trained is provided. In box 672, a training dataset for training the keypoint detection module is provided. The training dataset includes optical sensor training data and optical sensor training data, wherein keypoint elements of the patient station are labeled within the optical sensor training data. Providing the training dataset may include, for example, generating optical sensor training data included in the training dataset as synthetic optical sensor training data using a 3D digital scale model of the patient station. In box 674, the keypoint detection module is trained to detect, as output, the labeled keypoint elements of the patient station within the optical sensor training data of the training dataset in response to receiving the optical sensor training data of the corresponding training dataset.

[0168] For example, the labeled optical sensor training data may include two-dimensional horizontal coordinate values ​​of corresponding keypoint elements of the patient station within a first coordinate system. These two-dimensional horizontal coordinate values ​​can describe the position of the corresponding keypoint element in a plane parallel to the sensor plane of the optical sensor system. The training may also include training a keypoint detection module to determine, in response to receiving optical sensor training data of a corresponding training dataset, the two-dimensional horizontal coordinate values ​​of the labeled keypoint elements of the patient station within the optical sensor training data of the training dataset as output.

[0169] For example, the label may also include a third vertical coordinate value of the corresponding keypoint element in a first coordinate system. The third vertical coordinate value describes the distance of the position of the corresponding keypoint element from the sensor plane of the optical sensor system. The training may also include training a keypoint detection module to determine, in response to receiving optical sensor training data of a corresponding training dataset, the third vertical coordinate value of the labeled keypoint element of the patient station within the optical sensor training data of the training dataset as output.

[0170] For example, a keypoint detection module can be trained to determine confidence levels. These confidence levels can be, for example, estimates trained using a loss function that quantifies the deviation of the detected keypoint elements from the ground truth values ​​of keypoint elements provided by optical training data. Confidence levels can be, for example, functions of the output of the keypoint detection module. The maximum, minimum, average, or integral of the detection module's output can be examples of quantities that can be used to derive confidence measures within a training distribution provided by multiple optical training data sets.

[0171] Figure 7An exemplary optical sensor training data 418 is illustrated for training a keypoint detection module to detect keypoint elements 122 of a patient table. The optical sensor training data 418 is synthetic training data generated using a three-dimensional digital scale model 500 of the patient table. The three-dimensional digital scale model 500 includes an upper surface 126 having a tilted surface 128. Keypoint elements 122 are distributed, for example, on the upper surface 126. The keypoint elements 122 are distributed circumferentially around the tilted surface 128, for example, along the contour line of the patient table. The optical sensor training data 418 can be generated, for example, by adding random image elements 520. The random image elements 520 can be added, for example, to the background of the three-dimensional digital scale model 500. For example, the random image elements 520 can also be added at least partially as random obstacles to the three-dimensional digital scale model 500, such that at least some of the keypoint elements 122 are covered by the random image elements 520. Therefore, the key point detection module can be made robust to random obstacles because a typical patient table can be covered with additional materials, such as mattresses, padding, blankets, coil interfaces, and other materials.

[0172] Additionally, the optical sensor training data 418 includes labels 504 that mark keypoint elements 122 within the optical sensor training data 418. For training the keypoint detection module, for example, additional optical sensor training data 418 can be provided during training without the labels 504 of the keypoint elements 122 as input to the keypoint detection module. The keypoint detection module is trained to detect keypoint elements 122 in the optical sensor that can be provided without the labels 504 in the training data 418, which are labeled with the labels 504 of the labeled optical sensor training data 418.

[0173] For example, label 504 may include the two-dimensional horizontal coordinate values ​​of the corresponding keypoint element 122 of the patient table in a first coordinate system. The two-dimensional horizontal coordinate values ​​can describe the position of the corresponding keypoint element in a plane parallel to the sensor plane of the optical sensor system. Therefore, the optical sensor training data 418 can also be used, for example, to train the keypoint detection module to determine the two-dimensional horizontal coordinate values ​​of the labeled keypoint element 122 of the patient table as output.

[0174] For example, label 504 may also include a third vertical coordinate value of the corresponding keypoint element 122 in the first coordinate system. The third vertical coordinate value describes the distance of the position of the corresponding keypoint element 122 from the sensor plane of the optical sensor system. Therefore, the optical sensor training data 418 may also be used, for example, to train the keypoint detection module to determine a third vertical coordinate value as output, in addition to the two-dimensional horizontal coordinate value of the labeled keypoint element 122 of the patient table.

[0175] Figure 8An exemplary detection of keypoint elements 122 in optical sensor data 414 of a patient table 120 is illustrated. The patient table 120 is part of a medical imaging system (e.g., an MRI system). The patient table 120 includes an upper surface 126 having an inclined surface 128. The inclined surface is configured for a patient to lie on. Keypoint elements 122 of the patient table 120 are distributed, for example, on the upper surface 126. The keypoint elements 122 of the patient table 120 are distributed, for example, circumferentially around the inclined surface 128 along the contour line of the patient table. Additionally, additional materials 124, such as mattresses, padding, blankets, coil interfaces, and other materials, may be arranged on the patient table 120. A keypoint detection module is configured to detect keypoint elements 122 in the optical sensor data 414.

[0176] Figure 9 Another exemplary detection of key point element 122 in optical sensor data 414 is illustrated. Figure 9 The optical sensor data 414 includes an empty patient stage 120 of a medical imaging system 100 having an upper surface 126 with a tilted surface 128. Detected keypoint elements 122 are represented as Gaussian probability heatmaps, for example, having varying colors overlaid on the stage image. The size of individual heatmaps allows adjacent keypoint elements to have overlapping signal tails. Due to the overlap, a... Figure 9 The strip structure shown schematically represents the detected key point element 122.

[0177] Figure 10 Another exemplary detection of key point element 122 in optical sensor data 414 is illustrated. Figure 10 The optical sensor data 414 corresponds to Figure 9 The optical sensor data is 414, the only difference is that... Figure 10 The patient in the 120 ambulance is not empty. Figure 10 In this setting, the patient lies on a patient table 120. Additionally, extra materials 124, such as mattresses, padding, blankets, coil interfaces, and other materials, are arranged on the patient table 120. These extra materials 124 may, for example, at least partially cover some of the key element 122. However, as... Figure 10 As indicated by the schematically shown strip structure, the key point detection module is configured to robustly detect key point elements 122 in the optical sensor data 414. The optical sensor data 414 illustrates an exemplary examination performed during clinical use, in which obstacle verification confirms valid calibration.

[0178] Figure 11An exemplary computing device 400 for calibrating a first coordinate system of the field of view of an optical sensor system is illustrated. The computing device 400 is shown to include a computing component 402. The computing component 402 is intended to represent one or more processors or processing cores or other computing elements. The computing component 402 is shown connected to a hardware interface 406 and a memory 404. The hardware interface 406 enables the computing component 402 to exchange commands and data with other components (e.g., the optical sensor system). The hardware interface 406 may, for example, enable the computing component 402 to control the optical sensor system to acquire optical sensor data. The hardware interface 406 may, for example, further enable the computing component 402 to control a medical imaging system.

[0179] The computing system 404 is also shown connected to a user interface 408, which may, for example, enable an operator to control and operate the computing device 400, and via the computing device 400, control and operate optical sensors and / or medical imaging systems. The user interface 408 may, for example, include output and / or input devices that enable a user to interact with the computer 400. Output devices may, for example, include a display device configured to display magnetic resonance images 426. Input devices may, for example, include a keyboard and / or mouse, which enable a user to insert control commands for controlling the computing device 400, and via the computing device 400, insert optical sensors and / or medical imaging systems.

[0180] Memory 404 is shown as containing machine-executable instructions 410. Machine-executable instructions 410 enable computing unit 402 to perform control tasks, such as an optical sensor system, to perform digital tasks, and to perform various signal data processing tasks. Machine-executable instructions 410 can, for example, cause computing unit 402 and therefore computing device 400 to... Figure 1 or Figure 2 The method is used to calibrate the first coordinate system of the field of view of the optical sensor system. Machine-executable instructions 410 can, for example, enable the computing unit 402 and therefore the computing device 400 to further execute... Figures 3 to 6 Methods and / or based on Figure 7 Training data for the synthetic optical sensor is generated 418. For example, instruction 410 may enable computing unit 402 and thus computing device 400 to control the medical imaging system.

[0181] The memory 404 is also shown to include a keypoint detection module 412. The keypoint detection module 412 is configured to detect one or more keypoint elements of a target structure within the optical sensor data 414 in response to receiving optical sensor data 414 from the optical sensor. The keypoint detection module 412 may also be configured, for example, to determine two-dimensional horizontal coordinate values ​​of the detected keypoint elements in a first coordinate system. The two-dimensional horizontal coordinate values ​​describe the position of the detected keypoint element in a plane parallel to the sensor plane of the optical sensor system. For example, the keypoint detection module 412 may be configured to determine, in addition to the two-dimensional horizontal coordinate values, a third vertical coordinate value of the detected keypoint element in the first coordinate system. The third vertical coordinate value describes the distance of the detected keypoint element's position from the sensor plane of the optical sensor system.

[0182] The memory 404 is also shown to contain optical sensor data 414. The optical sensor data 414 has been acquired using an optical sensor configured for monitoring a medical imaging system.

[0183] The memory 404 is also shown as containing three-dimensional coordinate values ​​416 describing the position of the detected keypoint element in a second coordinate system of the medical imaging system. Using the coordinate values ​​describing the position of the detected keypoint element in a first coordinate system of the optical sensor data 414 and the coordinate values ​​416 describing the position of the detected keypoint element in the second coordinate system, one or more transformation parameters 420 for the transformation from the first coordinate system to the second coordinate system are determined.

[0184] The memory 404 may optionally also contain training data 418, such as synthetic training data, which is configured to train the keypoint detection module 412 to detect one or more keypoint elements of a target structure within the optical sensor data 414 in response to receiving optical sensor data 414 from the optical sensor.

[0185] Figure 12 This is an exemplary medical imaging system 100 that includes a computing device 400. The computing device 400 is configured to calibrate a first coordinate system for the field of view of the optical sensor system 140. Figure 12 The computing device 400 is, for example Figure 11 400 computing devices.

[0186] Medical imaging system 100 includes, for example, an optical sensor system 140 and a patient stage 120 configured to monitor the medical imaging system 100. A sensor plane 146 of the optical sensor system 140 is also indicated. The optical sensor system 140 may include one or more optical sensors. Medical imaging system 100 includes, for example, an exemplary magnetic resonance imaging system 152.

[0187] Alternatively, the medical imaging system 100 may include, for example, a computed tomography system or an X-ray imaging system. The medical imaging system 100 (e.g., having an MRI system 152) may be included by a radiotherapy (TR) system. The X-ray imaging system may be, for example, part of an image-guided therapy (IGT) system, or the X-ray imaging system may be, for example, a diagnostic X-ray (DXR) system.

[0188] Patient 118 is shown supported by patient table 120. Patient table 120 is movable in at least two spatial directions 130, 132. For example, patient table 120 is movable between a lower minimum position and an upper maximum position in the vertical direction 130. For example, patient table 120 is movable in the horizontal direction 132. By moving patient table 120 in the vertical direction 132, patient 118 lying on patient table 120 is moved, for example, into magnet 154 of MRI system 152. For example, patient table 120 is movable in three spatial directions.

[0189] The MRI system 152 is controlled by a computer 400. The MRI system 152 includes a magnet 154. The magnet 154 is a superconducting cylindrical magnet with a bore 156 passing through it. It is also possible to use different types of magnets. For example, split cylindrical magnets and so-called open magnets can also be used.

[0190] Within the bore 156 of the cylindrical magnet 154, there exists an imaging isocenter 158 for the MRI system 152. In the imaging isocenter 158, the magnetic field is sufficiently strong and homogeneous to perform magnetic resonance imaging.

[0191] A set of magnetic field gradient coils 160 is also present within the bore 156 of the magnet, which is used to acquire preliminary magnetic resonance data for spatial encoding of the magnetic spin within the bore 156 of the magnet 154. The magnetic field gradient coils 160 are connected to a magnetic field gradient coil power supply 162. The magnetic field gradient coils 160 are intended to be representative. Typically, the magnetic field gradient coils 160 contain three separate sets of coils for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply 162 supplies current to the magnetic field gradient coils 160. The current supplied to the magnetic field gradient coils 160 is controlled as a function of time and can be ramped or pulsed.

[0192] Furthermore, the MRI system 152 may include an RF transmitter coil 161 for manipulating the orientation of magnetic spins. The RF transmitter coil 161 may also be referred to as a transmitter antenna. The RF transmitter coil 161 is connected to an RF transmitter 163. It should be understood that the RF transmitter coil 161 and the RF transmitter 163 are representative. The RF transmitter coil 161 may have multiple transmitter elements, and the RF transmitter 163 may have multiple transmitter channels.

[0193] Transmitter 163, gradient controller 162, one or more light sources 142, optics, and optical sensors 144 are shown as a hardware interface 406 connected to computer 400. Computing device 400 is intended to represent one or more computing devices. Computing device 400 is configured to acquire medical imaging data, such as MRI images using MRI system 152, as part of the control system of medical imaging system 100. Computing device 400 is also configured to control one or more light sources 142 to adjust the illumination conditions of medical imaging system 100 during the acquisition of optical sensor data 414 using optical sensor 144. Optical sensor 144 is configured for monitoring medical imaging system 100.

[0194] Although the invention has been described and illustrated in detail in the accompanying drawings and the foregoing description, such description and illustration are to be regarded as illustrative or exemplary rather than restrictive; the invention is not limited to the disclosed embodiments.

[0195] Those skilled in the art, through studying the accompanying drawings, disclosure, and claims, will be able to understand and implement other variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. Although specific measures are recited in dissimilar dependent claims, this does not imply that combinations of these measures cannot be advantageously used. Computer programs may be stored and / or distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but computer programs may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A method for calibrating a first coordinate system of the field of view of an optical sensor system (140), the calibration comprising registering the first coordinate system with a second coordinate system of a medical imaging system (100) using a computing device (400), the computing device (400) including a key point detection module (412) configured to detect key point elements (122) of a patient stage (120) of the medical imaging system (100) within optical sensor data (414) acquired using the optical sensor system (140) as output in response to receiving optical sensor data (414) acquired using the optical sensor system (140), the optical sensor system (140) being configured to supervise the medical imaging system (100). The method includes: The system receives optical sensor data (414) acquired using the optical sensor system (140), including optical sensor data (414) from the patient station (120). The key point detection module (412) is used to detect the key point elements (122) within the optical sensor data (414). The optical sensor data (414) is used to determine the two-dimensional horizontal coordinates of the detected key point element (122) in the first coordinate system. The two-dimensional horizontal coordinates describe the position of the detected key point element (122) in a plane parallel to the sensor plane (146) of the optical sensor system (140). The third vertical coordinate value of the detected key point element (122) in the first coordinate system is determined, and the third vertical coordinate value describes the distance of the detected key point element (122) from the sensor plane (146) of the optical sensor system (140). Receive three-dimensional coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system of the medical imaging system (100). One or more transformation parameters (420) for the transformation from the first coordinate system to the second coordinate system are determined using the coordinate values ​​describing the position of the detected key point element (122) in the first coordinate system and the coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system.

2. The method according to claim 1, wherein the transformation described by the one or more transformation parameters (420) includes one or more of the following: one or more rotations, one or more translations, one or more distortions, and scaling.

3. The method according to any one of the preceding claims, wherein the received optical sensor data (414) includes optical sensor data (414) of the patient table (120) positioned relative to the optical sensor system (140), and for the optical sensor data of the patient table (120) positioned relative to the optical sensor system (140), a depth map describing the distance of the detected key point element (122) from the sensor plane (146) of the optical sensor system (140) is received, the third vertical coordinate value of the detected key point element (122) being determined using the depth map.

4. The method according to any one of the preceding claims, the method further comprising receiving a first reference value of a horizontal distance between the detected keypoint elements (122) of the patient table (120) determined in first optical sensor reference data, the first optical sensor reference data including optical sensor data (414) of the patient table (120) positioned at a lower first reference position having a first vertical reference distance of the detected keypoint elements (122) from the sensor plane (146) of the optical sensor system (140), The system receives a second reference value for the horizontal distance between the detected keypoint elements (122) of the patient table (120) determined in second optical sensor reference data, the second optical sensor reference data including optical sensor data (414) of the patient table (120) positioned at an upper second reference position having a second vertical reference distance of the detected keypoint elements (122) from the sensor plane (146) of the optical sensor system (140). Determining the third vertical coordinate value of the detected key point element (122) includes: The received optical data is used to determine the value of the horizontal distance between the detected key point elements (122) of the patient table (120). The third vertical coordinate value is estimated using the determined horizontal distance value, the first reference value of the received horizontal distance and the first vertical reference distance, and the second reference value of the received horizontal distance and the second vertical reference distance.

5. The method according to claim 4, wherein the lower first reference position is the lower minimum position of the patient table (120), the upper second reference position is the upper maximum position of the patient table (120), and the estimation of the third vertical coordinate value includes interpolation of the third vertical coordinate value.

6. The method according to any one of the preceding claims, wherein the optical sensor system (140) receives optical sensor data (414) comprising optical sensor data (414) of the patient table (120) in a first horizontal position and optical sensor data (414) of the patient table (120) in a second horizontal position, the first horizontal position and the second horizontal position of the patient table (120) being spaced apart from each other by a known distance, Detecting the key point element (122) within the optical sensor data (414) includes: The key point element (122) is detected within the optical sensor data (414) of the patient table (120) in the first horizontal position, and the key point element (122) is detected within the optical sensor data (414) of the patient table (120) in the second horizontal position. Determining the third vertical coordinate value of the detected key point element (122) includes performing triangulation using the position of the key point element (122) detected in the optical sensor data (414) of the patient table (120) in the first horizontal position and the position of the key point element (122) detected in the optical sensor data (414) of the patient table (120) in the second horizontal position.

7. The method according to any one of the preceding claims, wherein the optical sensor system (140) comprises at least two optical sensors arranged spaced apart from each other at a known fixed baseline. The received optical sensor data (414) includes first optical sensor data (414) acquired using a first optical sensor among the at least two optical sensors and second optical sensor data (414) acquired using a second optical sensor among the at least two optical sensors. Detecting the key point element (122) within the optical sensor data (414) includes: The key point element (122) is detected within the first optical sensor data (414) and the key point element (122) is detected within the second optical sensor data (414). Determining the third vertical coordinate value of the detected key point element (122) includes performing triangulation using the position of the key point element (122) detected in the first optical sensor data (414) and the position of the key point element (122) detected in the second optical sensor data (414).

8. The method according to any one of the preceding claims, in addition to detecting the key point element (122), the key point detection module (412) is further configured to output the third vertical coordinate value of the detected key point element (122) in response to receiving the optical sensor data (414), wherein determining the third vertical coordinate value of the detected key point element (122) includes: Use the key point detection module (412).

9. The method according to any one of the preceding claims, wherein the key point elements (122) of the patient table (120) are distributed on the surface of the patient table (120), particularly along the outline of the patient table (120).

10. The method according to any one of the preceding claims, the method further comprising checking the calibration, the checking of the calibration comprising: The coordinates of the key point elements (122) detected within the optical sensor test data acquired using the optical sensor system (140) are determined in the first coordinate system. The optical sensor test data includes optical sensor data (414) of the patient table (120) positioned at the test location. For the patient table (120) positioned at the test location, the coordinate values ​​of the key point element (122) in the second coordinate system are received. The coordinate values ​​of the first coordinate system are compared with the coordinate values ​​of the second coordinate system using one or more transformation parameters (420) determined for the transformation between the first coordinate system and the second coordinate system. In response to detecting that the deviation between the compared coordinate values ​​exceeds a predefined threshold, a recalibration is applied, which includes: initiating another calibration to determine one or more alternative transformation parameters (420), or using one or more other previously determined transformation parameters (420) as one or more alternative transformation parameters (420).

11. The method according to any one of the preceding claims, wherein the method further comprises: In addition to the optical sensor data (414) used for the calibration, a plurality of additional optical sensor data (414) acquired using the optical sensor system (140) are also received, including additional optical sensor data (414) of the patient station (120). The key point detection module (412) is used to detect the key point elements (122) within the additional optical sensor data (414). The optical sensor data (414) used for the calibration is selected from a set of received optical sensor data (414), which includes additional optical sensor data (414) in addition to the selected optical sensor data (414). The selection includes checking whether the confidence value determined for a predefined number of key point elements (122) detected within the selected optical sensor data (414) exceeds a predefined threshold.

12. The method according to any one of the preceding claims, further comprising: A training dataset is provided for training the keypoint detection module (412), the training dataset including optical sensor training data, wherein keypoint elements (122) of the patient table (120) are marked in the optical sensor training data. In response to receiving the optical sensor training data of the corresponding training dataset, the key point detection module (412) is trained to detect labeled key point elements (122) of the patient table (120) within the optical sensor training data of the training dataset as output.

13. A computer program including machine-executable instructions (410) for calibrating a first coordinate system of the field of view of an optical sensor system (140), the calibration including registering the first coordinate system with a second coordinate system of a medical imaging system (100) using a keypoint detection module (412), the keypoint detection module being configured to detect keypoint elements (122) of a patient stage (120) of the medical imaging system (100) within optical sensor data (414) acquired using the optical sensor system (140) as output in response to receiving such optical sensor data (414), the optical sensor system (140) being configured to supervise the medical imaging system (100). The machine-executable instructions (410) are executed by the processor (402) of the computing device (400), causing the processor (402) to control the computing device (400) to perform a method including the following operations: The system receives optical sensor data (414) acquired using the optical sensor system (140), including optical sensor data (414) from the patient station (120). The key point detection module (412) is used to detect the key point elements (122) within the optical sensor data (414). The optical sensor data (414) is used to determine the two-dimensional horizontal coordinates of the detected key point element (122) in the first coordinate system. The two-dimensional horizontal coordinates describe the position of the detected key point element (122) in a plane parallel to the sensor plane (146) of the optical sensor system (140). The third vertical coordinate value of the detected key point element (122) in the first coordinate system is determined, and the third vertical coordinate value describes the distance of the detected key point element (122) from the sensor plane (146) of the optical sensor system (140). Receive three-dimensional coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system of the medical imaging system (100). One or more transformation parameters (420) are used to determine the transformation from the first coordinate system to the second coordinate system, using the coordinate values ​​describing the position of the detected key point element (122) in the first coordinate system and the coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system.

14. A computing device (400) for calibrating a first coordinate system of the field of view of an optical sensor system (140), the calibration comprising registering the first coordinate system with a second coordinate system of a medical imaging system (100), the computing device (400) comprising a key point detection module (412) configured to detect key point elements (122) of a patient stage (120) of the medical imaging system (100) within optical sensor data (414) acquired using the optical sensor system (140) as output in response to receiving optical sensor data (414) acquired using the optical sensor system (140), the optical sensor system (140) being configured to supervise the medical imaging system (100). The computing device (400) includes a processor (402) and a memory (404) therein storing machine-executable instructions (410). The execution of the machine-executable instructions (410) by the processor (402) of the computing device (400) causes the processor (402) to control the computing device (400) to perform a method including the following operations: The system receives optical sensor data (414) acquired using the optical sensor system (140), including optical sensor data (414) from the patient station (120). The key point detection module (412) is used to detect the key point elements (122) within the optical sensor data (414). The optical sensor data (414) is used to determine the two-dimensional horizontal coordinates of the detected key point element (122) in the first coordinate system. The two-dimensional horizontal coordinates describe the position of the detected key point element (122) in a plane parallel to the sensor plane (146) of the optical sensor system (140). The third vertical coordinate value of the detected key point element (122) in the first coordinate system is determined, and the third vertical coordinate value describes the distance of the detected key point element (122) from the sensor plane (146) of the optical sensor system (140). Receive three-dimensional coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system of the medical imaging system (100). One or more transformation parameters (420) are used to determine the transformation from the first coordinate system to the second coordinate system, using the coordinate values ​​describing the position of the detected key point element (122) in the first coordinate system and the coordinate values ​​describing the position of the detected key point element (122) in the second coordinate system.

15. A medical imaging system (100) comprising a computing device (400) according to claim 14, an optical sensor system (140) configured to monitor the medical imaging system (100), and a patient table (120), wherein the medical imaging system (100) is one of: a magnetic resonance imaging system (152), a computed tomography system, or an X-ray imaging system.

Images