Method and apparatus for registration

Abstract

The invention relates to a registration method for determining a position and orientation of an object relative to a position detection system, wherein the method comprises the following steps: - capturing an image of a surface of the object with an image sensor unit comprising at least one motion sensor and / or at least one position sensor, - determining a capturing position of the image sensor unit in a coordinate system of the position detection system by processing motion sensor signals provided by the motion sensor and / or position signals provided by the position sensor, - generating a surface model of the object photogrammetrically from the captured image, and - transforming the surface model generated photogrammetrically into the coordinate system of the position detection system.

Description

Technical Field

[0001] The present invention relates to a registration method and registration apparatus for determining the position and orientation of an object relative to a position detection system. Background Technology

[0002] For example, position detection systems for navigation of medical devices (such as surgical instruments) are known in the medical field. Such position detection systems can be optical, ultrasonic, or electromagnetic position detection systems, and are used to detect the position and orientation of position sensors relative to the position detection system.

[0003] For example, electromagnetic position detection systems are known, in which a field generator produces an alternating electromagnetic field and provides it to a position sensor comprising a coil. The current induced in the coil by the alternating electromagnetic field depends on the orientation of the corresponding coil relative to the alternating electromagnetic field. If a movable device is equipped with such a position sensor, for example in the form of a sensor coil, the placement point and position of the device relative to a reference sensor, which may also include a coil, can be determined. Such a reference sensor is preferably rigidly connected to a part of the patient's body (or otherwise a different object).

[0004] To support surgeons in navigating surgical instruments within a patient's body, such position detection systems are typically used to detect the position of surgical instruments equipped with position sensors and display the instrument's position in cross-sectional images of the body part obtained through tomography.

[0005] To achieve this, the position values ​​provided by the instrument's position sensor must be transmitted to the coordinates of the patient's tomographic image. For example, it is known to generate a local anatomical image of the surface of a body part from the patient's tomographic image, so that points on the surface of the local anatomical image (hereinafter also referred to as the model surface) are correlated with those points on the surface of the actual body part, respectively contacted by a pointer instrument or a sensing instrument. Therefore, a transformation instruction for transforming the position values ​​detected by the position detection system into model coordinates can be generated within the scope of the registration method. For this purpose, multiple points on the actual surface of the body part are sensed, and associated position values ​​representing points on the actual surface are correlated with points on the model surface, while maintaining their relative positions to each other in a manner that produces minimal possible error. A transformation instruction specifying how the detected position values ​​are transformed into the coordinates of the local anatomical image (also referred to herein as the local anatomical model), and thus also into the coordinates of the tomographic image or model, is thus generated.

[0006] Patient registration refers to establishing a transformation function that corresponds positional data detected during surgery to positional information in image data obtained before surgery, such as through tomography. For example, for the purpose of patient registration, as described above, a patient model is detected and a transformation function is established that, within the scope of the registration method, corresponds the detected positional data and the resulting patient model to positional information about image data obtained before surgery.

[0007] To determine the transformation function (i.e., registration), the same geometric features in the model and (e.g., obtained via tomography) image data are established in the corresponding coordinate systems. These two coordinate systems are then correlated using these features. Surface registration using a pointer instrument is commonly used. Here, the patient's skin surface acts as the corresponding feature. During the procedure, the skin surface is sensed using a pointer instrument, and the skin surface is aligned with the skin surface extracted from the image data.

[0008] A non-tactile registration method is also described.

[0009] The non-tactile registration method for registering patient image data, as described in US 9208561 B2, utilizes an image sensor unit (single-focal-length camera or 3D camera) as a reference sensor. The image sensor unit generates a surface model of the skin surface, optically detects the reference sensor, and correlates its position with the skin surface model. Since the reference sensor is also tracked by the position detection system, the transformation of points on the skin surface model from the coordinate system of the surface model to the coordinate system of the position detection system can be determined, and registration can be calculated from this transformation to the patient image data.

[0010] Another non-tactile registration method utilizes a structured light or laser emitter / receiver unit rigidly integrated with an optical position detection system. The optical position detection system typically includes a light emitter / receiver unit and a position sensor configured to reflect light emitted by the emitter / receiver unit. The emitter / receiver unit then detects the reflected light, from which the position and orientation of the position sensor can be obtained. Thus, the surface model generated by the structured light or laser emitter / receiver unit is known in the coordinate system of the optical position detection system. Summary of the Invention

[0011] The objective of this invention is to provide an improved non-tactile registration method and an improved registration device.

[0012] Regarding the registration method, this objective is achieved through a registration method for determining the position and orientation of an object relative to a position detection system, wherein the method includes the following steps:

[0013] - Capture images of the object's surface using an image sensor unit that includes at least one motion sensor and / or at least one position sensor.

[0014] - The capture position of the image sensor unit is determined by processing the motion sensor signal provided by the motion sensor and / or the position signal provided by the position sensor.

[0015] - Generate a surface model of the object from the captured image using photogrammetry, and

[0016] - Transform the surface model generated by photogrammetry into the coordinate system of the position detection system.

[0017] Within the framework of this specification, the term "capture position" refers to the position of the image sensor unit when capturing an image of an object. For example, the term "capture position" refers to the position of the entrance pupil of the camera, which is part of the image sensor unit, when capturing an image.

[0018] In these steps, the order of "capturing an image of the object's surface using the image sensor unit," "determining the capture position of the image sensor unit," and "generating a surface model of the object from the captured image using photogrammetry" is irrelevant. Therefore, an image of the object's surface can be captured first, and the capture position of the image sensor unit determined only afterward. Alternatively, the capture position of the image sensor unit can be determined even after the surface model of the object has been generated using photogrammetry. However, the capture position of the image sensor unit is required to transform the photogrammetrically generated surface model into the coordinate system of the position detection system. This will be explained in more detail below.

[0019] This invention incorporates the understanding that, in general, non-tactile registration methods have the advantage of avoiding problems that arise with known registration methods using pointer instruments, such as the potential deformation of a patient's soft skin during contact with the pointer instrument. Furthermore, since the optical generation of the surface model allows for the simultaneous detection of a larger portion of the surface of a body part or object, registration can be performed more quickly and accurately.

[0020] When capturing an image of a surface, it is preferable to position the image sensor unit at a sufficient distance from the surface of the object. This typically means that the electromagnetic field of the electromagnetic position detection system is too weak at the capture location of the image sensor unit to allow the position of the image sensor unit to be determined using the electromagnetic position detection system.

[0021] The general concept for determining the position and orientation of an object relative to a position detection system can be described as follows:

[0022] Preferably, a surface model of the object is generated by the image sensor unit using photogrammetry. The surface model can be represented by a point cloud, wherein the coordinates of the points in the point cloud are defined in the coordinate system of the surface model. The coordinate system of the surface model is the intrinsic coordinate system of the image sensor unit, or more precisely, the intrinsic coordinate system of the camera included in the image sensor unit. Preferably, the coordinate system of the surface model is defined such that its origin is located at the position of the entrance pupil of the camera of the image sensor unit.

[0023] Points representing the point cloud of an object's surface have coordinates in both the coordinate system of the surface model and the coordinate system of the position detection system.

[0024] By generating a surface model from the captured image using photogrammetry, the coordinates of the points in the point cloud are first obtained in the coordinate system of the surface model. The coordinates of these points in the coordinate system of the position detection system can then be obtained through transformation.

[0025] To perform such transformations, transformation functions can be used to transform the coordinates of points in a point cloud defined in the coordinate system of a surface model into coordinates in the coordinate system of a position detection system. Transformation functions can include or be represented by transformation matrices.

[0026] To obtain the transformation function used to transform the coordinates of the image sensor unit's coordinate system to the coordinates of the position detection system, the position and orientation of the image sensor unit at the moment of image capture are required. This is because the coordinate system of the image sensor unit is centered on the image sensor unit, and specifically centered on the entrance pupil of the image sensor unit.

[0027] In terms of coordinates for the position detection system, the position of the image sensor unit (i.e., the capture position) can be obtained through sensors (e.g., motion sensors or position sensors) arranged on the image sensor unit. Typically, such sensors are offset relative to the entrance pupil of the image sensor unit. Therefore, the coordinate system of the image sensor unit, with the capture position as its origin, is offset from the coordinate systems of the motion sensors and / or position sensors, where the coordinates of the capture position are defined by a non-trivial vector. This offset (i.e., the vector) can be determined through calibration and can be represented by the coordinates of the image sensor unit's coordinate system or by the coordinates of the sensors arranged on the image sensor unit. Thus, by means of calibration, the coordinates of the capture position in the coordinate systems of the motion sensors and / or position sensors can be determined. The coordinate system of the image sensor unit can then be "attached" with its origin (i.e., the capture position in the coordinate system of the surface model) to the coordinates of the capture position in the coordinate systems of the motion sensors and / or position sensors.

[0028] Therefore, the coordinates of points representing the point cloud of the surface model in the coordinate system of the surface model can be transformed to the coordinate system of the position detection system via the coordinate system of the motion sensor or position sensor.

[0029] Preferred variations of the registration method according to the invention are described below.

[0030] If the image sensor unit includes a position sensor, the translation and rotation of the image sensor unit can be determined by detecting the position and orientation of the position sensor. Therefore, the coordinates and orientation of the capture position in the coordinate system of the position sensor can be directly determined within the coordinate system of the position detection system. The rotation of the image sensor unit can then be determined based on the orientation of the position sensor's coordinate system.

[0031] If the image sensor unit includes only a motion sensor (and does not include a position sensor), the translation and rotation of the image sensor unit relative to the position detection system can be determined indirectly. One way to indirectly determine the position and orientation of the image sensor unit is to move the image sensor unit along a path from a position with known coordinates in the coordinate system of the position detection system, or to the position itself. Points on the path of the motion sensor can be associated with this known position in the coordinate system of the position detection system. Therefore, the translation and rotation of the image sensor unit can be determined by detecting the position and orientation of the motion sensor's coordinate system relative to the coordinate system of the position detection system. Thus, the position and orientation of the motion sensor's coordinate system relative to the coordinate system of the position detection system can be determined. By calibrating the motion sensor to the position of the incident pupil, the coordinates of the capture position in the motion sensor's coordinate system can be determined. This allows the determination of the coordinates of the capture position in the coordinate system of the detection system.

[0032] The coordinate systems of the motion sensor and / or position sensor have a fixed relationship with the coordinate system of the surface model because the relative distance and orientation between the camera of the image sensor unit and the motion sensor and / or position sensor arranged on the image sensor unit remain constant. Therefore, a calibration vector representing the offset between the coordinate system of the surface model and the coordinate system of the motion sensor and / or position sensor can be determined. This can be accomplished by calibrating the camera (e.g., the position of the camera's entrance pupil) to the coordinate system of the motion sensor and / or position sensor. Calibration may include determining a transformation function between the coordinate system of the surface model and the coordinate system of the motion sensor or position sensor. The corresponding coordinates of the capture position in the coordinate system of the motion sensor and / or position sensor can then be determined.

[0033] Specifically, if the image sensor unit is positioned relatively close to the position detection system, the capture position can be determined using a position sensor attached to the image sensor unit, which can be positioned together with the position detection system to determine the position and orientation of the image sensor unit. After calibrating the position sensor to the position of the incident pupil of the image sensor unit, the coordinates of the capture position determined in the coordinate system of the position sensor can be transformed to the coordinate system of the position detection system. In these embodiments, the position and orientation of the image sensor unit can be directly tracked using an electromagnetic or optical position detection system. However, using only a position sensor to determine the capture position of the image sensor unit does not allow for determining capture positions where the electromagnetic field of the electromagnetic position detection system is too weak (i.e., outside the workspace of the position detection system).

[0034] Therefore, in some embodiments of these embodiments where the image sensor unit includes a position sensor, it is advantageous to also include a motion sensor. If the image sensor unit includes both a position sensor and a motion sensor, the position sensor can be used, for example, to determine the starting position of a user-selected path along which the image sensor unit moves. The path along which the image sensor unit moves can then be tracked using either the position sensor or the motion sensor, or both.

[0035] Alternatively or additionally, the capture position can be determined by using a position sensor to detect the position and orientation in the coordinate system of a position detection system that is separately positioned from the image sensor unit. Such a reference position sensor can be attached to the object itself or placed at a relative distance from the object, such that the reference position sensor is visible in the image along with the object when an image of the object is captured. The reference position sensor then provides a reference to the coordinates in the coordinate system of the position detection system in the image, thereby allowing points on the surface model to be correlated with their coordinates in the coordinate system of the position detection system. Thus, the coordinates of any point on the surface model can be transformed into the corresponding coordinates in the coordinate system of the position detection system. This method can also be combined with embodiments of registration methods, wherein the image sensor unit includes a position sensor and / or a motion sensor.

[0036] Alternatively or additionally, the capture position can be determined by fixing the image sensor unit to a predefined position, for example, defined by the position of the image sensor unit holder known in the coordinate system of the position detection system. The predefined position can also be defined by directly mounting the image sensor unit to the field generator of the electromagnetic position detection system. With the image sensor unit fixed to the predefined position at coordinates known in the coordinate system of the position detection system, an image can be captured. The capture position can then be calibrated to the predefined position, for example, the position of a position sensor included in the holder. The coordinates of the capture position in the coordinate system of the position detection system can then be determined. The image sensor unit fixed to the predefined position may also include the position sensor and / or motion sensor itself. An image can also be captured such that a reference position sensor attached to or placed relative to the object is visible along with the object in the captured image.

[0037] In various embodiments of the registration method according to the invention, and particularly in such embodiments where the image sensor unit includes a position sensor for registration, the registration method may include the following steps:

[0038] - Provide at least one image sensor unit, the at least one image sensor unit including at least one motion sensor for detecting linear acceleration and / or rotational speed over time independently of the position detection system.

[0039] In some of these various embodiments where the image sensor unit includes a motion sensor, the step of "determining the capture position of the image sensor unit" includes the following sub-steps.

[0040] -The image sensor unit moves along the spatial path relative to the position detection system.

[0041] - The path of the image sensor unit is recorded by processing the motion sensor signal provided by the motion sensor.

[0042] - Associating at least one point of the determined path with coordinates in the coordinate system of the position detection system, and

[0043] - The capture position of the image sensor unit is determined based on the determined path and the known spatial relationship between at least one point on the determined path and the coordinates.

[0044] These sub-steps refer to determining the position of the image sensor unit as it moves relative to the position detection system. These sub-steps can be performed before or after an image is captured.

[0045] The user can select the path along which the image sensor unit moves; that is, it does not need to follow a predetermined path.

[0046] The sub-step of associating at least one point of the determined path with coordinates in the coordinate system of the position detection system, and determining the capture position of the image of the captured object by the image sensor unit based on the determined path and the known spatial relationship between the determined path and at least one point on the determined path and the coordinates, is used to determine the position and orientation of the recorded path in the coordinate system of the position detection system. Specifically, the sub-step of associating at least one point of the determined path with coordinates in the coordinate system of the position detection system operates independently of whether an image has been captured. Therefore, the path of the image sensor unit can be recorded without capturing an image or before capturing an image. However, in order to determine the capture position, especially the capture position in the coordinate system of the position detection system, it is necessary to determine the path along which the image sensor unit has moved to or away from the capture position, and the spatial relationship between at least one point on the determined path and the coordinates in the coordinate system of the position detection system is known.

[0047] Associating at least one point on the defined path with coordinates in the coordinate system of the position detection system can be achieved in various ways. For example, a spatial relationship between a point on the path and the coordinates in the coordinate system can be established by moving an image sensor unit along the path from a known position in the coordinate system of the position detection system, or by moving the image sensor unit to the known position. Once the image sensor unit reaches a position with known coordinates in the coordinate system of the position detection system, a known spatial relationship is established between the coordinates in the coordinate system of the position detection system and at least one point on the defined path. When at the known position, the motion sensor can be reset.

[0048] The step of generating a surface model of an object from at least one captured image by photogrammetry requires only the captured image and can be performed at any time after the image has been captured.

[0049] In various embodiments where the image sensor unit includes a motion sensor, and in other variations of the registration method where the image sensor unit includes an additional position sensor, the step of "transforming the surface model generated by photogrammetry to the coordinate system of the position detection system" includes the following sub-steps.

[0050] - By calibration, at least one point of the generated surface model is correlated with the coordinates in the coordinate system of the motion sensor and / or position sensor.

[0051] For photogrammetry, the camera's entrance pupil (also known as the front node) is typically chosen as the reference for photogrammetric generation of the surface model. The entrance pupil of the image sensor unit has a fixed relative distance and orientation with the motion sensors and / or position sensors included in the image sensor unit. This results in an offset between the coordinate system of the image sensor unit and the coordinate system of the sensor. This offset can be determined through calibration.

[0052] The capture position, such as the position of the incident pupil, is preferably set as the origin of the coordinate system of the image sensor unit. The capture position in the coordinate system of the motion sensor and / or position sensor can be determined by determining the offset between the image sensor unit and the position or motion sensor.

[0053] Thus, the points of the generated surface model, initially defined by the coordinates in the coordinate system of the image sensor unit, can be correlated with the corresponding coordinates in the sensor's coordinate system, and subsequently with the corresponding coordinates in the coordinate system of the position detection system.

[0054] This sub-step requires that a surface model has been generated and therefore an image has been captured. This sub-step can be performed at any time after the surface model has been generated.

[0055] Preferably, this sub-step of relating at least one point of the generated surface model to coordinates in the coordinate system of the motion sensor and / or position sensor includes...

[0056] - A transformation function is determined through calibration, which is used to transform the coordinates between the coordinate system of the motion sensor and / or the coordinate system of the position sensor and the coordinate system of the image sensor unit, and

[0057] - Determine the coordinates of the capture position of the image sensor unit in the coordinate systems of the motion sensor and / or the position sensor, respectively.

[0058] Therefore, the coordinates of the capture position in the coordinate system of the motion sensor and / or position sensor are obtained. When the position and orientation of the coordinate system of the motion sensor and / or position sensor, and therefore the sensor, relative to the coordinate system of the position detection system, are known (or can be determined), the coordinates of the capture position in the coordinate system of the position detection system can also be determined. By using a transformation function, the coordinates of the surface model generated by photogrammetry in the coordinate system of the surface model (of the image sensor unit) can be transformed to the coordinate system of the position detection system.

[0059] In particular, registration methods based on various embodiments of an image sensor unit including a motion sensor can overcome the limitations of current non-tactile registration methods. For example, the non-tactile registration method described in US 9208561 B2 requires a position sensor for detecting the position and orientation in the coordinate system of a position detection system to be coupled to or constitute an image sensor unit. With the aid of a position sensor, the position and orientation of the image sensor unit can be directly determined in the coordinate system of the position detection system.

[0060] This invention includes another understanding: current non-tactile registration methods, where an image sensor unit is coupled to or includes a position sensor, only work when the position sensor can actually be reliably detected by the position detection system. Therefore, the application of such non-tactile registration methods is generally limited to the workspace of the position detection system. The workspace of the position detection system is, for example, a space where the light or electromagnetic field of the position detection system has a certain field strength and is sufficiently uniform to reliably determine the position and orientation of the position sensor. However, if the image sensor unit coupled to or including the position sensor is at a distance from the position detection system (where the workspace of the light or electromagnetic field of the position detection system is relatively weak or relatively non-uniform), current non-tactile registration methods are expected to lose accuracy and may no longer be applicable reliably.

[0061] Registration methods according to various embodiments of the image sensor unit including at least one motion sensor do not require the image sensor unit itself to include a position sensor or be coupled to a position sensor. Linear acceleration and rotational speed of a moving body can be detected using a motion sensor, or preferably a motion sensor arrangement including several motion sensors (e.g., an inertial sensor consisting of an accelerometer, a gyroscope, or also a magnetometer). The motion sensor provides a motion sensor signal representing the linear acceleration and rotational speed detected over time. The motion sensor signal can be processed, for example, by a tracking unit, such as by integration, to determine the path along which the image sensor unit moves. The path along which the image sensor unit moves relative to the position detection system includes a sequence of positions over time. In particular, this path is a freely chosen path relative to the position detection system. Tracking of the movement of the image sensor unit by means of the motion sensor operates independently of the position detection system.

[0062] Although the motion sensor operates independently of the position detection system, in particular, the position and orientation of an object relative to the position detection system can still be determined using non-tactile registration methods according to various embodiments of the image sensor unit including at least one motion sensor. This is because the path along which the image sensor unit moves is recorded by processing the motion sensor signal provided by the motion sensor, and because at least one point on the determined path is related to coordinates in the coordinate system of the position detection system. Therefore, for at least one point on the determined path, coordinates in the coordinate system of the position detection system can be determined. To determine the position and orientation of the object relative to the position detection system, specifically, it is necessary to determine the capture position of the image of the object captured by the image sensor unit within the coordinate system of the position detection system. When determining the capture position of the image sensor unit including the motion sensor in the coordinate system of the position detection system, at least the determined path of the image sensor unit and the known spatial relationship between the determined path and at least one point on the determined path and the coordinates must be known.

[0063] Furthermore, the position of the motion sensor, or in the case of sensor arrangement, must be calibrated to the entrance pupil of the camera in the image sensor unit. If the image sensor unit includes more than one camera, or if a camera includes more than one entrance pupil, the entrance pupils must be calibrated to each other and to one or more motion sensors. If the image sensor unit also includes additional sensors, such as one or more position sensors, the entrance pupils must also be calibrated to these position sensors. Calibration refers to determining the transformation function used for the transformation between the coordinate systems of two elements (e.g., entrance pupils and / or sensors). These transformation functions must typically be determined once because the relative distances and orientations between the corresponding entrance pupils and / or sensors do not change after they have been installed, for example, in the image sensor unit.

[0064] In this regard, it has been emphasized that the image sensor unit preferably includes a 3-D camera, such as an optical stereo camera, an optical multi-camera, or a time-of-flight camera.

[0065] Images captured from the capture location are used to generate a surface model via photogrammetry. The points of this surface model are then correlated with coordinates in the coordinate systems of a motion sensor and / or a position sensor through calibration. This may include determining a transformation function to transform the coordinates in the motion sensor and / or position sensor coordinate systems to corresponding coordinates in the coordinate system of the surface model.

[0066] After calibration, points on the surface model defined in the coordinate system of the surface model can be assigned coordinates in the coordinate system of the position detection system. Specifically, points on the surface model defined in the coordinate system of the surface model can be assigned coordinates in the coordinate system of the position detection system that have a known spatial relationship with at least one point on the path. This spatial relationship can be established by moving the image sensor unit from a known position in the coordinate system of the position detection system or by moving the image sensor unit to said position. After calibration, the generated surface model can be transformed to the coordinate system of the position detection system and thus registered to that coordinate system.

[0067] Using the non-tactile registration method according to the invention, and specifically, using the registration method according to various embodiments of the image sensor unit including a motion sensor, the position and orientation of an object relative to the position detection system can be determined with high accuracy. Furthermore, the registration method can be performed relatively easily and quickly, and (particularly with respect to various embodiments of the image sensor unit including a motion sensor), without being spatially restricted by the workspace of the position detection system. Therefore, a particular advantage of the registration method is that the image sensor unit can move relatively freely and independently of the workspace of the position detection system, while still reliably determining the position and orientation of the object relative to the position detection system. This thus allows for capturing an image of the object at the most suitable capture location. Typically, the most suitable capture location can be located outside the workspace of the position detection system, and can now be reliably approached.

[0068] Preferably, the position of the camera's entrance pupil is calibrated to the position of the sensor contained in the image sensor unit. Since the position and orientation of the sensor relative to the position detection system can be determined, the coordinates of the captured position in the coordinate system of the image sensor unit can be transformed to the coordinate system of the position detection system.

[0069] The location of the sensor when capturing an image can be the start or end point of the path, but it can also correspond to any other point on the path between the start and end points of the trajectory.

[0070] The known spatial relationship between at least one point on the determined path and its coordinates in the coordinate system of the position detection system can be the relative distance from at least one point on the determined path to its coordinates in the coordinate system of the position detection system. Preferably, the coordinates in the coordinate system of the position detection system are previously known or defined before the image sensor unit is moved. For example, if the image sensor unit moves from a known position in the coordinate system of the position detection system or moves to the known position, a known spatial relationship is established between the coordinates in the coordinate system of the position detection system and at least one point on the determined path. Based on this known spatial relationship and the known orientation of the path relative to the position detection system, any other points on the recorded path can be transformed into the coordinate system of the position detection system.

[0071] Preferably, the relative distance from at least one point of the determined path to its coordinates in the coordinate system of the position detection system, and the orientation of the path in the coordinate system of the position detection system, are obtained by associating at least two points of the determined path with different coordinates in the coordinate system of the position detection system. In some embodiments, this requires associating at least two points of the determined path with different coordinates in the coordinate system of the position detection system. Based on the relative distance from at least one point of the determined path to its coordinates in the coordinate system of the position detection system and the orientation of the path in the coordinate system of the position detection system, the relative distance of any point on the path can be determined in the coordinate system of the position detection system, particularly the position of the sensor when capturing an image.

[0072] The registration method according to the invention can be performed using different position detection systems, such as optical, electromagnetic, or ultrasonic-based position detection systems. However, preferably, the registration method according to the invention is performed using an electromagnetic position detection system.

[0073] Coordinates having a known spatial relationship with at least one point on the path can be defined by, for example, the position of a field generator or light source within the position detection system itself. For instance, the position of the field generator can be considered the origin of the coordinate system spanned by the electromagnetic position detection system. Coordinates having a known spatial relationship with at least one point on the path can also be defined by the position of a position sensor located at a distance relative to the field generator or light source, for example, at a fixed location. Such a position sensor can be attached to an object. The position sensor can also be included in an image sensor unit holder for an image sensor unit. Coordinates having a known spatial relationship with at least one point on the path, the position sensor is included in a touch point. The image sensor unit can move to and contact the touch point. As the image sensor unit moves away from the touch point along the path, each point on the path is preferably associated with the touch point, and specifically with the position sensor of the touch point. If the image sensor unit itself includes a position sensor, the coordinates having a known spatial relationship with at least one point on the path can also be defined by the position sensor detected within the coordinate system of the position detection system and thus by the specific position of the image sensor unit or the recorded path. For example, if the image sensor unit itself further includes a position sensor for detecting position and orientation in the coordinate system of the position detection system, then any position of the position sensor can be defined as coordinates having a known spatial relationship with at least one point on the path. Starting from this coordinate, the image sensor unit can move along the path and record the path by processing motion sensor signals provided by the motion sensor.

[0074] The surface model of the object generated by photogrammetry from at least one captured image can be calculated by the image sensor unit or, alternatively, by the position determination unit as part of the corresponding registration device.

[0075] The object (from which the image is captured) can be a part of the patient's body, such as the patient's face.

[0076] In some embodiments of various embodiments in which the image sensor unit includes a motion sensor, the image sensor unit moves along a path that begins at the coordinates of a known location in the coordinate system of the position detection system and ends at the location where the sensor captures the image. Preferably, the location where the sensor captures the image is then determined in the coordinate system of the position detection system based on the recorded path and at least the known spatial relationship between the starting point of the path and the coordinates.

[0077] The coordinates at which the image sensor unit begins to move in the coordinate system of the position detection system are located within the workspace of the position detection system. The image sensor unit can be moved away from this known position, even to a location outside the workspace. Because the path along which the image sensor unit moves begins at a known starting position, the relative distance from any point on the path to the known starting position can be calculated if the orientation of the path in the coordinate system of the position detection system is known. This means that any point on the path can be determined in the coordinate system of the position detection system, specifically any sensor position relative to the captured image of the position detection system located on this path. A portion of the path may lie outside the workspace, and since the path is recorded by processing motion sensor signals provided by the motion sensor, any point on the path outside the workspace can still be determined in the coordinate system of the position detection system. In particular, capture positions also located outside the workspace of the position detection system can still be determined in the coordinate system of the position detection system.

[0078] In some alternative embodiments of various embodiments in which the image sensor unit includes a motion sensor, the path along which the image sensor unit moves begins at the capture position and ends at the known coordinates in the coordinate system of the position detection system, and the image sensor unit moves along this path after the image has been captured. Preferably, the capture position is then determined in the coordinate system of the position detection system based on the recorded path and at least the known spatial relationship between the endpoint of the path and the coordinates.

[0079] In these alternative embodiments, the steps of moving the image sensor unit and capturing the image are reversed in timing compared to the previously described embodiments, wherein the image sensor unit moves along a path that begins at the coordinates in the coordinate system of the position detection system and ends at the capture position. Both the previously described embodiments and these alternative embodiments can produce the same results.

[0080] In these alternative embodiments where the image sensor unit also includes a motion sensor, the capture location may be located outside the workspace of the position detection system. However, when recording the path from the capture location to the endpoint depicted in the coordinate system of the position detection system by processing motion sensor signals provided by the motion sensor, if the orientation of the path in the coordinate system of the position detection system is known, the relative distance from any point, and specifically from the capture location to the endpoint, can be calculated based on the motion sensor signals after the image has been captured.

[0081] By moving the image sensor unit to a known position in the coordinate system of the position detection system or from said known position, a spatial relationship is established between the coordinate system of the position detection system and the movement path of the motion sensor. Generally, in embodiments where the image sensor unit includes a motion sensor, the capture position of the captured image can be determined solely based on the known spatial relationship between the coordinates in the coordinate system of the position detection system and points on the path recorded by processing motion sensor signals provided by the motion sensor. In this case, a position sensor is not required to determine the capture position. Specifically, the desired spatial relationship can be achieved by moving the image sensor unit along the path from the capture position to a known endpoint in the coordinate system of the position detection system, or by moving the image sensor unit along the path from a known starting point in the coordinate system of the position detection system to the capture position.

[0082] The registration method according to the invention may further include the following steps: detecting movement of the object during image capture, and providing a movement signal representing the detected movement and a timestamp by means of a position sensor to detect the position and orientation in the coordinate system of the position detection system arranged on the object. If movement of the object is detected during image capture, an alarm signal may be triggered to notify the user that the object has moved during image capture.

[0083] A suitable position sensor for implementing this step can be configured to provide a sensor signal representing the detected movement and a timestamp. The sensor signal can, for example, be transmitted to the position determination unit of the registration device. The position determination unit can be configured to compensate for the movement of the object detected during image capture by applying the detected movement as a correction value to the position of an image sensor unit having the same timestamp as the corresponding sensor signal. Thus, at least one point of the generated surface model can be correlated with coordinates in the coordinate system of the motion sensor and / or the position sensor using the corrected capture position.

[0084] In particular, in these embodiments where the image sensor unit does not include a position sensor for detecting the position and orientation in the coordinate system of the position detection unit, or in embodiments where the image sensor unit is located outside the workspace of the position detection system, the path along which the image sensor unit moves is preferably recorded only by processing motion sensor signals provided by the motion sensor.

[0085] However, if the image sensor unit also includes an optional position sensor, the registration method may include the step of detecting the position and orientation of the position sensor constituting the image sensor unit in the coordinate system of the position detection system. The position and orientation of the image sensor unit in the coordinate system of the position detection system can be calculated from the determined position and orientation of the position sensor.

[0086] If the image sensor unit includes a motion sensor and a position sensor, preferably, the starting or ending point of the path (the path along which the image sensor unit moves and is recorded by processing motion sensor signals) is determined as coordinates in the coordinate system of the position detection system by detecting the position and orientation of the position sensor. Thus, any point on the path, and specifically the location where the sensor captures the image, can be spatially associated with a known position in the coordinate system of the position detection system.

[0087] If the image sensor unit also includes a position sensor, the position and orientation of the image sensor unit can be directly tracked using the position sensor in the coordinate system of the position detection system. Therefore, as long as the image sensor unit, including the position sensor, moves within the workspace of the position detection system, its position and orientation can be directly determined in the coordinate system of the position detection system. Furthermore, by determining the position and orientation of the position sensor, the starting point of the trajectory tracked by the motion sensor can be directly determined in the coordinate system of the position detection system. The fact that the starting point of the trajectory can be determined in the coordinate system of the position detection system can be signaled to the user, for example, on a display or acoustically. Additionally, the path along which the image sensor unit moves can be recorded or tracked by processing both the motion sensor signal provided by the motion sensor and the position signal representing the position value provided by the position sensor. For example, it is advantageous for the motion sensor to take over or support the position sensor in certain situations, or to be used to correct the position value provided by the position sensor, in order to record the image sensor's path.

[0088] As long as the image sensor unit moves within the workspace of the position detection system, the position and orientation of the object relative to the position detection system can be determined based on both the motion sensor signal provided by the motion sensor and the position signal representing the position value provided by the position sensor.

[0089] It would be beneficial to improve the accuracy of recording the path along which the image sensor unit moves by comparing both motion sensor signals and position signals.

[0090] If the image sensor unit moves outside the workspace, the position sensor stops providing reliable position values. However, even while leaving the workspace, the path of the image sensor unit can still be reliably recorded by processing the motion sensor signals provided by the motion sensor. For example, the motion sensor can take over path tracking from the position sensor. Thus, the space in which the image sensor unit can move to determine the position and orientation of an object relative to the position detection system increases relative to the workspace of the position detection system. This allows for capturing images of the object at an optimal location, which may typically be outside the workspace of the position detection system.

[0091] The registration method according to the invention may further include the steps of: capturing an image of the object and a reference position sensor positioned relative to the object; and generating a surface model and the reference position sensor from the captured image using photogrammetry, such that the position of the reference position sensor in the coordinate system of the photogrammetrically generated surface model can be determined, while providing position information based on the coordinates of the coordinate system of the position detection system. The reference position sensor thus provides a reference point for calibration. Such a reference position sensor can also be used for a position sensor attached to the object. Similarly, the reference position sensor can be used to define coordinates with a known spatial relationship to points on a path. In particular, if the position sensor is not attached to the object and the image sensor unit does not include a position sensor, it may be advantageous to capture an image that shows the reference position sensor and the object, allowing the reference position sensor to be identified in the coordinate system of the photogrammetrically generated surface model. The reference position sensor is then photogrammetrically generated in the coordinate system of the object's surface model and provides another previously known reference position within the coordinate system of the position detection system. Therefore, registration accuracy can be improved.

[0092] In addition, if

[0093] - The reference position sensor is positioned relative to the object, and

[0094] - Generate the position of the reference position sensor in the coordinate system of the surface model using photogrammetry.

[0095] The position information provided by the coordinates of the position detection system can be used to verify the determined position of the surface model in the coordinate system of the position detection system after the surface model has been transformed to the coordinate system of the position detection system. This is done by calculating the deviation between the determined position based on the processed motion sensor signal and / or position signal and the position of the reference position sensor in the coordinate system of the position detection system.

[0096] Preferably, the imaging characteristics of the image sensor unit are known in advance.

[0097] Although the image sensor unit preferably includes a 3-D camera, in some embodiments it is advantageous if the image sensor unit includes a single-focus image sensor. If the image sensor unit includes a single-focus image sensor, preferably, in the registration method, multiple images of the object are captured at different capture positions located on the trajectory while the single-focus image sensor is moved. Therefore, in particular, if the image sensor unit includes a single-focus image sensor, the image sensor unit is preferably moved while capturing multiple images of the surface of the object at different capture positions. For example, the image sensor unit can be continuously guided on the surface of the object to be detected during the registration process. Here, the relative position of the image sensor unit with respect to the object (more precisely: with respect to the position detection system)—i.e., the position of the image sensor unit—and the information detected by photogrammetry (i.e., generally, the optically detected image data represents the captured image) are recorded.

[0098] Advantageously, continuous registration can be performed in such a way that a reference position sensor stationary relative to the object is no longer needed. Instead, a motion sensor or (if otherwise) a position sensor that makes up the image sensor unit will assume the function at the appropriate time.

[0099] Preferably, positional information, represented by, for example, motion sensor signals and / or position values, along with optically detected image data, is fed into an iterative algorithm. This algorithm detects objects (edges, lines, circles, etc.) in the optically detected image data (e.g., within a single image) and is also able to correlate said objects with each other across various individual images. Thus, a list of objects recorded by an image sensor unit, including a monofocal image sensor, from different viewing directions and positions is generated.

[0100] Preferably, another algorithm capable of determining the spatial location of the object is then used from this data. In the case of an image sensor unit including a monofocal image sensor, the object must have been recorded from at least two different viewing directions and positions. The object's location can then be determined by triangulation.

[0101] If a sufficient number of objects and their locations are identified, regular surface registration can be performed.

[0102] While using a single-focus image sensor unit is advantageous in some cases, in many other cases, the registration method can be improved by using a preferred 3D camera as the image sensor unit. With a 3D camera, the surface to be inspected can be recorded in three dimensions from the outset. Reconstructing the surface form from two-dimensional image data becomes redundant, as is required when the image sensor unit includes a single-focus image sensor as described above. Known 3D cameras record the three-dimensional surface form by means of the propagation time of positively measured infrared light pulses. Since individual points on the surface of the object are at different distances from the infrared light source of the image sensor unit and also from its infrared image sensor, this results in pulse propagation times of varying lengths between the light source and the image sensor. These pulse propagation times (or phase shifts) contain information related to the distance between the image sensor unit and the corresponding points on the surface of the object. This type of recording is also known as a TOF (Time-of-Flight) process.

[0103] The image sensor unit can also be implemented as an optical stereo camera or optical multi-camera that records the surface of the object to be inspected in three dimensions. An optical stereo camera can be implemented as a camera assembly of two or more cameras mounted on a fixed, known base. If the image sensor unit includes more than one entrance pupil, the entrance pupils are preferably calibrated to each other.

[0104] Natural lighting, i.e., in-situ lighting, is preferably used to perform photogrammetric inspection of the surface.

[0105] However, in some cases, it may be beneficial to project a pattern onto the surface of an object while simultaneously capturing an image to produce an enhanced image record of the object's surface.

[0106] In such cases, the registration method preferably includes the following steps: when capturing an image, projecting a pattern detectable by the image sensor unit onto the surface of the object.

[0107] To project a pattern onto an object, the image sensor unit can (preferably rigidly) be connected to the pattern projector of the registration device. The relative position between the image sensor unit and the pattern projector, as well as the imaging characteristics of the pattern projector, are preferably known beforehand. Alternatively, the pattern projector can also be connected (preferably rigidly) to a motion sensor and / or a position sensor such that the relative position between the image sensor unit and the pattern projector can always be determined based on motion sensor signals and / or position values ​​provided by the motion sensor and / or position sensor of the pattern projector, respectively, and motion sensor signals provided by the motion sensor of the image sensor unit. The pattern projector can then advantageously move independently of the image sensor unit, enabling the image sensor unit to record a particularly meaningful single image in each case with well-evaluable pattern distortion.

[0108] A pattern projector can be used to project an artificial structure (pattern, such as a stripe pattern) of known size onto a target surface within the capture area of ​​an image sensor unit. The pattern projected onto the surface can be optically detected by the image sensor unit, resulting in a single image displaying the pattern projected onto the surface, where the shape of the surface causes distortion of the pattern. A three-dimensional surface can be constructed in each single image based on the distortion of the pattern. That is, the total surface can be determined by combining the three-dimensional partial surfaces of the single images using overlapping regions. This total surface can then be used for registration with a conventional surface.

[0109] The pattern projector can be configured to project patterns using infrared or ultraviolet light. In this case, the image sensor unit is preferably equipped with an infrared-sensitive or ultraviolet-sensitive image sensor. This embodiment variation is advantageous because the projected pattern is invisible to the surgeon and therefore does not cause interference.

[0110] A self-adhesive patterned film can be provided as an alternative to or in addition to a pattern projector. This film is adhered to the target area. Due to the known pattern on the film, the shape of the film can be detected, and therefore the surface structure can also be detected.

[0111] The registration method may also include the step of placing multiple infrared markers on the surface of the object.

[0112] Infrared markers can be detected by an image sensor unit equipped with an infrared-sensitive image sensor. By positioning the infrared markers arranged on the surface of an object, a local surface anatomy can be generated photogrammetrically from the captured image based on the infrared markers. This allows for surface detection and the generation of a surface model of the object photogrammetrically, even when visible light is insufficient for surface detection.

[0113] The registration method may further include the step of assigning a timestamp to each captured image. For example, the image sensor unit may be configured to transmit the captured images and timestamps together to the location determination unit of the registration device.

[0114] The registration method may further include the step of transmitting a position value and a timestamp. This step can be implemented using a position detection system configured to transmit the position value and timestamp together to the position determination unit of the registration device. Alternatively, the position determination unit of the registration device may be configured to request a position value from the position detection system and / or a motion sensor signal from a motion sensor, and to append a timestamp to the corresponding one of the position value and / or motion sensor signal.

[0115] The motion sensor can be configured to transmit the motion sensor signal and a timestamp together to the tracking unit of the registration device. For example, if a position sensor is arranged on an object and configured to detect whether the object has moved when an image is captured, preferably, the position value provided by the position sensor, the image captured by the image sensor unit, and the timestamp are transmitted together to the position determination unit. The position determination unit can be configured to assign the position value to the image carrying the same timestamp. The capture position of the captured image, i.e., the correction value for the position of the incident pupil relative to the object, can then be corrected by the detected movement represented by the position value as a correction value for the capture position.

[0116] Alternatively, the registration method may further include the step of streaming image data representing the captured image. This step can be implemented using an image sensor unit configured to stream image data representing the captured image to a position determination unit of the registration apparatus. The image stream can be displayed on a monitor of the registration apparatus.

[0117] In the registration method according to the invention, an image can also be captured on an automated trigger. The automated trigger can be based on the analysis of streaming image data from the image sensor unit. For example, the image sensor unit can analyze the streaming image data and, if an object is identified relative to the image sensor unit at a certain location, trigger an automated trigger. For example, the image sensor unit can capture an image on the automated trigger based on the analysis of the determined position of the image sensor unit relative to a reference position sensor attached to the object. For example, if the image sensor unit is at a certain position relative to the reference position sensor, trigger an automated trigger and capture an image.

[0118] The registration method may further include determining the distortion of the alternating electromagnetic field by photogrammetric detection of multiple position sensors or movable position sensors located at different positions, wherein the movable position sensors move while the image is captured by the image sensor unit, and the positions and orientations of the multiple position sensors or the movable position sensors in the coordinate system of the position detection system can be directly determined by the position detection system.

[0119] To determine the possible distortion of the alternating electromagnetic field of the position detection system in all previously described embodiments of the registration method, multiple position sensors or movable position sensors at different locations are used. These movable position sensors move during photogrammetric detection of the surface of an object or body part, and their positions are also detected by photogrammetry. The distortion of the alternating electromagnetic field can then be established from the positions detected by the corresponding photogrammetry of the movable position sensors or multiple position sensors, and from the positions determined by the respective position sensors themselves by means of the position detection system. It can then be said that the alternating electromagnetic field itself is measured by photogrammetry.

[0120] Regarding the registration device, the objective of this invention is achieved through a registration device for determining the position and orientation of an object relative to a position detection system. The device includes a position detection system, at least one image sensor unit, a tracking unit, a position determination unit, and a coordinate transformation unit.

[0121] A position detection system is configured to determine the position and orientation of a position sensor, and at least one image sensor unit includes at least one motion sensor. The image sensor unit is configured to capture an image of an object and to generate a surface model of the object from at least one captured image using photogrammetry. The motion sensor is configured to provide a motion sensor signal representing a sequence of positions of the moving image sensor unit relative to the position detection system over time.

[0122] The tracking unit is configured to record the path of the image sensor unit by processing motion sensor signals provided by the motion sensor.

[0123] The position determination unit is configured to correlate at least one point of the determined path with coordinates in the coordinate system of the position detection system, determine the capture position of the image sensor unit based on the determined path and the known spatial relationship between the at least one point on the determined path and the coordinates, and correlate at least one point of the generated surface model with coordinates in the coordinate system of the motion sensor and / or the position sensor through calibration.

[0124] The coordinate transformation unit is configured to transform the generated surface model into the coordinate system of the position detection system.

[0125] The device may also include a pattern projector connected to or including at least one image sensor unit, wherein the pattern projector is configured to project a pattern onto the body.

[0126] Preferably, the image sensor unit includes a stereo camera, a multi-camera setup, or a time-of-flight camera. However, in some embodiments, the image sensor unit may also include a single camera.

[0127] Preferably, the image sensor unit is connected to the position determination unit. Preferably, the motion sensor is connected to the tracking unit. The position determination unit is also connected to the position detection system. The position determination unit is connected to the coordinate transformation unit. At least some of the connections between the components can be implemented as Universal Serial Bus (USB) connections. However, at least some of the connections between the components can also be implemented as wireless connections, such as Bluetooth connections.

[0128] Motion sensors can be sensor devices or arrays of gyroscopes and accelerometers that transmit motion sensor signals representing motion parameters of six degrees of freedom (e.g., three linear accelerations and three angular velocities).

[0129] The components of the registration device are preferably elements of a computer or other data processing system.

[0130] Optionally, the apparatus may further include an image sensor unit holder comprising a shape-fit adapter configured to repeatedly accept image sensor units in predefined and fixed positions. The position and orientation of the image sensor unit holder relative to the position detection system are preferably known a priori or can be determined by means of the position detection system.

[0131] For example, an image sensor unit holder can be attached so that the position and orientation of the position detector can be detected by the position detection system. From the detected position and orientation of the position sensor, the position and orientation of the image sensor unit holder can be calculated. If the image sensor unit is held by the holder, the position and orientation of the image sensor unit relative to the position detection system can also be calculated. Advantageously, the image sensor unit holder equipped with the position sensor can be freely positioned within the workspace of the position detection system and serves to provide a known position in the coordinate system of the position detection system, along which the image sensor unit can move from or to the known position, said path, which can be recorded by processing, for example, motion sensor signals. Thus, the capture position of the captured image can be determined, and this capture position is used to determine the position and orientation of the object relative to the position detection system.

[0132] If the image sensor unit holder does not include a position sensor, the position and orientation of the image sensor unit holder relative to the position detection system are preferably known a priori by fixing the image sensor unit holder relative to the position detection system. The image sensor unit holder is then fixed at a constant relative distance and orientation with respect to the position detection system, or even directly with respect to the position detection system. It is then sufficient to calibrate the image sensor unit holder to the position detection system once, such that the position and orientation of the image sensor unit holder relative to the position detection system are known.

[0133] According to another aspect of the invention, which can be implemented independently of the other aspects described herein and thus constitutes the invention itself, the objective of the invention is achieved by an alternative registration method for determining the position and orientation of an object relative to an electromagnetic position detection system. The method according to this aspect includes the following steps:

[0134] - Provides an image sensor unit that is rigidly fixed to the field generator of an electromagnetic position detection system.

[0135] - A transformation function is determined through calibration, which is used for the transformation between the coordinate system of the surface model and the coordinate system of the electromagnetic position detection system.

[0136] - The image sensor unit is used to capture an image of the object's surface.

[0137] - Generate a surface model of the object from the captured image using photogrammetry.

[0138] - Transform the surface model generated by photogrammetry into the coordinate system of the position detection system.

[0139] This alternative registration method does not require the use of one or more motion sensors and / or position sensors to determine the coordinates of the captured position in the coordinate system of the position detection system. Since the image sensor unit is integrated with the field generator in the fixed device, it is only necessary to calibrate the image sensor unit to the field generator to determine the position and orientation of the object relative to the position detection system, and thus determine the transformation function required to transform the coordinates of the image sensor unit's coordinate system to the coordinate system of the position detection system. However, optionally, the image sensor unit may include a motion sensor and / or a position sensor. Preferably, the image sensor unit includes an optical stereo camera, an optical multi-camera system, or a time-of-flight camera.

[0140] An alternative registration method according to this other aspect may include the following steps: when capturing an image, projecting a pattern detectable by the image sensor unit onto the surface of the object.

[0141] According to this alternative registration method, it may also include assigning a timestamp to each captured image.

[0142] Alternatively or additionally, an alternative registration method according to this other aspect may include streaming image data representing the captured image from the image sensor unit to the tracking unit.

[0143] Optionally, the alternative registration method according to this other aspect may include the following steps:

[0144] - Capture images of the object and a reference position sensor positioned relative to the object, the reference position sensor being used to detect the position and orientation in a coordinate system of a position detection system arranged on the object, and

[0145] - Generate a surface model and a reference position sensor from the captured image using photogrammetry, such that the position of the reference position sensor in the coordinate system of the photogrammetrically generated surface model is determined, while providing position information based on the coordinates of the position detection system.

[0146] Regarding the registration apparatus, the objective of the present invention is also achieved by an alternative registration apparatus for determining the position and orientation of an object relative to a position detection system, which can be implemented independently of the other aspects described above, and thus constitutes the invention in itself. This alternative registration apparatus includes...

[0147] - An electromagnetic position detection system, comprising a field generator for generating an alternating electromagnetic field.

[0148] - An image sensor unit rigidly fixed to the field generator, wherein the image sensor unit is configured to capture an image of the object and generate a surface model of the object from at least one captured image using photogrammetry, and

[0149] - A calibration unit configured to determine a transformation function through calibration, the transformation function being used for the transformation between the coordinate system of the surface model and the coordinate system of the electromagnetic position detection system, and

[0150] - Coordinate transformation unit, which is configured to use a transformation function to transform the coordinates representing the surface model in the coordinate system of the surface model to the coordinate system of the position detection system.

[0151] Typically, the coordinate system of a position detection system is spanned by or associated with the field generator or light source of the corresponding position detection system.

[0152] This alternative registration device may also include a pattern projector connected to or comprising an image sensor unit, wherein the pattern projector is configured to project a pattern onto the body. The image sensor unit of this alternative registration device may include a motion sensor and / or a position sensor. Attached Figure Description

[0153] Preferred embodiments of the invention will be described below with reference to the accompanying drawings. In the drawings:

[0154] Figure 1 A flowchart illustrating a registration method for determining the position and orientation of an object relative to a position detection system is shown.

[0155] Figure 2 A schematic block diagram of the registration device is shown.

[0156] Figure 3 The image sensor unit for capturing images of a patient's head is shown, wherein the image sensor unit includes a motion sensor;

[0157] Figure 4 The image sensor unit for capturing images of a patient's head is shown, wherein the image sensor unit includes a motion sensor and a position sensor for detecting position and orientation in the coordinate system of the position detection system;

[0158] Figure 5 An image sensor unit is shown mounted to an image sensor unit holder, which is rigidly or movably fixed to a position detection system.

[0159] Figure 6 An image sensor unit rigidly mounted to a position detection system is shown, wherein the image sensor unit and the position detection system are arranged and mounted to an image sensor unit holder.

[0160] Figure 7 A flowchart illustrating a registration method according to an alternative embodiment is shown.

[0161] Figure 8 This schematically illustrates how, when an image sensor unit includes a position sensor, the position and orientation of an object relative to a position detection system can be determined.

[0162] Figure 9 This schematically illustrates how the position and orientation of an object relative to a position detection system can be determined when the image sensor unit consists only of a motion sensor. Detailed Implementation

[0163] exist Figure 1 The diagram shows a flowchart illustrating a registration method according to the concept of the present invention.

[0164] The registration method includes the following steps

[0165] I1: Capture an image of the object's surface using an image sensor unit comprising at least one motion sensor and / or at least one position sensor.

[0166] I2: The capture position of the image sensor is determined by processing the motion sensor signal provided by the motion sensor and / or the position signal provided by the position sensor.

[0167] I3: Generates a surface model of the object from the captured image using photogrammetry, and

[0168] I4: Transform the surface model generated by photogrammetry into the coordinate system of the position detection system.

[0169] Steps I1, I2, and I3 can be performed in a different order. For example, in some variations, the capture position can be determined before capturing the image, or the surface model can be generated by photogrammetry before determining the capture position. However, the surface model must be generated by photogrammetry before transforming it to the coordinate system of the position detection system.

[0170] Preferably, the image sensor unit includes a time-of-flight camera, an optical stereo camera, or an optical multi-camera that records the surface of an object in three dimensions, and determines the position and orientation of the object relative to the position detection system.

[0171] In a preferred embodiment, step I1, which uses an image sensor unit to capture an image of the surface of an object, includes the following sub-steps: providing at least one image sensor unit, which includes at least one motion sensor for detecting linear acceleration and / or rotational speed over time independently of the position detection system (S1).

[0172] The motion sensor may be an inertial sensor including an accelerometer, gyroscope, or magnetometer. Furthermore, several of these sensors may be included in an image sensor unit to form a motion sensor arrangement. When the image sensor unit is moved, each motion sensor provides a motion sensor signal representing the linear acceleration or rotational speed detected over time.

[0173] Alternatively or in place of a motion sensor, the image sensor unit may include a position sensor for determining the position and orientation of the image sensor unit relative to the position detection system.

[0174] Step I2, which determines the capture position of the image sensor unit, may include several sub-steps. In a preferred embodiment, determining the capture position of the image sensor unit can be achieved through the following sub-steps.

[0175] S2: The image sensor unit moves along the spatial path relative to the position detection system.

[0176] S3: The path of the image sensor unit is recorded by processing the motion sensor signal provided by the motion sensor.

[0177] S4: Associate at least one point of the determined path with coordinates in the coordinate system of the position detection system, and

[0178] S5: Determine the capture position of the image sensor unit based on the determined path and the known spatial relationship between at least one point on the determined path and the coordinates.

[0179] Sub-steps S2, S3, S4, and S5 are used to determine the position of the image sensor unit based on the coordinates of the position detection system coordinate system.

[0180] For example, a known spatial relationship between points on the path and coordinates in the coordinate system of the position detection system can be established by moving the image sensor unit from a position with known coordinates in the coordinate system of the position detection system or by moving the image sensor unit to the position.

[0181] In some variations of step I2, a position sensor may be used instead of a motion sensor to determine the position and orientation of the image sensor unit relative to the position detection system. In such embodiments, the capture position can be determined by detecting the position and orientation of the position sensor. The position sensor constitutes the image sensor unit.

[0182] In another alternative to step I2, the capture position can also be determined by capturing the object and an image of a sensor attached to or positioned relative to the object.

[0183] However, in another alternative for determining the capture position (step I2), the image sensor unit can be fixed to a known predefined position in the coordinate system of the position detection system. This predefined position can be the position of an image sensor unit holder calibrated to the position detection system or include a position sensor for detecting position and orientation relative to the position detection system. If the position sensor, such as a position sensor arranged on the object, has been recorded with the object, the position sensor is generated photogrammetrically in the coordinate system of the object's surface model. Therefore, the coordinates in the position detection system associated with the position of the position sensor can be directly correlated spatially with the coordinates in the coordinate system of the generated surface model.

[0184] Step I4, which transforms the surface model generated by photogrammetry to the coordinate system of the position detection system, may include a sub-step (S6) of calibrating at least one point of the generated surface model to coordinates in the coordinate system of the motion sensor and / or the position sensor.

[0185] For example, a transformation function can be determined through calibration to transform the coordinates between the coordinate system of the motion sensor and / or position sensor and the coordinate system of the image sensor unit. This may include determining the coordinates of the capture position of the image sensor unit in the coordinate systems of the motion and / or position sensors, respectively. The transformation function thus determined can be used to transform the surface model generated by photogrammetry into the coordinate system of the position detection system.

[0186] Sub-step S6 is used to assign the points of the surface model to coordinates in the sensor's coordinate system.

[0187] The step of transforming the generated surface model into the coordinate system of the position detection system can be performed at a much later time than some of the previous sub-steps (e.g., on another day).

[0188] In different embodiments of the registration method, only some of the steps in capturing an image I1 of the object's surface using an image sensor unit, determining the capture position I2 of the image sensor unit, and transforming the photogrammetrically generated surface model into the coordinate system of the position detection system I4 include the sub-steps detailed above. For example, in one embodiment, the step of determining only the capture position of the image sensor unit I2 includes the sub-steps described above. In other embodiments, a motion sensor is not used; for example, a position sensor is used to determine the coordinates of the capture position in the coordinate system of the position detection system. In these embodiments, the step of transforming the photogrammetrically generated surface model into the coordinate system of the position detection system I4 may still include the sub-step S6 specified above.

[0189] exist Figure 2 The diagram shows a schematic block diagram of a preferred embodiment of the registration device 200.

[0190] The registration device includes a position detection system 202, an image sensor unit 204 including at least one motion sensor 206, a tracking unit 208, a position determination unit 210, and a coordinate transformation unit 212.

[0191] Position detection system 202 is configured to determine the position and orientation of a position sensor (not shown). Position detection system 202 can be an optical, electromagnetic, or ultrasonic-based position detection system. Preferably, position detection system 202 is an electromagnetic position detection system, which includes a field generator for generating an alternating electromagnetic field. If position detection system 202 is an electromagnetic position detection system, the position sensor used to detect the position and orientation in the coordinate system of the position detection system includes at least one coil. Position detection system 202 is connected to position determination unit 210 to transmit a position signal representing the position value provided by the position sensor to the position determination unit for further processing.

[0192] Image sensor unit 204 is configured to capture images of an object (not shown) and generate a surface model of the object using photogrammetry from at least one captured image. Image sensor unit 204 may include a single-focus image sensor. To determine the position and orientation of the object relative to the position detection system, the image sensor unit including the single-focus image sensor is preferably continuously guided across the surface of the object to be detected during the registration process. As the image sensor unit including the single-focus image sensor is moved, at least two images are captured from different viewing directions and viewing positions. The object position can then be determined by means of triangulation. However, preferably, the image sensor unit includes a stereo camera or a multi-camera arrangement or a time-of-flight camera that records the surface of the object to be detected in three dimensions. An optical stereo camera may be implemented, for example, by a camera assembly of two or more cameras mounted on a fixed, known base. Image sensor unit 204 is connected to an optional pattern projector 214. Image sensor unit 204 is also connected to a position determination unit 210 to provide the photogrammetrically generated surface model for further processing.

[0193] Motion sensor 206 is configured to provide motion sensor signals representing a sequence of positions of the moving image sensor unit 204 relative to the position detection system 202 over time. By processing such motion sensor signals provided by motion sensor 206, a path can be recorded along which the image sensor unit 204 moves during registration. The image sensor unit 204 may include only one motion sensor 206 or an arrangement of several motion sensors configured to detect linear acceleration and rotational speed as the image sensor unit 204 moves relative to the position detection system 202. Motion sensor 206 is connected to tracking unit 208 to transmit motion sensor signals to tracking unit 208 for recording the path along which the image sensor unit 204 moves.

[0194] The tracking unit 208 is connected to the image sensor unit 204 and the position determination unit 210, and is configured to record the path along which the image sensor unit 204 moves relative to the position detection system 202 by processing motion sensor signals provided by the motion sensor. The tracking unit 208 is connected to the motion sensor 206 that makes up the image sensor unit 204.

[0195] The position determination unit 210 is configured to correlate at least one point of the determined path with coordinates in the coordinate system of the position detection system 202. Furthermore, the position determination unit 210 is configured to determine the capture position of the image of the object captured by the image sensor unit 204 based on the determined path and the known spatial relationship between the determined path and the coordinates of at least one point on the determined path and the coordinates in the coordinate system of the position detection system 202. The position determination unit 210 is also configured to correlate at least one point of the generated surface model with coordinates in the coordinate system of the motion sensor by means of calibration. The position determination unit 210 is connected to the position detection system 202 to receive position signals and has access to the tracking unit 208 to further process the path along which the image sensor unit 204 has moved, as recorded by the tracking unit 208. The position determination unit 210 is also connected to the coordinate transformation unit 212 to provide information about the spatial relationship between the position of the generated surface model and the position of the image sensor unit 204 relative to the position detection system, the position of the image sensor unit relative to the position detection system corresponding to the determined relative position of the motion sensor 206. In addition, the position determination unit 210 is connected to the image sensor unit 204 to access the surface model generated by the image sensor unit 204 by photogrammetry.

[0196] The coordinate transformation unit 212 is connected to the position determination unit 210 and is configured to transform the generated surface model into the coordinate system of the position detection system 202.

[0197] The device 200 also includes an optional pattern projector 214 connected to the image sensor unit 204. The pattern projector 214 is configured to project a pattern onto the body.

[0198] At least some of the connections between the components of the registration device 200 can be implemented as Universal Serial Bus (USB) connections. At least some of the connections between the components of the registration device 200 can also be implemented as wireless connections, such as Bluetooth connections.

[0199] exist Figure 3The image sensor unit 16', including a motion sensor 16', is shown. This image sensor unit is configured to capture images of a patient's head 12. The head 12 and the image sensor unit 16' are positioned relative to a position detection system 10, which is configured to determine the position and orientation of a reference position sensor 18 rigidly fixed to the position detection system 10. The reference position sensor 18 is positioned relative to the head 12 such that if an image of the head is captured via the image sensor unit 16', the position sensor 18 will also be visible in that image. If a surface model of the object is generated, for example, by the image sensor unit 16' using photogrammetry, the reference position sensor will also be generated photogrammetrically along with the surface model. The reference position sensor 18 can be identified in the coordinate system of the photogrammetrically generated surface model and thus provides a reference to the coordinates in the coordinate system of the position detection system. Therefore, the points of the point cloud of the surface model can be directly linked to and assigned corresponding coordinates in the coordinate system of the position detection system.

[0200] However, providing this reference sensor 18 is not necessary, but optional. In fact, the position and orientation of the head relative to the position detection system 10 can be determined without providing the reference position sensor 18. This is achieved by processing the motion sensor signal provided by the motion sensor 14' to record the path along which the image sensor unit 16' moves. The recorded path consists of a sequence of positions of the moving image sensor unit 16' detected by the motion sensor 14' over time. Because at least one point on the recorded path has a known spatial relationship with the coordinates in the coordinate system of the position detection system, the recorded path can be determined in that coordinate system. Thus, the capture position of the image sensor unit 16' that has captured the image can be determined in the coordinate system of the position detection system 10. After calibrating the position of the entrance pupil of the image sensor unit to the position of the motion sensor 14', the coordinates of the surface model generated by photogrammetry in the coordinate system of the surface model by the image sensor unit 16' can be transformed into the coordinate system of the position detection system 10.

[0201] exist Figure 4 The image sensor unit 16 is shown, which includes a motion sensor 14' and an additional position sensor 14'" for detecting position and orientation in the coordinate system of the position detection system 10. Another optional position sensor 14 is attached to the head 12 and will determine the position and orientation of the head relative to the position detection system 10. If an image of the head 12 is captured, the image will show the other position sensor 14 and the head. When a surface model of the head 12 is generated using photogrammetry, the position sensor 14 can be identified in the coordinate system of the surface model and can be used to link points of the surface model to the coordinate system of the position detection system 10.

[0202] As long as the image sensor unit 16 moves within the workspace of the position detection system 10, the position and orientation of the position sensor 14” attached to the image sensor unit 16 can be directly determined in the coordinate system of the position detection system 10 by detecting the position and orientation of the position sensor 14”. However, if the image sensor unit 16 moves outside the workspace of the position detection system 10, the position and orientation of the image sensor 10 relative to the position detection system 10 can no longer be determined by detecting the position sensor 14”.

[0203] However, when leaving the workspace of the position detection system 10, the capture position of the captured image of the head 12 can still be determined by processing the motion sensor signal provided by the motion sensor 14'. Thus, the path along which the image sensor unit 16 moves can be recorded. At least one point must be associated with coordinates in the coordinate system of the position detection system 10 in order to determine the coordinates of the capture position in the coordinate system of the position detection system 10. The capture position, i.e., the position of the incident pupil relative to the object, can then be determined in the coordinate system of the position detection system 10. Through calibration, points on the surface model generated by photogrammetry from the image of the head 12 can be associated with coordinates in the coordinate systems of the motion sensor and / or the position sensor. After determining, for example, the position and orientation of the coordinate system of the position sensor relative to the coordinate system of the position detection system 10, the coordinate system of the surface model can be aligned with the coordinate system of the position detection system 10. This works independently of whether the image sensor unit moves within the workspace of the position detection system 10.

[0204] exist Figure 5 The image sensor unit 16” is shown mounted to the image sensor unit holder 11, which is fixed to the position detection system 10. (The last sentence appears to be incomplete and possibly contains errors.) Figures 1 to 4 In contrast to the described embodiment, the image sensor unit 16” is not equipped with a motion sensor or a position sensor. However, at least one of a motion sensor or a position sensor may optionally be incorporated into the image sensor unit 16”. This can be advantageous if the image sensor unit holder 11 can move relative to the position detection system, for example, similar to a pivot arm, such that the capture position is not fixed relative to the position detection system.

[0205] If the image sensor unit holder 11 is rigidly fixed to the position detection system 10, such as the field generator of the electromagnetic position detection system, it may be sufficient to calibrate the capture position of the image sensor unit 16” to the position detection system 10 in one go, so that the capture position is known in the coordinate system of the position detection system 10.

[0206] The image sensor unit 16” can be a single camera, a time-of-flight sensor (TOF sensor), or a camera assembly of two or more cameras based on a fixed, known base, i.e., a stereo camera.

[0207] The image sensor unit 16” includes a pattern projector (not shown) for projecting a pattern 22 onto the patient’s head. Projecting the pattern 22 onto the surface of the head 12 produces an enhanced surface image recording.

[0208] In addition, position sensor 14 is attached to the patient's head 12. When an image of the head is captured, the position sensor will also be visible in that image. Position sensor 14 will be generated from the image along with the head 12 using photogrammetry, and can therefore also be used to link points on the surface model to the coordinate system of position detection system 10.

[0209] exist Figure 6 The image sensor unit 16”' is rigidly fixed to the field generator 10 of the electromagnetic position detection system. The fixing device for the image sensor unit 16”' and the field generator 10 is mounted to the image sensor unit holder 11'. In different embodiments (not shown), the fixing device for the image sensor unit and the field generator is not mounted to the image sensor unit holder, but can be moved freely, for example, by the user manually.

[0210] In the illustrated embodiment, the image sensor unit holder 11' is positioned relative to the head 12 to determine the position and orientation of the head relative to the position detection system 10. The image sensor unit holder 11' may be mounted in a fixed position or may be movable relative to the head 12. For example, the image sensor unit holder 11' may be implemented as a pivot arm.

[0211] Advantageously, if the field generator 10 and the image sensor unit 16”' are combined in a fixing device, the position and orientation of the image sensor unit 16”' in the coordinate system of the position detection system spanned by the field generator 10 can be determined without using motion sensors and / or position sensors. When the image sensor unit is rigidly fixed to the field generator, it is not necessary to use motion sensors and / or position sensors to determine the position and orientation of the object relative to the position detection system.

[0212] In particular, it is not necessary to arrange position sensors relative to objects visible in the captured image so that the surface model generated by photogrammetry can be transformed into the coordinate system of the position detection system.

[0213] In practice, within the fixed setup, the field generator 10 itself can be considered as a position sensor to which the image sensor unit 16"' can be calibrated. After calibration with the field generator 10, the position and orientation of the image sensor unit relative to the position detection system (e.g., the position of the incident pupil) are known. Based on the known spatial relationship between the image sensor unit and the field generator, the surface model generated by photogrammetry can be directly transformed into the coordinate system of the position detection system.

[0214] For example, a transformation function can be determined through calibration for the transformation between the image sensor unit 16”' (more precisely, the entrance pupil of the camera of the image sensor unit 16”') and the field generator. Using these transformation functions, the coordinates of the surface model generated by photogrammetry, defined in the coordinate system of the camera of the image sensor unit 16”', can be transformed into coordinates representing the surface model in the coordinate system of the position detection system.

[0215] The image sensor unit 16”' preferably includes a 3-D camera and may optionally be equipped with a motion sensor and / or a position sensor.

[0216] An optional position sensor 14 is attached to the head 12, preferably visible in an image of the head 12 captured by the image sensor unit 16”'. By determining the position and orientation of this position sensor in the coordinate system spanned by the field generator 10, points in the coordinate system of a surface model of the head 12 generated by photogrammetry from the image captured by the position sensor 14 can be assigned coordinates in the coordinate system of the position detection system.

[0217] exist Figure 7 The diagram shows a flowchart illustrating a registration method according to an alternative embodiment.

[0218] An alternative registration method for determining the position and orientation of an object relative to an electromagnetic position detection system, wherein the method includes the following steps:

[0219] A1: An image sensor unit that is rigidly fixed to the field generator of an electromagnetic position detection system.

[0220] A2: Through calibration, determine the transformation function between the coordinate system used for the surface model and the coordinate system of the electromagnetic position detection system.

[0221] A3: Generate a surface model of the object from the captured image using photogrammetry.

[0222] A4: Capture an image of the object's surface using the image sensor unit.

[0223] A5: Transform the surface model generated by photogrammetry into the coordinate system of the position detection system.

[0224] Figure 8 This schematically illustrates how the position and orientation of an object relative to the position detection system 800 can be determined when the image sensor unit 802 includes a position sensor 804.

[0225] At the capture position (i.e., the position of the incident pupil when the image is captured), an image of the object is captured, and a surface model 806 of the object is generated from the captured image by photogrammetry.

[0226] The position and orientation of the position sensor 804 are known in the coordinate system of the position detection system 800.

[0227] However, the coordinate system of the position sensor 804 and the coordinate system of the image sensor unit 802 are offset. The coordinate system of the image sensor unit 802 is preferably defined such that its origin is located at the position of the entrance pupil 808 of the image sensor unit 802. The offset can be determined by calibration. This may include determining a transformation function for the transformation between the coordinate systems of the position sensor 804 and the image sensor unit 802. Taking into account the offset, the coordinates in the coordinate system of the image sensor unit 802 can be transformed to the coordinate system of the position detection system 800, so that the position and orientation of the object 806 relative to the position detection system 800 can be determined.

[0228] Figure 9 This schematically illustrates how the position and orientation of an object relative to the position detection system 900 can be determined when the image sensor unit 902 includes only the motion sensor 904 and does not include a position sensor.

[0229] The position and orientation of motion sensor 904 cannot be determined by position detection system 900. However, motion sensor 904 can provide motion sensor signals that can be processed to record the path along which image sensor unit 902 has moved from or to the capture position. To determine the position and orientation of the path in the coordinate system of position detection system (and thus the capture position of the image of captured object 906), a known spatial relationship needs to be established between at least one point on the path and a position 908 with known coordinates in the coordinate system of position detection system 900.

[0230] This can be accomplished by moving the image sensor unit 902 from the capture position to the known position 908, or vice versa, such that the known position 908 is the start or end point of the path. Therefore, the start or end point of the path has known coordinates in the coordinate system of the position detection system. Thus, the coordinates of this path, initially defined in the coordinate system of the motion sensor, can be transformed to the coordinates of the position detection system 900 by using the fact that the coordinates of the start or end point of the path are known in both the coordinate system of the motion sensor and the coordinate system of the position detection system. Therefore, the coordinates of the capture position in the coordinate system of the image sensor unit 902 can be transformed to the coordinates in the coordinate system of the position detection system.

[0231] The coordinate system of motion sensor 904 is offset from the coordinate system of image sensor unit 902, the origin of which is preferably located at the entrance pupil 910 of image sensor unit 902. To determine the coordinates of the origin of image sensor unit 904's coordinate system within the coordinate system of position detection system 900, this offset must be determined, for example, through calibration. Calibration allows the determination of a transformation function that transforms the coordinates in the coordinate system of image sensor unit 902 to the coordinate system of motion sensor 904. Once this transformation function is determined, the coordinates in the coordinate system of image sensor unit 902 can be transformed to the coordinate system of position detection system 900, taking into account this determined transformation function.

Claims

1. A method for non-tactile registration of objects using a position detection system, the method comprising: A surface model of the object is generated by photogrammetry based on one or more images of the object's surface captured using an image sensor unit positioned at the capture location, wherein the image sensor unit includes a motion sensor. The capture position of the image sensor unit is determined by processing the motion sensor signal from the motion sensor. and Associating the surface model of the object with the position detection system, wherein associating the surface model of the object with the position detection system includes: Associate at least one point of the generated surface model with the coordinate system of the motion sensor; and The position of at least one point on the surface model in the coordinate system of the motion sensor is transformed into the position in the coordinate system of the position detection system. Associating at least one point of the generated surface model with the coordinate system of the motion sensor includes: transforming (i) the position of the at least one point on the surface model in the coordinate system of the surface model to (ii) its position in the coordinate system of the motion sensor based on a calibration vector, wherein the calibration vector represents the offset between the coordinate system of the surface model and the coordinate system of the motion sensor.

2. The method according to claim 1, wherein the image sensor unit comprises an optical stereo camera.

3. The method of claim 1, wherein the image sensor unit comprises one or more cameras in a multi-camera arrangement.

4. The method according to claim 1, wherein the image sensor unit comprises an infrared image sensor.

5. The method of claim 4, wherein the image sensor unit is connected to a pattern projector, wherein the pattern projector is configured to project a pattern onto the object while the infrared image sensor captures the image, wherein the projected pattern has a known size such that the pattern is distorted by the shape of the surface when projected onto the surface of the object and is used to establish a three-dimensional surface of the object.

6. The method of claim 4, wherein the method further comprises: Multiple infrared markers are arranged on the surface of the object; The infrared marker is detected using the infrared image sensor; and By locating the infrared markers, a local surface anatomy of the surface model is generated from the captured image based on the infrared markers through photogrammetry.

7. The method of claim 1, wherein the motion sensor is configured to detect at least one of linear acceleration and rotational speed of the image sensor unit.

8. The method of claim 1, wherein the surface model comprises a point cloud.

9. The method of claim 1, wherein determining the capture location comprises: The image sensor unit moves along a spatial path relative to the position detection system. The path of the image sensor unit is recorded by processing the motion sensor signal provided by the motion sensor. Associating at least one point of the path with coordinates in the coordinate system of the position detection system, and The capture position of the image sensor unit is determined based on the path and the known spatial relationship between at least one point on the path and the coordinates.

10. The method of claim 9, wherein the path begins at a known location in the coordinate system of the position detection system and ends at the capture location.

11. The method of claim 9, wherein the path begins at the capture location and ends at a known location in the coordinate system of the location detection system.

12. The method according to claim 1, wherein the position detection system is an electromagnetic position detection system.

13. The method of claim 12, wherein the position detection system has a workspace having a predetermined electromagnetic field strength.

14. The method of claim 13, wherein the method includes capturing the one or more images using the image sensor unit at a capture location located outside the workspace.

15. The method of claim 14, wherein capturing the one or more images using an image sensor unit located at the capture position further comprises: Analyze the streaming image data of the image sensor unit; and Image capture is automatically triggered on an automated trigger based on the identification of the object at a specific position relative to the image sensor unit.

16. The method of claim 14, wherein capturing the one or more images using an image sensor unit located at the capture position further comprises: Analyze the streaming image data of the image sensor unit; and Based on the positional relationship between the reference position sensor attached to the object and the image sensor unit, image capture is automatically triggered on the automated trigger.

17. The method of claim 1, wherein the motion sensor is configured to track the position of the image sensor unit along a path in which the image sensor unit moves.

18. The method of claim 1, wherein the object is the face of the patient depicted in the captured image.

Images