3D marker elements, medical products, tracking systems, and location detection methods for spatial tracking

Abstract

The present invention relates to a surgical 3D marker element (1) having a surface (2) on or within which a two-dimensional optical pattern (4) for spatial position detection of a medical product (100) is permanently incorporated or applied. The optical pattern (4) takes the form of an M-row and N-column matrix (6) having corresponding pattern elements (8) as pattern fields (8), and the optical pattern (4) has at least four pattern elements (10), of which at least two pattern elements (10) are designed to be different in order to provide an optical reference for position detection of the 3D marker element (1) via an optical tracking system (500). The present invention also relates to a medical product (100), a tracking system (500), a position detection method, a computer-readable storage medium, and a computer program according to the related claims.

Description

Technical Field

[0001] The present disclosure relates to a surgical (optical) 3D marker element having a surface, particularly a sterilizable or sterilized surface for surgical use, and having a two-dimensional optical pattern on the surface that can be used for spatial pose recognition of medical products. Further, the present disclosure relates to medical products, tracking systems, pose recognition methods, and computer-readable storage media and computer programs according to the general concepts of the independent claims.

Background Art

[0002] In order to perform spatial tracking and pose estimation of an object (e.g., an instrument or device with a handpiece) based on a 2D image / 2D photograph, an accurate appearance model of the object is required. Usually, central and characteristic points, or keypoints, of the object are recognized and compared with a (stored) basic model / template. Based on knowledge of the correspondence from 2D to 3D or from 3D to 3D, it is possible to calculate or estimate the pose of the object, i.e., its position and orientation. It is also necessary that the image be captured by a calibrated camera in order to estimate the Euclidean pose of the object based on the image.

[0003] Currently, there are various conventional solutions for spatial tracking of surgical instruments, which are based on the following tracking methods and systems: - Infrared markers are attached around the handle of the instrument and detected by a camera. In particular, at least four infrared markers are attached in a spherical (rigid) manner. - Planar optically visible markers or optical patterns such as QR codes (registered trademarks) are attached to the handle of the instrument or around it and detected by a camera. - Colored or elongated markers are attached to the tip of the instrument and used for tracking.

[0004] However, surgical instruments typically lack sufficient distinguishing features on their surfaces, making it impossible to estimate their pose using conventional computer vision algorithms. Their appearance also presents challenges. For example, metal surfaces are often highly reflective and smooth, making it impossible, or almost impossible, to extract visual features from images (due to low contrast, etc.).

[0005] As a result, unless other geometric objects or similar structures, such as rigid bodies, are present on the instrument, it is often difficult to accurately determine the instrument's position, which threatens patient safety during surgery. Furthermore, current surgical instruments need to be more sophisticated and advanced. [Overview of the Initiative]

[0006] Therefore, the object of this disclosure is to provide surgical 3D marker elements, medical products, tracking systems, pose recognition methods, as well as computer-readable storage media and computer programs that avoid, or at least mitigate, the shortcomings of the prior art and, in particular, perform accurate and effective spatial pose recognition via optical detection. Part of a further object is cost-effective manufacturing and adaptation, particularly for medical products and tracking systems. Also, part of another object is to provide additional data optically (encrypted) in addition to pose recognition.

[0007] This objective is addressed by the features of claim 1 with respect to the surgical 3D marker element according to the present invention, by the features of claim 8 with respect to the medical product according to the present invention, by the features of claim 12 with respect to the tracking system according to the present invention, by the features of claim 15 with respect to the pose recognition method according to the present invention, and by the features of claim 16 with respect to the computer-readable storage medium according to the present invention.

[0008] Therefore, the basic idea is to integrate a specially adapted and optically detectable optical pattern directly onto or within a surface. Based on a 2D or 3D image of a 3D marker element having a defined optical pattern, pose recognition can be performed via a tracking system, particularly a navigation system equipped with a camera.

[0009] In other words, such marker designs or 3D marker elements are proposed that can be printed or engraved on or within the surface of medical products, particularly surgical instruments. These defined 3D marker elements enable the calculation of poses (including instrument poses, if applicable) within six degrees of freedom (6-DOF). The 3D marker elements may be placed on part or component of a medical product, particularly on parts of instruments, such as the tip of a suction cannula or a navigation pointer / navigation indicator, and may be printed, engraved, or even laser-engraved. In this way, pose detection elements can be created easily and effectively, optimized in terms of installation space (without the need to add other elements), and may be attached to the medical product, for example, or directly integrated into the medical product by laser engraving onto its surface.

[0010] Specifically, a surgical 3D marker element having a surface, particularly a sterilizable surface, is proposed for spatial pose recognition of medical products, on or within the surface, on which a two-dimensional optical pattern is permanently applied or incorporated, in particular, printed or engraved (e.g., by laser engraving) on ​​the surface. Thus, the optical 3D marker element on or within a visible (outer) surface provides a pose detection criterion. According to this disclosure, the optical pattern (provided on or within the surface) is configured in the form of a matrix having M rows and N columns (i.e., an M × N matrix like a table), and comprises corresponding (individual) pattern elements / modules as pattern fields of the matrix. The optical pattern includes at least four pattern elements (as pattern fields), of which at least two pattern elements are configured to differ from each other in order to provide (as a minimum requirement) an optical criterion for pose detection (position and orientation, i.e., 6 degrees of freedom - 6DoF) of the 3D marker element via an (external) optical tracking system.

[0011] This section describes design guidelines for manufacturing (visual) 3D marker elements that can be used, in a sense, particularly for tracking surgical instruments. These, especially linear 3D marker elements, are mapped, for example, to a cylindrical or flat surface, i.e., applied or introduced onto this surface. The optical pattern of the 3D marker element is divided into M × N (particularly uniformly sized) cells or pattern fields. The visual appearance of each pattern element is created, in particular, from a dictionary of predefined pattern element types. Recognition of the optical pattern and visible pattern elements (i.e., cells or pattern fields filled with pattern elements) may be performed using conventional image processing algorithms based on machine learning or deep learning.

[0012] In particular, for optical patterns with more than four pattern elements, it is advantageous that not all pattern elements need to be visible for pose determination. At least four key points must be visible to determine the 6D position (3D position and 3D direction) of a marker. Therefore, the optical pattern does not need to be fully visible for recognition, identification, and spatial tracking.

[0013] In other words, with respect to 3D marker elements, special, particularly linear marker designs are applied or introduced onto or inside surfaces, especially cylindrical or flat surfaces of the instrument, and are in particular printed or engraved. The marker designs are created to uniquely identify each cell or pattern field by utilizing the visual characteristics of the cells or the visual differences of the pattern elements.

[0014] In particular, the optical patterns of the two different 3D marker elements can be configured differently for each different instrument model to ensure distinction between types of instruments. Therefore, a specific instrument model can be identified by the design of the optical pattern of the 3D marker elements.

[0015] The term "position" refers to a geometric location in three-dimensional space, specifically defined using coordinates in the Cartesian coordinate system. In particular, a position can be specified by three coordinates: X, Y, and Z.

[0016] On the other hand, the term "orientation" refers to the arrangement (alignment) in space (for example, in terms of position). Orientation can also be described as an arrangement involving direction or rotation in three-dimensional space. In particular, orientation may be specified using three angles.

[0017] The term "pose" encompasses both position and orientation. Specifically, a pose can be represented by six coordinates: three position coordinates (X, Y, and Z) and three angular coordinates representing orientation.

[0018] Advantageous embodiments are described in the dependent claims and are specifically described below.

[0019] According to one embodiment, the surface may be curved, and preferably part of a cylindrical side surface. This means that the surface is not a flat / horizontal surface, or does not need to be flat.

[0020] According to one embodiment, the optical pattern may be configured in the form of a rectangular matrix in which corresponding rectangular pattern fields each have the same contour shape, and in particular having at least 8 rows and / or at least 5 columns. Such a standardized optical pattern can be easily and efficiently implemented and reflects standardized geometric relationships. Pattern elements obtainable from a dictionary are then inserted into individual pattern fields of the matrix. In particular, such a configuration can be easily applied or inserted into a surface and can achieve reliable spatial tracking.

[0021] In particular, the surface (having the optical pattern) may be configured to be horizontal / planar (i.e., flat), or it may be configured to be curved, such as part of a cylindrical surface.

[0022] According to one embodiment, the matrix may have a non-square configuration. This means, in particular, that the matrix is ​​not a QR code (registered trademark).

[0023] According to one embodiment, different pattern elements are arranged irregularly across a portion of the optical pattern, a large portion of the optical pattern, or the entire optical pattern, and in particular, they are not arranged in rows or columns consisting solely of identical pattern elements. This means that the matrix is ​​not a barcode in particular.

[0024] According to one embodiment, the matrix may have more rows than columns. Preferably, the matrix may have at least 1.2, 1.5, or 2 times the number of columns, and / or up to 5, 4, or 3 times the number of columns. Such a configuration is particularly advantageous for mounting on shaft fixtures that have a fairly elongated shape, such as cylindrical shaft fixtures.

[0025] In particular, an optical pattern may have exactly two, or more than two, preferably exactly three, exactly four, or exactly five different pattern elements (as a pattern dictionary), where these pattern elements differ from one another in at least one color or color perception (e.g., different contrast), and more preferably, each differs from the others in a different color by a circle, or ellipse, or point, or by a rectangle or rectangular outline, located in the center of the pattern field. Here, the term “color” refers to black and white. When creating a single 3D marker element, the first thing to decide is how many pattern elements that element should have. For example, if only binary pattern elements are used, the optical pattern can be easily and reliably distinguished, although it will naturally need to be expanded in rows and columns to “encode” additional data such as the type of instrument. In particular, two high-contrast colors are suitable for distinction and engraving on a surface, and there are two "cases": one where the pattern field is not engraved and forms a first pattern element, and another where the second pattern field is engraved and forms a second pattern field of a different color (or color perception) (for example, it may look different from a smooth surface due to a matte finish, or it may appear dark because light is only partially reflected). The number of pattern elements may also be increased through other geometric elements such as circles and dots (for example, in the case of laser engraving), which allows optical patterns to be easily and cost-effectively created on the surface.

[0026] According to further embodiments, the optical pattern has exactly two distinct pattern elements as binary pattern elements (which can also be said to form entries in a dictionary of binary pattern groups), and has the following preferred embodiments: a first pattern element of a first color, particularly white pattern field (particularly unmarked), and a second pattern element of a second color, particularly black pattern field (particularly marked) (where it is important that the contrast of the pattern fields is particularly different so that they can be distinguished by the camera image); or the first pattern element is a bright, particularly white pattern field, further having a dark, particularly black dot or ellipse in the center, and the second pattern element is a dark, particularly black pattern field, further having a bright, particularly white dot or ellipse in the center; or the first pattern element is a bright, particularly white pattern field, further having a dark, particularly black rectangle, particularly square in the center, and the second pattern element is a dark, particularly black pattern field. In this embodiment, the binary cell type or pattern element type is formed from two distinct visual appearances that form the pattern elements of the optical pattern.

[0027] Alternatively, the optical pattern preferably has exactly three different pattern elements as ternary pattern elements and has the following preferred embodiments: a first pattern element as a bright, particularly white, pattern field, a second pattern element as a dark, particularly black, pattern field, and a third pattern element of yet another (different) color, particularly red; or, as a bright, particularly white, pattern field, a first pattern element having a dark, particularly black, point or ellipse in the center, as a dark, particularly black, pattern field, a second pattern element having a bright, particularly white, point or ellipse in the center, and a third pattern element as a bright, particularly white, pattern field; or, as a bright, particularly white, pattern field, a first pattern element having a dark, particularly black, rectangle, particularly a square, in the center, as a dark, particularly black, dark pattern field, a second pattern element having a bright, particularly white, rectangle, particularly a square, in the center, and a third pattern element as a bright, particularly white, pattern field having a dark, particularly black, rectangle outline in the center. Since the ternary cell type or pattern element type is formed from three different visual appearances, more information can be integrated by a matrix of the same size compared to binary pattern elements (or binary dictionary entries).

[0028] Alternatively, the optical pattern preferably has exactly four different pattern elements as quaternary pattern elements, particularly a first pattern element as a white pattern field, a second pattern element as a black pattern field, a third pattern element of a first color, particularly red, and a fourth pattern element of a further second color, particularly green, or a first pattern element as a white pattern field having a black point in the center, a second pattern element as a black pattern field having a white point in the center, a third pattern element as a white pattern field, and a fourth pattern element as a black pattern field. The quaternary cell type is formed from four different visual appearances.

[0029] According to one embodiment, the optical pattern has at least 3 rows and 3 columns (3×3), whereby the central pattern element is adjacent to (surrounded by) 8 pattern elements, and these pattern elements together form a pattern word and can be uniquely identified. The marking is obtained by assigning each cell to a predefined dictionary type. In particular, for pattern elements or cells on a linear grid, there are 8 directly adjacent cells. The concatenation of the cell type and the 8 directly adjacent cell types forms a so-called cell word / pattern word. In particular, each cell within the matrix is connected to a different word. This special property enables the development of an efficient image processing algorithm that can identify each cell of the marker using only local image information. In particular, each cell / pattern element of the marker can be uniquely identified by analyzing the adjacent cell type / pattern element type.

[0030] Preferably, different pattern elements can be distinguished by color and / or the center of the circle, and / or the corners, and / or the contour. In a sense, different 3D marker elements may be created using different dictionaries or a dictionary containing different groups of entries with visual features. Also, for example, using binary pattern elements, different 3D marker elements may be created from the same dictionary (or group of entries, such as a group of entries for binary pattern elements). This feature enables different 3D marker elements to be associated with different medical products, especially instruments. It should be noted that for the problem of pose determination (position determination), one or a combination of all visual features available in the dictionary, especially features such as the center of the circle, corners, edges, and contours, can be used.

[0031] According to one embodiment, the first part of the optical pattern can be configured to be offset and / or tilted with respect to the second part of the optical pattern. This prevents the first part and the second part from being visible simultaneously. Such a form occurs especially when the surface is not configured to be flat / plane / horizontal.

[0032] According to one embodiment, the optical pattern does not have to contain encoded information. This means that the optical pattern is used primarily for the pure visual recognition of a pose and / or is not a barcode and / or QR code (registered trademark) in particular.

[0033] With respect to medical products, particularly medical instruments or devices, that have or have optical pose recognition during patient treatment, the objective is achieved by providing the medical product with a surgical 3D marker element according to the Disclosure, particularly in the distal part of the medical product. Preferably, the 3D marker element is directly integrated into the medical product, and the optical pattern is, for example, printed or incorporated into the surface of the medical product, or engraved, particularly laser engraved, within the surface.

[0034] In one embodiment, the medical product may be a surgical instrument having a distal instrument tip, particularly a scalpel, trocar, or navigation pointer, and the 3D marker element may be positioned in a region of less than 50%, particularly less than 30%, and particularly preferably less than 10%, of the total length of the longitudinal axis away from the distal tip, particularly directly within the region of the instrument tip, to provide spatial pose recognition of the instrument tip, in particular, the 3D marker element may be positioned on a surgical scalpel as an instrument having the 3D marker element on the flat surface of the cutting blade. Thus, when the optical marker element is positioned distally, for example on a pointer, the pose when it enters the field of view of a digital surgical microscope used for tracking and corresponding navigation can be detected, and this pose can be appropriately integrated into digital data. This special configuration enables simple and safe tracking.

[0035] According to another embodiment, the medical product may have a cylindrical contour having a cylindrical surface corresponding to the distal portion, and the 3D marker elements may be arranged partially, particularly all around, in the circumferential direction, and may be engraved. Preferably, the medical product is a medical suction device having a distal suction tube / suction cannula, and the 3D marker elements having an optical pattern are provided on the suction tube, in particular the optical pattern is printed on the surface of the suction tube or engraved within the surface. In particular the suction tube may be used, for example, as a pointer for defining feature points. Such forms of 3D marker elements extend the functional possibilities of the medical product.

[0036] In particular, medical products may be medical instruments with a special distal end, especially a conical shape, and an optical template pattern in the form of a rectangular matrix may be mapped via a linear transformation matrix to fit / deform (distort) the special shape, especially a cone, and may exist on or within the surface as a deformed (distorted) fitted / projected optical pattern. It can also be said that the template design can be deformed or cut out to map as an optical pattern on any surface. For example, if a digital template of an optical pattern is available as a rectangular matrix containing the corresponding pattern elements, this template can be fitted to the target surface via the corresponding matrix transformation (linear algebra) and applied or inserted into the target surface. For something similar to an elastic rubber surface (known as a balloon), if a uniformly stretched rectangular matrix with pattern elements is first printed, and then applied to the object, the optical pattern will be distorted accordingly, but its structure will still be recognizable. In particular, the shapes of special instruments, such as tweezers and conical instruments, require special, non-linear optical patterns. Such designs can be achieved, in particular, by masking or mapping a linear marker design template to the shape of the surface, especially the shape of the instrument being tracked. Alternatively, a linear marker design template can be deformed as an optical pattern on the instrument surface by deforming a linear cell grid. Markers can be applied to or inserted into any type of surface, particularly by deforming and / or cutting linear marker designs.

[0037] With respect to surgical tracking systems, particularly navigation systems, for surgical procedures on patients, the object of the present invention is solved by comprising a system component including a surgical 3D marker element or medical product (having a corresponding optical pattern) according to the present disclosure and a tracking unit / tracking device having at least one camera, preferably at least two cameras such as a stereo camera for 3D imaging, the cameras being configured to optically detect the 3D marker element having an optical pattern, determine a pose (relative to the tracking unit) based on the detected optical pattern, incorporate the detected pattern into the navigation system, and visually output the pose by a display device. The tracking system is specifically configured to use an optical pattern for pose recognition, in particular by providing the tracking system with definitions and relationships of the optical pattern so that it can be processed accordingly.

[0038] In particular, the surgical tracking system may further include a memory device storing a dictionary of pattern elements(s), especially dictionaries of binary pattern elements and / or ternary pattern elements and / or quaternary pattern elements, and the control unit may be configured to read the optical pattern based on the dictionary and determine the pose accordingly. Here, preferably, a portion of the optical pattern is already sufficient to determine the pose.

[0039] According to one embodiment, any portion of an optical pattern having at least four pattern elements can be used for pose determination. This means that, in order to determine the pose, only a portion containing at least four pattern elements, rather than the entire pattern, needs to be visible.

[0040] According to one embodiment, the optical pattern can preferably be used for pose determination purely optically, without using information encoded in the optical pattern in particular. This means that no additional information encoded in the optical pattern is required for clear pose determination.

[0041] Preferably, a dataset associated with the optical pattern is stored in a memory device, and in particular, includes a conversion from the pose of the optical pattern to the operating point of a medical product, such as the tip of an instrument (e.g., in the form of a 1 cm vector or indication in the direction of orientation (e.g., longitudinal axis), with a 3 mm offset from the center of the longitudinal axis (e.g., angled)), thereby detecting the position of the operating point and the orientation of the distal end. The control unit may be configured to determine the operating point and the orientation of the distal end based on the recognition of the optical pattern pose and the stored conversion, and to output them visually as a view (screen display) on a display device. This allows the surgeon to obtain important information for the procedure.

[0042] With respect to a pose recognition method for pose recognition of a 3D marker element having an optical pattern or a medical product according to the present disclosure, the object of the present invention is solved by the following steps: creating a 2D image by a camera of a tracking unit or a 3D image by at least two cameras of a tracking unit and detecting each visible pattern element of the optical pattern; a control unit identifying each pattern element based on a dictionary containing different pattern elements, in particular binary pattern elements, or ternary pattern elements, or quaternary pattern elements; determining the correspondence between the 2D position in the 2D image and the position of the template of the 3D marker element having an optical pattern, or the correspondence between the 3D position in the 3D image and the position of the template of the 3D marker element having an optical pattern; and performing n-point perspective position determination (estimation) on the 2D image for pose determination or 3D-3D point determination for pose determination of the optical pattern. In other words, determining / calculating, or at least estimating, the position of such markers involves recognizing each cell / pattern element visible in the image, identifying each cell to give a correspondence between its 2D position on the image and its position on a theoretical 3D marker design, and using perspective n-point position estimation based on the calculated correspondence. Pose / position determination can be extended to a 3D camera or stereo camera and involves recognizing each visible cell in two stereo images and identifying each cell to give a correspondence between its 2D position on the image and its position on a theoretical 3D marker design. The 3D cell position is determined by triangulation of two positions on the stereo images, and the pose of the 3D marker element is estimated using 3D-3D point matching.

[0043] With respect to computer-readable storage media or computer programs, the object of the present invention is solved by including instructions that, when executed by a computer, cause the computer to perform each step of the pause recognition method according to the present disclosure.

[0044] Any disclosure relating to the 3D marker element in this disclosure applies to the medical product, tracking system, and pose recognition method in this disclosure, and similarly, any disclosure relating to the medical product, tracking system, or pose recognition method applies to the 3D marker element or any other preamble thereof. [Brief explanation of the drawing]

[0045] The present disclosure will be described in more detail below with reference to preferred embodiments using the drawings.

[0046] [Figure 1] Figure 1 shows a schematic front view of a surgical tracking system equipped with a medical product having a 3D marker element according to a first preferred embodiment.

[0047] [Figure 2] Figure 2 shows a sketch of an unfilled rectangular matrix with a rectangular pattern field for an optical pattern.

[0048] [Figure 3] Figure 3 shows an exemplary 3D marker element with an optical pattern, with a magnified view of a portion of the area, including the central pattern element and eight adjacent pattern elements, for illustrative purposes.

[0049] [Figure 4] Figures 4a to 4c show examples of binary pattern elements, where hatching represents the perception of different colors.

[0050] [Figure 5] Figures 5a and 5c show examples of ternary pattern elements, respectively.

[0051] [Figure 6] Figures 6a and 6b show examples of quadrivalent pattern elements, respectively.

[0052] [Figure 7]Figure 7 shows a schematic image of a template for a binary optical pattern on the surface of a medical product.

[0053] [Figure 8] Figure 8 shows a schematic image of a template for a quaternary optical pattern on the surface of a medical product.

[0054] [Figure 9] Figure 9 shows a flowchart of a pose recognition method according to a preferred embodiment.

[0055] The drawings are schematic and are intended solely to aid in understanding the invention. Identical components are denoted by the same reference numerals. Features of the various embodiments are interchangeable. [Modes for carrying out the invention]

[0056] Figure 1 shows a schematic front view of a surgical tracking system 500, which, according to a preferred embodiment, takes the form of a navigation system for tracking surgical instruments, particularly during surgical procedures on a patient (not shown).

[0057] The tracking system 500 has a tracking unit 502 equipped with a camera 504 in the form of a surgical microscope for spatial tracking. Furthermore, the tracking system 500 has a medical product 100 of a preferred embodiment, which takes the form of a medical device 104 of a suction device equipped with a distal suction tube 110, and is spatially tracked with respect to its pose, i.e., position and orientation, using the camera 504. The suction tube 110 has a surgical 3D marker element 1 of a preferred embodiment at its distal end 108. For example, information on the position and / or orientation of the medical product 100 relative to the surgical microscope is sufficient to adjust the pose of the surgical microscope. Optionally, the surgical tracking system 500 has an additional external navigation system comprising a surgical microscope, particularly a marker on the microscope head, and an external camera (not shown), particularly a stereo camera, for detecting and evaluating the pose of the marker on the surgical microscope. This allows for additional tracking of the pose of the surgical microscope relative to the patient.

[0058] This surgical 3D marker element 1 has a sterilizable surface 2, specifically printed or engraved, into which a two-dimensional optical pattern 4 is permanently incorporated for spatial pose recognition of a medical product 100. Specifically, the optical pattern 4 is composed in the form of a matrix 6 with M=10 rows and N=5 columns, each containing corresponding pattern elements 8 (10×5=50) as a pattern field 8.

[0059] The optical pattern 4 has 50 pattern elements 10, more than four of which are configured differently to provide an optical reference for pose detection of the 3D marker element 1 by the optical tracking system 500. In particular, the optical pattern is at least partially arranged around the suction tube 110 so that the camera 504 can detect at least a portion of the optical pattern 4 from each side of the suction tube 110 or at each pose. The pose can also be determined based on this portion of the optical pattern 4.

[0060] The tracking system 500 is configured to optically detect a 3D marker element 1 having an optical pattern 4, determine a pose based on the detected optical pattern 4, incorporate the pose into its navigation system or its navigation function, and combine it with preoperative image data such as CT images or MRI images to visually output the latest pose to the surgeon via a display device 506. In this way, the suction tube 110 can be displayed in a microscope image, and to ensure reliable and good navigation, the precise pose of the suction tube 110 can also be displayed in a cross-section of MRI data in another view. In particular, landmarks may be set, and the suction tube 110 may be used as a pointer. The suction tube 110 does not need to be precisely fitted; in this configuration, a commercially available suction tube can be used, and the optical pattern can be directly imprinted on the distal surface 2 using a laser and then optically detected. Thus, the 3D marker element provides an efficient and cost-effective way to adapt and upgrade already approved medical products to support pose recognition. This allows medical products to be taken to the next level of functionality without requiring complex additional parts that would normally require time and cost to approve. However, the optical pattern can be positioned in close proximity to the operating point of the medical product and can also extend over a wide area, thus enabling highly accurate pose recognition. This allows pose recognition to be performed even if the tip of the instrument or the tip of the suction tube 110 is obscured, and furthermore, pose recognition can be reliably performed from different directions (even if part of the optical pattern 4 is obscured).

[0061] Figure 2 shows an example of a matrix having M x N cells or pattern fields into which pattern elements can be inserted (see Figure 3). In Figure 2, the matrix is ​​a rectangle with square pattern fields 8, all of which have the same outline and are composed in the same manner.

[0062] Next, moving to Figure 3, these pattern fields 8 are provided with pattern elements 10, and in Figure 3, the binary pattern element 10 consists of a first bright (in this case, white) pattern element 10 and a second dark (in this case, black) pattern element 10 (hatching is intended to represent dark colors, especially black). As a result, an optical pattern 4 is obtained consisting of the first and second pattern elements 10 arranged according to the definition rules, so to speak, that optically encodes information or makes it detectable (by a computer).

[0063] In Figure 3, for illustrative purposes, a 3x3 pattern element cell group is shown by a border in the central region of the optical pattern 4 (left portion), and is shown individually in the right portion of Figure 3 for illustrative purposes. The central pattern element 10 (dark portion) is surrounded by eight other adjacent pattern elements. These nine pattern elements 10 form a pattern word. In particular, the tracking system 500 can determine, based on the pattern word, that a binary pattern element 10 is being used.

[0064] Figures 4a and 4c each show different (paired) binary pattern elements 10. Figure 4a shows a bright (e.g., white) pattern field 8 as a pattern element, and a dark (e.g., black, with hatching intended to represent dark or black) pattern field as pattern element 10. In Figure 4b, a filled circle (which can also be described as a dot) is provided in the center of the pattern field. Alternatively, as shown in Figure 4c, a rectangle in the shape of a square may be used instead of a circle.

[0065] Figures 5a to 5c show further embodiments (three sets) of the ternary pattern element 10. These are similar to the binary pattern elements in Figures 4a to 4c, but differ in that they include an additional pattern field 8 containing yet another pattern element 10.

[0066] Figures 6a and 6b show alternative ternary (quadrivalent) values ​​on the quadrivalent pattern element 10.

[0067] Each pair, triplet, and quadruple in these groups may be stored as a group entry in the pattern dictionary. This allows the tracking system 500 to recognize each pair, triplet, and quadruple, and the tracking system can then determine various pattern elements 10 based on this pattern dictionary, and subsequently determine at least one pose.

[0068] Figure 7 shows an optical pattern template (left portion) and an example of inserting the corresponding optical pattern 4 onto the surface 2 of the suction tube 110. The pattern is positioned so as to wrap around or enclose the cylindrical surface of the suction tube, forming a 3D marker element 1 integrated with the medical device. In particular, the optical pattern 4 contains additional information (encoded) regarding the type of device / instrument and can be combined with data associated with the device or instrument.

[0069] Figure 8 shows a further embodiment of a template for a 3D marking element with an optical pattern (left portion), which is applied to the distal portion 108 of a suction tube 110. In this embodiment, a quat (four elements) having an ellipse and nested ellipses is used as the quatreple pattern element 10. Even with only two "colors" or contrasts, four different elements can be created by using different designs of the pattern element 10, and these elements can be appropriately provided for at least one pose recognition on an instrument or device. Above the suction tube 110 on the right side of Figure 8, the group of pattern elements 10 is shown again for illustrative purposes (as a set of pattern element letters, so to speak).

[0070] Figure 9 shows a flowchart of a pose recognition method for pose recognition of a 3D marker element 1 equipped with an optical pattern 4, in particular a pose recognition method for the tracking system 500 described above. This method has the following steps.

[0071] In the first step S1, a 2D image is created by the camera 504 of the tracking unit 502, or a 3D image is created by at least two cameras of the tracking unit 502, and each visible pattern element 10 of the optical pattern 4 is detected.

[0072] In the next step S2, the control unit 510 identifies each pattern element 10 based on a dictionary containing different pattern elements 10, particularly binary pattern elements 10.

[0073] Next, in step S3, the correspondence between the 2D position in the 2D image and the position of the template of the 3D marker element 1 having the optical pattern 4, or the correspondence between the 3D position in the 3D image and the position of the template of the 3D marker element 1 having the optical pattern 4 is determined.

[0074] Finally, in step S4, n-point perspective position determination is performed on the 2D image for pose determination, or 3D-3D point determination is performed for pose determination of optical pattern 4.

[0075] Subsequently, the position and orientation of the 3D marker element 1 are determined using the optical pattern 4, and may be incorporated into the navigation accordingly.

[0076] In particular, by using the defined optical pattern 4, accurate and clear pose recognition can be performed using only one (conventional) 2D image (i.e., a two-dimensional image). [Explanation of Symbols]

[0077] 1 3D marker element 2 Surface 4 Optical pattern 6 Matrix 8 Pattern field / cell field 10 Pattern element 100 Medical product 102 Instrument 104 Medical device 106 Instrument tip 108 Distal part 110 Suction tube 500 Tracking system 502 Tracking unit 504 Camera 506 Display device 508 Storage device 510 Control unit S1 Image creation step S2 Pattern element identification step S3 Correspondence relationship determination step S4 Pose determination execution step

Claims

1. A surgical 3D marker element (1) for spatial pose recognition of a medical product (100), having a surface (2), particularly a sterilizable surface (2), wherein a two-dimensional optical pattern (4) is permanently applied or incorporated on or within the surface (2), particularly printed or engraved. The optical pattern (4) is configured in the form of a matrix (6) having M rows and N columns, with the corresponding pattern elements (8) as pattern fields (8). A surgical 3D marker element comprising at least four pattern elements (10), of which at least two pattern elements (10) are configured to provide an optical reference for pose detection of the 3D marker element (1) via an optical tracking system (500).

2. In the surgical 3D marker element (1) described in claim 1, The surface (2) is a curved surface, preferably a part of a cylindrical side surface, and is a surgical 3D marker element.

3. In the surgical 3D marker element (1) according to claim 1 or claim 2, The optical pattern (4) is composed of a rectangular matrix (6) in which each corresponding rectangular pattern field (8) has the same contour shape, and is characterized in that it has at least 8 rows and / or at least 5 columns, making it a surgical 3D marker element.

4. In the surgical 3D marker element (1) according to any one of claims 1 to 3, The matrix (6) is a surgical 3D marker element characterized by having a non-square configuration.

5. In the surgical 3D marker element (1) according to any one of claims 1 to 4, A surgical 3D marker element characterized in that the different pattern elements (10) are irregularly arranged over a part of the optical pattern (4), a large part of the optical pattern (4), or the entire optical pattern (4), and in particular, they are not arranged in rows or columns consisting only of identical pattern elements.

6. In the surgical 3D marker element (1) according to any one of claims 1 to 5, The matrix is ​​characterized in that the number of rows is greater than the number of columns, preferably the number of rows is at least 1.2 times, 1.5 times, or 2 times the number of columns, and / or the number of rows is up to 5 times, 4 times, or 3 times the number of columns, and is a surgical 3D marker element.

7. In the surgical 3D marker element (1) according to any one of claims 1 to 6, The optical pattern (4) has exactly two or more different pattern elements (10), preferably exactly three, exactly four, or exactly five different pattern elements (10). Surgical 3D marker elements, characterized in that the pattern elements (10) are distinguished from each other by at least one color or color perception, and more preferably by a circle, ellipse, or dot located in the center of the pattern field (8), or by a rectangle or rectangular outline located in the center of the pattern field (8), each in particular by a different color.

8. In the surgical 3D marker element (1) according to any one of claims 1 to 7, The optical pattern (4) has exactly two different pattern elements (10) as binary pattern elements (10), In the two different pattern elements (10) described above, The first pattern element (10) is a bright, especially white, pattern field (8), and the second pattern element is a dark, especially black, pattern field (8), or The first pattern element (10) is a bright, especially white, pattern field (8) and further has a dark, especially black, dot or black ellipse in the center, the second pattern element (10) is a dark, especially black, pattern field (8) and further has a white dot or white ellipse in the center, or A surgical 3D marker element characterized in that the first pattern element is a bright, particularly white, pattern field (8), and further has a black rectangle, particularly a square, in the center, and the second pattern element (10) is a black pattern field (8).

9. In a surgical 3D marker element (1) according to any one of claims 1 to 8, The optical pattern (4) has exactly three different pattern elements (10) as ternary pattern elements (10), In the three different pattern elements (10) described above, The first pattern element (10) may be a white pattern field (8), the second pattern element (10) may be a black pattern field (8), and the third pattern element (10) may be another color, particularly red, or The first pattern element (10) is a white pattern field (8) and further has a black dot or ellipse in the center, the second pattern element (10) is a black pattern field (8) and further has a white dot or ellipse in the center, the third pattern element (10) is a white pattern field (8), or A surgical 3D marker element characterized in that the first pattern element (10) is a white pattern field (10) and further has a black rectangle, particularly a square, in the center; the second pattern element (10) is a black pattern field (8) and further has a white rectangle, particularly a square, in the center; and the third pattern element (10) is a white pattern field (8) and further has a black rectangular outline in the center.

10. In a surgical 3D marker element (1) according to any one of claims 1 to 9, The optical pattern has exactly four different pattern elements (10) as quaternary pattern elements (10), In particular, in the four different pattern elements (10), The first pattern element (10) is a white pattern field (8), the second pattern element (10) is a black pattern field (8), the third pattern element (10) is another first color, especially red, the fourth pattern element (109) is another second color, especially green, or A surgical 3D marker element characterized in that the first pattern element (10) is a white pattern field (8) and has a black dot in the center, the second pattern element (10) is a black pattern field (8) and has a white dot in the center, the third pattern element (109) is a white pattern field (8), and the fourth pattern element (10) is a black pattern field (8).

11. In a surgical 3D marker element (1) described in any one of the prior claims, The optical pattern (4) has at least 3 rows and 3 columns, so that the central pattern element (10) is adjacent to eight pattern elements (10), together forming a pattern word that can be uniquely identified, and is a surgical 3D marker element.

12. In a surgical 3D marker element (1) described in any one of the prior claims, A surgical 3D marker element characterized in that different pattern elements (10) are distinguished by color and / or the center of a circle and / or an angle and / or an outline.

13. In a surgical 3D marker element (1) described in any one of the prior claims, A surgical 3D marker element characterized in that the first portion of the optical pattern (4) is configured to be offset and / or inclined with respect to the second portion of the optical pattern (4), thereby preventing the first portion and the second portion from being viewed simultaneously.

14. In a 3D surgical marker element (1) described in any one of the prior claims, A surgical 3D marker element characterized in that the aforementioned surface (2) is not planar.

15. In a 3D surgical marker element (1) described in any one of the prior claims, The optical pattern (4) is characterized in that it does not contain encoded information, thereby providing a surgical 3D marker element.

16. In a 3D surgical marker element (1) described in any one of the prior claims, A surgical 3D marker element characterized in that the optical pattern (4) is not a barcode and / or QR code (registered trademark).

17. A medical product (100), particularly a medical instrument (102) or medical device (104) having optical pose recognition during patient treatment, The medical product (100) has a surgical 3D marker element (1) according to any one of claims 1 to 16, particularly in the distal part of the medical product (100), Preferably, the 3D marker element (1) is directly integrated into the medical product (100), and the optical pattern (4) is printed on the surface of the medical product or incorporated into the surface, preferably by laser engraving, the medical product.

18. In the medical product (100) described in claim 17, The medical product (100) is a surgical instrument (102), particularly a scalpel or trocar, having a distal instrument tip (106), wherein the 3D marker element (1) is positioned in a region of less than 50%, particularly less than 30%, of the total length of the distal instrument tip (106), particularly directly on the region of the instrument tip (106), to provide spatial pose recognition of the instrument tip (106), and is particularly positioned on a surgical scalpel as an instrument having the 3D marker element on the flat surface of the cutting blade.

19. In the medical product (100) according to claim 17 or claim 18, The medical product (100) has a cylindrical contour having a cylindrical surface (2) corresponding to the distal portion (108), The 3D marker element (1) is arranged partially, particularly around the entire circumference, in the circumferential direction and is particularly engraved. Preferably, the medical product (100) is a medical suction device (104) having a distal suction tube (110), A medical product characterized in that the 3D marker element (1) having the optical pattern (4) is provided on the suction tube (110), and in particular the optical pattern (4) is printed on the surface (2) of the suction tube (110) or engraved within the surface (2).

20. In a medical product (100) according to any one of claims 17 to 19, The medical product (100) is characterized in that the distal portion (108) is configured as a medical instrument (104) having a special, particularly conical shape, and an optical template pattern in the form of a rectangular matrix is ​​mapped via a linear transformation matrix to conform to the special, particularly conical shape, and exists on or within the surface (2) as a conformed optical pattern (4).

21. A surgical tracking system (500) for surgical procedures on a patient, in particular a navigation system, A surgical 3D marker element (1) according to any one of claims 1 to 16, or a medical product (100) according to any one of claims 17 to 20, A surgical tracking system characterized by a tracking unit (502) having at least one camera (504), preferably a stereo camera, wherein the camera is configured to optically detect the 3D marker element (1) having the optical pattern (4), and to determine a pose based on the detected optical pattern (4), and in particular the tracking unit (502) is configured to incorporate the detected optical pattern into a navigation system and to visually output the pose by a display device (506).

22. In the surgical tracking system (500) according to claim 21, The surgical tracking system (500) further comprises a storage device (508) which stores dictionaries of pattern elements, in particular dictionaries of binary pattern elements and / or tertiary pattern elements and / or quaternary pattern elements, A surgical tracking system characterized in that a control unit (510) is configured to read the optical pattern (4) based on the dictionary and determine the pose.

23. In the surgical tracking system (500) according to claim 22, A surgical tracking system characterized in that a portion of the optical pattern (4) is sufficient to determine the pose.

24. In a surgical tracking system (500) according to any one of claims 21 to 23, A surgical tracking system characterized in that any portion of the optical pattern (4) having at least four pattern elements (10) can be used for pose determination.

25. In a surgical tracking system (500) according to any one of claims 21 to 23, A surgical tracking system characterized in that the optical pattern (4) can preferably be used for pose determination purely optically, and in particular without using information encoded in the optical pattern (4).

26. In a surgical tracking system (500) according to any one of claims 21 to 25, A dataset associated with the optical pattern (504) is stored in a storage device (508), which in particular includes the conversion of the pose of the optical pattern (4) to the operating point of the medical product (100), thereby detecting the position of the operating point and the orientation of the distal part (108). A surgical tracking system characterized in that the control unit (510) is configured to determine the orientation of the operating point and the distal portion (108) based on the pose recognition of the optical pattern (4) and the stored conversion, and to visually output the operating point and the orientation as a view by the display device (506).

27. A pose recognition method for pose recognition of a 3D marker element (1) having an optical pattern (4) as described in any one of claims 1 to 16, or for pose recognition of a medical product (100) as described in any one of claims 17 to 20, The following steps are included: Steps include creating a 2D image from the camera (504) of the tracking unit (502) or a 3D image from at least two cameras of the tracking unit (502) (S1), and detecting each visible pattern element (10) of the optical pattern (4); Step (S2) of the control unit (510) identifying each pattern element (10) based on a dictionary containing different pattern elements (10), particularly binary pattern elements (10), or ternary pattern elements (10), or quaternary pattern elements (10); Step (S3) of determining the correspondence between a 2D position in the 2D image and the position of the template of the 3D marker element (1) having the optical pattern (4), or the correspondence between a 3D position in the 3D image and the position of the template of the 3D marker element (1) having the optical pattern (4); A pose recognition method, step (S4) of performing n-point perspective position determination in the 2D image for pose determination, or 3D-3D point determination for pose determination of the optical pattern (4).

28. A computer-readable storage medium, A storage medium that, when executed by a computer, includes instructions causing the computer to perform each step of the pause recognition method described in claim 27.

Images