Apparatus and method for detecting direction and position of a marker in three-dimensional space

Abstract

This invention describes a possibility for detecting the orientation and position of markers in three-dimensional space using at least one marker arrangement. The marker arrangement includes at least two marker units, with illumination devices for each marker unit arranged along a path. Each marker unit has at least three illumination devices, which are configured as markers and / or communication elements. At least one optical image capture unit is provided, adapted to record an image of the marker arrangement. An evaluation unit is provided, adapted to precisely determine the orientation and position of the marker arrangement from exactly one image from one of the optical image capture units. A first marker unit type with at least three markers and at least one second marker unit type with exactly two markers and at least one communication element are provided. The evaluation unit is configured to identify the marker units and definitively determine the orientation and position of the marker arrangement using the identified markers.

Description

Technical Field

[0001] An apparatus for detecting the orientation and position of a marker in three-dimensional space is described. Therefore, during the process of detecting the orientation and position of a marker, the orientation and position of the marker relative to a defined spatial coordinate system are determined. The invention also relates to a method for detecting orientation and position using the described apparatus and a computer program product. Background Technology

[0002] The proposed device has at least one marking arrangement, which can be specifically fixed to an object. Therefore, by determining the orientation and position of the markings on the marking arrangement, the orientation of the object can then be calculated. The marking arrangement includes at least two marking units, each having an optically active illumination device arranged along a path (in the sense of a straight, non-curved path). Preferably, the illumination devices of one or each marking unit are arranged exactly along a straight path, while the marking units are arranged in a non-collinear manner. Each marking unit has at least three illumination devices designed as marking and / or communication elements.

[0003] Furthermore, while markers and communication elements are always optically active elements, they differ at least functionally, as will be described later. Throughout this document, the term "illumination device" is also understood to refer to an optically active element, i.e., an illumination element that is turned on or reflects light. Illumination elements present on the marker unit but not turned on or reflecting light are not considered (optically active) illumination devices, i.e., not markers or communication elements. This means that a number of operational illumination devices are arranged in the operating marker unit, which can be reliably detected by the optical image capture unit. According to a preferred embodiment, these are light-emitting devices, such as LEDs, that can be turned on or off, and are therefore optically active. However, in principle, an uncovered light-reflecting surface can also be optically active in the sense of an illumination device. Therefore, a marker unit suitable for use in the device is configured such that, during the operation of the marker unit, for example, because the reflective surface of the illumination device is not covered, the illumination device suitable for this purpose is activated and emits light or reflects light falling upon it.

[0004] Other lighting elements that are not turned on during operation (i.e., turned off during operation) or do not reflect light (e.g., due to a cover) are not considered lighting devices. This does not preclude turning off all, for example, motorized lighting devices (i.e., temporarily de-energizing) when the marking unit is not operating and the device is not used for position detection.

[0005] The device also includes at least one optical image capture unit and an evaluation unit. The optical image capture unit is configured to capture an image of the marker arrangement, for example, using a digital camera. The evaluation unit is adapted to definitively determine the orientation and position of the markers and / or communication elements on the marker arrangement from a precise image from one of the optical image capture units. For this purpose, the evaluation unit learns the position of the illumination device (marker or communication element) on the marker unit and the position of the marker unit with the mark / communication element on the marker arrangement. According to a preferred embodiment, the evaluation unit may be configured to determine a unique orientation and position from the markers.

[0006] According to a preferred, generally known embodiment, each marking unit can form an LED (light-emitting diode) illumination device that can be turned on and off. This illumination device is preferably a linear array of several (at least three) LEDs that, when turned on, emit light waves preferably in the optically visible or non-visible wavelength range. A particularly preferred embodiment provides infrared LEDs. Preferably, these LEDs can be arranged on a circuit board at defined intervals, for example, arranged as an LED path array with equal spacing, or arranged at predetermined intervals in specific locations (if desired, individually fixed to the circuit board).

[0007] According to the present invention only, any arrangement of markings and / or communication elements on a straight path is considered a marking unit; therefore, markings and / or communication elements not on such a path do not constitute a marking unit according to the present invention. If only the projection of an optically active lighting device lies on the path connecting two other optically active lighting devices, but the optically active lighting devices are arranged on another plane, then this optically active lighting device is not located on the path connecting the other two lighting devices in the sense of the present invention.

[0008] The marking unit can be arranged in its own (separate) housing, and several separate housings together constitute the marking arrangement. However, according to the invention, several marking units (e.g., two or three marking units) can also be combined with each other in a defined arrangement within a common housing to form the marking arrangement according to the invention.

[0009] Systems and methods are known from publications EP 1813911 A1, US2005 / 0201613 A1, EP 1498688 B1, WO2004 / 114112A1, US2008 / 0111985 A1, WO 2006 / 069748 A1, US 5227985 A, US 7742895B2, and DE 102014012693 B4. These disclosures determine the orientation and position of an object in space by capturing an arrangement of markers fixed to the object. For this purpose, optically visible markers are provided in a fixed geometric arrangement. Many different arrangements are possible for evaluation. Most proposed arrangements set at least four markers, with at least three spanning a plane and at least one marker located outside this plane. The at least three markers arranged in a plane can be located on at least two non-parallel straight lines spanning the plane. Such a marker arrangement can reliably determine the orientation and position of the marker arrangement in space from a single captured two-dimensional image of the marker arrangement, wherein the two-dimensional image is captured by an optical image capture unit (e.g., a camera, especially a digital camera).

[0010] DE 102014012693 B4 describes a reliable but complex system and method. Position and orientation determination is based on a single two-dimensional image of a marker arrangement with at least seven markers attached to an object. The system also includes an image capture unit for acquiring a two-dimensional image of the object or the marker arrangement on the object, and an evaluation unit for definitively determining the object's orientation and position based on the acquired image. The seven markers of the marker arrangement are in a fixed spatial relationship with each other, with six markers forming a plane and the seventh marker positioned outside or at a distance from the plane. The six markers are grouped such that they lie on two distinct straight lines intersecting at a 90° angle. The first straight line includes at least four markers, and the second straight line includes at least two additional markers. In a top view of the plane, the seventh marker outside the plane also lies on the first straight line, specifically on the side of the second straight line away from at least two markers on the first straight line. This is crucial for definitively assigning image markers, i.e., the markers depicted in the image, thereby reconstructing the markers' orientation and position in space. During the evaluation process, several homographies were calculated for the possible assignments. From these homographies, a location determination was reconstructed for each case. Based on this, the average reprojection error of all image markers (compared to the known actual arrangement of the markers on the marker arrangement) was calculated to determine each location. The homography matrix with the smallest error was correct and used to determine the unique location.

[0011] This evaluation method yielded good results in practice, but it was costly in terms of image evaluation and the computational power of the system, especially because it required calculating more than one homography matrix and applying a Levenberg-Marquardt solver each time to estimate the orientation. Furthermore, the replication errors of coplanar and non-coplanar markers were calculated, and the minimum of the matches was used to evaluate the orientation. This complex computation, particularly considering a large number of markers, led to problems in real-time tracking of fast-moving objects, as the computational power of practical systems was insufficient, especially if the system monitored not just one marker arrangement, but multiple marker arrangements, or even if these markers might be in a single image.

[0012] In theory, each marker arrangement can be distinguished from other marker arrangements captured in an image by different labels. However, if these markers (e.g., in the form of pasted labels) are not fully visible in the image, the evaluation of the surrounding environment of the marker arrangements in the image often leads to ambiguity when tracking moving objects. In principle, it is also conceivable to place markers on marker arrangements in geometrically different ways. However, this results in a more complex evaluation of markers in many different arrangements.

[0013] In industrial environments, for real-time applications, especially on production lines, it is sometimes essential to determine the orientation and position of marker placement very quickly. For this, the evaluation of camera images and the determination of orientation and position must be fast enough to be directly applied to the production process, such as for motion control. Finding the intersection shape of markers placed on a plane is time-consuming and slows down position determination. The intersection shape of markers on a plane results in a relatively large planar area, making it difficult to place markers within a 360° azimuth angle, especially since elevated markers can overlap with other markers. This leads to varying detection accuracy depending on the camera's angle of view. Summary of the Invention

[0014] Therefore, the object of the present invention is to provide a simplified possibility for orientation and position detection of marker units in the aforementioned apparatus for orientation and position detection, which enables rapid evaluation and high accuracy and detection rate even at very different camera angles.

[0015] This objective is achieved by the apparatus and method of the present invention for detecting the orientation and position of a marker in three-dimensional space. Specifically, at least one marker unit is configured to belong to a first marker unit type having at least three, or preferably exactly three, markers, while at least one additional marker unit belongs to a second marker unit type having exactly two markers and at least one communication element. At least one communication element in the second marker unit type is arranged between the two markers. Preferably, in the case of several communication elements, all communication elements are arranged between the two markers. According to the invention, no marker unit should be collinear with another marker unit of the same marker arrangement, i.e., they should not be located on the same straight line.

[0016] To achieve good orientation detection, at least one marker unit of the first marker unit type and at least one marker unit of the second marker unit type are arranged in a non-coplanar manner. The individual linear marker units can be positioned relatively freely within the marker arrangement according to use and application; thus, according to a particularly preferred embodiment, the marker units do not overlap in the marker arrangement, that is, the marker units do not intersect each other. This constraint greatly simplifies the assignment of illumination devices detected in the image to the markers used for orientation and position detection. Therefore, according to the invention, the minimum marker arrangement can consist of exactly one marker arrangement of the first type and exactly one marker arrangement of the second type (where "type" refers to the type of marker unit). According to the invention, it is preferred that other illumination devices are not considered in image evaluation, excluding, for example, status indicator LEDs. In this marker arrangement with exactly two marker units, the marker units (i.e., paths with illumination devices) are not arranged in parallel, and therefore the marker units are not coplanar with each other. This minimum marker arrangement can be implemented, for example, simply with marker units having their own housings that can be freely and individually fixed to the body. The evaluation unit must then be trained in the arrangement of marker units, for example using the methods described later. In addition, the arrangement of the marker cells can also be measured and sent as a measurement value to the evaluation cell.

[0017] According to the present invention, the evaluation unit is designed to identify marker units of a first marker unit type and a second marker unit type. This allows for rapid detection of the marker arrangement and determination of its orientation and position through simple image evaluation, even though the individual marker units in the marker arrangement may be freely arranged. To this end, the present invention provides an evaluation unit suitable for definitively determining the orientation and position of the marker arrangement using only the markers of the identified marker units of the first and second marker unit types. This allows for highly flexible identification of marker units in different arrangements, for example, those arranged around an object, based solely on a few known patterns in the recorded image corresponding to the first or second type of marker units, and enables orientation and position detection from any viewpoint, particularly azimuth.

[0018] According to a preferred embodiment, the marks of each of the first and second mark unit types are identical for all mark arrangements of the device. Specifically, this means that the marks in the mark units of the first mark unit type and the mark units of the second mark unit type are each arranged in the same position. With respect to the marks, according to a preferred embodiment, all mark units of the first and second mark unit types are therefore identical. This makes it possible, particularly in systems or devices with a large number of mark arrangements, to use the same algorithm to determine the orientation and position of the mark arrangements. Different mark arrangements are then distinguished by communication elements, which can be explicitly identified (e.g., due to unique relative arrangements within the device). These communication elements are arranged in the mark units of the second type of mark unit such that the encoding of the communication element relative to the mark including the communication element is different for all second type mark units operating in the same device.

[0019] The differences between the markers and communication elements (which may be implemented in hardware by the same lighting elements) mean that the arrangement of the lighting elements in the evaluation unit is known for each marker unit type, i.e., the positions of the lighting elements relative to each other are known, and the function "marker" or "communication element" is assigned or can be assigned to each position. According to a particularly preferred embodiment of the invention, an exact one position of the lighting device in each marker unit type is assigned to only one of the functions "marker" or "communication element," i.e., there is no dual assignment of functions.

[0020] According to a preferred embodiment of the invention, each tag unit of the first tag unit type and the second tag unit type may have three to seven active lighting devices, i.e., tagging and / or communication elements, and more lighting elements may be provided if necessary. However, in a preferred embodiment, the tag units are arranged in operation to activate no more than three to five active lighting devices.

[0021] A preferred embodiment limits the number of markers (active lighting devices with the function of "markers") to a maximum of five, more preferably a maximum of four, and particularly preferably a maximum of three. The more markers provided, the longer it takes to determine the orientation and position of the markers. Accordingly, a particularly preferred embodiment, according to the invention, arranges a marker unit of the first marker unit type having exactly three lighting elements and exactly three markers.

[0022] According to a further embodiment, preferably in combination with the preceding embodiments, the marker unit of the second type of marker unit may have 6 to 14 illumination elements, preferably 8 to 12, particularly preferably 10, of which (preferably exactly) two are used as markers and 2 to 5 are used as communication elements. Preferably, all second type marker units used in a system or device differ in the arrangement of their communication elements. This means that in each marker unit of the second marker unit type, different illumination elements are activated as communication elements to at least enable identification.

[0023] In a preferred embodiment, the linear arrangement of the lighting devices in the first and second marker unit types may also have different lengths. For example, the linear arrangement of the lighting devices in the second marker unit type may be shorter than that in the first marker unit type. The first and second marker unit types are paths consisting of at least three lighting devices, each defined at both ends by an end lighting device (end mark) assigned a "marking" function. Between the end marks of each marker unit type, at least one additional lighting device is arranged on the path connecting the end lighting devices. In a preferred embodiment, the path of the first marker unit type is longer than the path of the second marker unit type. This can be particularly advantageous in marker orientation detection.

[0024] More communication elements increase the amount of data that can be communicated per time unit. This can be utilized if, in addition to identifying the marker arrangement, the communication elements are also used to transmit other data during time-division multiplexing. However, this also increases the number of different communication modes and prolongs the time required to read data from the communication modes. Furthermore, the number of active lighting devices (markers and / or communication elements) increases the power consumption of each marker unit and marker arrangement. Generally, more communication elements are unnecessary, especially if the communication elements are only used to identify the marker arrangement and / or marker units. In marker units of the second marker unit type, at least two communication elements and (preferably exactly) two marks are beneficial for distinguishing between the two marker unit types.

[0025] According to a particularly preferred embodiment of the invention, since the power supply for the marking units (which may also belong to a marking arrangement comprising multiple marking units) is provided by a rechargeable battery, it is particularly advantageous to limit each marking unit to having three active markings. For example, if the marking units are attached to a large workpiece, they can also be attached to the power supply of the large workpiece (e.g., generally an electrically operated machining tool). In this case, the power supply is not a decisive factor limiting the number of optically active markings and / or communication elements in use. However, in the case of marking arrangements equipped with energy storage devices, it is necessary to provide a corresponding power interface or charging interface for charging the energy storage device, such as a wired charging connection or a wireless charging option in the form of an inductive charging interface, for example. This is more expensive in terms of manufacturing and maintenance.

[0026] With the marker units in the marker arrangement according to the present invention, it is possible to inversely calculate the orientation and position of the marker arrangement in space from the markers recorded in a two-dimensional image using a calibrated camera (in the sense of the optical image capture unit of the present invention). Various solutions for this purpose are known to those skilled in the art. These methods include solution methods such as P3P (Perspective-3-Point), particularly the known algebraic solution algorithm for the perspective-three-point problem (AP3P method) or the iterative solution algorithm for the perspective-three-point problem (Iterative P3P), which are known to those skilled in the art and are described, for example, in the following publications: AP3P: Tong Ke and Stergios Roumeliotis. An efficient algebraic solution to the perspective-three-point problem. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on. IEEE, 2017; P3P: Xiao-Shan Gao, Xiao-Rong Hou, Jianliang Tang, and Hang-Fei Cheng. Complete solution classification for the perspective-three-point problem. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 25(8): 930-943, 2003. These well-known algorithms, provided the previously defined conditions are met, can find the spatial orientation and position of an object based on the correspondence between the object's markings and its two-dimensional image in a calibrated camera image.

[0027] To make these algorithms more robust to outliers in point reconstruction from images, a method called RANSAC (Random Sample Consensus) is used. This method establishes initial associations between detected labeled units in an image (belonging to a first and a second labeled unit type). These initial associations can be iteratively improved until all relationships between detectable labeled units in the image are known, thus identifying the labeled units. With this information, the relationships between the labeled units can be evaluated, thereby determining the orientation and location of the labeled units in space. The RANSAC procedure is an iterative method for estimating the parameters of a mathematical model from a set of observations that includes outliers, where the outliers cannot affect the estimates. Therefore, it can also be interpreted as an outlier detection method. An explanation of this algorithm can be found, for example, in the following paper: Martin A. Fischler, Robert C. Bolles: Random Sample Consensus: AParadigm for Model Fitting with Applications to Image Analysis and Automated Cartography, Communications of the ACM, June 1981, https: / / doi.org / 10.1145 / 358669.358692.

[0028] The application of these methods is known to those skilled in the art. They can be applied to the present invention as described, and, if necessary, adapted within a skilled optimization framework.

[0029] In principle, this invention is independent of specific types of algorithms (known in the art) used for orientation and position detection.

[0030] A particular advantage of the marking arrangement proposed according to the present invention is that, through a simple and structurally clear marking arrangement, the actual orientation and position of the marking arrangement (i.e., relative to a human movement sequence) can be determined in real time. This orientation and position relate to typical industrial manufacturing and assembly processes, medical applications (e.g., operations performed by remotely controlled endoscopes), or similar applications (e.g., inspection of industrial components). Therefore, real-time tracking of the marking arrangement is also possible. This allows for real-time tracking of the movement of tools, devices, or main components bearing the marking arrangement.

[0031] In many applications, the evaluation speed is sufficient to satisfy the movements performed by manipulators (controlled mechanical motion devices, commonly known as robots), thus this invention can be widely used. Due to the simplicity and flexibility of its structure, the evaluation of image points, including orientation and position detection, is simpler and faster than in the prior art. Unlike the cross-shaped designs known in the prior art, 360° orientation detection can also be easily achieved, for example, by arranging the marker units on a cylindrical surface. A significant advantage of this invention is that the recorded images are similar regardless of the azimuth angle around the axis of symmetry of the arrangement (i.e., the axis of the cylinder in the case of an arrangement that is also a cylindrical surface), and have constant accuracy across the entire 360° azimuth angle. This is particularly suitable for a preferred arrangement of linear marker units in which there are no intersections between the marker units.

[0032] According to a preferred embodiment of the invention, the marking arrangement (in the sense of one, more or each of at least one marking arrangement of the device) includes at least two marking units of a first marking unit type and at least one marking unit of a second marking unit type, which are fixed to each other in a defined arrangement manner known to the evaluation unit.

[0033] In a preferred embodiment, the marker arrangement has exactly two marker units of the first marker unit type and exactly one marker unit of the second marker unit type, in that sense, there are no other marker units or markers. Preferably, this applies to all marker arrangements of the device, so that the marker positions in all marker arrangements are identical, and each preferred marker arrangement of the device differs from another marker arrangement of the device (only) in that the marker unit of the second marker unit type includes exactly one communication element. Several of these marker arrangements can be fixedly arranged to each other in a common structure (e.g., in a frame or housing) and together constitute a multi-marker arrangement in which several marker arrangements are thus fixedly arranged to each other. Since the structure of the marker arrangements is always identical, and the arrangement of all illumination devices (i.e., markers) for direction and position detection is identical, position detection can be performed using particularly efficient and fast algorithms.

[0034] However, one embodiment of the marker arrangement according to the invention is also conceivable, in which at least one second-type marker unit is always located between two first-type marker units. If more second-type marker units are arranged adjacent to each other in the marker arrangement, the communication rate can be significantly improved, for example, in a large system (the large device according to the invention) where many different marker arrangements must operate together, or where high data transmission rates are required, for example, because a large amount of additional information must be transmitted, whether through data transmission with only one code or through several different coding sequences, where the data information is transmitted one after another. Another embodiment can be configured such that on a body with a non-planar surface (e.g., a cylinder, cone, or other substantially arbitrary shape), several first-type and second-type marker units are arranged such that at least one first-type marker unit and one second-type marker unit (arranged in a regular repeating or arbitrary arrangement) are arranged adjacent to each other, and are visible in photographs. In the case where the marker units are arranged parallel to each other, for example on a cylindrical surface, it is preferable that two first-type marker units are arranged adjacent to one second-type marker unit, so that all the marker units are not coplanar. A preferred embodiment of this type can be an arrangement on a cylindrical surface, wherein the marking units of the first and second types are arranged in parallel on the entire surface or at least on a portion (relative to the circumference) along the cylindrical axis.

[0035] In a particularly preferred embodiment of the marker arrangement, such as an embodiment with exactly three marker units (two of the first type and one of the second type), the paths of the illumination devices for the marker units of the first marker unit type lie in a common plane, preferably forming two opposite sides of a parallelogram. The end marker of each marker unit forms an endpoint, and the path of the illumination device ends at the end marker. In a particularly preferred embodiment, the straight path connecting the illumination devices (endpoints) can form a rectangle with an end marker at each corner of the rectangle. Furthermore, the paths of the marker units of the second marker unit type may be shorter than those of the first marker unit type, i.e., the distance between the end markers of the second type may be shorter than the distance between the end markers of the first type. In such an embodiment, the marker units of the second type can be arranged within the parallelogram or rectangle spanned by the marker units of the first type, or more generally within the surface shape formed by the connection of the end markers of the marker units of the first type. This makes it easier to find the markers in an image of the marker arrangement, and then perform orientation and position detection accordingly.

[0036] In a particular embodiment, the marker units of the second type of marker unit can be arranged outside the plane spanned by the two marker units of the first type of marker unit, specifically elevated relative to this plane. Elevated arrangement refers to arranging the illumination device of the marker unit in a viewing direction perpendicular to the plane (spanned by the first type of marker unit), where the illumination device is visible. With such an arrangement, the markers of the second type of marker unit will certainly not be coplanar with the markers of the first type of marker unit. This improves the accuracy of orientation and position detection. However, according to a preferred embodiment, the path of the illumination device of the second type of marker unit can be located in a plane parallel to the plane spanned by the markers of the first type of marker unit. This is particularly advantageous for orientation detection of markers in space.

[0037] In a preferred embodiment of the invention, the straight paths of the lighting devices for the marking units of the marking arrangement are arranged in parallel. This is also abbreviated herein as parallel arrangement of marking units. This expression should be understood synonymously. This means that the straight lines along the (straight) paths of the lighting devices for all the marking units of the marking arrangement do not intersect, and the marking arrangement is particularly a marking arrangement having exactly two marking units of a first marking unit type (also called "first type marking unit" by synonym) and exactly one marking unit of a second marking unit type (also called "second type marking unit" by synonym). A straight line should not be understood as infinitely long in a mathematical sense, but rather as an extension of the straight path arrangement that does not intersect within a certain length. For the purposes of this invention, paths arranged at small angles (e.g., not exceeding 5°) to each other are also considered parallel. This particularly includes manufacturing tolerances.

[0038] The selection of marker units or the parallel arrangement of all marker units in a marker arrangement represents a very effective criterion for selecting a marker arrangement belonging to a marker unit, and is also a very precise boundary condition for algorithms applied to orientation and position detection. This is particularly applicable to two marker units of the first type. In this embodiment, the second type of marker unit can also be arranged in any position relative to the first type. However, it is particularly preferred that it be arranged between marker units of the first type, which makes marker assignment particularly easy. However, the invention is not limited to this preferred embodiment, which can be very useful, especially when there are exactly three marker units in the marker arrangement.

[0039] In principle, it is suggested that marker units of the first marker unit type be arranged adjacent to marker units of the second marker unit type in the marker arrangement (either throughout the entire marker arrangement or only in a portion of the marker arrangement). Adjacent arrangement means that a marker unit of the first marker unit type is arranged transversely to the longitudinal direction of the marker unit on at least one of the two sides, and preferably, a marker unit of the first marker unit type is also arranged on each of the two sides, which is also in the longitudinal direction. In the longitudinal direction, this means that the angle (minimum amount) between the paths of the first and second marker units is less than 45°, and the straight paths (adjacent to each other in the transverse direction of the straight paths) have at least 50% overlap (in the longitudinal direction of the paths).

[0040] In this particularly preferred embodiment, at least some or all of the marker units of the marker arrangement are arranged in parallel. In principle, this embodiment can also be used (as previously described) to combine multiple marker arrangements into a common structure, thus forming a multi-marker arrangement. However, the invention is not limited thereto, and in principle, the number of marker units of the first and / or second marker unit types can be freely selected in the marker arrangement without the need to arrange a single marker arrangement multiple times in a multi-marker arrangement. In a spatial arrangement, the marker arrangement is particularly preferably formed as a protrusion, which is another embodiment described below.

[0041] Therefore, according to one embodiment of the present invention, a plurality of mark arrangements consisting of at least one first type of mark unit and one second type of mark unit can be combined into a common structure in which the individual mark arrangements are arranged relative to each other in a defined manner and form a multi-mark arrangement, the outer contour of the common structure forming a convex body.

[0042] A simple geometric possibility is that the markers (or corresponding lighting devices) arranged in each marking unit along a straight path are situated on a sleeve with a circular, elliptical, or similar curved cross-section, wherein all paths with lighting devices are arranged parallel to each other in the axial direction of the sleeve perpendicular to the cross-sectional area. This is, for example, when all paths with lighting devices (markers or communication elements) are arranged parallel to each other along the axial direction of a cylindrical sleeve. This forms a convex body.

[0043] In this invention, a convexity exists if a straight line connecting two marks (even lighting devices) that are not adjacent or collinear within the same marking unit is arranged within or along the outer contour of the body. If the shape of the body is not as clearly defined as a cylindrical body, an outer contour can be formed such that, in each case, the three marks (or lighting devices) closest to each other are connected by a triangular surface. In this way, an outer contour is formed in each case, upon which the presence of a convex outer contour can be checked. Openings are retained in, for example, the edge regions of the end faces of the body, and are closed by triangular regions starting from the intersection of the respective contour surfaces. Such a multi-marker arrangement can be arranged around any body, and the convex shape ensures optimal visibility of the marks used for orientation and position detection.

[0044] According to a preferred embodiment, the evaluation unit is configured to perform the methods described below or parts thereof. In particular, the methods described below or certain embodiments thereof may be particularly applicable to specific embodiments described above.

[0045] Therefore, the present invention also relates to a method for detecting the orientation and position of a marker in three-dimensional space, particularly a method using an embodiment of the aforementioned apparatus, wherein an evaluation unit is configured to perform the method or a portion thereof. According to the invention, the method particularly includes the following steps, which can be performed in the order described below. However, the order may be changed if technically feasible without departing from the subject matter of the invention.

[0046] - Capture an image of at least one marker arrangement, comprising at least one marker unit of a first marker unit type and at least one marker unit of a second marker unit type, each marker unit comprising at least three optically active illumination devices arranged along a path. According to a preferred embodiment, the first type of marker unit may have a fixed number of illumination devices, particularly three, arranged at the same position in each marker unit and functioning as a marker. In the case where the illumination devices are arranged along a path, the second type of marker unit has illumination devices as markers (also referred to as end markers), particularly at two opposite ends. According to a preferred embodiment of the device according to the invention, all other illumination devices of the second type of marker unit (arranged between two markers) may serve as communication elements, preferably arranged differently from each marker unit of the second type according to the invention, particularly by optically activating or turning on different illumination devices respectively. These may also be selected such that the first and second types of marker units can be distinguished by (optically active) illumination devices, based on the position and / or number of illumination devices, particularly preferably at least based on the number of illumination devices. Therefore, the second type of marker unit preferably has exactly two to five, i.e., exactly two, three, four, or five communication elements and exactly two markers (at the beginning and end of the path, respectively).

[0047] - Detect lighting devices in the captured image. Using traditional image detection software, each lighting device is identified as a lighting point in the image, and its unique two-dimensional image coordinates (x, y, z) are assigned. B y B Image coordinates, also known as pixel coordinates, are assigned to points in the image. These image coordinates can then be used for further evaluation. Examples of this will be presented later.

[0048] - Detected lighting devices are assigned to at least one tag unit of a first tag unit type, wherein the lighting devices are specifically designed as tags, and detected lighting devices are assigned to at least one tag unit of a second tag unit type, wherein the lighting devices are designed as both tags and communication elements. This is accomplished by evaluating the image coordinates of the detected lighting devices and evaluating whether the selection of certain lighting devices in the image (i.e., their image coordinates) matches or may match the tag arrangement in the first tag unit type or the second tag unit type (known in the evaluation unit of the execution step). Appropriate selection criteria are specified for this purpose.

[0049] - Assign functional identifiers or functional communication elements to lighting devices detected in the image. This can be done, in particular, on the basis of assigning lighting devices to one of the two types of marker units.

[0050] - Determining the direction and position of the marker arrangement based on the markers, that is, determining the arrangement of the markers on the marker arrangement based on the lighting device having the specified function of "marker", which is the arrangement of the markers on the marker unit and the arrangement of the marker unit in the marker arrangement, are synonymous in the sense of this invention. According to the invention, this can be accomplished using the algorithm and RANSAC method already described.

[0051] According to the invention, preferably, assigning marker units to a first marker unit or a second marker unit may include the step of determining all straight paths that have exactly three illumination devices (i.e., a total of three markers and / or communication elements) in the captured image. This means that all exactly three illumination devices are clustered along the entire straight path having three (optically active) illumination devices, regardless of whether these illumination devices are markers or communication elements, wherein these illumination devices (markers and / or communication elements) can be reliably detected in the image and arranged along the straight path. These paths are recorded and can be saved, for example, as 3-tuples, whose image coordinates can exactly represent each of the three markers / communication elements in the list, for example. Thus, the 3-tuples describe a straight path with three markers / communication elements that can be identified in the captured image. All 3-tuples determined in this way constitute the entire straight path. For example, the 3-tuple is of the form [x B (1), y B (1); x B (2), y B (2); x B (3), y B (3)], where the numbers in parentheses are the numbers of the detected lighting devices (markers / communication elements). These paths preferably end at a lighting device in each case, with another lighting device between them. The lighting device where a straight path ends is also referred to as an endpoint.

[0052] If, in a captured image, more than three markers or communication elements are arranged along a straight path, then each possible sub-combination with exactly three markers or communication elements (i.e., three lighting devices) is considered one of the defined straight paths with exactly three markers / communication elements, and each possible sub-combination is recorded in the whole of the straight paths, for example, in a list of 3-tuples. In the following text, for simplicity, a straight path is sometimes simply referred to as a path. Therefore, these terms are synonymous.

[0053] In a preferred apparatus according to the invention, the lengths of the marker units of the first and second marker unit types (the paths for arranging the illumination devices, respectively) are substantially the same, i.e., particularly differing by no more than 10%, preferably no more than 5%. It can be assumed that, despite the possibility of perspective distortion during image capture, the path lengths (the distances between the endpoints of the paths formed by the end markers, respectively) are substantially the same. This is particularly applicable when the typical length of the marker units is between three and fifteen centimeters, preferably between three and six centimeters, and the distance between the camera (image capture unit) and the captured marker arrangement is at least more than one meter, preferably between two and five meters or two and ten meters. This is common for typical applications. In this case, as a selection criterion for paths that may belong to the marker arrangement, those paths can be determined from all determined straight paths with comparable distances between the endpoints. Such paths can be combined into data packets, and if necessary, particularly when several different marker arrangements are detected in an image, several data packets can be determined in one image.

[0054] In the device according to the invention, the marker units (which are respectively the paths for arranging the lighting devices) are arranged in parallel, and this parallel arrangement can serve as an optional or further selection criterion for paths that may belong to a marker arrangement. Due to perspective distortion during image capture, it cannot be assumed that paths extending mathematically to infinite length do not intersect in the two-dimensional coordinate system of the image.

[0055] In a very simple evaluation, paths that intersect directly in the image coordinate system can be excluded as non-parallel paths. In a preferred variant of this selection criterion, it can be checked whether extending a straight path by a certain extension factor (e.g., extension factors of 1.5 and 5 at one or both ends) results in the extended paths intersecting. Again, in this case, such paths that intersect as extended paths can be excluded as non-parallel paths, and the extension factor can also be parameterized, depending particularly on the application and / or acquisition method. The appropriate selection of the extension factor may also be outside the preferred range described above and is within the capabilities of those skilled in the art. Such paths can also be combined to form data packets. If, for example, in a marker arrangement with only two marker units, parallel straight paths are excluded, a similar reverse criterion can be used.

[0056] If these criteria are applied to the image of the lighting fixtures, and especially if these criteria and / or further criteria are combined, the data packets for selecting paths can be chosen solely based on the evaluation of the lighting fixture image. This effectively leads to efficient pre-selection of lighting fixtures belonging to the marker arrangement. Pre-selection of lighting fixtures, since their arrangement essentially serves as lighting for marker units, significantly increases the evaluation speed because the time-consuming direction and position detection calculations can be limited to possible marker groups. This allows for practical real-time applications within detected motion sequences, such as determining when a marker arrives at a certain location (tracking).

[0057] Knowing the arrangement and number of marker units in the marker arrangement allows for the determination of further selection criteria. For example, in one embodiment of the device, a path arrangement can be searched in the image if a total of three marker units are combined into a marker arrangement, and each side of a marker unit of the second marker unit type has a marker unit of the first marker unit type. Specifically, paths can be selected from a selected path data packet, where at least one path of lighting devices with different relative arrangements is arranged between two straight paths of lighting devices with similar relative arrangements. Thus, the middle path can be a path of the second marker unit type, while the outer straight paths can be paths of the first marker unit type. As a further criterion for paths of the first marker unit type, if the first and second types of marker units can be so clearly distinguished, then the relative proportion of the spacing between the lighting devices on the path can also be a criterion. Paths that satisfy this classification criterion can then be assigned to possible marker unit types. This is particularly applicable if all marker units have substantially the same marker unit arrangement according to a particularly preferred embodiment of the invention.

[0058] According to a more general approach, the invention can be configured such that path allocation for a second marker unit type is performed before a determined straight path is assigned to a first marker unit type. This includes communication elements, the arrangement of which allows for the explicit identification of the marker arrangement. According to the invention, the marker arrangement and the relative arrangement of the marker units in the system are known in the evaluation unit.

[0059] By understanding the arrangement of the first and second type of marker units within the identified marker arrangement, paths potentially belonging to the identified marker arrangement can be determined in the captured image. Therefore, marker units belonging to this type of marker arrangement can be searched specifically near the second type of marker units. In practice, this also allows for rapid and reliable identification of potential marker units through image evaluation of the recorded images. Therefore, according to a particularly preferred embodiment, paths belonging to the marker type of the second marker unit can be selected from the determined straight paths, and the identifier of the marker arrangement can be derived from the communication elements of the second marker unit. Thus, the structure of the marker unit and the arrangement of the markers on the marker unit are known.

[0060] In a particularly preferred embodiment, a search area can thus be defined in the recorded image based on the markings of the second type of marking unit, and markings can be searched within that search area. If lighting devices are found in the search area of ​​the recorded image, they are assigned as markings to the marking unit and can be used to determine direction and location.

[0061] In a simplified embodiment, the straight paths of the second type of marker units can be identified by the fact that they have a significantly different proportion from the paths of the first type of marker units in the arrangement of the lighting device.

[0062] If the second type of marker unit has exactly two markers as end markers of the lighting device path, and at least two communication elements arranged on the path between the markers, and the two markers, i.e., the end markers, are identical for all second type of marker units in the device (i.e., the entire system or device), then a simpler and more secure detection possibility arises. On the other hand, the communication elements provided between the (end) markers are different for each marker unit in the system.

[0063] According to the invention, these second-type marker units can be readily determined by selecting from a set of defined straight paths those that share the same endpoints (detected illumination devices) in the captured image. In the case of at least two communication elements, at least two straight paths with the same endpoints and different illumination devices are correspondingly found on the path between the endpoints. Thus, the endpoints (illumination devices) are assigned the "marking" function, and the illumination devices located between the endpoints are assigned the "communication element" function. Therefore, from an image, the second-type marker units can be reconstructed using the encoding of the communication elements, and the encoding can be read out, for example, by comparing it with a known encoding pattern. This can be done solely from the captured image without the need for orientation and / or position detection.

[0064] The communication element can display a unique identifier (identification) of the tag unit, through which the tag arrangement can be uniquely identified, for example. Furthermore, according to the invention, the communication element can also be used to convey other information, such as information from several consecutively recorded images, by assembling and decoding coded data in the evaluation unit. This information may include, for example, information about the tool to which the tag unit is connected, such as the tool's battery status, the functions performed by the tool, or detected data values. Those skilled in the art are familiar with the specific possibilities in this regard, for example, from DE102019114531A1.

[0065] According to a preferred embodiment of the invention, after determining a straight path belonging to the second marker unit type and knowing the arrangement of the marker units of the now determined marker arrangement, a search area can be defined in which straight paths belonging to the first marker unit type are searched. If such a marker unit is found, the marker arrangement can be considered reliably identified. Now, by assigning the "marker" function to the lighting devices detected in the image, the lighting devices detected in the image are now assigned to the remaining markers of the marker unit as much as possible.

[0066] Then, based on the understanding of the arrangement of the marks on the mark arrangement, these marks are used, for example, by the algorithm described above, to determine the direction and position of the mark arrangement.

[0067] In cases where the arrangement of marker units in a marker arrangement is not known in advance, for example, because the individual marker units of the first and second types of marker units are not collinearly fixed to the object in a substantially arbitrary manner to form the marker arrangement, preferably at least one marker is not located in a common plane with the remaining markers of the marker units in the marker arrangement, the present invention proposes a method, particularly relating to a proposed method for detecting the orientation and position of markers in three-dimensional space, which is performed at least once before performing the previously described orientation and position detection method to understand the position of the markers and communication elements.

[0068] The method according to the invention is used to teach the arrangement of markers and communication elements in a marker arrangement, wherein at least one marker unit of a first marker unit type and at least one marker unit of a second marker unit type are positioned, each marker unit having at least three optically active illumination devices arranged along a path. In the marker units of the first and second marker unit types, the arrangement of the illumination devices used as markers or communication elements is known in advance. It is preferable to use marker units of the first and second marker unit types described above.

[0069] The following steps are performed in the method according to the present invention.

[0070] - A marker arrangement with fixed marker units is moved relative to an image capturing device, and a series of images are acquired during the movement, wherein all marker units are visible to the image capturing device for a predetermined or pre-predictable minimum duration and with a predetermined or pre-predictable minimum rotation, i.e., visible in each image of the acquired image sequence. The relative movement of the marker arrangement can be a translation and / or rotation performed by the marker arrangement or the image capturing device.

[0071] - Detecting potential marker arrangements in an image sequence. According to the invention, this can be accomplished through image evaluation related to the previously described method, specifically detecting straight paths in the recorded image that have a total of three illumination devices, and then assigning the straight paths to potential marker units of either a first marker unit type or a second marker unit type.

[0072] - Tracking latent marker units across multiple images in an image sequence. According to the invention, tracking of latent marker units can be achieved such that the motion is small compared to the frame rate (i.e., correspondingly slow rotation and / or translation occurs according to the invention). Then, in the context of image detection, confidence can be considered based on the fact that the latent marker units and their distances do not change relatively.

[0073] - Using the basic and already explained RANSAC method, it is possible to establish initial relationships between subarrays and improve the relationships of the entire image sequence through numerical optimization iterations until the full relationship between the entire image sequence and rotational motion is known.

[0074] - The arrangement of the marker cells is derived from all the relationships based on geometric allocation. This procedure is known to those skilled in the art in image evaluation methods.

[0075] This determines the arrangement of the marker units in the marker arrangement, and then the orientation and position detection method described above can be performed. According to the present invention, a method for teaching the arrangement of markers and communication elements in the marker arrangement can be performed at least once before repeatedly performing the orientation and position detection of the markers, if the arrangement is not yet known.

[0076] To avoid interference and improve accuracy, during the implementation of the proposed method for teaching the arrangement of markers and communication elements in a marker arrangement, it is best to ensure that no lighting devices are arranged in the image area of ​​the image capture unit during the acquisition of image sequences, and that these lighting devices are not arranged in a marker unit. Attached Figure Description

[0077] Further advantages, features, and application possibilities of the invention will become apparent from the following description and drawings of preferred embodiments. Therefore, all described and / or depicted features, or any technically reasonable combination thereof, are subject to the subject matter of this invention, independent of their summary in the described or depicted embodiments or claims.

[0078] The attached diagram shows:

[0079] Figure 1 A preferred embodiment of the device according to the invention is illustrated.

[0080] Figure 2 yes Figure 1 The side view showing the arrangement of the markings.

[0081] Figure 3 This is a side view of another embodiment of the marking arrangement according to the present invention.

[0082] Figure 4 The illustration shows an image taken during a preferred embodiment of the method according to the invention, in which two markings are arranged.

[0083] Figure 5 This schematically demonstrates the implementation used according to Figure 4 A preferred embodiment of the image method determines a straight path with three lighting devices.

[0084] Figure 6 yes Figure 1 The conceptual representation of the spatial arrangement of markers and communication elements in the marker arrangement; and

[0085] Figure 7 It is similar to Figure 3 The spatial arrangement of markers and communication elements in the illustrated marker layout is a conceptual representation. Detailed Implementation

[0086] Figure 1 The apparatus 1 is illustrated in diagram form. This apparatus 1 is used to detect the orientation and position of marks in a three-dimensional space with a mark arrangement 2. In this example, the mark arrangement 2 includes a total of three mark units 11, 12, each mark unit 11, 12 having an illumination element 10 arranged along a path 13. The illumination element 10 is optically active when it is turned on and emits light, or preferably when emitting infrared light. The illumination element 10 can be configured, in particular, as an LED capable of being turned on or off, and can be arranged, for example, on a circuit board. In the optically active state, the illumination element 10 is also referred to as an illumination device 20, which... Figures 4 to 7 Displayed in dark black. Figure 1In the process, all lighting elements 10 are in the off state. When the device 1 is in the working state, each marking unit 11, 12 has at least three lighting devices 20, i.e., the lit lighting elements 10, wherein the lighting devices 20 are designed as marking 21 and / or communication elements 22.

[0087] The device 1 includes an optical image capture unit 3, which is specifically designed as a camera (digital camera), and the optical image capture unit 3 is calibrated in space. The image capture unit / camera 3 is configured to acquire an image 90 of the marker arrangement 2. The camera 3 is connected to an evaluation unit 4, which is adapted in a manner already described to accurately determine the orientation and position of the marker arrangement 2 from an image 90 from the optical image capture unit 3.

[0088] The marking arrangement 2 has two marking units 11 of the first marking arrangement type, each marking unit 11 having three lighting elements 20 in an operational state. Therefore, in the example shown here, all three lighting elements 10 of the two marking units 11 are turned on and function as markings 21. This function... Figure 1 The markings are represented by the cross symbols shown in the lighting element 10. Therefore, all the marking units 11 in the work form three lighting devices 20 in exactly the same position. This can usually be identified by the relative spacing of the lighting devices 20 in the captured image.

[0089] Between the two marker units 11 of the first type of marker unit, marker units 12 of the second type of marker unit are arranged in the marker arrangement 2, wherein, in operation, the marker unit 12 has exactly two markers 21 (displayed as cross marks in the illumination element 10) and at least one communication element 22. The function of this "communication element" is... Figure 1 The symbol "X" in the lighting element 10 is used to indicate this. However, in terms of hardware, the lighting element 10 is preferably designed the same, regardless of its function as a "marker" or "communication element".

[0090] In the second type of marker unit 12, two lighting devices 20, serving as markers 21, are arranged at both ends of the marker unit; therefore, the two lighting devices 20 are also referred to as end markers. This limitation applies universally herein. Between these (exactly two) markers 21, at least one lighting device 20 of the marker unit 12 is arranged, preferably at least two to five lighting devices 20, and these lighting devices 20 function as communication elements 22.

[0091] According to a preferred embodiment of the device 1 of the invention, the communication element 22 of at least one tag unit 12 of the second type of tag arrangement 2 is used to uniquely identify the tag arrangement 2. This is achieved by the fact that in the device 1, all the tag units 2 used are configured to display a unique combination of communication elements 22.

[0092] All illumination elements 10, and corresponding all marks 21 and (if applicable) communication elements 22 of mark units 11, 12, are located on path 13, and (all) communication elements 22 are arranged between two marks 21 of mark unit 12. This is beneficial for evaluation. Furthermore, the distances between the end marks (in the sense defined above) are different for mark units 11 of the first mark unit type and mark units 12 of the second mark unit type; preferably, the distance between the end marks of the second type of mark unit 12 is smaller than the distance between the end marks of the first type of mark unit 11. Moreover, all mark units 11, 12 of mark arrangement 2 are arranged in parallel, that is, the paths 13 of their illumination elements 10 are parallel. A particularly preferred variation is the illustrated one, in which the first mark unit type mark unit 11 spans a rectangle, and the second mark unit type mark units 12 are arranged symmetrically along the central axis of the rectangle. The resulting arrangement of marks 21 can be easily identified in the recorded image 90 by appropriate selection criteria and selected for evaluation.

[0093] According to Figure 2 As can be seen in the side view of the marking unit 2, the second type of marking unit 12 is arranged at a higher position between the first type of marking units 11. This ensures that the marking unit 12 is not arranged coplanarly with the two marking units 11, which span a plane.

[0094] Knowing the arrangement of the marker units 11 and 12 in the marker arrangement 2, the evaluation unit 4 is adapted to identify the marker unit 11 of the first marker unit type and the marker unit 12 of the second marker unit type, or equivalently to identify their marker 21, and (preferably specifically) to determine the orientation and position of the marker arrangement 2 using the marker 21 of the identified first marker unit type and second marker unit type marker units 11 and 12.

[0095] Figure 3A multi-marker arrangement 5 is shown, in which several marker arrangements 2 are arranged in the shape of a hollow cylinder, such that the illumination element 10 is at least approximately located on an imaginary cylindrical surface 6. Therefore, the multi-marker arrangement 5 represents an embodiment of a protrusion, which can reliably detect the orientation and position of the multi-marker arrangement 5 even if there is only one camera 3 in space at almost any location of the multi-marker arrangement 5. Therefore, a preferred application of such a multi-marker arrangement 5 is that it is fixed around a working tool.

[0096] Reference Figure 4 and Figure 5 In the context of applying the proposed method for determining orientation and position, the basic procedure for image evaluation is described below.

[0097] Figure 4 An image 90 from a camera or optical image capture unit 3 is shown, which is fed to an evaluation unit 4 for evaluating and performing the method according to the invention.

[0098] Image 90 shows two marker arrangements 2, 2' located at different distances and orientations in space. In particular, marker arrangements 2, 2' can be fixed to objects not shown in space, such as tools or main body parts, to enable the tracking of motion sequences.

[0099] In the image, only the lighting device 20 is visible as a light point. This applies whether it's the path 13 where the lighting device 20 is arranged, or... Figure 4 The illumination elements 10, which are shown as dashed circles at the positions of the marked units 11 and 12 (not turned on, and therefore in a non-optically active state), are not visible. Furthermore, all the illumination devices appear identical in image 90. Different shapes (circular for marker 21, square for communication element 22) cannot be discerned from the image itself. This information is primarily determined during the image evaluation process described below.

[0100] In one processing step, all illumination devices 20 in image 90 are detected and assigned two-dimensional image coordinates (x, y, x). B y B This allows for the identification of each lighting device 20 in the captured image 90.

[0101] Subsequently, all straight paths 30-01, 30-02, 30-03, 30-04, 30-05, 30-06 and 30-11, 30-12, 30-13, 30-14, 30-15, 30-16 are identified in image 90. Each straight path has a total of three lighting devices 20, such as... Figure 5The diagram illustrates this. All parallel straight paths 30 combine to form data packets 31-1 and 31-2, which may belong to marking arrangements 2 and 2'. Furthermore, straight paths 30-01, 30-02, 30-03, 30-06, 30-11, 30-12, 30-13, and 30-16 are selected from data packets 31-1 and 31-2; these straight paths are of approximately the same length, i.e., there is a considerable distance between their endpoints (end markers). In the example shown, straight paths 30-04 and 30-05, and 30-14 and 30-15 are significantly shorter. Therefore, these straight paths cannot be assigned to any marking arrangement 2 and 2' and are removed from data packets 31-1 and 31-2, respectively.

[0102] Among the remaining straight paths 30 in data packets 31-1 and 31-2, straight paths 30-02, 30-03, and 30-12, 30-13 of the lighting devices 20 are selected that have the same endpoints (i.e., endpoint markers in the sense of the lighting devices 20 arranged at the ends of the straight path) and different positions between the endpoints. It is worth noting that these paths 30-02, 30-03, and 30-12, 30-13 are superimposed in image 90 (corresponding to...). Figure 4 Their endpoints have the same image coordinates (for each data packet 31-1 and 31-2 respectively). These paths are in Figure 5 The information is displayed side-by-side to illustrate this assessment step.

[0103] Therefore, paths 30-02 and 30-03 can be assigned to a second type of tag unit 12 of tag arrangement 2, while paths 30-12 and 30-13 can be assigned to a second type of tag unit 12 of another tag arrangement 2'. Thus, tag 21 and communication element 22 can be assigned. Furthermore, tag unit 2 or 2' can be clearly identified from the pattern of communication element 22.

[0104] The lighting devices 20 for the remaining paths 30-01, 30-06, 30-11, and 30-16 are assigned according to the marking unit 11 of the first type. Then, the lighting devices 20 are the corresponding markings 21.

[0105] Based on the marker 21 now assigned to each detected marker arrangement 2, 2', the evaluation unit 4 calculates the orientation and position of the marker arrangements 2, 2' in a manner that has been explained and is generally known to those skilled in the art.

[0106] Figure 6 The three-dimensional arrangement of markers 21 and communication elements 22 in marker arrangement 2 is shown, with five communication elements 22.

[0107] Figure 7The diagram shows the three-dimensional arrangement of the marker 21, designed as a convex marker arrangement 7, and the communication element 22, which is in conjunction with... Figure 3 The arrangement of the multiple markers shown is basically similar to that of 5.

[0108] Markers 21 and communication elements 22 are disposed on a cylindrical surface 6, on which first-type marker units 11 and second-type marker units 12 are always arranged alternately. In addition to the markers 21 at opposite ends, each marker unit 12 has five communication elements 22, each with a different code, so that even if only one second-type marker unit 12 is picked up by the image capture unit 3, orientation identification of objects connected to the marker arrangement 7 can be performed.

[0109] Figure label:

[0110] 1 device

[0111] 2, 2' Marker Arrangement

[0112] 3. Image capture unit of digital camera

[0113] 4. Evaluation Unit

[0114] 5. Multi-marker arrangement

[0115] 6. Imaginary cylindrical surface

[0116] 7. Arrangement of marks forming convex bodies

[0117] 10 Lighting elements

[0118] 11. First type of tag unit

[0119] 12. Marking unit of the second marking unit type

[0120] 13 Paths

[0121] 20 lighting fixtures

[0122] 21 Mark

[0123] 22 Communication Components

[0124] 30. Straight path derived from image evaluation

[0125] 31 Data packets consisting of straight paths

[0126] 90 captured images

Claims

1. An apparatus for detecting the orientation and position of a marker (21) in three-dimensional space, comprising: - At least one marking arrangement (2, 7), the marking arrangement (2, 7) comprising at least two marking units (11, 12), wherein lighting devices (20) are arranged along a path (13), each marking unit (11, 12) having at least three lighting devices (20), the lighting devices (20) being designed as markings (21) and / or communication elements (22), - At least one optical image capturing unit (3), adapted to capture an image (90) of the marked arrangement (2, 7), and - Evaluation unit (4), adapted to determine the orientation and position of the marker arrangement (2, 7) from exactly one image (90) obtained from one of the optical image capture units (3), Its features -At least one tag unit (11) belongs to a first tag unit type having at least three tags (21), - At least one additional tag unit (12) belongs to a second tag unit type, which has exactly two tags (21) and at least one communication element (22), the at least one communication element (22) being arranged between the two tags (21). - At least one marker unit (11) of the first marker unit type and at least one marker unit (12) of the second marker unit type are arranged in a non-coplanar manner. - The evaluation unit (4) is adapted to identify the marking units (11, 12) of the first marking unit type and the second marking unit type, and to clearly determine the direction and position of the marking arrangement (2, 7) using the markings (21) of the marking units (11, 12) identified by the first marking unit type and the second marking unit type.

2. The apparatus of claim 1, wherein, The marking arrangement (2, 7) includes two marking units (11) of the first marking unit type and one marking unit (12) of the second marking unit type.

3. The apparatus of claim 2, wherein, The second type of marker unit (12) is arranged outside the plane spanned by the two first type of marker units (11).

4. The apparatus of any one of claims 1 to 3, wherein, The paths (13) of the lighting devices (20) of the marking units (11, 12) of the marking arrangement (2, 7) are arranged parallel to each other.

5. The apparatus of any one of claims 1 to 3, wherein, In the marker arrangement (2, 7), a marker unit (11) of the first marker unit type is always arranged adjacent to a marker unit (12) of the second marker unit type.

6. The apparatus of any one of claims 1 to 3, wherein, Several mark arrangements (2) are combined to form a common structure in which each mark arrangement (2) is arranged relative to each other in a defined manner and forms a multi-mark arrangement (5), wherein the outer contour of the common structure forms a convex body.

7. A method for detecting the direction and position of a marker (21) in a three- dimensional space, using a device (1) according to any one of claims 1 to 6, characterized in that, The method includes the following steps: - Capture an image (90) of at least one marker arrangement (2, 7), the marker arrangement (2, 7) including at least one marker unit (11) of a first marker unit type and at least one marker unit (12) of a second marker unit type, each marker unit including at least three lighting devices (20) arranged along the path (13); - Detect the lighting device (20) in the captured image (90); - The detected lighting device (20) is assigned to at least one tag unit (11) of a first tag unit type, wherein the lighting device (20) is specifically designed as a tag (21); - The detected lighting device (20) is assigned to at least one tag unit (12) of the second tag unit type, wherein the lighting device (20) is designed as a tag (21) and a communication element (22); - Assign the marking function and / or communication element function to the lighting device (20) detected in the image (90); -Based on the understanding of the arrangement of the marker (21) on the marker arrangement (2, 7), determine the direction and position of the marker arrangement (2, 7).

8. The method according to claim 7, characterized in that, Assigning the detected lighting devices (20) to the marking units (11, 12) of the first marking unit type or the second marking unit type includes the step of determining all straight paths (30) in the captured image (90), the straight paths (30) having exactly three lighting devices (20) in total.

9. The method according to claim 8, characterized in that, From all the determined straight paths (30), identify those paths whose endpoints are equidistant from each other.

10. The method according to claim 8 or 9, characterized in that, Identify the parallel straight paths from all the determined straight paths (30).

11. The method according to claim 8 or 9, characterized in that, Select a straight path belonging to the second marker unit type (12) from the determined straight path (30), and obtain the identifier of the marker arrangement (2, 7) from the communication element (22) of the marker unit (12) of the second marker unit type.

12. The method according to claim 11, characterized in that, A search area is defined around a straight path (30) belonging to the second marker unit type, in which a straight path (30) belonging to a marker unit (11) of the first marker unit type is searched.

13. A method for teaching the arrangement of markers (21) and communication elements (22) in a marker arrangement (2, 7), wherein at least one marker unit (11) of a first marker unit type and at least one marker unit (12) of a second marker unit type are positioned, each marker unit having at least three lighting devices (20) arranged along a path (13), the arrangement of the lighting devices used as markers (21) or communication elements (22) in the first and second marker unit types (11, 12) can be known in advance, characterized by the following steps: - The marker arrangement (2, 7) with fixed marker units (11, 12) is moved relative to the image capture unit (3), and the image sequence is captured during the movement; -Detect the potential label arrangement (2, 7) in the image (90) of the image sequence; - Track potential labeled units (11, 12) on several images (90) of the image sequence; - Create initial relationships between the labeled units (11, 12) and iteratively improve the relationships over the entire image sequence until all motion-related relationships over the entire image sequence are known; - The arrangement of the marker units (11, 12) is derived from all relations based on geometric allocation.

14. The method according to claim 13, characterized in that, During the recording of the image sequence, no lighting device (20) is arranged in the image area of ​​the image capture unit (3), and the lighting device (20) is not arranged in the marking units (11, 12).

Images