Pose determination in parallel kinematics using reference markers

Abstract

A parallel kinematic system comprising: a camera (110) and a marking area (150) with mutually distinguishable markings, wherein the camera (110) is configured to observe the marking area (150) at different poses of the parallel kinematic system, wherein the pose of the parallel kinematic system can be determined based on an image of the marking area taken with the camera (110), if the image contains at least n arbitrary markings in one direction, where n is greater than or equal to 1;where a distance, D, between any two markings adjacent in direction satisfies the following formula: FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn,wobeitmdie Länge einer der Markierungen ist,FOVmineine Länge des Ausschnitts des Markierungsbereichs (150) ist, der bei einem Minimalabstand der Kamera (110) von dem Markierungsbereich (150) in das Sichtfeld (400) der Kamera (110) fällt, wobeider Minimalabstand ein minimaler Abstand unter den Abständen ist, die der Markierungsbereich (150) von der Kamera (110) durch Änderungen der Pose entfernt sein kann.;

Description

[0001] The present invention relates to devices and methods for determining the pose of parallel kinematics.

[0002] A common problem in controlling and regulating a robot is determining its current pose (position and orientation).

[0003] Especially in parallel kinematics, the pose—that is, the position and orientation of, for example, the moving work platform of a hexapod—often cannot be calculated directly and / or exactly from the lengths of the driven legs or joint angles. Instead, numerical methods, such as iterative optimization procedures, are necessary. However, these methods are time-consuming and computationally intensive, and the achievable accuracy depends significantly on the initial estimate used. Furthermore, they are usually based on measuring the leg lengths and / or joint angles using internal sensors (e.g.,...)Incremental sensors for measuring the leg lengths of a hexapod fail to detect influences such as misalignment, deformation, play, and backlash in the legs, joints, and / or the moving platform itself (even with absolute sensors in the legs), and therefore cannot be taken into account by any subsequent numerical method. Furthermore, incremental sensors, for example, require complex reference runs to achieve a zero position for the legs, and there are specific parallel kinematic systems for which no numerical methods are available.

[0004] However, if external optical incremental and absolute sensors are used, a large number of sensors are usually necessary to measure all degrees of freedom. Likewise, due to the limited measuring range of the sensors, large positioning movements of the parallel kinematics are often not achievable.

[0005] When using 6D measurement technology via photogrammetry, a large number of images from as many different positions as possible are often necessary to generate an accurate 3D scan. Since a comparatively large area needs to be captured, only relatively low accuracy is achievable, or an extremely high camera resolution is required to detect changes in position, e.g., in the nanometer range.

[0006] Moldagalieva, Akmaral, et al. “Computer vision-based pose estimation of tensegrity robots using fiducial markers.” 2019 IEEE / SICE International Symposium on System Integration (SILM). IEEE, 2019, discloses a method based on “fiducial tags” that attempts to keep the entire workspace of the robot within the camera's field of view. This necessitates a very large camera field of view, requiring the camera to be positioned quite far from the moving work platform. Since the achievable accuracy decreases with increasing distance between the camera and the tag, given a certain camera resolution, this method can only determine the kinematic pose with low accuracy.

[0007] The present invention is therefore based on the objective of improving the determination of the pose of parallel kinematics.

[0008] The problem is solved according to the invention by the features of the independent claims. Some advantageous embodiments are the subject of the dependent claims.

[0009] The invention is based on the idea of ​​placing several reference markers on a parallel kinematic system in such a way that at least one of the reference markers or a certain minimum number of reference markers is always in the camera's field of view.

[0010] According to a first aspect of the present invention, a parallel kinematic system is provided. The parallel kinematic system comprises a camera and a marking area with mutually distinguishable markings, wherein the camera is configured to observe the marking area in different poses (or even in all possible poses) of the parallel kinematic system. The pose of the parallel kinematic system can be determined based on an image of the marking area captured by the camera, provided that the image contains at least n arbitrary markings in a direction, where n is greater than or equal to 1. The distance D between any two markings adjacent in that direction satisfies the formula [formula missing in original text]. FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn, where t m the length of one of the markers is FOV minthe length of the section of the marking area that falls into the camera's field of view at a minimum distance of the camera from the marking area, and the minimum distance is the minimum distance among the distances that the marking area can be from the camera by changes in pose.

[0011] According to a second aspect of the present invention, a method is provided for applying mutually distinguishable markings to a parallel kinematic system in a marking region, such that the pose of the parallel kinematic system can be determined based on an image of the marking region if at least a predetermined number, n, of markings are located in a direction on the image. The method comprises a step of determining the distance D between any two markings adjacent in direction according to the formula: FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn, where t mthe length of one of the markers is FOV min The length of the portion of the marking area that falls within the camera's field of view at a minimum distance from the camera, where the minimum distance is the minimum distance among all the distances the marking area can be from the camera by changing its pose. The method further comprises a step of applying adjacent markings at the specified distance.

[0012] According to a third aspect of the present invention, a parallel kinematic system is provided. The parallel kinematic system comprises a camera and a marking area with mutually distinguishable markings attached thereto, wherein the camera is configured to observe the marking area in different poses (or even in all possible poses) of the parallel kinematic system. The distance D between any two markings adjacent in one direction satisfies the formula D≤FOVmin−(n+1)∗tmn, where t m the length of one of the markers is FOV minThe length of the portion of the marking area that falls within the camera's field of view at a minimum distance from the marking area is defined as the minimum distance among all possible distances that the marking area can be from the camera by changing the pose. Furthermore, the markings are located in different planes, and the pose of the parallel kinematics can be determined based on an image captured by the camera, provided that this image contains at least a number, n, of markings in the direction, where n is greater than or equal to 2.

[0013] According to a fourth aspect of the present invention, a method is provided for applying mutually distinguishable markings to a parallel kinematic system in a marking area, such that the pose of the parallel kinematic system can be determined based on a camera image of the marking area if at least a predetermined number, n, of the markings are located in a direction on the image, where n is greater than or equal to 2. The method comprises a step of determining the distance D between any two markings adjacent in direction according to the formula: D≤FOVmin−(n+1)∗tmn, where t m the length of one of the markers is FOV minThe length of the portion of the marking area that falls within the camera's field of view at a minimum distance from the camera, where the minimum distance is the minimum distance among all the distances that the marking area can be from the camera by changing its pose. The method further comprises a step of placing adjacent markings at the specified interval, positioned such that the markings are located in different planes.

[0014] In general, in embodiments of the first to fourth aspects, the length FOV can min the equation FOVmin=(gminƒ−1)lSensor suffice, whereby g min the minimum distance is, l Sensor where f is the length of the camera's sensor, and f is the camera's focal length.

[0015] In general, in embodiments of the first and third aspects, the markings can be arranged in the marking area according to a regular arrangement scheme.

[0016] In general, in embodiments of the second and fourth aspects, the markings can be applied in the marking area according to a regular arrangement scheme during the application step.

[0017] In general, in embodiments of the first and third aspects, the marking area can be located on the underside of the working platform of the parallel kinematics, and the camera can be mounted in or on a base of the parallel kinematics and directed towards the underside of the working platform.

[0018] In general, in embodiments of the first and third aspects, the marking area can be located in or on a base of the parallel kinematics, and the camera can be mounted on the underside of the working platform of the parallel kinematics, and the camera can be directed towards the base of the parallel kinematics. Alternatively, in embodiments of the first and third aspects, in the step of mounting the marking area, it can be located in or on a base of the parallel kinematics, and the camera can be mounted on the underside of the working platform of the parallel kinematics and directed towards the base of the parallel kinematics.

[0019] In general, in embodiments of the second and fourth aspects, in the step of attaching the marking area, this can be attached to the underside of the work platform of the parallel kinematics, and / or the camera can be attached in or to a base of the parallel kinematics and pointed towards the underside of the work platform. Alternatively, in embodiments of the second and fourth aspects, in the step of attaching the marking area, this can be attached in or to a base of the parallel kinematics, and the camera can be attached to the underside of the work platform of the parallel kinematics and pointed towards the base of the parallel kinematics.

[0020] In general, in embodiments of the first to fourth aspects, the length t m a marking of the equation tm≥px∗p∗tb∗(gmaxƒ−1) suffice, where p is a camera-dependent value greater than or equal to 2 and less than or equal to 5, px is the length corresponding to a sample value from the camera, g max where f is the maximum distance among the distances that the marking area can be from the camera by changing the pose, f is a focal length of the camera, and t b The number of information units in the marker.

[0021] In general, in embodiments of the first to fourth aspects, the markings may be reference markers, such as ARToolKit markings, ArUco markings, QR codes or, in particular, AprilTag markings.

[0022] In general, in embodiments of the first to fourth aspects, each of the markings can consist of several squares, the squares corresponding to the units of information and each square containing a bit that can be encoded.

[0023] Further details, advantages and features of the invention will become apparent from the following description and the drawings, to which express reference is made with regard to all details not described in the text. The drawings show: Fig. 1a and b schematic three-dimensional representations of an exemplary parallel kinematic system. Fig. 2a and b schematic sectional views of an exemplary parallel kinematic system. Fig. 3. A flowchart showing exemplary steps for placing markers in the marking area. Fig. 4 A schematic representation of the visible portion of the marking area when the pose of the kinematics changes. Fig. 5 A schematic representation of a minimum field of view of a marking area at various deflections in the x and y directions and a corresponding maximum field of view; the distances between the markings are chosen so that at least 1 and in the limiting case 4 markings are fully visible. Fig. 6 A schematic representation of a minimum field of view at various deflections in the x and y directions and a corresponding maximum field of view; the distances between the markings are chosen so that at least 4 and in the limiting case 9 markings are fully visible. Fig. 7a and b schematic representations of marking areas in which the markings are evenly spaced. Fig. 8 is an example marking. Fig. 9 the field of view of a marking area where, in the limiting case, more (i.e., (n x + 1) = (n y+ 1) = 3 in the x- or y-direction) as the desired minimum number n x = n y = 2 markings in the x and y directions are fully visible. Fig. 10 a field of view which is slightly deflected downwards and to the left compared to the field of view in Fig. The number is shifted by 9; therefore, there are only n again. x = n y = 2 markings in the x and y directions are fully visible. Fig. 11a the visibility area of ​​a marking area where, in the limiting case, more (i.e. a total of 4) than the desired minimum number n x = n y = 1 where markings in the x and y directions are fully visible; Fig. 11b the field of vision from Fig. 11a after increasing the distances between the markings in the x and y directions by 2t m , which means that in borderline cases no marking is fully visible anymore. Fig. 12 a schematic representation for determining the distance between the markings when taking rotations into account. Fig. 13a to Fig. d Marking areas where the markings are arranged in different planes. Fig. 13e a schematic representation of a periodic pattern according to which the markings can be placed in three different levels. Fig. 14 a schematic representation of the enlargement of the work area that can be covered by using multiple levels in which markers are placed.

[0024] The present invention relates to parallel kinematics on which markings are attached, and to methods for attaching markings to parallel kinematics. Parallel kinematics

[0025] In robotics, a fundamental distinction is made between the main classes of serial and parallel kinematics. Hybrid kinematics, which combine elements of both, also exist. While serial kinematics consist of a series of links (e.g., linear and / or rotary axes) forming an open kinematic chain, the parallel kinematics considered in this application consist of a number of closed kinematic chains. In practice, parallel rod kinematics, rod actuators, and / or rotary actuators are frequently used for the parallel axes of motion, coupling two planes that move relative to each other. Thus, each drive is directly connected to the (end) effector (e.g., a tool carrier). As a result, unlike serial kinematics, the drives are not burdened with the masses of all subsequent links and drives.Since all drives move simultaneously, i.e., in parallel to each other, the loads are distributed more evenly across all guide elements. The resulting low moving masses enable extreme dynamics with high speeds and accelerations, while maintaining high mechanical precision. Another difference from serial mechanisms is that in parallel kinematics, the drives, especially the motors and gearboxes, remain stationary. This optimizes not only the dynamics and performance of such robots but also their energy efficiency. Parallel kinematics are therefore frequently used when simple motion sequences with high repeatability and speed are required. Classic examples of parallel kinematics are hexapods and delta robots.It should be noted at this point that the example of a hexapod frequently used in the present application serves only for illustration and that what has been said generally also applies to other parallel kinematics. Examples of implementation

[0026] According to one embodiment of the present invention, a parallel kinematic system is provided. The parallel kinematic system comprises, as exemplified in Fig. 1, Fig. 2a and Fig. Figure 2b shows a camera 110 configured to observe a marking area 150 of the parallel kinematics moving in sync with a pose of the parallel kinematics. The parallel kinematics further comprise mutually distinguishable markings attached to the parallel kinematics in the marking area 150.

[0027] The markings are located in the marking area either: - placed with a distance D between each pair of adjacent markings in one direction, which corresponds to the formula FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn, suffices, wherein the pose of the parallel kinematics can be determined based on an image of the marking area taken with camera 110, if the image contains at least n arbitrary markings in the direction, where n is greater than or equal to 1; or - placed in different planes, with the distance D between any two adjacent markings of the formula in one direction D≤FOVmin−(n+1)∗tmn suffices, and the pose of the parallel kinematics can be determined based on an image taken with the camera (e.g., any one or any single one), provided that this image contains at least a number, n arbitrary, of the markings in the direction, where n is greater than or equal to 2.

[0028] Here, as explained in more detail below, t refers to mthe length of one of the markers, and FOV min a length of the minimum field of view of the camera 110.

[0029] Accordingly, in another embodiment, a method for applying mutually distinguishable markings to a parallel kinematic system in a marking area is provided. Such a method is described in Fig. 3 is shown and includes a step S310 of determining a distance in accordance with either the formula - FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn; or the formula - D≤FOVmin−(n+1)∗tmn.

[0030] The method further comprises a step S320 of placing adjacent markers in a given direction at a specified distance D such that the pose of the parallel kinematics can be determined based on a camera image of the marking area, provided that at least a predetermined number, n, of the markers are located in the direction shown in the image. In other words, if at least the predetermined number, n, of markers in the direction shown in an (arbitrary) captured image is present, the poses can be determined based on the image. In general, the markers in the placement step S320 can be placed in different planes, particularly if n is greater than or equal to 2 and / or the second of the two formulas above applies. D≤FOVmin−(n+1)∗tmn is used.

[0031] By directly determining the position or pose of the moving work platform using markers, it is no longer necessary to solve the forward kinematics using complex numerical methods (relieving the controller of this burden, although higher bandwidth is required depending on the frame rate and the identification time). Furthermore, influences such as offset, deformation, play, and backlash in the legs, joints, or the moving platform itself can be detected. Absolute position determination becomes possible, eliminating the need for reference runs. It also allows for direct control of the position of the moving work platform, rather than just the driven joints (e.g., only the leg lengths in a hexapod). Moreover, only one sensor (camera) is used to measure six degrees of freedom, thus eliminating the need for the complicated alignment of multiple sensors.In particular, it is possible to use only one sensor image, i.e., one sensor signal, which further simplifies pose determination. Since at least one marker (or the necessary minimum number) is visible in every pose of the parallel kinematics, such direct measurement can achieve high accuracy in pose determination across the entire working range of the robot.

[0032] It should be noted that the following detailed description applies equally to the parallel kinematics according to the invention as well as to the attachment methods according to the invention. camera

[0033] In general, parallel kinematic systems according to the invention can include a camera. However, the present invention is not limited to this, as parallel kinematic systems according to the invention can also include no camera. A parallel kinematic system can, for example, comprise only a mounting location, mounting device, and / or mounting bracket to which a camera can be attached according to the invention (i.e., attached such that the camera is directed towards the marking area). Likewise, it is possible that a parallel kinematic system is intended solely for use with a camera (of a specific focal length) positioned at a particular location and directed towards the marking area.

[0034] Furthermore, it should be noted that the term camera is to be understood broadly here and includes any devices for optical image recording, in particular both cameras with and cameras without optics (e.g. a “pinhole”).

[0035] Furthermore, the camera may include a suitable lens (possibly with an extension tube) that can, for example, be screwed onto the camera to focus it on the movable work platform. The term "camera" also includes such a potentially interchangeable lens. Therefore, when the present application refers to the focal length of the camera, this may include or be the focal length of an optical system and / or a lens. The same applies to the other camera parameters used here.

[0036] Furthermore, the camera can include one or more extension tubes that reduce the distance to the focal plane. The reduced working distance g' can be calculated using the following equations: gƒp'=11ƒ−1b+zr b=11ƒ−1gƒp where zr denotes the width of the intermediate ring.

[0037] The camera is set up to observe the marked area in different, or even all, possible poses of the parallel kinematics. The camera and the marked area can be positioned, but do not have to be, so that the marked area is observable in all possible poses. For example, it may not be intended to actually move to certain theoretically achievable poses and / or to precisely determine them using the markings. In particular, it may be that pose determination based on the markings and the camera is only intended for poses within a specific work area. This could, for example, be the work area for performing a specific task (or a specific part of a task) for which particularly precise pose determination is necessary.

[0038] Here, "observing" means that the camera is pointed, or can be pointed, at the marked area, i.e., it can take pictures of it. These pictures can be used, as explained below, to determine the pose of the kinematics. It should be noted, however, that this does not mean that the camera must always have the entire marked area in its field of view. The camera can, for example, as explained in more detail below, only have a portion of the marked area in its field of view.

[0039] The camera can be mounted in or on a base of the parallel kinematic system. The base could be, for example, a base plate or platform of the kinematic system or the hexapod, in or on which the camera can be mounted and / or fixed. The camera can be pointed towards the underside of the work platform, especially if the marking area is located there.

[0040] This is also in Fig. 1 as well as in Fig. 2a and Fig. Figure 2b illustrates this. As shown there, camera 110 is built into the base platform 120 and pointed towards the underside of manipulator platform 140, where the AprilTag array 150 is also located. It should be noted again here that lens 130 can also be considered part of camera 110.

[0041] The camera can then be positioned so that, in the hexapod's zero or "home" position, it is perpendicular to the moving platform (or the marking area). The zero position can, in particular, be a pose where the moving work platform is parallel to the base plate. Alternative positioning and / or orientation of the camera is also possible, as long as the marking area remains visible to the camera in different poses (or even in all possible poses).

[0042] As explained below in the description of the placement of the marking area, if the position of the camera and the marking area are "reversed", the camera can also be placed at a moving location, e.g. on the end effector, in particular the underside of the work platform. Field of view, minimum field of view (FOV) min , and maximum field of view (FOV) max

[0043] The field of view (the abbreviation "FOV" used here is derived from the English term "Field of View") refers to the area captured by the camera. It has the same shape as the corresponding sensor of the camera. For the sake of simplicity and clarity, only the case of a rectangular and / or square sensor will be explicitly described below, but the invention is not limited to such a sensor.

[0044] In the case of a rectangular sensor, the field of view has the same aspect ratio as the sensor, resulting in FOVx=(gƒ−1)∗Sensx and FOVy=(gƒ−1)∗Sensy, where f denotes the focal length of the camera, FO x The width of the field of view is called FOV. y the height of the field of view, sensitivity x the sensor width is designated, and Sens y The sensor height is referred to as the field of view. The field of view thus corresponds to the area or section of the marked area that the camera can observe at a specific distance from the marked area. A larger sensor can therefore "capture" a larger field of view at the same working distance or object distance g. In other words, the dimensions of the sensor are related to the inverse of the image scale. 1 / bg=(gƒ−1) scaled to preserve the field of view (b denotes, as usual, the image distance and 1 / f = 1 / b + 1 / g).

[0045] To ensure that one or more markers are always within the field of view, the minimum field of view can be used when determining the marker spacing as explained below. According to the formulas above, the minimum length FOV is calculated as follows: min of the field of vision FOVx,min=(gminƒ−1)∗Sensx and FOVy,min=(gminƒ−1)∗Sensy, Here, the distance g, also referred to as the minimum distance, is min The minimum, or smallest, distance among the distances that the marking area can be from the camera. The minimum distance g min This corresponds to the minimum object distance of the moving marking area that can be achieved by changing the pose (in other words, by moving the robot) within a specific, possibly application-limited area of ​​the pose space. Therefore, g must be... minThis does not refer to the absolute minimum distance if, for example, there is no intention to actually move to certain theoretically achievable poses and / or to precisely determine them using the markings. In other words, the minimum distance can refer to the smallest distance at which pose determination should be carried out based on the markings and the camera. It can, in particular, refer to the minimum working distance, i.e., the minimum distance required to perform a specific task for which precise pose determination is necessary.

[0046] The lengths FOV x,min and FOV y,minThese are accordingly the width and height of the portion of the marked area that falls within the camera's field of view at a minimum distance from the marked area. In addition to the sensor width and height, the camera's minimum field of view is therefore defined by the focal length f, which is determined, for example, by the lens used, and the minimum distance of the marked area from the camera.

[0047] In particular, if the sensor is square and / or if the marking area can rotate due to changes in the pose relative to the sensor, it may also be useful to use only one FOF length. min to work with the minimum field of view. In other words, the length of the minimum field of view, i.e., the length of the portion of the marking area that falls within the camera's field of view at a minimum distance from the marking area, can also be determined by the following formula. FOVmin=(gminƒ−1)lSensor, for l Sensor the smaller of the two lengths sensor x and sensor y to choose. In other words, FOV corresponds to min the smaller (or one that is not larger) of the two lengths FOV x,min and FOV y,min .

[0048] For the sake of simplicity, the following will often assume only a minimal FOV length. min , for example, the case of a square sensor. However, it should be noted that the following applies to FOV. min said generally also applies to FOV x,min , and FOV y,min This applies. In other words, FOV can be min to FOV x,min , to FOV y,min , or it concerns the smaller of the two lengths (unless it is clear from the context that this is not the case). The same applies to l. Sensor regarding sensor x and sensor y .

[0049] Analogous to the minimum field of view, the dimensions of the maximum field of view, which is the field of view at the maximum working distance g, can be max acts, will be determined to FOVx,max=(gmaxƒ−1)∗Sensx and FOVy,max=(gmaxƒ−1)∗Sensy,

[0050] The maximum working distance max This is the maximum distance among the distances that the marker area can be from the camera due to changes in pose. Analogous to the minimum distance mentioned above, this is the maximum distance of the marker area from the camera when the robot is moving.

[0051] If the sensor is square and / or the marking area can rotate due to changes in the sensor's pose, then... FOVmax=(gmaxƒ−1)LSensor are used, with L Sensor the longer of the two lengths sensor x and sensor y to choose. Marking area

[0052] In general, the marker area can be positioned so that it moves with the pose of the kinematics. The marker area can therefore be attached, for example, to the end effector, whose position and orientation are defined by the pose. In the case of a hexapod, the marker area can thus be located, in particular, on the moving work platform, e.g., in and / or symmetrically around the center of the work platform. Specifically, if the camera is mounted in or intended to be mounted in a base plate of the hexapod, the marker area can be attached to the underside of the work platform of the parallel kinematics (attachment method).

[0053] The marking area can, for example, first be created on a separate additional plate. This additional plate can then be attached to the work platform (e.g., screwed on) so that it is visible from below through the aperture.

[0054] It should be noted at this point that the present application usually only explicitly describes, as an illustrative example, the case of a fixed camera and a moving marking area. However, the present invention is not limited to a moving marking area. More precisely, it is possible to interchange the described position (i.e., the mounting location) of the camera and the marking area. For each embodiment of the present invention explicitly described here, there is therefore also a corresponding further embodiment in which the positions of the camera and the marking area are interchanged, and to which the present invention also refers. The camera then moves along with the camera, and the marking area is stationary. For example,It is possible that the marking area is located in the base of the parallel kinematics (and therefore does not move) and the camera is located on the underside of the hexapod's work platform (and therefore moves with it).

[0055] It should also be noted that the abbreviation ATA (from the English for AprilTagArray) is used below for the marking area, but this does not necessarily refer to a specific arrangement of the markings and / or the use of AprilTags. By using an array of tags, only a small image area (field of view) is required for the camera. This allows the camera to be positioned significantly closer to the work platform, which increases the achievable accuracy. Dimensions of the marking area

[0056] The size of the marking area can be adjusted to the position and / or the field of view of the camera so that the camera always observes a section of the marking area, even when the robot is moving within the intended frame.

[0057] This is in Fig. 4 for a sensor with rectangular dimensions, i.e., a rectangular field of view 400 with width and height FOV x or FO y , shown. As can be seen, the dimensions of the marking area 150, i.e. the width and height, are shown with ATA. x or ATA y The depicted field of view 400 is centered within the marking area 150 and corresponds to a specific pose of the robot, e.g., a resting pose, home pose, home position, and / or reference position. As illustrated by the double arrows, the marking area 150 can move relative to the field of view 400 when the robot moves. Here, S denotes xThe range of motion in the x-direction, i.e., both "right" and "left" from a centered (rest) position of the hexapod, as shown. Similarly, S denotes y The range of motion in the y-direction, i.e., both "up" and "down". In other words, the considered range of motion of the hexapod in the horizontal and vertical directions is 2S. x or 2S y To ensure that the field of view remains within the ATA range during such movements, the dimensions of the ATA can be calculated according to the hexapod's range of motion. This calculation uses the field of view at maximum distance (g). max ) to the camera results (designed for the larger field of view, therefore also suitable for smaller fields of view at g) min ). This results in ATAx=FOVx.max+2∗Sx and ATAy=FOVy.max+2∗Sy for the horizontal or vertical length of the marking area.

[0058] The relationships between minimum field of view, maximum field of view, positioning distances and the dimensions of the marking area will now be explained again using Fig. 5 and Fig. Figure 6 illustrates. As can be seen, in Fig. 5 and Fig. Six marking areas, 500 and 600 respectively, each containing a plurality of markings 510 and 610 respectively, are shown. Marking areas 500 and 600 differ primarily in the density of the markings, or rather the distances between markings, which will be discussed further below.

[0059] Furthermore, in Fig. 5 and Fig. 6. Three areas are marked in each case. Areas 550 and 650 represent an example of a field of view at minimum distance g. minThe camera is positioned at the center of the marking area, pointing from the marking field. Areas 550 and 650 are therefore without any deflection in the direction of the positioning distances from a centered pose; the hexapod / robot is at the zero point, e.g., its resting pose. Areas 560 and 660 represent a corresponding field of view at maximum working distance g. max and maximum deflections S x and S y (towards the upper left). As can be seen, these fields of view are larger than the other fields of view shown. For areas 570 and 670, this is the field of view at minimum working distance at g. min . and maximum deflections S x and S y (to the upper right). Arrangement of markings in the marking area

[0060] In general, the tag area contains multiple tags. Specifically, the tag area can be an array (a field) of fiducial tags, e.g., AprilTags. These can be arranged within the tag area according to a regular arrangement scheme. For example, the tags can be, as in Fig. 7a and Fig. Figure 7b illustrates that the markings are placed, or have been placed, according to the points of a two-dimensional grid, particularly at regular intervals. However, the marking area can also consist of a grid of markings, where the markings are arranged, for example, on concentric circles around a center point.

[0061] It is important that the grid is set up and the camera's field of view (including the lens) is adjusted so that at least one marker, or a specific number of markers, are always fully visible. The marking area should also be large enough so that at least one marker, or the desired number of markers, are fully visible even in the hexapod's extreme positions. Furthermore, the exact position of each marker on the array should be known to determine the pose. Markings

[0062] As already indicated, parallel kinematics includes mutually distinguishable markers (also called tags), or rather, the application procedure includes a step of applying mutually distinguishable markers in a marker area. Mutually distinguishable here means that any two markers can be differentiated, i.e., it is possible to uniquely identify a marker based on an image captured by the camera. The previously described pose of the kinematics could also be used for this purpose.

[0063] The ability to differentiate between the individual tags (and their known position on the array) makes it possible to deduce the exact location (position and orientation) of the moving platform from a single tag.

[0064] In general, the markings can be reference markers, such as ARToolKit markings, ArUco markings, QR codes, or especially AprilTag markings.

[0065] In particular, AprilTags, which are a specific system of reference markers (also known as "fiducial tags"), have become established in robotics. They can be considered a special type of QR code and, similar to QR codes, have a specific shape and layout for identification, error correction, preventing misdetections, and ensuring recognition in case of obstruction. Compared to typical QR codes, however, AprilTags contain less data and are specifically designed for robust identification at long distances and rapid decoding of their precise position and orientation relative to the camera, which can be particularly advantageous for real-time robotics applications. An example of an AprilTag is shown in Fig. 8 shown.

[0066] However, the present invention is not limited to a specific type of markings and / or tags. In general, any type of optical marking can be used, as long as the markings are mutually distinguishable and the pose of the kinematics can be determined with them, as described below. Length of a marker

[0067] The length of one of the markings is defined in the present application as t m designated. t m Here, denotes an actual length, and can therefore be specified in meters or millimeters, for example. Generally, all markings can represent the same length t. m They can be square. However, the present invention is not limited to this. The markings can, for example, also be rectangular and / or not actually utilize the entire area of ​​a rectangle, e.g., if they are round.

[0068] Furthermore, t bthe number of information units of the marking in the direction of length t m . t b This refers, for example, to the number of bits that are encoded next to each other in the direction in which the length t is also encoded. m is measured. While t b that is, the width and / or height of a marker is represented in, for example, bits, t m This represents the "actual" width and / or height of the marking. The size t b is therefore, in contrast to t m , unitless or dimensionless. Here again, it is simplified to assume that the markings are square, i.e., the number of bits t b in both directions defining the respective square is the same.

[0069] The term "information units" here refers to individual areas from which the marker can be composed, each capable of encoding information. The markers can, for example, be like those shown in... Fig. Figure 8 shows that the unit consists of several squares, with the squares corresponding to the units of information and one bit encodeable in each square. Thus, one unit of information can correspond to a single bit or square representing a day in April. However, the present invention is not limited to this. For example, it is possible to encode more than one bit of information in a unit, for example by using colors and / or different heights.

[0070] To ensure the markings are easily visible, the length t can be adjusted. m a marking can be determined according to the following equation: tm≥px∗p∗tb∗(gmaxƒ−1)

[0071] Here, the pixel length (px) refers to the length of the sensor that corresponds to one sample taken by the camera. px is therefore the pixel size of the camera, e.g., in meters. For example, if the sensor has a length of in the x-direction , , the sensor x and is the number of samples or pixels in the x-direction with Npx This is referred to as the pixel length (sensor). x (as explained above, this refers to the length of the camera sensor in the x-direction): px=Sensorx / Npx

[0072] N px This corresponds to the image resolution in the x-direction (i.e., the number of pixels in the x-direction), and a smaller px value therefore corresponds to a higher camera resolution. For a non-square sensor, there would be an analogously defined pixel length py in the y-direction and also a separate t. m and t b for the y-direction. However, it is also possible that the pixel lengths in the x- and y-directions (px = py) are the same, even if the sensor dimensions in the x- and y-directions are different (sensor). x ≠ Sensor y ). In the following, it is assumed for simplicity that the camera pixels are square, so at least px = py holds true.

[0073] The variable p here corresponds to the desired minimum number of samples per unit of information according to the Nyquist-Shannon sampling theorem and is preferably 5. In general, however, p can be chosen differently depending on the camera and / or application, but is usually greater than or equal to 2 and less than or equal to 5. For example, p = 2 may be more suitable for a monochrome camera, while a value of p = 3 to 4 may be more suitable for an RGB camera.

[0074] Is the minimum size of an April day t m Thus determined, the AprilTags can be used at any distance less than or equal to the maximum working distance g ≤ g max still be detected well enough. Distance D between markings

[0075] In general, it is possible to determine (e.g., calculate) the tag density based on the specified "desired number" of tags that should always be visible, as well as the given system parameters, in particular the focal length. Such a "desired" or specified number is denoted by "n" (or ni) in this application. x and n y (if a distinction is explicitly made between the x and y directions) and can generally be any integer greater than or equal to 1 (e.g., n = 1, n = 2, etc.). n x and n y They can generally be the same or different. The desired minimum number of tags on the image is therefore n * n or, if a distinction is made between the x and y directions, n x * n y .

[0076] For example, it may be sufficient for n of the markers to be visible in one direction (e.g., x- or y-direction) on an image of the marking area to determine the pose of the kinematics based on that image. Specifically, the number n of markers can correspond to the minimum number of markers that must be visible in one direction (e.g., x- or y-direction) on an image of the marking area to determine the pose of the kinematics. In other words, n can be the minimum number of markers necessary (and sufficient) so that the pose can always be determined based on an image containing n markers in the direction under consideration. "Always" here means that, of course, a sufficient number of markers must also be visible in the other direction (which may be a different number of markers than the number in the direction under consideration).

[0077] For example, n xand n y These are the minimum number of markers that must be visible in the x and y directions of an image for the pose to be determined based on that image. The pose can then always be determined if both (i) n x Markings in the x-direction, as well as (ii) n y Markings are located in the y-direction on the image. The word "sufficient" in this context refers to the direction being considered and does not mean that a certain number of markings in the other direction might not also be necessary. Similarly, the statement means that the pose can always be determined if n x Markings in the x-direction indicate that this is possible, unless n y Markings can be seen in the y-direction.

[0078] It is therefore possible that if there are fewer than n markers in an image, the pose can no longer be uniquely determined (at least not for an image taken in any given pose). However, the present application is not limited to such a "minimal" n. The number n can, for example, be larger than theoretically necessary to determine the kinematic pose, perhaps to improve the reliability / robustness of the recognition. The number n can, for example, be predetermined by the choice of markers used or determined based on the choice of markers.

[0079] For example, the distances, D x and D y , calculate between adjacent tags in the x and y directions based on a specified number of tags that should always be fully visible in at least the x and y directions. The markers can be, for example, as in Fig. 7a indicated, at regular intervals D xand D y They must be arranged in the x- or y-direction. The term "adjacent" then refers to the nearest marker in the x- or y-direction.

[0080] By ensuring the visibility of at least the desired number of markers (e.g., one), the position of the hexapod's moving platform can be detected for each desired pose. Simultaneously, the large distances described here allow for the use of fewer different markers and / or a reduction in the number of markers appearing in a single image captured by the camera. This simplifies and speeds up the detection / identification of the marker in an image and the determination of the pose.

[0081] It should first be noted that, although not every distance D will be mentioned below max , D X and D yIt is explicitly stated that what is said applies in the same, i.e., analogous, way to the distances D max , D X and D y The following applies between markings. Between D x and D y This can be distinguished, for example, if neither the marking area rotates relative to the camera nor the sensor dimensions are square. For a square sensor, D would be x = D y = D, and if relative rotations are possible, the smaller of the two distances would be D x and D y to use (and, as explained in more detail below, through 2 to share).

[0082] For example, the desired number n can be determined. x a tags per row (horizontally arranged tags, "x-direction") and the desired number n y Specify the number of tags per column (vertically arranged tags, "y-direction"). To ensure that the desired number of tags (n) is always displayed, x and n y) is within the camera's field of view, the distance D can be calculated based on this. x between the tags in the x-direction and the distance D y The distance between the tags in the y-direction is calculated according to: Dx≤Dxmax=FOVx−(nx+1)∗tmnx Dy≤Dymax=FOVy−(ny+1)∗tmny

[0083] Therefore, n x * n y April days are within the field of view. Since the field of view increases with increasing working distance, the field of view at g can be used for the calculation. min be used. More precisely, FOV x,min and FOV y,min for FOV X or FOV y It is noted that FOV x,min ≥ (n x + 1) * t m and FOV y,min ≥ (n y + 1) * t m This should apply so that non-negative distances Dxmax or Dymax result.

[0084] The maximum distances specified above ensure the desired minimum number of markers across the entire work area. At the same time, relatively large intervals are permitted, which facilitates the identification of individual markers and saves processing time.

[0085] Especially if it is sufficient that only one marking is fully visible at any given time, the distance D (D here stands in particular for D) can be used. x and / or D y ) between any two adjacent markings in the area (FOVmin3tm) / 2 <D≤FOVmin−2tm This allows for the use of a large day interval, while still ensuring that at least one day is sufficiently visible. Limiting case – enlargement of Dmax(Dxmax,Dymax)

[0086] If the distances D x and D y As described above, there is a limiting case where more than n are determined. x * n yTags are fully visible; more precisely, it may be possible that up to (n x + 1) * (n y + 1) Markings are fully visible. This is in Fig. 9 for case n x = n y Figure 2 illustrates this. As can be seen, nine tags are fully visible, meaning they are entirely within the field of view (950). The limiting case here refers to the transition of columns or rows of tags from the field of view (i.e., when, for example, a new row has just moved in on one side, but the row on the other side has not yet started to move out again).

[0087] As in Fig. As illustrated in Figure 10, this is no longer the case with minimal displacement; that is, with a small displacement (in the x and y directions), only n x *n yTags, in this example only four tags, are fully visible and therefore completely within the field of view 1050.

[0088] Since the camera typically has a finite resolution, this limiting case can be used to determine the maximum distances. Dxmax and Dymax and therefore also D x and D y to increase each by one pixel size (px). This results in DX≤Dxmax=FOVX,min−(nx+1)∗tmnx+px(gminƒ−1) Dy≤Dymax=FO y,min−(ny+1)∗tmny+py(gminƒ−1)

[0089] The distance between the markers is therefore shifted by the smallest possible resolvable distance, namely px or py. This ensures that, purely geometrically, there are always n x * n y Tags are (fully) in the field of vision, as the others are not fully in the field of vision.

[0090] A better camera (with a smaller pixel size px or py) would allow for a possible increase in the distances between the April days D. X or and D y Contrary to this, adding the px or py term to the formulas above results in the multiple visible tags being shifted apart by exactly one pixel in the limiting case. This means that exactly one pixel-wide row of one of the two edge tags is missing, and it is no longer fully visible. The better the camera, the smaller the resolved pixel, and therefore the closer the tags should be to each other if only one pixel of an edge tag is to be missing in the limiting case. This means that a denser array is needed to fully utilize the higher resolution of a better camera. It should be noted that the above applies to a (predefined marker length t). mThis applies. If the file size is adjusted to the better resolution (smaller px or py), then smaller tags can be used when using a better camera, according to the formula above. tm=px∗p∗tb∗(gmaxƒ−1). t m where the smaller pixel length px or py of the better camera is now used. In particular, by inserting the px-dependent expression for t m in the above formula for the distances D X or D y to see that the distances can usually be increased by using a smaller pixel size, if the marker length t m is adjusted accordingly to the better camera.

[0091] On the other hand, it is also possible to use this higher camera resolution to increase the distances between the markings. Generally, a unit of information has a length t. m / t b, and one sample value corresponds to the length of the marker at a distance g from the camera. px∗(gƒ−1). Therefore, the working distance g P(g)=tmtb∗px(ƒg−ƒ)

[0092] Samples are taken with the camera of an information unit. As can be seen, using a camera with a smaller pixel size (px) leads to an increase in the number of samples taken, P. In general, if the minimum number of samples taken is P(g) max If the number of markers is greater than the specified minimum number p, the markers can be made smaller, thus reducing P, and / or the distances between the markers can be increased. In particular, if the distances are to be increased, (Pp) samples at each of the two information units at the edge can be omitted. The distance between the markers can thus be, for example, DX≤Dxmax=FOVX,min−(nx+1)∗tmnx+(2P(g)−2p+1)∗px∗(gminƒ−1) can be increased. For P(g), a conservative approach is possible, e.g., P(g max ) can be used. However, since the n x Since markings will generally not all be fully visible at the minimum working distance, P(g) can also be used here. min ) be used.

[0093] If you increase the distance or the maximum distance further to DX≤Dxmax=FOVx−(1−nx)∗tmnx so in the limiting case (n x - 1) * (n y - 1) completely visible. One row of markings has already moved out of view, while the next has not yet begun to move into view. Therefore, in the limiting case, fewer, not more, markings are now visible; for n x = n y = 2 would therefore only be visible for one day in the limiting case with such an increased distance.

[0094] Generally, it may not be necessary to see an entire day to identify it and / or determine its position. The distance Dxmax moves as in Fig. 11a and Fig. 11b illustrates, in the area (between the boundaries) FOVx−(nx+1)∗tmnx≤Dxmax≤FO x−(1−nx)∗tmnx

[0095] More precisely, D X in Fig. 11a is set equal to the lower limit in the formula above (corresponding to the left side), and in Fig. 11a is set at the upper limit (corresponding to the right side). The field of view is 1400 in Fig. 11a and b are the same size; only the distance between the markings has been changed. As in Fig. 11b indicated the area in which Dx=Dxmax or Dy=Dymax in the process, a length of 2t is changed. mThe boundaries of the range represent the extreme cases in which either (n x + 1) Tags are visible (lower limit, corresponding to the left expression of the equation above), or (n x - 1) Tags can be seen (upper limit, corresponding to the right-hand expression of the equation above). In particular, when choosing n x = n y = 1 at the upper limit, no day can be fully seen.

[0096] In general, the lower limit always requires at least n x · n y Tags fully visible and, in the limiting case, more tags fully visible, namely up to (n x + 1) · (n y + 1). The upper limit is at most n x · n y Tags are fully visible. That is, in the limiting case, fewer tags are fully visible, namely up to (n x - 1) · (n y - 1).

[0097] For example, will n x = n yIf = 3 is chosen, and the distance corresponding to the lower limit is used, then there are always at least n x - n y = 3 · 3 = 9 tags are fully visible in the field of view. In the limiting case, i.e., in certain poses, more than 9 tags are fully visible (up to 16). With the same choice of n x , n y Using a distance corresponding to the upper limit, at most 9 tags are completely in the field of view, but in the limiting case less than 9 tags (up to 4).

[0098] If it is not necessary to see a marker completely in order to identify it and / or determine its position, the distance can therefore be used. Dxmax up to almost 2 tons m can be enlarged. In other words, if, for example, it is sufficient to see only the fraction 0 < R ≤ 1 of a tag, the distance between the tags can be increased according to DX=Dxmax=FO x−(nx+1)∗tmnx+2tm(1−R) be determined. Consideration of rotations

[0099] If the marker area can rotate relative to the sensor, the smaller of the two field-of-view lengths (FOV) is used. x and FOV y used. Furthermore, the distinction between D and D should no longer be made. X and D y They can be distinguished. Therefore, the same distance D = D will be used. X = D y used for the horizontal and vertical directions. The distance can be determined according to any of the formulas above, but it is then adjusted to account for rotations. 2 divided, i.e. reduced in size. The overall result is therefore... D≤min{Dxmax / 2,Dymax / 2}

[0100] This will now be discussed in relation to Fig. 12 explained in more detail. Fig. Figure 12 represents squares 1201, 1202, 1203, and 1204 of the April tags without considering rotation. As can be seen, the distances in the x and y directions between adjacent tags are still different. The distance in the y direction is smaller in this example.

[0101] Taking into account rotations around the center of the FOV, the spacing is adjusted so that all the AprilTags lie on a radius within the field of view. This results in squares 1251, 1252, 1253, and 1254, which illustrate AprilTags with adjusted spacing. As indicated, the distance D is chosen such that the resulting diameter 2r corresponds to the distance in the y-direction, i.e., less than or equal to 2r. Dymax is, for example 2r=Dymin=FOVy−(ny+1)∗tmny=2D This applies. The distance is therefore calculated as follows: D=FOVy−(ny+1)∗tm2ny For n y = 1 means this explicitly: D=FOVy−2tm2 3D-ATA

[0102] In some embodiments, the markings are placed in different planes within the marking area. In particular, two adjacent markings are placed in different planes. However, not all adjacent markings need to be in different planes. This is in Fig. 13a, Fig. 13b, Fig. 13c and Fig. Figure 13d illustrates this. As can be seen, the markings 1350 are placed in the respective marking areas 1300 on different levels.

[0103] The term "different planes" refers to the fact that the xy-planes of the individual markers are located at different heights or depths. The markers are thus offset in the "z-direction," where the z-direction is orthogonal to the previously described xy-plane. In other words, for a given pose, the different planes are at different working distances, particularly at different distances from the camera. Therefore, for a given pose, the markers of different planes have different object distances.

[0104] The use of a 3-dimensional marking area (“3D-ATA”) makes it possible to increase the working area in which markings can still be reliably detected by the camera. This allows for a depth of field (also called depth of focus) across the entire intended working area of ​​the parallel mechanism. As in Fig. 14, illustrated, this corresponds to the adjustment range ±S z in the z-direction. For a given camera setup with a specific focal length and any extension tubes, a working distance g results. fp to the front lens of the objective, where the plane is sharply focused (focal plane 1450). Starting from this distance, the ATA is adjusted in small steps of ±S. z The tags are moved and the number of detected tags is recorded. For example, if enough tags are detected at a setting range of ±5.5 mm, but the desired setting range is ±6.5 mm, then 1 mm is still "missing" to cover the desired working area of ​​the hexapod. This can be achieved by placing the tags on multiple levels on the ATA, rather than on a single level.

[0105] In Fig. 14. The manipulator platforms 1440 and 1460 (e.g., the movable platform of the hexapod, here representing the marking area 1300) are shifted by a length ±δ from the focus plane 1450, which is located at a distance g. fp The distances to the camera are shown. These are the measured distances at which sufficient AprilTags are still detected. Manipulator platforms 1430 and 1470 represent the displacements of the manipulator platform corresponding to the maximum desired displacement of the manipulator platform in the z-direction, thus corresponding to the adjustment ranges ±S. z of the hexapod. This is what needs to be achieved. In order to cover the entire desired working area (i.e., to be able to create a sharp image of a marker), the differences ±Δ are overcome by placing tags on a surface with a difference of Δ. + The material is applied to a raised surface or to a surface recessed by Δ_. The distances are calculated as follows: Increase by Δ + = S z - δ + Deepening by Δ_= S z - δ_

[0106] In general, this can be done as in Fig. 14 shown, also δ + = δ_ = δ or Δ = Δ + = Δ Δ _ apply; the raised level is then Δ= S z - δ is increased relative to the focal plane and the recessed plane by (the same length) Δ = S z - δ deepened. More than two layers can also be created. Generally, for example, 2k layers can be used, with the markers placed in layers that are around ±Δi=Δk∗i With i = {1, ..., k}, the markings are raised or lowered in the z-direction with respect to the focus plane 1450. If markings are also placed in the zero plane (i.e., in the focus plane 1450, where Δ0=0), this results in 2k + 1 planes. In the x- and y-directions, the markings can be assigned to the different planes periodically, following a regular pattern. In particular, the markings can be assigned to the different planes such that any two adjacent markings lie in different planes. However, this is not mandatory, and a pattern in which some adjacent markings lie in the same plane is also conceivable (see...). Fig. 13e).

[0107] It should also be noted that in the 3D-ATA versions, the markings are arranged (closely) or the distance D is determined (application method) in such a way that two or more markings are always in the camera's field of view. Therefore, n xand / or n y greater than 1 is chosen and the distance D between any two adjacent markings thus satisfies the following formula: 2D≤FOVmin−3tm n x and n y are chosen in accordance with the assignment of the markers to the levels, in particular the number of levels. In particular, n x and n y Based on the assignment of markers to the levels, the selection is such that markers from different levels are always visible. In particular, the number of markers that are always visible can be greater than or equal to the number of different levels, and the markers can be assigned to the levels in such a way that one marker from each level is always visible. This ensures that a marker is always visible and can be focused sharply enough. The day intervals D X , D y It can therefore be such that n always exists x * n yTags are in the field of view, and the tags can be up to n x * n y be or will be arranged on different levels.

[0108] For example, the markings can be on three different levels, as in Fig. 13e are shown and assigned. Fig. Figure 13e shows a consistent pattern that repeats after four markers in the x and y directions, and can therefore be continued accordingly. "0" corresponds to the zero plane / focus plane, "+" to the plane raised above the zero plane, and "-" to the recessed plane. The zero plane is thus used as a frame much more frequently than the other two planes. The field of view can then be chosen, for example, so that four markers in the x direction and four in the y direction are always fully visible (i.e., a total of 16 markers), ensuring that at least one marker from each of the three planes is always visible. Pose determination based on an image of a marker

[0109] As already mentioned, the markings or reference markers can be used to determine the pose (position and orientation) of the kinematics. For this purpose, an image of the marked area, or more precisely, an image of the currently visible portion of the marked area, corresponding to the camera's current field of view, is taken.

[0110] It should first be noted that the term "pose of the kinematics" in this application refers, for example, to the pose of an end effector of the respective kinematics. The end effector is, for example, the last link in a kinematic chain. It is generally the component or assembly responsible for performing the actual handling task. In other words, the effector brings about the actual interaction of the robot (i.e., the kinematics) with its environment. An end effector can be, in particular, a tool, a tool carrier, a gripper, or a platform to be moved (e.g., in the case of hexapods). Furthermore, it should be noted that the pose of the kinematics is determined as it was at the time the image was taken.

[0111] More precisely, the markings can be configured such that the pose of the parallel kinematics can be determined based on a camera image of the marking area, provided the image contains at least n arbitrary markings in one direction, where n is greater than or equal to 1. Specifically, for n = 1, the pose of the parallel kinematics can be determined for each marking based on a camera image of that marking. In other words, determining the pose may require only a single arbitrary marking in the captured image. The image may therefore contain no other markings than this one, and regardless of which marking this one is, the pose can be determined.In general, as explained above, it may also be sufficient and / or necessary for the image of the marking area to contain at least n markings in one direction, where n can also be greater than one. As before, in this case it doesn't matter which n markings are in the image in that direction (as long as there are at least n markings in the corresponding direction and, as explained above, a sufficient number of markings are also visible in the other direction).

[0112] This uses the known position of the sensor or camera that captured the image. The distinguishability between individual tags, which allows for unique identification (family and individual), can also be used, potentially enabling the recognition and differentiation of multiple tags in a single image. Furthermore, the fact that the position and orientation of a marker in space relative to the camera can be determined based on the captured image can be utilized. The known position of each marker on the array or on the kinematics can then be used to deduce the precise position and orientation of the moving platform from the position of a single tag.From a single image, the position and rotation of the captured marker(s) relative to the camera can be determined. Using the known position / orientation of the camera and the marker's position on the kinematics, the pose of the kinematics can then be determined. This allows for an absolute determination of the position of, for example, the moving platform of a hexapod. Markers that enable this are, in particular, the aforementioned AprilTags.

[0113] Position detection using AprilTags can be automated in several steps. An example procedure consisting of 9 steps for AprilTag detection and determining the tag's pose relative to the camera is shown below: Step 1 ("Decimate"): The image captured by the camera is reduced in size by an arbitrary factor N, which can be defined at runtime. Only every Nth row and column is copied into a new image, which is then processed in subsequent steps. The original image is needed again in the "Refinement" and "Decode" steps. Step 2 (“BlurSharpen”): In this step, the reduced image can either be blurred or sharpened using a Gaussian filter. The strength of the filter can be adjusted using a parameter that can be set before starting the process. The sign of the parameter determines whether the image is blurred or sharpened. Step 3 (“Threshold”): The image is segmented into bright and dark areas, as well as areas with low contrast. A local thresholding method can be used for this. First, the image is divided into four-by-four-pixel tiles, with each minima and maxima forming a new image. The minima image is then eroded, and the maxima image is dilated. The threshold is then calculated as the average between the minimum and maximum values. If the difference is too small, the contrast is insufficient. Step 4 (“Connected Components Labeling”): Related segments are grouped into components and assigned a unique label. A UnionFind data structure can be used for this. Step 5 (Gradient Cluster): In this step, all subpixels located on the boundary between a light and a dark component (edge ​​pixels) are captured. Eight-neighborhood is used for this. A separate list of subpixels is maintained for each component combination, storing their position and edge direction. A hash table is used to match the data to the appropriate list. Step 6 (“Quad”): First, the midpoint of an edge sequence is determined using a bounding box that encompasses all pixels in the list. Then, the pixels are sorted by the angle around the midpoint, and duplicate entries for a position are removed. Next, the algorithm searches for the vertices of the quadrilateral (here, the marker is assumed to be quadrilateral). To do this, a straight line is fitted to a window consisting of consecutive edge pixels. This window is then shifted across the entire sequence. The vertices of the marker are located at the points where the largest fitting errors occur. Finally, straight lines are fitted to the segments between the vertices, and the intersection points yield the final vertices of the quadrilaterals. Step 7 (“Refinement”): Using the original image, the edges of the found quadrilaterals are rescanned to increase the accuracy, which was affected by the image reduction. The algorithm searches for the steepest gradient along the normal at points evenly distributed along the edge. The number of points is one-eighth (for t). b = 8) of the edge length. This provides support points for recalculating lines along the edge, the intersection points of which yield the new vertices. Step 8 (“Decode”): First, the homograph between the image coordinates and the detected quadrilaterals (markers) is calculated. Based on this, sampling points are projected onto the original image. The sampling points at the edge of the tag have a known color (black / white). This allows a model of the color gradient to be created, from which the threshold values ​​for the actual data points are generated. The tag family provides information about the location of the known points and the data points. Decoding a valid ID also reveals the tag's orientation. Step 9 (“Pose Estimation”): This step utilizes the camera parameters. After the homography has been calculated, the position and rotation relative to the camera can also be determined. Using an iterative procedure, the rotation matrix and the translation vector are calculated. Example with numerical values

[0114] The following list contains most of the parameters mentioned here, along with example values. Work area:

[0115] The distance g fp The distance between the lens or front lens and the focal plane can be determined, for example, using measurement techniques. It describes the distance between the lens and the AprilTag array at which the array is sharply focused. With a hexapod's position in the z-direction of S z This results in a working range of g fp ± S z . gmin=gƒp−Sz gmax=gƒp+Sz gmin≤g≤gmax Are the positioning ranges of the parallel mechanism in the x, y, and z directions given by S x = 17 mm, S y = 16 mm, or S z = 6.5 mm, so with an example distance g fp = 20mm a working area g min ≤ g ≤ g max from: gmin=gƒp−Sz=20 mm−6.5 mm=13.5 mm gmax=gƒp+Sz=20mm+6.5mm=26.5mm

[0116] Camera and field of view: The camera also provides the following parameters: Focal length f = 8 mm Pixel size px = 3.45 µm Sensor dimensions in the x-direction Sens x = 8.446 mm Sensor dimensions in the y-direction Sensor y = 7.066 mm

[0117] The dimensions of the field of view (FOV) can be calculated from the camera parameters to determine the minimum and maximum distances. For the field of view at the minimum working distance g min surrendered FOVxmin=(gminƒ−1)∗Sensorx=(13.5 mm8 mm−1)∗8.446 mm=5.81 mm FOVymin=(gminƒ−1)∗Sensory=(13.5 mm8 mm−1)∗7.066 mm=4.86 mm Similarly, the field of view can be affected at maximum working distance g. max to be determined: FOVxmax=(gmaxƒ−1)∗Sensorx=(26.5 mm8 mm−1)∗8.446 mm=19.52 mm FOVymax=(gmaxƒ−1)∗Sensory=(26.5 mm8 mm−1)∗7.066 mm=16.34 mm Dimensions of the marking area

[0118] The above example values ​​result in the following dimensions for the marking area: ATAx=FOVx.max+2∗Sx=19.53mm+2*17mm=53.53mm ATAy=FOVy.max+2∗Sy=16.34mm+2*16mm=48.34mm Day size:

[0119] The minimum size of an April day is then calculated depending on the maximum working distance as follows: tm≥px∗p∗tb∗(gmaxƒ−1)≥3.45 μm ∗5∗8∗(26.5 mm8 mm−1)≥0.32 mm Distances between AprilTags:

[0120] According to the specifications of n x and n y The spacing of the April tags will be adjusted at minimum field of view (FOV). min calculated according to: Dx=FOVxmin−(nx+1)∗tmnx Dy=FOVymin−(ny+1)∗tmny

[0121] If we assume the known dependencies for FOVxmin and t m Substituting the above equation, the distance between the April tags can be expressed in general form as follows: Dx=FOVxmin−(nx+1)∗tmnx =(gƒp−Szƒ−1)∗Senx−(nx+1)∗px∗p∗tb∗(gƒp+Szƒ−1)nx

[0122] Are n x = n y Given that 1 is the desired number of AprilTags in the x or y direction, the following results numerically using the example values ​​above: Dx=5.81mm−(1+1)∗0.32mm1=5.17mm Dy=4.86mm−(1+1)∗0.32mm1=4.22mm

[0123] For n x = n y = 3 would, however, D x = 1.51 mm and D y = 1.194 mm.

[0124] Taking rotation into account, the distances for n decrease. x = n y = 1 on: Dx=Dy=D=4.86mm−(1+1)∗0.32mm2=2.984mm

[0125] In summary, the present invention relates to parallel kinematics and methods for manufacturing parallel kinematics. A parallel kinematic system according to the invention comprises mutually distinguishable markings that are applied to the parallel kinematic system in a marking area. The marking area is a region of the kinematic system that moves with the pose of the kinematic system.

[0126] According to one aspect of the present invention, the markings are placed at a distance in a direction such that n markings are always fully visible in that direction, and the pose of the parallel kinematics can be determined based on an image captured by the camera that contains at least n markings in that direction. A corresponding method of placement relates to the appropriate placement of markings.

[0127] According to another aspect of the present invention, the markings are placed at a distance that ensures that n or more markings are fully visible in one direction, the markings are placed in different planes, and the pose of the parallel kinematics can be determined based on an image taken with the camera that contains at least n arbitrary markings in that direction. A corresponding placement method relates to the appropriate placement of markings.

Claims

[1] A parallel kinematics, comprising: a camera (110) and a marking area (150) with mutually distinguishable markings, wherein the camera (110) is configured to observe the marking area (150) at different poses of the parallel kinematics, wherein the pose of the parallel kinematics can be determined based on an image of the marking area taken with the camera (110), if the image contains at least n arbitrary markings in one direction, where n is greater than or equal to 1; where A distance, D, between any two adjacent markings in the direction of the following formula is sufficient: FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn, wobei t m the length of one of the markings is, FOV mina length of the section of the marking area (150) that falls into the field of view (400) of the camera (110) at a minimum distance of the camera (110) from the marking area (150), wherein The minimum distance is a minimum distance below the distances that the marking area (150) can be from the camera (110) by changes in pose. [2] The parallel kinematics according to claim 1, wherein the length FOF min The following equation is satisfied: FOVmin=(gminƒ−1)lSensor, where G min the minimum distance is, l Sensor a length of the camera sensor (110) is, and f is the focal length of the camera (110). [3] The parallel kinematics according to claim 1 or 2, wherein the markings are arranged in the marking area (150) according to a regular arrangement scheme. [4] The parallel kinematics according to any one of claims 1 to 3, wherein the marking area (150) is attached to an underside of a work platform (140) of the parallel kinematics and the camera (110) is attached in or to a base (120) of the parallel kinematics and is directed towards the underside of the work platform (140), or the marking area (150) is attached in or on the base (120) of the parallel kinematics, and the camera (110) is attached to a bottom side of the work platform (140) and directed towards the base (120) of the parallel kinematics. [5] The parallel kinematics according to any one of claims 1 to 4, wherein the length t m A marking of the following equation suffices: tm≥px∗p∗tb∗(gmaxƒ−1), where p is a value dependent on the camera (110) greater than or equal to 2 and less than or equal to 5, px is the length that corresponds to one sample value of the camera (110), G maxa maximum distance is among the distances that the marking area (150) can be from the camera (110) by changes in pose, f is a focal length of the camera (110), and t b The number of information units in the marker. [6] The parallel kinematics according to any one of claims 1 to 5, wherein the markings are reference markers, such as ARToolKit markings, ArUco markings, QR codes or, in particular, AprilTag markings. [7] The parallel kinematics according to any one of claims 1 to 6, wherein each of the markings consists of several squares, the squares corresponding to the information units and each square containing a bit that can be encoded. [8] A method for applying mutually distinguishable markers to a parallel kinematics in a marking area (150) such that the pose of the parallel kinematics can be determined based on an image of the marking area taken with a camera if at least a predetermined number, n, of the markers are located in one direction on the image, where n is greater than or equal to 1, the method comprising: Determine (S310) a distance, D, between any two adjacent markings in the direction according to the following formula: FOVmin−(n+2)∗tmn+1 <D≤FOVmin−(n+1)∗tmn, wobei t m the length of one of the markings is, FOV min a length of the section of the marking area (150) that falls into the field of view (400) of the camera (110) at a minimum distance of the camera (110) from the marking area (150), wherein the minimum distance is a minimum distance among the distances that the marking area (150) can be from the camera (110) by changes in pose; and Apply (S320) adjacent markings at the specified distance. [9] A parallel kinematics, comprising: a camera (110) and a marking area (150) with mutually distinguishable markings, wherein the camera (110) is configured to observe the marking area (150) at different poses of the parallel kinematics, wherein A distance, D, between any two adjacent markings in one direction is sufficient according to the following formula: D≤FOVmin−(n+1)∗tmn, where t m the length of one of the markings is, FOV min a length of the section of the marking area (150) that falls into the field of view (400) of the camera at a minimum distance of the camera from the marking area (150), wherein the minimum distance is a minimum distance among the distances that the marking area (150) can be from the camera by changes in pose, the markings are located on different levels, and the pose of the parallel kinematics can be determined based on an image taken with the camera, if this contains at least a number, n, arbitrary of the markings in the direction, where n is greater than or equal to 2. [10] A method for applying mutually distinguishable markings to a parallel kinematics in a marking area (150) such that the pose of the parallel kinematics can be determined based on an image of the marking area taken with a camera if at least a predetermined number, n, of the markings are located in one direction on the image, where n is greater than or equal to 2, the method comprising: Determine (S310) a distance, D, between any two adjacent markings in the direction according to the following formula: D≤FOVmin−(n+1)∗tmn, where t m the length of one of the markings is, FOV min a length of the section of the marking area (150) that falls into the field of view (400) of the camera at a minimum distance of the camera from the marking area (150), wherein the minimum distance is a minimum distance among the distances that the marking area (150) can be from the camera by changes in pose; and Attach (S320) in the direction of adjacent markings at the specified distance, wherein The markings should be placed so that they are on different planes.

Images