Method and system for determining gestures

Abstract

Methods and systems for capturing and evaluating a plurality of two-dimensional diffraction patterns for the purpose of estimating a pose are illustrated. The plurality of two-dimensional diffraction patterns are generated by diffraction at the element (30) of coherent radiation having different wavelength components (141, 142). The evaluation device (20) is configured to estimate a posture of the element (30) based on the plurality of two-dimensional diffraction patterns (50).

Description

Technical Field

[0001] The present invention relates to a method and a system for pose estimation. The present invention particularly relates to a non-contact method and system for pose estimation using electromagnetic radiation, and components therefor. Background Art

[0002] Pose estimation of components has many applications, for example in manufacturing technology or medical engineering. An exemplary application area is pose estimation of workpieces (i.e., estimation of the translational and rotational degrees of freedom of workpieces). This can be conventionally performed using tactile methods or without contact.

[0003] Techniques for non-contact pose estimation of components require certain trade-offs. For example, in the case of a camera tracking system using multiple cameras or interferometric techniques, there is an interaction between the achievable accuracy and the achievable spatial dynamic range (i.e., the detection volume and / or the solid angle range). Achieving high accuracy over a large detection volume and / or a large solid angle range is challenging.

[0004] For camera-based detection using a single camera, it is generally not possible to clearly distinguish the characteristic markings of deflections in one degree of freedom from deflections in other degrees of freedom. For example, this can be partially overcome by using multiple cameras. In this regard, a camera tracking system with incoherent illumination is an example. In addition, in the case of a system with incoherent illumination, relatively high accuracy can generally be achieved in two translational coordinates, but the resolution in the remaining directions is usually much poorer.

[0005] Coherent illumination can provide improvements with respect to these drawbacks of systems with incoherent illumination. Measuring the relative phase angles of the wave trains involved also allows reconstruction of the pose in the depth direction (i.e., along the beam direction) with interferometric accuracy.

[0006] In addition to the accuracy achievable in up to six degrees of freedom, various conventional methods also differ in terms of the absolute accuracy achievable in each case. Typically, the intrinsic camera parameters and the extrinsic camera parameters are calibrated by known test standards. In a multi-camera system, the absolute positions of the individual cameras can be calibrated therefrom, and the absolute pose of the target object for pose estimation can be obtained accordingly. In the case of a single-camera system and a pose estimation target including a repetitive grating structure, absolute pose estimation is generally only possible using additional measurement techniques.

[0007] Accordingly, there is still a need for improved methods and systems for pose estimation of an object of interest. In particular, there is a need for systems and methods that provide improvements in terms of achievable accuracy and / or the dynamic range that enables pose estimation. In particular, there is a need for such systems and methods that allow for the estimation of an absolute pose (absolute translational position and / or rotational position relative to a reference frame) with high accuracy. There is also a need for elements that can be used in such pose estimation methods. SUMMARY OF THE INVENTION

[0008] According to the invention, systems and methods as defined in the independent claims are set forth. The dependent claims define preferred and advantageous exemplary embodiments.

[0009] According to one aspect, the invention relates to a pose estimation system. The system includes at least one detector configured to capture a plurality of two-dimensional diffraction patterns generated by diffraction of coherent radiation at an element. The system includes evaluation means configured to estimate the pose of the element based on the plurality of two-dimensional diffraction patterns. In this case, the coherent radiation has different wavelength components, and / or the element includes a diffraction structure that allows a bijective assignment of the plurality of two-dimensional diffraction patterns to exactly one pose.

[0010] Using diffraction makes it possible to increase the solid angle for which pose estimation is possible. The inherent geometric constraints of reflection techniques are relaxed. Using coherent radiation having different wavelength components allows for the resolution of ambiguities and the unambiguous estimation of an absolute pose. In an alternative or additionally, using the diffraction structure of the element that allows a bijective assignment of the plurality of two-dimensional diffraction patterns to exactly one pose allows for the resolution of ambiguities and makes a unique estimation of the absolute pose possible.

[0011] These different wavelength components may be phase-stable with respect to each other.

[0012] This also makes it possible to use the synthetic wavelength generated by the coherent superposition and interference of different wavelength components in pose estimation.

[0013] The evaluation means may be configured to estimate the pose of the element using the relative phase angles of these different wavelength components.

[0014] Using phase information simplifies the unambiguous pose estimation within a desired detection volume and over a desired detection solid angle range.

[0015] The at least one detector may include one or more detectors configured to acquire phase information. The detector or detectors configured to acquire phase information may include one or more interferometers.

[0016] The at least one detector may include one or more lensless cameras.

[0017] Obtaining the phase information simplifies the unambiguous pose estimation within the desired detection volume and over the desired detection solid angle range.

[0018] The evaluation device may be configured to determine the surface reconstruction of the component and / or the pose estimation of the component by digital holography.

[0019] This allows an initial value to be provided, which can then be refined by the evaluation device.

[0020] The evaluation device may be configured to use the diffraction images captured for the plurality of wavelength components to determine the surface reconstruction of the component and / or the pose of the component.

[0021] This allows an initial value to be provided, which can then be refined by the evaluation device.

[0022] The system may be configured to allow the relative phase angles of different wavelength components to have variable adjustability.

[0023] This can further improve the resolution of the pose estimation over a high dynamic range.

[0024] The evaluation device may be configured to perform pose estimation using reference data that depends on the configuration of the diffraction structure of the component. The reference data may include the design specifications of the diffraction structure of the component. The reference data may define the geometry of the diffraction structure of the component. For example, the reference data regarding the diffraction structure may be transferred from the system used to manufacture the component to the evaluation device.

[0025] The reference data may be such that they allow forward calculations of the expected diffraction patterns at each detector and for each wavelength component.

[0026] This facilitates pose estimation without the need for comprehensive reference measurement results.

[0027] The evaluation device may be configured to perform a multi-level evaluation process for pose estimation, which includes an initial pose estimation and a refinement of the pose estimation based on the plurality of two-dimensional diffraction patterns.

[0028] This allows high resolution achievable by using coherent measurement methods to be achieved in the iterations of an iterative process. The initial pose estimation can be used to reduce the computational complexity, for example when the expected diffraction pattern is determined by forward calculation using the knowledge of the geometry of the diffraction structure (e.g., which can be encoded in the reference data) and compared with the captured diffraction pattern.

[0029] The evaluation device may be configured to use reference data to refine the pose estimation.

[0030] This allows for high resolution achievable by using coherent measurement methods to be implemented during the iterations of an iterative process.

[0031] The evaluation device may be configured to determine at least one reference diffraction pattern expected at the at least one detector based on reference data and a pose estimate, and to refine the pose estimate based on a comparison of the at least one two-dimensional diffraction pattern with the at least one reference diffraction pattern.

[0032] This allows for pose estimation to be implemented with the high resolution of coherent measurement methods without the need for comprehensive reference measurements for calibration.

[0033] The evaluation device may be configured to determine multiple expected reference diffraction patterns and use these reference diffraction patterns to refine the pose estimate.

[0034] This allows for pose estimation to be implemented with the high resolution of coherent measurement methods without the need for comprehensive reference measurements for calibration.

[0035] The evaluation may include iterative refinement of the pose estimate.

[0036] Thus, the iterative method allows for efficient search for possible poses in the parameter space, particularly in a parameter space of up to 6 dimensions.

[0037] The system may further include a radiation source for generating coherent radiation or a radiation source arrangement having multiple sources. Different configurations for generating coherent radiation are possible. For example, the radiation source may be configured such that it outputs multiple coherent wavelength components simultaneously and optionally also generates these coherent wavelength components. In a further configuration, the multiple sources of the radiation source arrangement may be configured such that each of them outputs at least one of the coherent wavelength and components and optionally also generates at least one of them. The radiation source or radiation source arrangement may include at least one laser, frequency comb, and / or optical parametric oscillator. By using such a source, various degrees of freedom of the element can be determined with high accuracy. For example, a frequency comb capable of outputting multiple wavelength components, where each wavelength component is coherent and overall coherent with each other, may be advantageously used.

[0038] The at least one radiation source or the radiation source arrangement may be arranged stationary relative to the element. For example, the radiation source (e.g., the end of an optical fiber coupled to a laser or frequency comb) and the element may both be attached to the same carrier. For example, the carrier may be a workpiece, tool, or medical device, whose pose in the detection volume is intended to be determinable over a certain spatial rotation region.

[0039] The generation and evaluation of a diffraction pattern can be facilitated by using a radiation source or a radiation source arrangement that is arranged stationary relative to the diffraction element. In particular, it is no longer necessary to update the beam axis of the coherent radiation based on the current translation position of the diffraction element.

[0040] The diffraction element can be arranged movably relative to the at least one radiation source. The tracking mechanism of the system can be designed to update the beam axis of the coherent radiation such that the coherent radiation impinges on the diffraction element.

[0041] The at least one detector can include a plurality of channels for capturing different wavelength components.

[0042] Thus, these different wavelength components can be used to bijectively assign the plurality of diffraction patterns to exactly one pose.

[0043] The at least one detector can be configured to capture a synthetic wavelength formed by the coherent superposition of different wavelength components.

[0044] Thus, the further improvements in terms of magnification in resolution and / or dynamic range that can be achieved with wavelength components that are phase-stable with respect to each other can be used efficiently.

[0045] The at least one detector can include a plurality of detectors.

[0046] As a result, a further improvement in pose estimation can be achieved, for example, by being able to reduce the influence of possible shadows.

[0047] The surface normals of the detector surfaces of the plurality of detectors can be offset and / or tilted relative to each other.

[0048] As a result, a further improvement in pose estimation can be achieved, for example, by being able to reduce the influence of possible shadows.

[0049] The plurality of detectors can be positioned along the surface of the detection volume.

[0050] The evaluation device can be configured to computationally determine four, five, or six degrees of freedom of the element for pose estimation.

[0051] To refine the pose estimation determined using an additional measurement system, the evaluation device can be configured to computationally determine at least one degree of freedom of the element with a higher accuracy than can be achieved using the additional measurement system.

[0052] The system can also include an element at which the coherent radiation diffracts.

[0053] The element can be a diffraction element. The element can include a diffraction structure having a planar or volumetric arrangement of diffraction structure elements.

[0054] This enables a bijective assignment of the two-dimensional diffraction pattern to exactly one pose.

[0055] The diffractive element can be configured such that the two-dimensional diffraction pattern emerging at the at least one detector can be uniquely assigned to a pose of the diffractive element in the detection volume.

[0056] For this purpose, for example, an aperiodic volume diffraction structure can be defined in a way that depends on the dimensions of the detection volume, the desired solid angle range of resolvable rotations of the element, and the detector surface of the at least one detector.

[0057] This ensures that for the respective available detector configurations, a unique pose estimation within the desired dynamic range is possible.

[0058] The diffractive element can be configured such that it does not produce a repetition of the two-dimensional diffraction pattern of coherent radiation over a predetermined volume and / or over a predetermined solid angle range.

[0059] This ensures that for the respective available detector configurations, a unique pose estimation within the desired dynamic range is possible.

[0060] The diffractive element can be configured such that the diffractive element produces a diffraction pattern over a solid angle range of at least 2π, greater than 2π, greater than 3π or 4π.

[0061] This enables pose estimation even when the element rotates within a correspondingly large solid angle range.

[0062] The diffractive element can include a transparent or translucent material in which the diffraction structure is formed.

[0063] This can help to cover a relatively large solid angle range.

[0064] The diffractive element can include a glass or quartz material in which the at least one diffraction structure is formed.

[0065] Thus, good temperature stability of the measurement technique can be achieved.

[0066] The diffractive element can include a material with a coefficient of thermal expansion (CTE) value at room temperature not exceeding 100 ppb / K, not exceeding 50 ppb / K, not exceeding 20 ppb / K or not exceeding 10 ppb / K, such as a glass or quartz material with high temperature stability.

[0067] Thus, good temperature stability of the measurement technique can be achieved.

[0068] The diffractive element can have a faceted surface, for example a surface with one or more polyhedral parts.

[0069] This can promote diffraction into a relatively large solid angle range.

[0070] The diffraction structure can have a two-dimensional or three-dimensional diffraction structure.

[0071] This can facilitate a bijective assignment between the diffraction pattern and the pose. This also applies even if there is only one wavelength component, i.e., if the coherent radiation is selected as monochromatic radiation.

[0072] The diffraction structure can have a structure with a pseudo-random distribution, where the data used by the evaluation device depends on this pseudo-random distribution structure. In other words, the scattering centers can be randomly or pseudo-randomly distributed during the design process of the element, but the configuration of the diffraction structure is deterministic in the sense that it is known and can be used to evaluate this two-dimensional diffraction pattern or these two-dimensional diffraction patterns.

[0073] This contributes to the configuration of the element such that a unique assignment of the pose can be made with high accuracy even over a relatively large dynamic range (e.g., a relatively large detection volume and / or a relatively large solid angle range of possible rotations of the element).

[0074] The system can be an industrial manufacturing system, an industrial measurement system, or a medical engineering system.

[0075] The system can include a robot or any other actuator, such as a multi-axis robot, which is controllable to change the translational position and / or angular alignment of a workpiece, a tool, or a medical device. The pose of the workpiece, tool, or medical device can be estimated using the system according to the present invention. In this case, the diffraction element can be arranged on the workpiece, tool, or medical device, or on a movable part of the robot.

[0076] The system can include a human-machine interface by means of which the result of the pose estimation can be output.

[0077] In an alternative or additionally, the system can be configured to use the result of the pose estimation to control at least one actuator. The system can be configured such that an industrial manufacturing process, industrial quality control, and / or a medical engineering instrument are affected by the actuation of the at least one actuator.

[0078] According to another aspect of the present invention, a pose estimation method is elucidated, which includes the following steps: capturing a plurality of two-dimensional diffraction patterns using at least one detector, where the plurality of two-dimensional diffraction patterns are generated by diffraction of coherent radiation at an element; and estimating the pose of the element based on the plurality of two-dimensional diffraction patterns. The coherent radiation may include wavelength components with different wavelengths. In an alternative or additionally, the element may include a diffraction structure that allows a bijective assignment of one or more diffraction patterns generated by diffraction at the diffraction structure to exactly one pose.

[0079] Additional optional features of the pose estimation method and the effects achieved thereby respectively correspond to the features and effects described with reference to the system.

[0080] According to another aspect of the present invention, a diffraction element for pose estimation is elucidated, where the diffraction element includes a diffraction structure that allows a bijective assignment of one or more diffraction patterns generated by diffraction of coherent radiation at the diffraction structure to exactly one pose.

[0081] This achieves high resolution in pose estimation, including absolute pose estimation.

[0082] The diffraction element may include a transparent or translucent material in which the diffraction structure is formed.

[0083] This may help to cover a relatively large solid angle range. Shadow effects can be avoided.

[0084] The diffraction element may include a glass or quartz material in which the at least one diffraction structure is formed.

[0085] Thus, good temperature stability of the measurement technique can be achieved.

[0086] The diffraction element may include a material with a CTE value not exceeding 100 ppb / K, not exceeding 50 ppb / K, not exceeding 20 ppb / K, or not exceeding 10 ppb / K at room temperature, such as a glass or quartz material with high temperature stability.

[0087] Thus, good temperature stability of the measurement technique can be achieved.

[0088] The diffraction element may have a faceted surface, such as a surface with one or more polyhedral parts.

[0089] This can promote diffraction into a relatively large solid angle range.

[0090] The diffraction structure may have a two-dimensional or three-dimensional diffraction structure.

[0091] This enables a bijective assignment between the diffraction pattern and the pose. This also applies even if only one wavelength component is present, i.e., if the coherent radiation is selected to be monochromatic radiation.

[0092] The diffraction structure can have a pseudo-random distribution of structures, where the data used by the evaluation device depends on this pseudo-random distribution of structures. In other words, the scattering centers can be randomly or pseudo-randomly distributed during the design process of the element, but the configuration of the diffraction structure is deterministic in the sense that it is known and can be used to evaluate this two-dimensional diffraction pattern or these two-dimensional diffraction patterns.

[0093] This facilitates the configuration of the element, which enables a unique assignment of the pose with high accuracy even over a relatively large dynamic range (e.g., a relatively large detection volume and / or a relatively large solid angle range of possible rotations of the element).

[0094] According to another aspect of the invention, a device for use in estimating a pose is disclosed, the device comprising a diffraction element according to one aspect or an exemplary embodiment, and a radiation source or a radiation source arrangement of coherent radiation.

[0095] Different configurations of the radiation source or the radiation source arrangement for generating coherent radiation are possible. For example, the radiation source can be configured such that it outputs multiple coherent wavelength components simultaneously and optionally also generates these coherent wavelength components. In a further configuration, multiple sources of the radiation source arrangement can be configured such that each of them outputs at least one of the coherent wavelength and components and optionally also generates at least one of them. The radiation source or the radiation source arrangement can include at least one laser, a frequency comb, and / or an optical parametric oscillator. By using such a source, various degrees of freedom of the element can be determined with high accuracy. For example, a frequency comb capable of outputting multiple wavelength components can be advantageously used, where each wavelength component is coherent and overall coherent with each other.

[0096] The at least one radiation source or the radiation source arrangement can be arranged stationary relative to the element. For example, the device can include a carrier or a holding device, on which both the source / source arrangement and the element are arranged stationary. For example, the carrier can be a workpiece, a tool, or a medical device, whose pose in the detection volume is intended to be determinable over a certain spatial rotation region.

[0097] By using a radiation source or a radiation source arrangement arranged stationary relative to the diffraction element, the generation of the diffraction pattern and its evaluation can be facilitated. In particular, it is no longer necessary to update the beam axis of the coherent radiation according to the current translational position of the diffraction element.

[0098] The carrier or the holding device may be made of a material with low thermal expansion, such as invar alloy. The carrier or the holding device may be made of a material having a linear thermal expansion coefficient value of 10·10 -6 / K or less, 7·10 -6 / K or less, 5·10 -6 / K or less, or 3·10 -6 / K or less.

[0099] This allows for achieving high accuracy in pose estimation and robustness against thermal fluctuations.

[0100] According to one aspect, the present invention relates to an evaluation device for a pose estimation system. The evaluation device includes the following: an interface for receiving a plurality of two-dimensional diffraction patterns generated by diffraction of coherent radiation at an element; and a processing circuit configured to estimate the pose of the element based on the plurality of two-dimensional diffraction patterns.

[0101] Additional optional features of this evaluation device and the effects achieved thereby respectively correspond to the optional features described with reference to the pose estimation system.

[0102] According to another aspect of the present invention, machine-readable command codes are set forth, which, when executed by a programmable computing unit, perform the method according to one aspect or an exemplary embodiment of the present invention.

[0103] According to another aspect of the present invention, a storage medium storing machine-readable command codes is set forth, which, when executed by a programmable computing unit, perform the method according to one aspect or an exemplary embodiment of the present invention.

[0104] The method, system, and system components according to an exemplary embodiment of the present invention achieve various effects. Estimation of the absolute pose is possible. Estimating the absolute pose may include determining up to three translational degrees of freedom and / or up to three rotational degrees of freedom relative to a coordinate system defined by the at least one detector. Compared with various conventional methods for estimating the absolute pose, the pose estimation can be implemented with higher accuracy and within a larger dynamic range.

[0105] These methods, systems, and system components can be used in different fields. This includes pose estimation in an industrial environment, such as pose estimation in industrial manufacturing and / or quality control. These methods, systems, and system components can also be used in the context of medical engineering devices or medical engineering systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. In the drawings, like or identical reference numerals denote elements having like or identical configurations and / or functions.

[0107] Figure 1 A pose estimation system according to one exemplary embodiment is shown.

[0108] Figure 2 The system after the optical element has been translated and rotated is shown. Figure 1 system.

[0109] Figure 3 A pose estimation system according to one exemplary embodiment is shown.

[0110] Figure 4 A pose estimation system according to one exemplary embodiment is shown.

[0111] Figure 5 A pose estimation system according to one exemplary embodiment is shown.

[0112] Figure 6 is a schematic diagram of a diffractive element that can be used in the systems and methods according to the exemplary embodiments.

[0113] Figure 7 is a schematic diagram of a diffractive element that can be used in the systems and methods according to the exemplary embodiments.

[0114] Figure 8 is a schematic diagram of a diffractive element that can be used in the systems and methods according to the exemplary embodiments.

[0115] Figure 9 is a schematic diagram of a diffractive element that can be used in the systems and methods according to the exemplary embodiments.

[0116] Figure 10 is a flowchart of a method according to one exemplary embodiment.

[0117] Figure 11 is a flowchart of a method according to one exemplary embodiment.

[0118] Figure 12 is a flowchart of a method according to one exemplary embodiment.

[0119] Figure 14 is a schematic diagram of an evaluation device that can be used in the systems and methods according to the exemplary embodiments.

[0120] Figure 15 can be used by the Figure 14 evaluation device is a schematic diagram of a machine learning model.

[0121] Figure 16 and Figure 17 shows a source or radiation source arrangement that can be used in a pose estimation system and method according to the present invention.

[0122] Figure 18 is a schematic diagram of a unit that can be used in a system and method according to an exemplary embodiment.

[0123] Figure 19 is a schematic diagram of a diffraction element that can be used in a system and method according to an exemplary embodiment.

[0124] Figure 20 shows a system according to one exemplary embodiment. Detailed Description

[0125] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. In the drawings, like or identical reference numerals denote elements having like or identical configurations and / or functions.

[0126] Although the exemplary embodiments are described in the context of pose estimation in an industrial environment or medical engineering, the systems and methods described herein are not limited to these application areas.

[0127] The features of the exemplary embodiments may be combined with each other, unless this is explicitly excluded in the following description.

[0128] A pose estimation method and system according to an exemplary embodiment allow for the estimation of the absolute pose of an element. Estimating the absolute pose should be understood herein to mean determining up to three translational degrees of freedom and / or up to three rotational degrees of freedom in a reference frame, relative to which the element can move. For example, the reference frame may be defined by one or more detectors used in pose estimation for capturing two-dimensional diffraction patterns.

[0129] The captured diffraction patterns may each be speckle patterns.

[0130] The estimated pose may be the pose of the element, or the pose of a rigid part (such as a workpiece, tool, or surgical instrument) connected thereto and derived therefrom.

[0131] The pose estimation method and system may perform pose estimation using two-dimensional diffraction patterns in the far field. In this process, coherent radiation is radiated onto the element. In the art, the far field of a two-dimensional diffraction pattern caused by a diffraction structure should be understood to particularly refer to the distance between the detector and the diffraction structure of the optical element, which allows for the computationally determined two-dimensional diffraction pattern using the Fraunhofer approximation.

[0132] For example, the detector or each detector for capturing a two-dimensional diffraction pattern can be at a distance from the element that is at least twice the longest wavelength of the coherent radiation, three times the longest wavelength of the coherent radiation, five times the longest wavelength of the coherent radiation, or ten times the longest wavelength of the coherent radiation.

[0133] Figure 1 Fig. 10 shows a pose estimation system 10. The system 10 is configured to estimate the absolute pose of the optical element 30 in the detection volume 11. The system 10 can be configured to determine both the translational position and the rotation of the optical element 30 relative to a coordinate system 200 defined by at least one detector 12.

[0134] The system 10 includes at least the element 30, one or more detectors 12, and an evaluation device 20.

[0135] The element 30 is configured such that the pose can be unambiguously estimated based on the captured two-dimensional diffraction pattern. In other words, there is a bijection between any possible combination of the captured two-dimensional diffraction patterns and exactly one pose. For this purpose, the element can, for example, have a planar diffraction structure or a volume diffraction structure. The planar diffraction structure or the volume diffraction structure can have an aperiodic configuration, for example a random or quasi-random configuration, which is known to the system 10. It can be formed by techniques such as laser writing or three-dimensional printing. In addition to the aperiodic diffraction structure, the element 30 can optionally have additional structures, in particular additional diffraction structures. The additional diffraction structure can be periodic and can be used, for example, to determine an initial estimate of the pose, which is then refined by the techniques described in detail below.

[0136] The one or more detectors 12 are configured to capture a plurality of two-dimensional diffraction patterns generated by the diffraction of the coherent radiation 14 at the optical element 30. The coherent radiation 14 includes wavelength components 141, 142 having different wavelengths. Each of the wavelength components 141, 142 is coherent radiation of one wavelength. The different wavelength components 141, 142 can be phase-stable with respect to each other, but this is not mandatory.

[0137] The one or more detectors 12 are configured to capture the plurality of two-dimensional diffraction patterns in the far field in order to facilitate pose estimation in a computational manner. For this purpose, the one or more detectors 12 can be arranged around the detection volume 11 at a distance from the optical element 30. The position of each detector 12 can be defined to ensure that for each pose of the optical element 30 in the detection volume 11, the detector 12 captures one or more two-dimensional diffraction patterns in the far field.

[0138] Each detector 12 can include a two-dimensional arrangement of sensor pixels. These sensor pixels can be arranged on the detector surface. Various sensors known in the art can be used.

[0139] At least one detector 12 or a plurality of detectors 12 may have a plurality of color channels. The plurality of color channels may correspond to different wavelengths of the wavelength components 141, 142, but especially if the different radiation components are phase-stable with respect to each other, the plurality of color channels may also include at least one color channel corresponding to the composite wavelength resulting from the coherent superposition of two wavelength components 141, 142 that are phase-stable with respect to each other.

[0140] For example, if the coherent radiation has a first wavelength component 141 with a first wavelength λ1 and a second wavelength component 142 with a first wavelength λ2, the at least one detector 12 may be configured to capture radiation with a wavelength of λ1·λ2 / |λ1 - λ2|.

[0141] The coherent radiation diffracted at the optical element 30 is schematically represented as radiation 15 incident on the detector 12.

[0142] The evaluation device 20 is configured to evaluate the plurality of two-dimensional diffraction patterns captured by the one or more detectors 12. For pose estimation based on the two-dimensional diffraction patterns, the evaluation device 20 may also use data defining the configuration of the diffraction structure of the element 30. For example, the data used may originate from the production process of the diffraction structure. The data may define the arrangement of the diffraction structure elements in a planar diffraction structure or a volume diffraction structure produced, for example, by laser writing or 3D printing.

[0143] Reference is made below to Figures 11 to 15 describe in detail further features and operating modes of the evaluation device 20.

[0144] Figure 2 System 1 is shown after the element 30 has been translationally moved to a second position 17 in the coordinate system 200.

[0145] The translational movement from the first position 16 to the second position 17 causes a change in the two-dimensional diffraction pattern. System 10 may be configured to determine the first position 16 and the second position 17 in an absolute sense in each case, where the two-dimensional diffraction patterns captured in each case are evaluated. It is not necessary to use the two-dimensional diffraction pattern captured by the one or more detectors 12 when the element is in the first position 16 to determine the second position 17, and in particular the plurality of translational coordinates of the second position.

[0146] As Figure 2As schematically shown, the object coordinate system 18 of the optical element 30 can be rotated relative to the coordinate system 200 of the at least one detector. The rotation relative to the coordinate system 200 can be determined by up to three rotational degrees of freedom. For example, these rotational degrees of freedom can be specified by three Euler angles. The rotational position is also referred to herein as "orientation" to allow for a simple conceptual distinction from the translational position.

[0147] The system 10 can be configured to determine, in each case, the orientation of the optical element 30 in an absolute sense, wherein the two-dimensional diffraction patterns captured in each case are evaluated. A first orientation ( Figure 1 ) of the optical element can be determined based on at least a first two-dimensional diffraction pattern captured when the element is in the first orientation. A second orientation ( Figure 2 ) of the optical element can be determined based on a second two-dimensional diffraction pattern captured when the element is in the second orientation; the second orientation can be determined independently of the first two-dimensional diffraction pattern.

[0148] As Figure 3 and Figure 4 shown, the system can include a radiation source 13 or a radiation source arrangement 130 having a plurality of radiation sources 131, 132, which is configured to radiate coherent radiation 14 onto the element 30. The radiation source 13 or the radiation source arrangement 130 can include one or more lasers. The radiation source 13 or the radiation source arrangement 130 can be configured to radiate coherent radiation having a plurality of wavelength components onto the element 30.

[0149] The radiation source 13 or the radiation source arrangement 130 can include one or more lasers, each laser being configured to radiate a plurality of wavelength components onto the element 30. The radiation source arrangement 130 can include a plurality of radiation sources 131, 132, such as a plurality of lasers, each radiation source being configured to radiate exactly one wavelength component of the wavelength components onto the element 30.

[0150] The radiation source 13 can be configured to radiate coherent radiation 14 having different wavelength components onto the element 30. For example, the radiation source 13 can include a frequency comb generator, which is configured to radiate different wavelength components onto the element 30.

[0151] As Figure 3 and Figure 4 shown, the system can include a plurality of detectors 12, 121. The detectors 12, 121 can be arranged in a manner distributed around the detection volume 11. The surface normals of the detector surfaces (e.g., of the pixel arrays) of the respective detectors 12, 121 can be inclined relative to each other. These detector surfaces can each be oriented such that their surface normals generally point towards the center 17 of the detection volume 11.

[0152] Each detector of the detector 12 or of one, several or all of the detectors 12, 121 may be configured to acquire phase information. For this purpose, at least one interferometer may be provided. The surface reconstruction of the element 30 may be determined based on the phase information. In an alternative or additionally, the phase information may be used for a bijective assignment between the diffraction pattern (including amplitude and phase information) and the pose.

[0153] Figure 5 A variant of the system 10 is shown, in which the phase information is used for the holographic evaluation 50. For this purpose, the evaluation device 20 or a computing unit separate therefrom may be configured to evaluate the phase information of at least one wavelength component 141, 142 and advantageously to evaluate the phase information of a plurality of said wavelength components. The holographic evaluation may be carried out according to known digital holography principles. The holographic evaluation may be used as a starting point for a more accurate estimation of the pose based on the plurality of diffraction patterns.

[0154] As Figure 1 and Figure 5 shown, the system 10 may include one or more components that further utilize the pose determined by the evaluation device 20. In Figure 1 and Figure 5 a human-machine interface 19 and a controller 40 are schematically depicted, and the human-machine interface and the controller may receive and represent the estimated pose from the evaluation device 20 or use said pose for controlling operations. The controller 40 may be a controller for at least one actuator from an industrial manufacturing or quality control system or a medical engineering system.

[0155] The evaluation device 20 may be configured to output the determined pose to the human-machine interface 19, for example in order to visually represent the determined pose.

[0156] The evaluation device 20 may be configured to output the determined pose to the controller 40 in order to be able to control at least one actuator according to a control loop.

[0157] The evaluation device 20 may include a network interface and may be configured to transmit the determined pose via a local area network or a work flow network. For example, this may be used for recording the pose and / or monitoring the system 10 from a remote location.

[0158] The pose estimation system according to the present invention and the pose estimation method performed by the system achieve various advantages over conventional methods using coherent radiation. One such conventional method is so-called speckle localization, in which a partially reflective object is irradiated with coherent radiation and the resulting interference pattern is compared with a reference measurement. The geometry of the partially reflective surface from speckle localization is generally unknown. Thus, the method again requires a reference measurement. The partially reflective surface from speckle localization also limits the solid angle on which the pose can be estimated. Additionally, on the detector side, speckle localization has the following limitation: for the correlated reconstruction of pose changes, the characteristic interference pattern from the reference measurement can only be shifted by no more than the field of view of the camera, or alternatively requires complex additional reference measurements.

[0159] On the other hand, the pose estimation system and method according to the present invention allow for the estimation of the absolute pose without requiring a reference measurement. The high resolution of the coherent measurement method is maintained in this process.

[0160] The pose estimation technique according to the present invention particularly has the following effect: maintaining the accuracy of the coherent measurement method, but making it possible to cover a larger solid angle.

[0161] Figures 6 to 9 A schematic configuration of element 30 is shown. Element 30 includes a diffraction structure 31 ([ Figure 6 ) with diffraction structure elements arranged in a planar arrangement and / or diffraction structures 32, 33 ([ Figure 7 , Figure 8 and Figure 9 ) with diffraction structure elements arranged in a volume arrangement. Compared with a one-dimensional arrangement, the more complex configuration of the diffraction structure helps to more easily perform the bijective assignment of the pose and the diffraction pattern.

[0162] The technique according to the present invention also allows for defining the design specifications and related manufacturing parameters of element 30 such that a desired pose estimation within the dynamic range is possible. The technique according to the present invention also allows for implementing a measurement rule that transfers the accuracy embodied by element 30 to the coordinate system 200 of the detector and generally to the pose estimation of the optical element 30 (in up to 6 degrees of freedom or 6 dimensions of the pose space).

[0163] Element 30 can be configured such that diffraction patterns are generated at the detector 12 or detectors 12, 121 in the far field under coherent illumination, and these diffraction patterns allow for the bijective assignment of the diffraction pattern and the pose of element 30.

[0164] The diffractive structures 32, 33 can include aperiodic gratings or another aperiodic volume diffractive structure. Periodic gratings or any other periodic diffractive structure have the property under coherent illumination that the local interference pattern in the detector plane cannot be uniquely assigned to one pose. This drawback can be overcome with an aperiodic grating or any other aperiodic planar diffractive structure or volume diffractive structure. To determine a suitable configuration of the aperiodic planar diffractive structure or volume diffractive structure, the required information content of the diffractive structure can be estimated as follows: The surface of the detection volume 11 can be subdivided into a plurality of parts, each of which has the size of the detector surface of one of these detectors 12, 121. Each of these parts can be assigned a unique, spatially discrete "target" pattern. Different color channels can be considered in this process, for example by assigning target patterns to each color channel. A useful reference parameter in this context is the pixel size of the detector used. Based on these initial target diffraction patterns, the reciprocal distribution of the required grating volume of the optical element 30 corresponding to the coherent illumination can be calculated using the discrete Fourier transform. The minimum producible structure size (which can depend, for example, on the pixel size of the detector 12) and the desired maximum size of the optical element 30 into which the diffractive structure is introduced can be used as boundary conditions. Additional boundary conditions can be considered when determining the aperiodic planar diffractive structure or volume diffractive structure.

[0165] As determined above, the diffractive structure at which the coherent radiation is scattered defines the producible diffractive element. For the diffractive structure, both the far-field diffraction pattern of the diffractive element and the diffractive structure elements can be arranged quasi-randomly. Nevertheless, the diffractive structure is known and fully computable. This represents another difference from speckle localization, where the geometry of the partially reflecting surface is usually unknown.

[0166] For the manufacture of such a diffractive element 30, this allows for a targeted construction and verification of the spatial structure of the diffractive element 30. For example, microscopic methods can be used for this purpose. Alternatively or additionally, it becomes possible to verify the two-dimensional diffraction pattern at one or more detector levels.

[0167] The technical advantage of this approach is that one or several detectors can be used to determine poses that can cover a large solid angle, optionally even the full solid angle range. For this purpose, the evaluation device 20 can compare the captured two-dimensional diffraction pattern or the captured two-dimensional diffraction patterns with a reference diffraction pattern calculated for the element 30. Since the diffractive structure is specifically produced and / or the data defining the diffractive structure can be available to the evaluation device 20, the reference diffraction pattern can be calculated by forward propagation.

[0168] Another technical advantage of this technology according to an exemplary embodiment is that the diffraction structures 31, 32, 33 can be implemented in materials with low thermal expansion coefficients such as glass or quartz materials. This ensures improved insensitivity to temperature changes. Known glasses with low thermal expansion can be used. For example, it is possible to use glass or quartz materials with a coefficient of thermal expansion (CTE) value not exceeding 100 ppb / K, not exceeding 50 ppb / K, not exceeding 20 ppb / K, or not exceeding 10 ppb / K at room temperature (23 °C). Thus, a diffraction element 30 on the order of 1 cm can achieve an accuracy of <1 nm in the measurement standard within a temperature band of 1 K. When projected onto the detection plane of the detector 12 or these detectors 12, a lateral accuracy of <1 μm in pose can be achieved.

[0169] The element 30 can be configured such that there are diffraction images within a solid angle range of at least 2π, at least 3π, or a full solid angle of 4π. Shadows can be accepted.

[0170] As will be described in detail below, some requirements for the element 30, particularly for the aperiodic volume diffraction structure 30, can be relaxed.

[0171] For example, it is not mandatory to form a two-dimensional diffraction pattern uniquely associated with the pose in each color channel on the detection surface of the detector 12 for the entire surface of the detection volume 11. Ambiguity can be accepted, and the ambiguity can be resolved, for example, by using a redundant measurement system.

[0172] In addition to the aperiodic planar diffraction structure or volume diffraction structures 31, 32, 33, the element 30 can include at least one additional diffraction structure, such as a periodic grating, for resolving any ambiguity and / or for determining an initial estimate of the pose. These different diffraction structures can differ in terms of the average distance from the structural elements.

[0173] Due to the use of coherent radiation including wavelength components with at least two different wavelengths, the technology disclosed herein can be performed independently for multiple color channels. This relaxes the diffraction structure requirements. Much simpler diffraction structures can be used because the combination of diffraction patterns from different color channels can be used for pose estimation. If the radiation components with different wavelengths are phase-stable with respect to each other, it is also possible to generate a synthetic wavelength by interference that will further improve the accuracy and / or increase the dynamic range.

[0174] The diffractive element 30 need not be configured in such a way that it radiates a diffractive pattern over the entire solid angle of 4π. For example, radiation into a smaller solid angle range (e.g., into a half-space) may be sufficient for many applications. Alternatively, a plurality of diffractive elements 30 may be combined on a carrier, e.g., on different side surfaces of an industrial or medical tool, where each diffractive element 30 radiates into a solid angle range less than the 4π solid angle range. However, by using the plurality of diffractive elements 30, the pose of the carrier can still be estimated over the entire solid angle range.

[0175] The functionality of the evaluation device 20 for a system and method according to an exemplary embodiment is described in more detail below.

[0176] Figure 10 A flowchart of a method 60 according to an exemplary embodiment is shown. The method 60 can be automatically executed by the evaluation device 20.

[0177] At 61, the evaluation device 20 receives a plurality of two-dimensional diffractive patterns. The plurality of two-dimensional diffractive patterns are captured by at least one detector 12 in a plurality of color channels and are the result of the diffraction of coherent radiation at the diffractive structure of the element 30.

[0178] At 62, the two-dimensional diffractive patterns are processed to estimate the pose of the element 30. Data defining the configuration of the diffractive structure can be used for pose estimation. For example, such data can be obtained from the manufacturing process of the diffractive structure of the element 30.

[0179] At 63, the estimated pose is provided. This can include output via the human-machine interface 19 or providing the pose to at least one controller 40.

[0180] Figure 11 A flowchart of a method 70 according to an exemplary embodiment is shown. The method 70 can be automatically executed by the evaluation device 20.

[0181] At 71, the pose is estimated. The pose can be estimated by using the phase information of the wavelength components of the diffraction. The pose estimation can be determined using digital holographic interferometry. The pose estimation can also be determined based on the actuation signals of an actuator chain that moves the element 30 in the detection volume 11.

[0182] At 72, the pose estimation is refined. This can be done in various ways. For example, in the case of knowledge of the diffractive structure, a forward calculation can be performed that determines the diffractive patterns expected in each of the plurality of color channels for one or more possible candidate poses in the vicinity of the pose estimation.

[0183] At 73, it can be checked whether the desired target accuracy has been achieved. This check can include determining an error metric that quantifies the difference between the diffraction pattern captured in different color channels and the diffraction pattern determined by calculation for a candidate pose. Any suitable difference metric for the image can be used to quantify this difference.

[0184] If the desired target accuracy has not been achieved, the method can return to step 72.

[0185] Otherwise, an estimated pose can be provided at step 74, as explained with reference to step 73.

[0186] In addition to the aperiodic volume diffraction structure, the diffraction element 30 can also include additional features that can facilitate or be used for pose estimation.

[0187] Figure 12 A flowchart of a method 80 according to an exemplary embodiment is shown. The method 80 can be automatically executed by the evaluation device 20.

[0188] At 81, the evaluation device 20 processes the phase and intensity information from the two-dimensional diffraction pattern. This processing can include digital holographic reconstruction of the surface of the element 30. This processing can include pose estimation of the element 30.

[0189] At 82, the two-dimensional diffraction pattern is further processed using data related to the configuration of the diffraction structure of the element 30. This data can define the geometry of the diffraction structure of the element 30.

[0190] At 83, the estimated pose is provided. This can include output via the human-machine interface 19 or providing the pose to at least one controller 40.

[0191] Figure 13 A flowchart of a method 90 according to an exemplary embodiment is shown. The method 90 can be automatically executed by the system 10.

[0192] At 91, a plurality of diffraction patterns generated by diffraction of coherent radiation having at least two different wavelengths are captured.

[0193] At 92, digital holographic reconstruction of the surface of the element 30 is implemented as an initial pose estimation. For this purpose, the evaluation device 20 processes the phase and intensity information from the two-dimensional diffraction pattern.

[0194] At 93, a plurality of reference diffraction patterns are calculated using data related to the configuration of the diffraction structure of the element 30. The corresponding forward calculation can be implemented with little computational effort. The plurality of reference diffraction patterns are calculated for different wavelength components.

[0195] At 94, refine the pose. For this purpose, a distance metric for image comparison can be used to determine for which poses the reference diffraction pattern determined by calculation in different color channels is most similar to the actually captured diffraction pattern, i.e., the distance metric is as small as possible.

[0196] At 95, provide the estimated pose. This can include output via the human-machine interface 19 or providing the pose to at least one controller 40.

[0197] Figure 14 A block diagram representation of an evaluation device 20 that can be used in the system 10 and method according to an exemplary embodiment is shown.

[0198] The evaluation device 20 includes at least one first interface 21 by means of which a plurality of two-dimensional diffraction patterns 50 can be received. The at least one first interface 21 can be communicatively connected directly or via a data network (such as a wireless or wired local area network) to this detector 12 or these detectors 12. By means of the at least one first interface 21, the evaluation device 20 is also capable of receiving data 100 that depends on the diffraction structure of the element 30. For example, the data 100 can define the geometry of the diffraction structure as defined in the manufacture of the optical element 30 and inscribed in the element 30. The data 100 can include a configuration file for the manufacture of the optical element 30, for example in the form of an XML file, which is used during the manufacturing process.

[0199] The evaluation device 20 includes a processing circuit 24. The processing circuit 24 is configured to process the two-dimensional diffraction patterns 50 for pose estimation. Optionally, the processing circuit can process the data 100 that depends on the configuration of the diffraction structure by calculation. This allows the pose in the detection volume 11 to be estimated.

[0200] The pose estimation 25 performed by the processing circuit 24 can include processing the phase information obtained from the diffraction pattern. For example, digital holographic reconstruction can be performed and / or both the amplitude and phase information can be compared in order to match the actually captured diffraction pattern with the computationally determined reference diffraction pattern.

[0201] Regardless of whether phase information is also used in addition to the amplitude information, the assumed pose for the optical element 30 can be matched with the captured two-dimensional diffraction pattern 50 by the reference diffraction pattern determined by calculation according to the data 100 for pose estimation. The processing circuit 24 can determine the reference diffraction pattern for each of the plurality of candidate poses and the plurality of wavelengths of the coherent radiation by calculation in order to match these reference diffraction patterns with the captured two-dimensional diffraction pattern 50 in each case.

[0202] The results of the pose estimation 25 can be stored in the storage system 23 of the evaluation device 20 for further use and can be retrieved from there by the processing circuit 24 when necessary.

[0203] The processing circuit 24 may include one or more integrated circuits for performing the required processing on the two-dimensional diffraction pattern. The one or more integrated circuits may include, for example, any desired one or any desired combination of the following circuits or circuit components: application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), processor, controller, one or more quantum gates, circuits for quantum information processing.

[0204] The results of the pose estimation can be output via the at least one first interface 21 of the evaluation device 20 or via a second interface 22 different therefrom. For example, the results can be output to the human-machine interface or controller of an industrial or medical engineering system.

[0205] To perform the pose estimation, the evaluation device 20 can utilize at least one trained machine learning model 35.

[0206] Figure 15 FIG. is a schematic diagram of the machine learning model 35 that can be executed by the evaluation device 20. The machine learning model 35 includes an input layer 36, a plurality of hidden layers 38, and an output layer 37.

[0207] The input layer 36 can be configured to receive the pixel values of the diffraction pattern captured by the at least one detector. The output layer 37 can be configured to output information about the pose.

[0208] In a further configuration, the input layer 36 can be configured to receive both the pixel values of the diffraction pattern captured by the at least one detector and the reference pixel values determined by calculation, where the reference pixel values are determined by calculation based on the data 100 about the diffraction structure 31. The output layer 37 can then output a probability value that indicates whether the captured two-dimensional diffraction pattern corresponds to the same pose for which the reference pixel values have been determined by calculation.

[0209] The technical advantage of the technology described herein is that the labeled data required to train the machine learning model 35 can be determined computationally in large quantities based on data 100 regarding the diffraction structure. For example, for each of the multiple possible poses of the component 30 in the detection volume 11, the two-dimensional reference diffraction pattern expected at the detector 12 or the detectors 12, 121 for the corresponding pose can be determined computationally by a forward propagation method. The reference diffraction pattern determined computationally can then be labeled with the known pose for which the forward propagation has been performed. During training, which can utilize known techniques such as gradient-based methods, the machine learning model 35 can be trained to the effect of assigning the correct pose to each reference diffraction pattern by setting the adjustable parameters of the machine learning model 35.

[0210] More complex techniques can be used to implement the trained machine learning model 35. For example, a generative adversarial network (GAN) can be used to form and train the machine learning model 35.

[0211] The use of the machine learning model is optional. The evaluation device 20 can solve the inverse problem of the reconstruction from momentum space to position space for estimating the pose in a different way.

[0212] For example, the evaluation device 20 can be configured to computationally determine the expected diffraction pattern of the multiple color channels for each desired pose. A comparison with the captured two-dimensional diffraction pattern can then be made. The expected diffraction pattern (also referred to as the reference diffraction pattern) can be stored in the evaluation device 20 or a separate storage system in order to keep the latency in pose estimation as low as possible.

[0213] In an alternative or additionally, the evaluation device 20 can be configured to use an approximation method. In this case, the initial estimate of the pose, which does not need to rely on the diffraction pattern of the diffraction structure, can be further refined. The initial estimate can be received from an additional measurement system with a coarser resolution or from the controller of the actuator chain.

[0214] In an alternative or additionally, the evaluation device 20 can be configured to perform an iterative procedure for estimating the pose. The estimate of the pose can be refined more and more.

[0215] The above techniques can also be combined with each other.

[0216] The radiation source 13 or the radiation source arrangement 130 generates coherent radiation having multiple wavelengths. The radiation sources 13, 130 are sufficiently frequency-stable to allow the diffraction pattern to have a well-defined phase angle during the relevant measurement period in which the diffraction pattern is located.

[0217] Figure 16The configuration of the radiation source 13 that outputs wavelength components 141, 142 of different wavelengths is shown. Color demultiplexing can be used to capture diffraction patterns at multiple wavelengths. Thus, an additional degree of freedom is used in the detection. The diffraction pattern can be evaluated for different color channels of the detector. This can be done as described above. The data 100 according to the configuration that can define the diffraction structure can be determined computationally for different wavelengths separately to calculate the expected diffraction pattern.

[0218] The different wavelength components 141, 142 can be phase-stable with respect to each other. This additionally allows the use of a synthetic wavelength that can be generated by the interference between the individual wavelength components 141, 142. Similar to digital holography, this can achieve good precision.

[0219] The different wavelength components 141, 142 can be output by the same physical unit. For example, the radiation source 13 can include a frequency comb generator that outputs the wavelength components 141, 142.

[0220] As Figure 17 shown, the system can also include a radiation source arrangement 130 having a plurality of physically separate radiation sources 131, 132. For example, a first radiation source 131 that outputs a wavelength component 141 of a first wavelength and at least one additional radiation source 132 that outputs a wavelength component 142 of a second wavelength can be provided.

[0221] The systems and methods described in detail can be used for pose estimation in industrial manufacturing plants and / or quality control systems or medical engineering systems. The estimated pose can be provided to a controller 40, which actuates at least one actuator based on the estimated pose.

[0222] Figure 17 is a schematic diagram of the system 10 including the robotic arm 41. Instead of the robotic arm 41, other mechanisms can also be used to move the target object 42 in the working area.

[0223] The pose estimated by the evaluation device 20 can be supplied to a controller 40, which is configured to generate at least one control signal for an actuator (such as of a robotic arm) based on the estimated pose. In this way, the pose of the target object 42 (such as a workpiece, a tool, or a surgical instrument) can be checked and (if necessary) adjusted.

[0224] As Figure 17As shown, the diffractive element 30 can be firmly attached to the target object 42 or integrally formed with the target object. Since the diffractive element 30 is typically arranged offset relative to a predefined point of the target object 42 (e.g., the volume center point or the tool tip), it is generally necessary to determine all three translational coordinates and all three rotational coordinates of the diffractive element 30 in order to determine the position of the predefined point of the target object 42 in the detection volume.

[0225] The pose estimation system and method use elements 30 configured in a purposeful manner. The diffractive structure can be formed in such a way that if the data defining the configuration of the aperiodic volume diffractive structure is available for evaluation, the aperiodic volume diffractive structure allows a unique assignment of any desired pose in the detection volume based on at least one diffraction pattern. This data can be provided during the production of the diffractive element 30 for use by the pose estimation system or method.

[0226] Figure 18 A unit with an element 30 is shown that can be used in the system and method according to an exemplary embodiment.

[0227] In this case, the radiation source 13 or the radiation source arrangement is optionally firmly connected to the element 30 via a mechanical coupling element 110. For example, the coupling element can be made of a material such as invar or another material having a low thermal expansion (e.g., comparable to or less than that of invar). The coupling element 110 can be a carrier or a holding device. The coupling element can have a value of the linear thermal expansion coefficient at 23 °C of 10·10 -6 / K or less, 7·10 -6 / K or less, 5·10 -6 / K or less, or 3·10 -6 / K or less of a material.

[0228] The evaluation of the diffraction pattern can be simplified by this firm connection.

[0229] The element 30 can have different surface shapes and geometries. For example, the element 30 can have a geometry that helps to radiate the diffraction pattern over a desired solid angle range. For this purpose, the element 30 can be configured, for example, to have a faceted surface. The element 30 can have a surface including polyhedral-shaped regions. The faceted surface can have a polyhedral structure without symmetry or a statistical triangular shape.

[0230] Figure 19 is a schematic view of the element 30, which is produced (in particular formed) such that its surface has polyhedral-shaped regions 34. Thus, a characteristic can be achieved such that the diffraction pattern is radiated over a relatively large solid angle range 150.

[0231] The systems and methods described in detail can be used for pose estimation in industrial manufacturing plants and / or quality control systems or medical engineering systems. The estimated pose can be provided to a controller 40, which actuates at least one actuator based on the estimated pose.

[0232] Figure 20 FIG. 4 is a schematic view of a system 10 that includes a robotic arm 41. Instead of the robotic arm 41, other actuators or actuator chains can also be used to move a target object 42 in the working area.

[0233] The pose estimated by the evaluation device 20 can be supplied to a controller 40, which is configured to generate at least one control signal for an actuator (such as a robotic arm) based on the estimated pose. In this way, the pose of the target object 42 (such as a workpiece, tool, or surgical instrument) can be checked and, if necessary, adjusted.

[0234] As Figure 20 shown, the element 30 can be fixedly connected to the target object 42 or integrally formed with the target object. Since the diffraction element 30 is typically arranged offset with respect to a predefined point of the target object 42 (such as the volume center point or the tool tip), it is generally necessary to determine all three translational coordinates and all three rotational coordinates of the diffraction element 30 in order to determine the position of the predefined point of the target object 42 in the detection volume.

[0235] As described with reference to the accompanying drawings, the pose estimation system and method use an element that includes a diffraction structure. The diffraction structure is configured such that the diffraction patterns captured for different wavelengths are bijectively assigned to poses in the detection volume. The diffraction structure is advantageously known and follows reproducible design rules. This is different from diffraction structures that use random interference at two-dimensional surfaces or three-dimensional structures.

[0236] The diffraction image generated by the element can cover a solid angle of at least 2π, at least 3π, or 4π. Due to the design of the diffraction element and its application, shadows can be accepted.

[0237] The diffraction structure can be configured based on the configuration of the system 10, in particular based on the size of the detection volume 10 and based on the detector surface and / or pixel size and / or number of color channels of the at least one detector 12.

[0238] Calibration can be performed based on the configuration of the element 30 and / or the extrinsic camera parameters of the detector 12 or the detectors 12, 121.

[0239] The intensity distribution generated by the coherent illumination of the diffraction structure 31 at different wavelengths in the detection volume can be calculated. For this purpose, for example, the Gerchberg-Saxton method or some other technique that takes into account the angular range of irradiation and / or detection capable of coherent radiation can be used.

[0240] These diffraction patterns can be captured by at least one detector 12. Multiple detectors 12 can be used to cover all degrees of freedom, especially all three translational degrees of freedom and all three rotational degrees of freedom.

[0241] These systems and methods can be used not only to estimate the pose of the diffraction element, but also to determine higher torques. For example, these systems and methods can also be used to determine translational velocity and / or acceleration and / or rotational velocity and / or acceleration. These can also be output via the human-machine interface 19 or provided to the controller 40.

[0242] These systems and methods can use the estimate of the pose and refine this estimate. Digital interferometric holography can be used to determine this estimate. Alternatively or additionally, this estimate can be obtained from the kinematic chain of the actuator, and the evaluation device 20 can be communicatively connected to the kinematic chain and the kinematic chain changes the pose of the diffraction element 30.

[0243] The diffraction element 30 can be produced by laser writing or other techniques (such as 3D printing). Such techniques allow the formation of aperiodic volume diffraction structures with dimensions less than 5 cm, less than 3 cm, less than 2 cm, less than 1 cm, less than 5 mm, less than 3 mm, less than 1 mm, less than 0.5 mm, less than 0.3 mm or less than 0.1 mm. However, by using a conventional detector surface and pixel size, this aperiodic volume diffraction structure can achieve a bijective mapping of pose and diffraction pattern in a detection volume with a radius of at least 1 cm, at least 3 cm, at least 5 cm, at least 10 cm, at least 30 cm, at least 50 cm, at least 1 m, at least 2 m, at least 3 m or larger.

[0244] Although the exemplary embodiments have been described with reference to the accompanying drawings, modifications can be implemented in additional exemplary embodiments. For example, the following modifications can be used alternatively or cumulatively:

[0245] - Fewer than three rotational coordinates and / or fewer than three translational coordinates can be determined. For example, if high accuracy is only a concern for the angular degrees of freedom, the coherent illumination can be positioned such that the diffraction element moves relative to it. Then, a rough tracking of the coherent radiation is sufficient.

[0246] - If at least one auxiliary measurement system 90 is used to estimate the pose at a lower resolution, the requirements for the bijective assignment between the pose and the diffraction pattern can be relaxed.

[0247] - The diffraction image can cover the full solid angle, but this is not mandatory. For example, a half-space or a smaller angular range can be sufficient for the desired pose estimation for the corresponding application. This allows reducing the illuminated volume of the diffraction element.

[0248] Although descriptions of exemplary embodiments that can be used in systems for industrial manufacturing or quality control or medical engineering have been given, the disclosed technology can also be used in other application areas.

[0249] This disclosure also encompasses embodiments having any combination of the features elucidated or shown with respect to different embodiments. This disclosure also encompasses the individual features in the drawings, even if they are shown there in combination with other features and / or are not mentioned above or below. Alternatives of the embodiments described in the drawings and the specification and individual alternatives of their features can also be excluded from the subject matter of the present invention or the disclosed subject matter.

[0250] The expressions "comprising" and "having" and their derivatives refer to a non-exhaustive relationship and do not exclude the presence of other elements or steps. The indefinite article "a" or "an" and their derivatives do not exclude the presence of multiple corresponding elements. The functions of the multiple features elucidated in the claims can be implemented by a unit or a step.

[0251] Machine-readable command code that can be executed by a programmable circuit for the purpose of performing the method according to an exemplary embodiment can be stored and / or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, which is provided together with or as part of other hardware. The command code can also be distributed in other forms, for example, in the form of a modulated data signal sequence.

[0252] Exemplary embodiments of the present invention provide improved techniques for pose estimation that allow estimating the absolute pose (in particular up to six degrees of freedom) with interferometric accuracy.

Claims

1. A pose estimation system (10), comprising: at least one detector (12; 12, 121), the at least one detector being configured to capture a plurality of two-dimensional diffraction patterns (50) generated by diffraction of coherent radiation having different wavelength components (141, 142) at an element (30), and evaluation means (20), the evaluation means being configured to estimate the pose of the element (30) based on the plurality of two-dimensional diffraction patterns (50).

2. The system (10) according to claim 1, wherein, These different wavelength components (141, 142) are phase-stable with respect to each other.

3. The system (10) according to claim 2, wherein, The evaluation means (20) is configured to use the relative phase angles of these different wavelength components (141, 142) to estimate the pose of the element (30).

4. The system (10) according to claim 3, wherein, The system (10) is configured to allow the relative phase angle of these different wavelength components (141, 142) to have variable adjustability.

5. The system (10) according to any one of the preceding claims, wherein, The evaluation means (20) is configured to perform pose estimation using reference data (100), the reference data depending on the configuration of the diffraction structure of the element (30).

6. The system (10) according to any one of the preceding claims, wherein, The evaluation means (20) is configured to perform a multi-stage evaluation process for the pose estimation, the multi-stage evaluation process including an initial pose estimation and a refinement of the pose estimation based on the plurality of two-dimensional diffraction patterns (50).

7. The system (10) according to claim 6, dependent on claim 5, wherein, The evaluation means (20) is configured to use the reference data (100) to refine the pose estimation.

8. The system (10) according to any one of the preceding claims, further comprising a radiation source (13) for generating the coherent radiation or a radiation source arrangement (130) having a plurality of radiation sources, wherein, The radiation source (13) or the radiation source arrangement (130) includes at least one laser, a frequency comb, and an optical parametric oscillator.

9. The system (10) according to any one of the preceding claims, wherein, The at least one detector (12; 12, 121) includes a plurality of channels for capturing these different wavelength components (141, 142) and / or for capturing a synthetic wavelength formed by the coherent superposition of these different wavelength components (141, 142).

10. The system (10) according to claim 9, wherein, The at least one detector includes a plurality of detectors (12, 121), wherein the surface normals of the detector surfaces among the plurality of detectors (12, 121) are offset and / or tilted with respect to each other.

11. The system (10) according to any one of the preceding claims, wherein, The evaluation means (20) is configured to computationally determine six degrees of freedom of the element (30) for the pose estimation.

12. The system (10) according to any one of the preceding claims, further comprising the element (30), wherein, The element (30) includes a diffraction structure (31, 32, 33), the diffraction structure having a planar or volumetric arrangement of diffraction structure elements.

13. The system (10) according to any one of the preceding claims, wherein, The system (10) includes or is an industrial manufacturing system, an industrial measurement system, or a medical engineering system.

14. A pose estimation method, comprising: using at least one detector (12; 12, 121) to capture a plurality of two-dimensional diffraction patterns (50), wherein the plurality of two-dimensional diffraction patterns (50) are generated by diffraction of coherent radiation having different wavelength components (141, 142) at an element (30), and estimating the pose of the element (30) based on the plurality of two-dimensional diffraction patterns (50).

15. The method according to claim 14, wherein The method is performed by the system (10) according to any one of claims 1 to 13.

Images