SYSTEM, METHOD AND MARKER FOR DETERMINING THE POSITION OF A MOVING OBJECT IN SPACE

Description

[0001] This application claims priority from German patent application No. 10 2017 113 615.0, filed on June 20, 2017.

[0002] The invention relates to a system for 6D position determination of an object moving in space with 6 degrees of freedom.

[0003] The invention further relates to a method for 6D position determination of an object movable in space with 6 degrees of freedom.

[0004] There are a wide variety of industrial applications in which precise position determination is required to control the motion of an object. Such an object could be, for example, a robot arm, for example, in mirror manufacturing, whose 6-dimensional (6D) position must be precisely known and controlled. Other application examples concern the 6D position control of movable mirrors, for example, in a projection exposure system to generate a precise image of a reticle structure on the semiconductor substrate to be exposed. Current position control systems are often based on conventional sensors such as optical encoders, capacitive sensors, and eddy current sensors.

[0005] For the purposes of the present invention, a position of an object in space is understood to be a position according to N degrees of freedom of movement, where N can be 2, 3, 4, 5, or 6. For example, a 6D position is a position of the object in space according to 3 degrees of freedom of translation and 3 degrees of freedom of rotation. The term "position" thus also encompasses the orientation of the object in space.

[0006] Due to the improved performance of image sensors in recent years, position determination can also be carried out by capturing an image of the object with one or more cameras. DE 10 2009 009 372 A1 proposes a device and a method for monitoring the orientation of at least one mirror, for example in a projection exposure system for microlithography, in which a detection device comprising a camera detects a pattern provided by a pattern source with spatially and / or temporally variable light sources that are reflected by the mirror onto the detection device. The orientation of the mirror can be determined from the mirror image captured by the detection device. According to this document, the pattern provided by the pattern source can be a noise pattern, i.e.a pattern generated by a random process, such as surface areas of components in the vicinity of the mirror to be monitored, for example, painted or textured surfaces of the housing of a projection exposure system or the like. For the pattern comparison, i.e., the comparison of the pattern imaged in the detection device with the original pattern, various image or pattern recognition methods or comparable methods, in particular correlative methods, can be used.

[0007] Although the present invention can also be used in applications where very precise position determination in the sub-micrometer range or even in the sub-nanometer range is required, the present invention is particularly suitable for the position determination of a moving object in a large working volume, for example a size of 1 m 3< , and with large working distances between the measuring system and the object.

[0008] EP 1 473 549 A1 discloses a device for detecting a 2D position to measure the position of a first element relative to a second element. A 2D absolute scale comprises a 2D absolute scale pattern extending across the 2D scale range along each measurement axis of the scale. The 2D absolute scale pattern comprises a predetermined quasi-random pattern interleaved with a plurality of code sections along each axis. Each code section comprises a plurality of code elements indicating an absolute measurement value.

[0009] WO 2017 / 072281 A1 discloses a method for controlling the movements of articulated arms of an industrial robot using a motion control device. The motion control device is held by an operator in a position relative to the reference mark specified by the motion control device, and the reference mark is detected and, if necessary, read by the motion control device, and its distance from the reference mark is measured. The position and orientation of the motion control device are detected by internal sensors, e.g., inertial / acceleration sensors, or by external sensors, e.g., a 3D camera.

[0010] US 2008 / 0100622 A1 discloses a method for detecting a surface in a moving image, wherein the surface is covered with a pattern formed from a marker material.

[0011] JPH06203166 discloses a method for measuring the position of an object using three or more cameras, wherein the captured image information is fed to a neural network.

[0012] It is an object of the invention to provide a system and a method for determining the position of a moving object in space, which enables the most accurate and reliable position determination possible, particularly in large working volumes with large distances between the measuring system and the object.

[0013] According to the invention, a system according to claim 1 is provided for 6D position determination of an object movable in space with 6 degrees of freedom.

[0014] Furthermore, according to the invention, a method according to claim 12 is provided for determining the position of an object movable in space with 6 degrees of freedom.

[0015] The system and method according to the invention are camera-based. The system and method use a marker attached to the object to determine the object's position by capturing an image of the marker. The marker is flat and divided into a plurality of individual fields, each having a statistical noise pattern. The noise patterns can be different from one another or identical. If they are different, they can be detected by the image evaluation unit, for example, using a correlative method. It is understood that the system and method can use not just one marker, but also multiple markers, each attached to the object at different locations and each divided into a plurality of individual fields with different noise patterns.The image acquisition system comprises several cameras, especially if the object has a large number of degrees of freedom of movement, for example if it can rotate 360° around an axis, as can be the case with a robot arm of a machine.

[0016] Depending on the position of the moving object and thus of the marker, the image of the marker is captured from a different viewing angle by the preferably stationary image acquisition unit, resulting in geometrically different images of the pattern. In other words, the marker is subjected to a temporally varying projection transformation when projected onto an image sensor, for example. A reference image of the noise pattern is stored in the image evaluation unit, with the reference image representing, for example, the "undistorted" top view of the marker. The image evaluation unit is then designed to compare the current image of the marker with the reference image in order to determine the current position of the marker and thus of the object from the comparison.

[0017] Because the marker, which is preferably flat, i.e. not curved, has a plurality of individual fields, each of which has a statistical noise pattern, the computational effort in image evaluation can be reduced and the image evaluation can be significantly improved with regard to the precision of determining the position of the marker and thus of the object.

[0018] With the system and method according to the invention, it is fundamentally possible to determine the N-dimensional position of the marker, from which the N-dimensional position of the object can be determined. N can be in the range from 2 to 6.

[0019] Preferably, the noise patterns are such that the noise has a high bandwidth in the frequency domain.

[0020] This enables particularly precise localization of the individual marker fields in the image of the marker. The higher the frequency bandwidth of the noise, the more precisely the individual marker fields can be localized.

[0021] The noise patterns of the individual fields are preferably different from field to field, in particular at least approximately uncorrelated.

[0022] A very narrow autocorrelation function, ideally a δ or Dirac function, further increases the accuracy of the marker's and thus the object's position determination. This makes the method and system particularly suitable for correlation-based image analysis techniques to precisely localize at least one or more of the individual fields in the currently acquired image of the marker.

[0023] Preferably, the noise patterns are grayscale patterns, in particular with a probability density of gray values that is a Gaussian distribution. The grayscale patterns can, for example, be generated as black-and-white pixel patterns in a random process.

[0024] A Gaussian probability density function, or gray-value distribution, has the advantage of maximum entropy and therefore maximum information content for a given standard deviation. The given standard deviation refers to the gray-value range (intensity range) of the image acquisition unit. The power density of the noise can, for example, be chosen so that the 3σ value of the Gaussian distribution fits within the gray-value range of the image acquisition unit.

[0025] Preferably, the noise patterns contain white noise, which, as such, has a very narrow autocorrelation function. The narrower the autocorrelation function, the better correlation-based algorithms can localize the noise patterns. White noise has a δ or Dirac function as its autocorrelation function.

[0026] The degree of filling or covering the area of the marker with noise patterns should be as high as possible in order to determine the position as accurately as possible.

[0027] In a further preferred embodiment, the fields of the marker are polygonal, in particular rectangular, in particular square.

[0028] In this embodiment, the individual fields of the marker are bordered by straight lines. The surface of the marker itself is also preferably polygonal, in particular rectangular, in particular square, bordered. The straight border of the fields of the marker and / or the marker itself has the advantage that, when evaluating the current image of the marker, a rough determination of the projection parameters and a rough localization of the fields of the marker can initially be carried out with little computational effort. This can reduce the computational effort during the subsequent search for the exact projection parameters or fine localization of the fields by evaluating one or more of the imaged noise patterns.

[0029] The marker fields can be arranged in a matrix in rows and columns and can each have the same orientation to each other.

[0030] However, it is also possible to rotate and / or offset the marker fields relative to each other to eliminate ambiguities in the rough determination of the orientation of the marker and / or the fields in the current image of the marker. For example, in the case of rectangular or square fields, the individual fields can each be rotated by an angle relative to each other, so that no field has the same orientation as any of the others.

[0031] With more than square fields, polygonal fields in general, or fields with different numbers of corners, such ambiguities can also be avoided.

[0032] However, it is also possible that the individual fields of the marker are circular or elliptical.

[0033] In order to achieve the highest possible degree of filling of the marker area with noise patterns in the case of circular fields, it can be provided that the circular fields are offset from one another in such a way that the area of the marker is covered as densely as possible with the circular fields.

[0034] As already mentioned above, a preliminary rough determination of the marker's position based on the captured image of the marker is advantageous. In this context, the image analysis unit is designed to make a rough determination of the position of the marker currently captured by the image acquisition unit based on the detection of corners and / or edges of the marker and / or the marker's fields.

[0035] Additionally or alternatively, a Radon / Hough transform can also be calculated, which represents a robust method for detecting straight lines, circles, or any other parameterizable geometric shapes in a grayscale image. As already mentioned, a rough determination of the marker's position has the advantage of reducing the computational effort required for the subsequent exact position determination.

[0036] In connection with the above-mentioned measure, the image evaluation unit is preferably designed to carry out a fine determination of the position of the marker currently detected by the image acquisition unit based on a recognition of the noise patterns in comparison with the stored reference image of the noise patterns.

[0037] Fine-tuning can be performed, for example, by calculating a number of parameter sets for one or more noise patterns. A similarity measure, such as a merit function, can then be calculated for each of the assumed parameter sets by comparing the reference image (modified / distorted, shifted using the assumed parameters) with the corresponding currently acquired image of the marker. The parameter set that achieves the highest similarity (or lowest dissimilarity) is the parameter set that ultimately precisely locates the marker's field(s).

[0038] However, the (normalized) correlation coefficient can also be used as a similarity measure. The normalized correlation coefficient also allows an assessment of the current match quality, i.e., how well / reliably the currently evaluated field has been localized.

[0039] Likewise, to reduce computational effort, it is possible to choose an approach in which the search for the parameter set that uniquely locates the relevant marker field is first performed using a lower-resolution image of the noise pattern(s), and then refined using the higher-resolution image. This allows search areas to be narrowed down in the higher-resolution image and reduces computational effort.

[0040] In a further preferred embodiment, the image acquisition unit has a low-pass filter or a defocusing device.

[0041] While it is advantageous and preferable to provide the noise patterns of the marker fields in such a way that the noise has a very wide bandwidth to enable the most precise localization of the individual fields, it may be advantageous and advisable to reduce the bandwidth of the noise patterns by using a low-pass filter during image acquisition to avoid so-called "aliasing" effects. "Aliasing" can occur if the image currently captured by the image acquisition unit is severely distorted, for example, if the camera is viewing the surface of the marker with grazing incidence. In this case, it may happen that parts of the captured image are sampled at a very low spatial sampling rate. This can lead to incorrect localizations of the marker fields, which in turn can severely distort the position and / or orientation determination of the marker and thus of the object.

[0042] Instead of a low-pass filter in the beam path of the image acquisition unit, it can also be provided to defocus its optics in a targeted manner.

[0043] It is particularly advantageous if the noise patterns are generated during marker production or during the production of the noise patterns by low-pass filtering of white noise. This has the advantage that there is better control over the filtering during marker production than if the filtering is performed by optics in the beam path of the image acquisition unit. With low-pass filtering of white noise when generating the noise patterns during marker production, a good compromise can be found between the widest possible bandwidth of the noise patterns and sufficient low-pass filtering in order to, on the one hand, keep the autocorrelation function of the noise patterns as narrow as possible, and, on the other hand, to avoid or at least reduce aliasing effects as much as possible.

[0044] When determining the position of objects that have a large range of motion and for which images of the marker with correspondingly strong projection distortion are captured by the image acquisition unit, it is advantageous to use several cameras and / or several markers in the system and the method, at least for some degrees of freedom of movement, so that at least one camera always observes at least one marker with a projection distortion that is still acceptable.

[0045] Both the system according to the invention and the method according to the invention can be used for any moving object to determine its position and / or orientation. The object can be, in particular, a moving machine part, in particular a robot arm of a machine used in the manufacture of mirrors or other optical components.

[0046] However, the movable object can also be a movable component of an optical system itself, in particular a component of a projection exposure system for microlithography. For example, the movable component can be a movable mirror.

[0047] However, the invention can also be used in other areas, such as medical technology, robot-assisted surgery, etc.

[0048] Further advantages and features can be found in the following description and the attached drawing.

[0049] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

[0050] Embodiments of the invention are illustrated in the drawings and will be described in more detail below with reference to them. They show: Fig. 1 shows an embodiment of a system for determining the position of a movable object in space, the object being, for example, a robot arm of a machine, and a first embodiment of a marker for determining the position; Fig. 2 shows an image captured by an image capture unit of the system in Fig. 1 captured exemplary image of a robot arm in Fig. 1 attached marker; Fig. 3 is a block diagram showing an embodiment of a method for determining the position of a moving object in space by means of the system in Fig. 1 illustrated; Fig. 4 a practical example of a single noise pattern of the marker in Fig. 1 ; Fig. 5 shows a further embodiment of a marker for determining the position of a moving object; Fig. 6 shows a further embodiment of a marker for determining the position of a moving object; Fig. 7 shows a further embodiment of a marker for determining the position of a moving object, wherein the marker is Fig. 6 Fig. 8 shows an image acquisition unit of the system in Fig. 1 Acquired image of a marker with the image acquisition unit scanning the marker; and Fig. 9 shows two noise patterns of a marker, with the left noise pattern being unfiltered and the right noise pattern being low-pass filtered.

[0051] In Fig. 1 A system for determining the position of a movable object 12 in space, designated by the general reference numeral 10, is shown schematically. In the illustrated embodiment, the object 12 is a robot arm 14 of a machine 15. The robot arm 14 is movable, for example, translationally movable and rotationally movable. The current location of the robot arm 14 can be expressed, for example, in Cartesian coordinates x, y, z, and the current orientation as an angle of rotation about one or more of the axes x, y, z. The system 10 is fundamentally capable of determining the position of the object 12 in space according to 6 degrees of freedom of movement, i.e., 6-dimensionally. The working volume of the robot arm can be large, for example, 1 m 3 or more. In Fig. 1 A coordinate system with x, y and z axes is shown as an example, with arrows 15 x , 15 y and 15 z illustrating the degrees of freedom of rotation of the object 12 around the x, y and z axes.

[0052] The system 10 has a marker 16 attached to the object 12. The marker 16 is stationary with respect to the object 12, ie, it moves with the object 12 when it moves.

[0053] The system 10 further comprises an image capture unit 18 located remotely from the object 12. The image capture unit 18 comprises a camera 20, which is, for example, a video camera, and is equipped with an image sensor 22. The image sensor 22 can be embodied as a commercially available image sensor.

[0054] The camera 20 is arranged to capture an image of the marker 16, wherein the camera captures images of the marker 16 regularly or continuously in a temporal sequence, so that the changing positions of the object 12 can be continuously tracked. The camera 20 further has a lens 24 that images the marker 16 onto the image sensor 22.

[0055] The camera 20 can be fixed in place, at least during operation of the system 10. However, the camera 20 can also be movable.

[0056] The system 10 further comprises an image evaluation unit 26 which is connected to the image acquisition unit 18 and serves to evaluate the image of the marker 16 acquired by the camera 20 in order to determine the current position of the marker 16 and thus of the object 12.

[0057] While in Fig. 1 While the system 10 is shown with only one camera 20 and one marker 16 for reasons of simplicity, it is understood that the system 10 may have multiple cameras 20 and multiple markers 16. The multiple markers 16 may be mounted at different positions on the object 12, and the cameras 20 may be distributed in space so that the cameras 20 observe the marker(s) 16 from different viewing angles.

[0058] The system 10 can be designed as a pure measuring system for tracking the movements of the object 12, but can also be used for motion control or motion regulation for the object 12.

[0059] Depending on the position of the object 12 in space, a more or less projection-distorted image of the marker 16 is created on the image sensor 22 of the camera 20. Fig. 2 shows an example of such a projection-distorted image 28 of the marker 16 in Fig. 1 The goal is now to determine the exact position of object 12 from image 28 of marker 16 by determining the position of marker 16 in the coordinate system of camera 20. In the practical implementation of system 10, aspects such as camera calibration and imaging errors of camera 20 are taken into account, although this is not necessary for understanding the present invention. For understanding the invention, it is sufficient to consider camera 20 as a pinhole camera.

[0060] The result of the position determination by the image evaluation unit 26 is output by the image evaluation unit 26, as indicated by an arrow 27 in Fig. 1 is indicated, for example to a display not shown, or to a control or regulation for controlling / regulating the movement of the object 12.

[0061] In Fig. 1 Marker 16 is also shown in isolation and enlarged. "T" denotes the upper end of Marker 16, "L" the left end, "R" the right end, and "B" the lower end of Marker 16.

[0062] The marker 16 is flat. The marker 16 is preferably planar. A surface 30 of the marker 16 is in the Fig. 1 shown embodiment is square. Other surface shapes such as generally rectangular, generally polygonal, in particular polygonal, can also be considered. The surface 30 is divided into a plurality of fields 32, wherein in the embodiment in Fig. 1 a total of nine such fields 32 are present. However, the number of fields 32 is not critical. It can be smaller or larger than 9. The fields 32 are in the embodiment shown in Fig. 1 also square. However, the fields 32 can also be generally rectangular or generally polygonal.

[0063] The arrangement of the fields 32 is shown in Fig. 1 In the embodiment shown, the array is formed in a matrix-like manner in rows and columns, with the individual fields 32 each having the same orientation or rotational position.

[0064] Each of the fields 32 is filled with a statistical noise pattern 34. In the Figuren 1, 2 , 5 bis 9 For the sake of simplicity, the noise patterns 34 are shown as regular patterns and identical to one another. However, it is understood that the noise patterns 34 were generated by a stochastic process and thus do not exhibit any dominant periodicity and are also not regular. Furthermore, the noise patterns can be identical to one another. The noise patterns 34 can be advantageous from one field 32 to another, but can also be completely different. In other words, the noise patterns 34 can be unique from one another. A practical realization of an individual noise pattern 34 is shown in Fig. 4 shown.

[0065] The noise patterns 34 fill the area 30 of the marker 16 with as high a filling level as possible.

[0066] The noise patterns 34 are preferably as in Fig. 4 shown as an example, are formed as grayscale patterns. The noise patterns 34 preferably have the largest possible bandwidth in the (spatial) frequency domain. The noise patterns 34 are generated during the manufacture of the marker 16 and stored as a reference image in the image evaluation unit 26.

[0067] The cross-correlation function of all noise patterns 34 of the marker 16 is preferably as small as possible, ie the noise patterns 34 are preferably at least approximately uncorrelated from field to field 32.

[0068] The noise patterns 34 preferably contain broadband white noise with a maximum possible signal value. Maximum signal value here means that the image sensor 22 of the camera 20 does not run into the black or white boundary, but rather the grayscale range (intensity range) of the image sensor 22 is fully utilized. As already mentioned, the area 30 of the marker 16 should be filled with the white noise as much as possible.

[0069] White noise has a narrow autocorrelation function. The narrower the autocorrelation function, the better correlation-based algorithms can localize the noise patterns. 34 White noise has a δ-peak as its autocorrelation function. This is the narrowest autocorrelation function possible. Therefore, white noise is preferable for correlation-based localization algorithms.

[0070] Furthermore, the gray values of the noise patterns are Gaussian distributed. A Gaussian probability density function (distribution of gray values) exhibits maximum entropy and thus maximum information content for a given standard deviation. The given standard deviation refers to the gray value range (intensity range) of camera 20. The noise power can, for example, be selected such that the 3σ value of the Gaussian distribution fits within the gray value (intensity value) range of the camera.

[0071] Not necessarily always, but usually a noise pattern that is well suited for correlation-based localization algorithms is also suitable for other localization algorithms.

[0072] With reference to Fig. 3 An embodiment of the image evaluation unit 26 and a method are described how the image evaluation unit 26 evaluates the image of the marker 16 currently captured by the camera 20 in order to determine the position of the marker 16 in space and thus of the object 12 in space.

[0073] The processes running in the image evaluation unit 26 are preferably divided into two higher-level process steps 36 and 38, with process step 36 running first.

[0074] The first process step 36 is a rough determination, which serves to roughly locate the marker 16 and the fields 32 of the marker 16. The rough determination is performed using the currently captured image of the marker in the camera 20. This can be done, for example, by edge detection 40 of the marker 16 and / or the fields 32 of the marker 16, by corner detection 42 of the marker 16 and / or the fields 32 of the marker 16, and / or by calculating 44 a Radon / Hough transform. Steps 40, 42, and 44 can be efficiently implemented, for example, on an FPGA (Field Programmable Gate Array, i.e., a digital integrated circuit into which a logic circuit can be loaded). The information from steps 40, 42 and 44 are linked together in an information fusion step 46 in order to obtain improved localization information as part of the rough determination of the position of the marker 16.The information fusion step 46, or a portion thereof, may be executed in software on a microprocessor. This is because the analyses of edge detection 40, corner detection 42, and the Radon / Hough transform yield a moderate number of point coordinates and / or a list of parameterized lines rather than a large amount of image data.

[0075] The rough determination results in rough projection parameters and a rough localization of the fields 32 of the marker 16.

[0076] Following the rough determination 36, a fine determination 38 is performed, which serves to precisely localize the fields 32 in particular. The advantage of a rough determination prior to the fine determination is that the search areas for the fine localization of the fields 32 can be limited. This advantageously reduces the computing time.

[0077] The fine determination 38 comprises a fine adjustment 48 of the noise patterns 34 of the marker 16 based on the currently acquired image of the marker 16 and the reference image of the noise patterns 34 stored in the image evaluation unit 26. The fine determination 38 results in precise projection parameters and precise localizations of the fields 32 of the marker 16.

[0078] Finally, in a step 50, the position of marker 16 in space, and thus of object 12 in space, is calculated. The result is then output according to an arrow 52.

[0079] As part of all or part of the image analysis, it is also possible to use motion information from previous images of marker 16, specifically in steps 46, 48, and 50, as designated by reference numeral 54. The motion information from previously acquired images of marker 16 can be used to reduce computing times by further narrowing down search areas or to improve the tracking of the movement of object 12 by system 10.

[0080] If the rough determination of the position of marker 16 is performed without evaluating the noise patterns 34, for example, based only on corner and / or edge detection of marker 16 and / or fields 32, an ambiguity exists in the exemplary embodiment of marker 16 regarding a rotation of marker 16 by 90° in each case. The reason for this is the strictly symmetrical and regular arrangement of fields 32 with the same size and as squares with the same orientation.

[0081] Fig. 5 shows an embodiment of a marker 16a in which this ambiguity is avoided. In the marker 16a, the fields 32a are all oriented differently to one another. In the example shown with nine fields 32a, the fields 32a can, for example, be rotated by 9° from field to field. This eliminates the ambiguity with respect to a 0° rotation of the marker 16a or with respect to rotations of the marker 16a by integer multiples of 90°. This enables an unambiguous identification of the orientation of the marker 16a already in the rough determination phase according to Fig. 3 without evaluating the noise patterns 34. For example, the upper left corner of the marker 16a can always be identified, since only with respect to the upper left field 32a do the edges of the field 32a run parallel to the outer edges of the marker 16a.

[0082] Another possibility, not shown, to avoid ambiguities in the rough determination by edge and corner detection is that the fields 32 of the marker 16 have polygonal structures.

[0083] Examples are described below of how the fine determination 38 and thus the precise determination of the position of the marker 16 in space can be carried out based on an evaluation of the noise patterns 34 in the image of the marker 16 captured by the image capture unit 16.

[0084] The fine determination serves to localize the fields 32 and is performed by evaluating the noise patterns 34 in the camera image. The fine localization of the fields 32 can be implemented, for example, using an approach in which the parameter space (position parameters) is searched completely as part of a full search. However, since prior knowledge from the coarse determination according to process step 36 is available, the size of the search areas in the parameter space can be limited, which also limits the computing time.

[0085] A full search essentially means processing the noise patterns 34 with a number of assumed parameter sets, thereby generating an artificial camera image with the respective assumed parameters. A similarity measure (merit function) is then calculated for each of the assumed (searched) parameter sets by comparing the reference image of the noise patterns (master pattern), which is stored in the image evaluation unit, with the corresponding currently acquired image of the marker 16, whereby the master pattern is modified, distorted, or shifted using the assumed parameters. The parameter set that achieves the maximum similarity (or minimum dissimilarity) of the modified / distorted / shifted master pattern with the currently acquired noise pattern(s) 34 is the parameter set that ultimately finely localizes the field(s) 32.

[0086] Since a full search involves trying numerous parameter sets in cascade loops, the computation can be accelerated by running multiple parallel units, for example, on an FPGA. At first glance, all computation units require direct access to the acquired image data of marker 16. If a bottleneck occurs here, the image data can be copied / duplicated (for example, in multiple block RAM units within an FPGA) before the computation units use the data. Consequently, the computation units performing the similarity calculation do not all need to access the same memory.

[0087] The (normalized) correlation coefficient can be used as a similarity measure for determining the parameter set that ultimately localizes one or more or all of the fields 32. The normalized correlation coefficient also allows for an assessment of the current quality of the comparison, i.e., how well / reliably a currently evaluated field has been localized.

[0088] During the full search, the maximum of the similarity measure is determined to find the appropriate parameter set representing the position of field 32. The value of the maximum itself can be used as a measure of the quality of localization of a specific field. This value can, for example, also be transformed into a weighting factor that can be used in an estimation step when determining the position of marker 16 in space. With this approach, more reliably localized fields 32 are given more consideration than less reliable ones.

[0089] Since there are a large number of degrees of freedom for the full search, the calculation is still very complex.

[0090] For each pixel within the noise pattern 34, a distorted coordinate must be calculated, for example, using a calculation similar to the one disclosed in the article by R. Hartley and A. Zisserman: Multiple View Geometry, Tutorial, CVPR, June 1999: x ′ = x ′ 1 x ′ 3 = h 11 x + h 12 y + h 13 h 31 x + h 32 y + h 33 , y ′ = x ′ 2 x ′ 3 = h 21 x + h 22 y + h 33 h 31 x + h 32 y + h 33

[0091] In the projection transformation equations above, there is one degree of freedom for normalization. This allows for investigation into whether all projection transformation parameters need to be modified during the search. Such investigations can show that fixing some parameters (taking into account the result of the rough estimation) yields sufficiently good results while significantly reducing the required computational effort.

[0092] Another option is to explore a linearization of the projection equation, which can lead to a more efficient solution. This means that the coordinate of a projected point can be calculated more efficiently within a small search space around the roughly determined parameters obtained from the rough determination.

[0093] Another way to reduce computational effort is to adopt an approach in which the search is first performed on a lower-resolution image of the marker 16 in the camera, with the search subsequently being refined on the same higher-resolution image. Prior knowledge from the results of the lower-resolution image can be used to limit the search areas in the steps of locating the field(s) 32 in the higher-resolution image.

[0094] According to another embodiment, instead of using the correlation coefficient calculated from the image data as a similarity measure, an approach based on a census transform may be chosen to accelerate the localization of the field(s) 32. The census transform is described, for example, in the article by Fridtjof Stein: Efficient Computation of Optical Flow Using the Census Transform, Proceedings of the 26th DAGM Symposium, 2004, pages 79-86.

[0095] The fine determination (process step 38) for the exact localization of the fields 32 of the marker 16 based on the noise patterns 34 is particularly precise when, as already mentioned, the noise patterns have a large bandwidth in the frequency domain. The size of the bandwidth of the noise patterns directly determines the accuracy of the system 10. White noise patterns have a tight autocorrelation function, ideally a 2-dimensional delta pulse, and are therefore optimally suited for correlation-based block matching methods for localizing the fields 32.

[0096] Noise patterns with full-frequency, i.e., very broadband, content allow for a good estimation of motion-induced image blur for speed estimations. This means that the frequency content of the image is determined only by the camera's imaging behavior and motion-induced blur.

[0097] If the fields 32 as in Fig. 1 Since the fields shown have straight lines as their boundaries, an efficient rough pre-estimation of the projection parameters and marker position candidates is possible. An ellipse fit, as required for circular fields, is not necessary here. Instead, Radon / Hough transform-type analyses can be used to find the lines and field position candidates. Additionally or alternatively, the corners can be analyzed to find the frames or edges of the field(s). Mathematically and from an accuracy perspective, an ellipse fit may be better, however, Radon / Hough transform-type analyses and / or corner detection can be efficiently implemented in hardware / FPGA. A Radon transform calculation can be efficiently performed by considering the so-called "Central Slice Theorem" or "Projection Slice Theorem."

[0098] According to a further embodiment of a marker 16b, which in Fig. 6 As shown, the marker 16b can have circular fields 32b into which the area 30b of the marker 16b is divided. The fields 32b are each filled with a statistical noise pattern 34b, as in the case of the marker 16 and the marker 16a, respectively. Regarding the design of the noise pattern 34b, reference can be made to the description of the noise pattern 34 of the marker 16.

[0099] In the case of marker 16b, the high accuracy of the position determination of marker 16b can be achieved due to the high-bandwidth noise patterns 34b. The use of circular or elliptical fields 32b can be advantageous, since even under projective distortion, a circular shape results in a circular / elliptical shape, while the high measurement accuracy is achieved by the statistical noise patterns within the fields 32b.

[0100] A rough determination of the position and / or orientation of marker 16b can be achieved using standard methods for circle or ellipse detection. The detected ellipse parameters or the corresponding coordinate list can be used for position calculation in addition to the results of the fine localization of the noise pattern 34b.

[0101] In the case of the marker 16c with circular (or elliptical) fields 32b, according to the embodiment of a marker 16c in Fig. 7 The area coverage or the degree of filling of the area 30c of the marker 16c can be increased by arranging the fields 32c not strictly in rows and columns, but by offsetting them relative to each other in such a way that the highest possible area coverage of the area 30c of the marker 16c is achieved. In the arrangement according to Fig. 6 For example, twelve fields 32c can be accommodated on the area 30c instead of only nine fields 32b, with fields 32c and 32b being the same size.

[0102] Increasing the fill level of area 30c and thus increasing the number of fields filled with statistical noise patterns means there are more observation pixels. Increasing the number of observation pixels in the fields reduces the standard deviation in the localization of fields 32c.

[0103] The following is based on Fig. 8 and 9 another aspect is described.

[0104] Fig. 8 shows a picture 28' comparable to the picture 28 in Fig. 2 of the marker 16 on the object 12, whereby the camera 20 now captures the marker 16 with a strongly grazing incidence of the viewing direction. The result is that the image 28' of the marker 16 on the image sensor 22 of the camera 20 is severely projection-distorted. This means that at least parts of the image 28' are sampled at a lower spatial sampling rate.

[0105] This reduced sampling rate due to severe projection distortion of marker 16 leads to so-called "aliasing," especially when, as is preferred within the scope of the present invention, the bandwidth of the noise pattern frequency is maximized. Aliasing is an effect and, in the field of signal analysis, refers to errors that occur when the signal to be sampled contains frequency components higher than half the sampling frequency. Aliasing leads to poor or even completely incorrect localization or position determination of fields 32 in image 28' and thus to unusable position determination of marker 16.

[0106] Since aliasing occurs during image formation, it cannot be reversed in the image evaluation unit 26. However, this problem can be solved in the design of the system 10. The bandwidth of the noise patterns must be limited so that aliasing does not occur for projection distortions that are not too severe.

[0107] For very strong projection distortions, the position measurement results become very uncertain—at least for some degrees of freedom. The threshold for what constitutes "too strong" projection distortion must be determined during system development. The problem of aliasing can be solved, for example, by having the image acquisition unit 18 comprise multiple cameras 20 that observe the marker 16 from different viewing directions, and / or by having multiple markers 16 distributed around the object 12. If multiple cameras are present, they must be arranged such that at least one camera always observes a noise pattern 34 with a sufficiently high sampling rate to avoid aliasing or keep it small enough. If multiple markers 16 are used, they can be arranged such that at least one of the markers 16 is observed by the camera 20 without excessive projection distortion.

[0108] Another way to avoid aliasing is to limit the frequency band (essentially an anti-aliasing filter). This can be achieved, for example, by deliberately defocusing the optics of camera 20 (lens 24). Another possibility is to generate the noise patterns 34 by low-pass filtering white noise.

[0109] Bandlimiting by low-pass filtering the noise pattern during its generation is advantageous (to reduce aliasing effects). This approach provides better control over the filter applied to the noise pattern 34 compared to filtering by adjusting the lens during the acquisition of the marker 16 with the camera 20. Control over the filter applied to the noise pattern during its generation is advantageous because the filter applied to the noise pattern determines the autocorrelation function of the resulting noise pattern. This allows for a good balance between sufficient low-pass filtering and maintaining a narrow autocorrelation function.

[0110] Fig. 9 shows on the left an example of a statistical noise pattern 34 without low-pass filtering during generation, while on the right in Fig. 8The noise pattern 34 was generated using low-pass filtering. A 2D binomial filter, for example, can be used to smooth the noise pattern 34.

[0111] An alternative filter can be one that yields a rotationally symmetric autocorrelation function e^(-λ* sqrt(x 2< +y 2< )). λ determines the "narrowness" of the autocorrelation function, which relates to the bandwidth in the frequency domain. The x and y coordinates are the spatial coordinates in the image plane.

[0112] Anti-aliasing can be directed not only at the noise patterns 34, but also at the edges of the fields 32 and / or the edges of the marker 16 itself, since aliasing effects can also occur there.

Claims

1. System for the 6D position determination of an object (12) which is movable with 6 degrees of freedom in space, having one or more markers (16) which is / are intended to be applied to the object (12) and is / are applied to the object (12) in a stationary manner for 6D position determination, wherein the marker(s) (16) each has / have a surface (30) which is subdivided into a plurality of individual fields (32), wherein the fields (32) each have a statistical noise pattern (34), also having an image capture unit (18) which is remote from the object (12) and is arranged to capture an image (28) of the marker(s) (16), wherein the image capture unit (18) has a plurality of cameras (20) which observe the marker(s) (16) at different viewing angles, and having an image evaluation unit (26) which stores a reference image of the noise patterns (34) and is designed to locate at least one of the fields (32) in the currently captured image (28) of the marker(s) (16) by comparison with the reference image in order to determine a current position of the marker(s) (16) in space, wherein the image evaluation unit (26) is designed to coarsely determine the position of the marker(s) (16) currently captured by the image capture unit (18) by detecting corners and / or edges of the marker(s) (16) and / or of the fields (32) of the marker(s) (16).

2. System according to Claim 1, wherein the fields (32, 32a) of the marker(s) (16, 16a) are polygonal, in particular rectangular, in particular square, or wherein the fields (32b, 32c) of the marker (16b, 16c) are circular or elliptical.

3. System according to Claim 1 or 2, wherein the fields (32, 32b) of the marker(s) (16, 16b) are arranged in the form of a matrix in rows and columns.

4. System according to Claim 2, wherein the fields (32a, 32c) of the marker(s) (16a, 16c) are rotated and / or shifted relative to one another.

5. System according to one of Claims 1 to 4, wherein the noise patterns (34) of the individual fields (32) are different from one another, in particular are at least approximately uncorrelated.

6. System according to one of Claims 1 to 5, wherein the noise patterns (34) have broadband noise in the frequency domain.

7. System according to one of Claims 1 to 6, wherein the noise patterns (34) are greyscale patterns, and / or wherein the noise patterns (34) contain white noise.

8. System according to Claim 7, wherein a probability density of greyscale values within the noise patterns (34) is a Gaussian distribution.

9. System according to one of Claims 1 to 8, wherein the image evaluation unit (26) is designed to finely determine the position of the marker(s) (16) currently captured by the image capture unit (18) by detecting the noise patterns (34).

10. System according to one of Claims 1 to 9, wherein the image capture unit (18) has a low-pass filter and / or a defocussing device.

11. System according to one of Claims 1 to 10, wherein the noise patterns (34) of the marker(s) (34) are generated by subjecting white noise to low-pass filtering.

12. Method for the 6D position determination of an object (12) which is movable with 6 degrees of freedom in space, having the steps of: providing one or more markers (16) each having a surface which is subdivided into a plurality of individual fields (32), wherein the fields (32) each have a statistical noise pattern, storing a reference image of the noise patterns (34), applying the marker(s) (16) to the object (12) such that the marker(s) (16), in the state applied to the object (12), is / are stationary with respect to the object (12), capturing a current image of the marker(s) (16) on the object (12) by means of a plurality of cameras which observe the marker(s) (16) from different viewing angles, and evaluating the currently captured image of the marker(s) (16), wherein at least one of the fields (32) of the marker(s) is located in the currently captured image by comparison with the reference image in order to determine a current position of the marker(s) (16) in space, wherein the evaluation comprises coarsely determining the position of the currently captured marker(s) (16) by detecting corners and / or edges of the marker(s) (16) and / or of the fields (32) of the marker(s) (16).

13. Method according to Claim 12, wherein the object (12) is a movable machine part, in particular a robot arm (14).

14. Method according to Claim 12, wherein the object is a movable component of an optical system, in particular of a microlithographic projection exposure apparatus.