Position detector and method for 3D positioning

Abstract

To provide a position detector capable of determining a position in a shorter time.SOLUTION: A position detector 30 generates 3D position information of an object 20 in a position determination space 13 for the object. The position detector includes a camera 3 that has a lens 5 and an image sensor 4 to define an imaging area 10, and at least one first light deflecting element 1 arranged in the imaging area 10. The camera 3 and the at least one light deflecting element 1 are adapted to simultaneously generate at least two images of the position determination space 13 on the image sensor 4, a first image is generated by light beams deflected at the first light deflecting element 1, and the at least two images differ with respect to a viewing direction of the position determination space.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a position detector and a method for 3D positioning (measurement) of an object in a positioning space. A further aspect of the present invention relates to a robot system having a position detector according to the invention, a method for determining spatial coordinates, a method for operating a robot system, particularly a pipetting robot system, a method for determining the spatial coordinates of an object, and an application of the method and system.

Background Art

[0002] Position detectors are used in various situations where the position of an object in three coordinate directions should be accurately known. Below, pipette tip positioning (measurement) is discussed as a specific example to aid understanding of such situations.

[0003] Due to manufacturing variations in pipette production or assembly errors when picking up the pipette tip, the relative position of the pipette tip with respect to the pipette pickup tends to vary. Even if the pipette pickup is positioned accurately and reproducibly, for example by a pipetting robot, the position variation of the pipette tip may possibly prevent the pipette tip from reaching small objects such as wells in a well plate reliably enough. The quality and selection of the pipette tip material affect the range of variation of the position of the pipette tip and probably rule out the use of well plates with small well spacings.

[0004] A plate-shaped position detector with external dimensions on a standard microplate is known from patent specification EP1489425B1. This enables the positioning of a functional element, such as a pipette tip, by scanning beams from two light barriers whose beam directions are not parallel to the outer edge of the plate. The plate can be used in a manner in which an object is moved by a robotic system in the X, Y, and Z directions of orthogonal coordinates within the plate's region to determine its position, and the shading of the scanning beams is observed in synchronization with this movement. In this way, for example, the positioning of a pipette tip relative to the plate is possible.

[0005] A drawback of this method is that it takes a relatively long time to determine the position. Slower movement is required to increase the accuracy of the position determination. A further drawback is that the lateral movement of the robot system needs to be synchronized with the evaluation of the signal from the optical barrier receiver. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] EP1489425B1 [Overview of the Initiative]

[0007] The present invention aims to provide an apparatus and method that solves the shortcomings of the prior art. In particular, the present invention aims to enable position determination (measurement) to be performed in a shorter time. Furthermore, the invention aims to enable accurate position determination (measurement) in all three spatial dimensions.

[0008] The objective is solved by the position detector according to claim 1.

[0009] The position detector according to the invention is used to generate 3D position information of an object in a position determination (measurement) space. This position detector is used - A camera having a lens and an image sensor that form an imaging area, and - At least one first optical deflection element arranged in the imaging region, It is equipped with.

[0010] A camera and at least one optical deflection element are suitable for simultaneously generating at least two images of a positioning space in an image sensor, thereby generating a first image with a light beam deflected by the first optical deflection element. Here, the at least two images differ with respect to the field of view (viewing direction) of the positioning space.

[0011] The 3D position information of an object can be included, for example, in an image recording that can be recorded by an image sensor. The positioning space is designed in such a way that it can receive, at least partially, an object whose 3D position is determined. The beam path starting from the positioning space is imaged into a real image in the image sensor in at least two different directions by the position detector according to the invention. These two different directions correspond to different field of view directions of the positioning space. The final stage of imaging is performed by the camera lens. There is at least one first optical deflection element in front of the lens. The position detector is designed so that at least two such real images are generated in the image sensor simultaneously. At least one of the images is generated by a light beam deflected by the first optical deflection element. This optical deflection element is in the direct imaging area of ​​the camera. The optical deflection element can be, for example, a mirror or a deflection prism. The optical deflection element is designed to deflect the light beam in a defined direction so that a downstream lens along the beam path can image the deflected light beam into a real image. The optical deflection element can be designed to interact with the camera lens in a manner that allows for a magnification effect, for example. The optical deflection element may also be designed for distortion-free imaging. The first and second images can be generated in different regions of the image sensor. The two images can also partially overlap and be distinguishable by, for example, color, wavelength range, or polarization. The beam paths to the first and second images can influence the camera lens from different directions, or can be mixed in front of the camera, for example, via a semi-mirrored optical element.

[0012] The inventors recognized that the proposed position detector can be used to determine (measure) the 3D position information of an object in a very simple manner. Due to the simultaneous generation of two images from different viewing directions, unlike conventional two-dimensional images, it is also possible to determine (measure) information about a third spatial dimension. The determination of 3D position information can be performed in a short time.

[0013] The present invention is applicable to various situations in which the position of a reference point of an object, particularly a rectangular object, should be precisely known in three coordinate directions. As a specific example of such a situation, the positioning of a pipette tip is described in the introduction. The reference point of an object whose position can be determined by the present invention is, for example, the end of any rectangular object. Apart from the pipette tip, the above reference point can also be the tip of a drill, the center of the end face of a milling machine, the probe tip of a morph tracer, the tip of a soldering iron, etc. Embodiments of the position detector are derived from the features of claims 2-12. The features of the dependent claims can be combined in any way, as long as they do not contradict each other.

[0014] In one embodiment, the positioning space is located within the imaging range of the camera. In this embodiment, at least one of two images can be generated via a direct beam path from the positioning space to the camera lens. This embodiment can be particularly simple. For example, the positioning space may be located at the center of the imaging area of ​​the camera, and a single optical deflection element in the peripheral zone of the imaging area of ​​the camera may be used to obtain a first direction of observation in the positioning space, where the first direction may be substantially perpendicular to the direct direction of observation in the positioning space, i.e., in this case, a second direction of observation.

[0015] In one embodiment of the position detector, a first optical deflection element is designed to deflect light on a first flat surface that forms a first plane. This embodiment achieves an undistorted first image. Evaluating the 3D position information generated by the position detector is particularly easy in this embodiment. The first flat surface can be, for example, a flat mirrored surface or the plane of an optical prism. This first plane is an imaginary plane extending the first flat surface beyond its edges. This first plane may have, for example, a normal that forms an angle, for example, in the range of 20° to 50°, together with the optical axis of the camera. An angle slightly greater than 45° is suitable for providing a pair of orthogonal viewing directions, along with a direct viewing direction to the position determination space. Small tilt angles with respect to the optical axis, such as in the range of 0°-30°, when combined with the position of the first optical deflection element which is further away from the camera than the positioning space, are useful in providing a direction of view of an object that shows the sides of the object that are not visible in the direct field of view.

[0016] One embodiment of the position detector also includes a second optical deflection element. The camera and the second optical deflection element are suitable for generating a second image of at least two images by a light beam deflected by the second optical deflection element. In this embodiment, the first and second optical deflection elements can be arranged such that the beam paths from the object to the first and second images are essentially the same length. This allows both images to be in focus on the image sensor simultaneously. This embodiment can also be combined with an embodiment that allows for a direct field of view of the positioning space, thereby allowing a third image of the positioning space to be obtained by the camera's image sensor.

[0017] In one embodiment of the position detector, a second optical deflection element is designed to deflect light in a second plane that forms a second plane. This embodiment has the advantage that the first and second images are not distorted, as already described above with respect to the first image. The second plane is an imaginary plane that extends beyond its edges. The second optical deflection element may have the same characteristics as described above with respect to the first optical deflection element.

[0018] In a further embodiment of the position detector, which is a combination of features from two of the embodiments described above, both the first and second optical deflection elements are designed to deflect light in first and second flat plates forming first and second planes, respectively.

[0019] In particular, the first and second optical deflection elements may be two planar mirrors. For example, according to this embodiment, the position detector according to the invention can be implemented with only two optical deflection elements, especially mirrors, and as a result, no further optical elements other than the two optical deflection elements are required outside the camera.

[0020] In one embodiment of the position detector in which the first and second planes are formed as described above, the first and second planes intersect in an imaginary straight line perpendicular to the optical axis of the camera. In this embodiment, imaginary lines and optical axes can form a plane of symmetry of the device. In particular, such a plane of symmetry can pass through the positioning space. Thus, the first and second images are generated with respect to the object on the plane of symmetry via beam paths of equal length. The first and second flat surfaces can extend close to the plane of symmetry. This arrangement has the advantage of making good use of the camera's imaging area.

[0021] In a further embodiment of the position detector, the first and second planes form four spatial portions, where the camera and position determination space are located in the first spatial portion of these four spatial portions, where the aperture angle of the first spatial portion is in the range of 100°-140°. This embodiment makes it possible to generate 3D positional information that can be evaluated particularly well. This is due to the fact that this embodiment makes it possible to achieve mainly independent field of view directions for the positioning space with respect to the first and second images. In particular, the aperture angle of the first spatial portion can be in the range of 130.0° to 135.0°. Thus, the directions of observation can be substantially perpendicular to each other. For example, in a mirror-symmetric arrangement, the first and second planes can have normals that are inclined at approximately 23° to the optical axis of the camera, respectively. In this example, the aperture angle of the spatial portion is 134° (=180°-2*(23°)). In the case where the distance between the camera and the mirror is large and unclear (borderline case), the perpendicular direction of observation can be achieved by the inclination angle of the mirrors of 22.5° each (i.e., the aperture angle of the spatial portion of 135°).

[0022] In one embodiment of the position detector, the camera, first optical deflection element, and second optical deflection element are arranged with respect to the center point of the position determination space as follows: That is, the first beam path from the center point through the first optical deflection element and lens to the image sensor, and the second beam path from the center point through the second optical deflection element and lens to the image sensor, are arranged so that they extend orthogonally from the center point. This embodiment makes it possible to generate 3D positional information that is particularly good at evaluation.

[0023] One embodiment of the position detector further comprises at least one light source arranged to illuminate the position determination space. This light source can improve the quality of 3D positional information, for example, by reducing image noise in image recordings that include 3D positional information. For example, the light source may be designed to illuminate in flash mode. The light source may emit monochromatic light or light from a narrow wavelength band. This allows for the selective blocking and removal of light from sources other than the light source in front of or inside the camera.

[0024] A further embodiment of the position detector comprises, in addition to at least one light source, a first diffusing element for diffusing light. Here, the first diffusing element and the first light deflecting element are arranged opposite to each other with respect to the positioning space, and the at least one light source is arranged to indirectly illuminate the positioning space via the first diffusing element. In this embodiment, a shadow image in which the contour of the object is particularly clear is generated in the first image. Therefore, feature points of the object such as vertices can be recognized particularly well. Further, this embodiment is particularly suitable for determining (measuring) the 3D position of an object having a reflecting surface or an at least partially transparent object. Particularly, this embodiment may further comprise a second diffusing element, and the second diffusing element and the second light deflecting element are also arranged opposite to each other with respect to the positioning space, and the at least one light source is arranged to indirectly illuminate the positioning space via the second diffusing element. In this way, two similar shadow images are generated from two different viewing directions. The second light source may also be arranged to particularly illuminate the second diffusing element. In this case, the first and second light sources can, for example, vary the color or wavelength of the emitted light rays, which enables the two shadow images to be distinguishable at the image sensor even if the images overlap. For example, with a suitable light source, a first shadow image of an object against a red background and a second shadow image of the object from a different viewing direction and against a green background can be generated simultaneously at the image sensor.

[0025] One embodiment of the position detector comprises a housing surrounding a camera, a first light deflecting element, a positioning space, and optionally a second light deflecting element, where the housing has an access opening for inserting at least one end of an object into the positioning space. This embodiment particularly generates high-quality 3D position information. In this embodiment, stray light within the imaging range of the camera or interference caused by the object, which does not belong to the object for which the 3D position information is to be collected, is mostly removed. At the same time, the object whose position is to be determined (measured) can be easily exchanged through the access opening.

[0026] For example, the camera of the position detector has a lens with a focal length in the range of 50 mm to 200 mm. In particular, the lens may have the characteristics of a telephoto lens. The inventor recognized in this embodiment that 3D position information can be determined with particularly high precision. Due to the focal length within the above-mentioned range, the distortion of the image is very low, and a large depth of field can be achieved.

[0027] The robot system according to claim 13 is also within the scope of the invention. The robot system according to the invention comprises a position detector according to the invention. In particular, this robot system may be a pipetting operation robot system.

[0028] Furthermore, the object is solved by the method according to claim 14. The method according to the invention is a method for determining (measuring) the spatial coordinates of an object. The method comprises the following steps: a) Positioning at least a part of the object with respect to the camera; b) Generating at least two images of the object on the image sensor of the camera, wherein at least one of the images is generated by reflection of the object, and at least two images reproduce the object from different viewing directions, and c) Determining the spatial coordinates of the object from at least two images. By the method according to the invention, the spatial coordinates of the object can be determined very quickly.

[0029] A modification of the above method is used to determine the spatial coordinates of an object in the positioning space of a position detector according to the invention. This method comprises the following steps: a) Positioning at least a part of the object in the positioning space; b) Generating an image record by means of a camera, where the image record includes a first and a second image, and the first image is generated by a beam path deflected by a first light deflection element, c1) Evaluate the first image in the image recording generated in step b), where the first horizontal-vertical position of the object in the image is determined. c2) Evaluate the second image in the image recording generated in step b), where the second horizontal-vertical position of the object in the image is determined. c3) Calculate the spatial coordinates of the object as a function of the first and second horizontal-vertical positions.

[0030] As the basis for the calculation in step c3), for example, a calibration procedure can be performed beforehand, in which the object is moved by known coordinate values ​​in three independent spatial directions, and steps a), b), c1), c2), and c3) are performed. From the relationship between the position determined in the image recording and the known coordinate changes, a system of linear equations for the transformation coefficients between the horizontal-vertical position and spatial coordinates in the two images can be established and solved. Such a calibration procedure can be performed on an accurate cross table, for example, on a CNC machine with a positional resolution of 10 μm or better. The inventors recognize that the use of 3D positional information from image recordings by the position detector according to the invention leads to particularly accurate and reproducible spatial coordinates.

[0031] In one variation of the above method, step c3) includes calculating the spatial coordinates of the object as a function of the average values ​​of the first and second horizontal-vertical positions and the distance between the first and second horizontal-vertical positions. In this variation of the method, the first coordinate can be determined from the average of two vertical positions. The second coordinate can be determined from the average of two horizontal positions. The third coordinate can be determined from the distance between the first and second horizontal-vertical positions. This coordinate is independent of the first and second coordinates, and it is ensured that all combinations of the three coordinates reduce to spatial coordinates.

[0032] Furthermore, the scope of the invention includes a method for operating the robot system according to the invention. This method includes the following steps: - To determine the first spatial coordinates of the reference point of the first object, the steps of the method according to the invention are performed while the first object is held in a positioning space by a robotic system. - Determine the first parameter set of the first motion sequence in the robot system, thereby moving the reference point of the first object from the positioning space to the target position. - To determine the second spatial coordinates of the reference point of the second object, the steps of the method according to the invention are performed while the second object is held in the positioning space by the robotic system. - To obtain a second parameter set for the second motion sequence in the robot system, modify the first parameter set of the first motion sequence as a function of the difference between the first and second spatial coordinates. - Execute the second action sequence based on the second parameter set provided by the second object.

[0033] The method for determining the first spatial coordinates of the reference point of the first object can be performed before or after the first parameter set of the first motion sequence is formed. In a combination of the two steps, a kind of virtual zero point, the reference point, is formed in the positioning space, and the first motion sequence then precisely guides it to the desired target position. Further motion sequences can be corrected for deviations of the corresponding reference points of the second object, and of other objects, from the previously formed reference point, in such a way that the corresponding reference points of other objects are also guided to the target position. Variations in the precise position of the reference point, which may be, for example, the tip of a pipette, can be compensated for in this way. Such variations can be caused by, for example, geometric deviations of each object, such as the individual shape of a pipette. For example, a slight bend in a long, hollow needle is almost unavoidable, and this results in a deviation between the needle's pickup position and the needle's tip in a plane perpendicular to the needle's axis. Variations can also be caused by the fact that the reproducibility of object pick-up by the robotic system is not always perfect. For example, repeated connection of a pipette to a corresponding container leads to slight variations in the spatial position of the pipette tip relative to the container, even if the pipette is the same. The method of the invention compensates for all of these variations, regardless of the exact cause.

[0034] This invention is further directed toward the application of the inventive method for operating robotic systems.

[0035] In one application of this method, the robotic system is a pipette manipulation robotic system, and the first object is the pipette tip, and the second object is the pipette tip. In this application, high precision in the well approach is achieved independently of manufacturing variations in pipette tips. This allows for the use of less expensive pipettes with well plates having smaller hole spacing, such as standard well plates with 96, 384, or 1536 wells, without increasing the risk of pipetting. For example, the first object may be the tip of the first pipette, and the second object may be a second pipette tip that is different from the first pipette tip. For example, the second object may also be the tip of the first pipette, but after the first pipette tip has been used to penetrate the lid of a container. For example, a pipette tip in the form of a long metal cannula can be used to penetrate the lid of a container. This may involve a slight change in shape, which may change the position of the pipette tip. In this sense, the pipette tip before and after the puncture process are the first and second objects, and their 3D positions can be determined separately by the method according to the invention. For example, a pipetting robot can move a sequence of several pipettes, such as four or eight pipettes, in sequence. In this case, for example, the individual deviation from the target position can be determined for each of the pipettes in the position detector. Thus, for example, the entire sequence of pipettes can be moved together to a rough position, and dispensing from each pipette can be performed separately. Here, immediately before dispensing, individual fine adjustments to the position can be made based on the offset determined in the position detector.

[0036] In one variation of the application, the target position is above the MALDI target plate, that is, the target plate for matrix-assisted laser desorption and ionization. In this application, the analyte in solution can be pipetted with great precision to the corresponding portion of the target plate onto which the laser beam is directed in the following steps. Thus, pulses containing the analyte molecules are generated immediately without the need to search for the region on the target plate with the highest analyte concentration. This application is particularly suitable as a preparatory step for mass spectrometry with time-of-flight analysis (TOF) based on matrix-assisted laser desorption / ionization (MALDI-TOF). For example, a transport device with a carrier for the MALDI target plate is mounted on the same platform on which the position detector is attached. This transport device is used to transfer the target plate from the pipette operating position to the MALDI-TOF mass spectrometer. In this way, particularly high reproducibility can be achieved when the target plate is positioned where the laser beam of the MALDI-TOF mass spectrometer is directed.

[0037] Possible procedures include the following steps: - Picking up the tip of the pipette. - The first pipette tip is moved to the position determination space of the position detector to determine the first spatial coordinates of the pipette tip for the first time by the method of the invention. This forms a reference point for pipette tips to be used thereafter. - Move the pipette tip to the precise desired dispensing position above the MALDI target plate. The MALDI target plate is in the pipette operating position. The rough position may be approached in advance, for example, along a programmed path. Movement to the precise position may be controlled by the user using arrow keys in the so-called teaching mode of the robot controller. The precise target position may be recognized, for example, by a ring printed on the target plate, where the center of the ring corresponds to the target position. - The target coordinates are stored as the coordinates of the robot system. Therefore, the distance of all spatial coordinates between the reference point in the position detector and the target coordinates is known. - Pick up the tip of the second pipette. - The second pipette tip is moved to the position determination space of the position detector, and the second spatial coordinates of the pipette tip are determined by the method according to the invention. This results in the position of the second pipette tip being different from the previously determined reference point. -Approaching the target coordinates above the MALDI target plate, the distance between the reference point and the target coordinates in the position detector is taken into consideration, and the determined difference of the second pipette position relative to the previously determined reference point is taken into consideration.

[0038] In the same method, several target coordinates can be determined, for example, in addition to the dispensing position above the MALDI target plate, the coordinates of a specific well in the microplate where the analyte is received by aspiration can also be determined. Then, when determining control commands for the path from the well from which the analyte is picked up to the desired dispensing position above the MALDI target plate, the stored displacement can be taken into consideration in an effective pipetting procedure.

[0039] In another variation of the application, the target position is formed on the culture plate, and the second set of parameters in the second operation sequence is further adjusted based on the coordinates of the bacterial colonies on the culture plate. By modifying the application, it is possible to punch out precisely formed areas of bacterial colonies in microscopic images using the pipette tip and then analyze them. Therefore, manufacturing variations in the pipette tip have little effect on the positioning accuracy when punching out samples in the culture plate.

[0040] Further within the scope of the invention is a system for determining the spatial coordinates of an object in a position determination space, wherein the system comprises a position detector according to the invention and an evaluation unit. The image sensor and the evaluation unit are operably connected for the transmission of image data. The system is further configured to perform the method according to the invention for determining the spatial coordinates of an object. For example, the evaluation unit may be formed by a control computer of a robot system. The evaluation unit may also be located at a distance from the position detector. Image data may be transmitted, for example, via a wired or wireless computer network. [Brief explanation of the drawing]

[0041] Embodiments of the present invention will be described in more detail with reference to the drawings. [Figure 1] Figure 1 shows a schematic cross-section of the position detector according to the invention. [Figure 2] Figure 2 shows a schematic cross-section of an embodiment of the position detector. [Figure 3] Figure 3 shows a schematic cross-section of an embodiment of the position detector. [Figure 4] Figure 4 shows a schematic diagram of the system for determining the spatial coordinates of an object. [Figure 5] Figure 5 shows a flowchart illustrating a variation of the method for determining the spatial coordinates of an object. [Figure 6] Figures 6a)-c) show image recordings of the pipette tip at different positions in the position detection space, and Figures 6d)-f) show schematic cross-sections illustrating each position of the pipette tip. [Figure 7] Figure 7 shows a perspective view of an embodiment of the position detector. [Figure 8] Figure 8 shows a robot system having a position detector according to the invention. [Figure 9] Figure 9 shows a perspective view of the embodiment shown in Figure 7, along with a view of the housing. [Figure 10] Figure 10 shows a cross-section according to the embodiment shown in Figure 9. [Modes for carrying out the invention]

[0042] Figure 1 shows a schematic cross-section of the position detector 30 according to the invention. A camera 3 having an image sensor 4 and a lens 5 forms an imaging area 10 in front of the camera. A first optical deflection element 1, shown here as a mirror, is located in the imaging area 10 which can be directly imaged by the camera. The first optical deflection element 1 forms a first indirect field of view 11 to a position determination space 13 where the position of an object can be determined (measured). There is a second (another) field of view 12 from the camera to the position determination space 13. Here, a direct field of view of the position determination space 13 is shown. The first field of view 11 and the second field of view 12 have different field of view directions of the position determination space. In this arrangement of the camera 3 and optical deflection element 1, first and second images in the position determination space can be simultaneously generated in the image sensor 4 from the first field of view 11 and the second field of view 12.

[0043] Figure 2 shows an embodiment of the position detector 30. In this embodiment, the first optical deflection element 1 and the second optical deflection element 2 are arranged in the imaging area 10 of the camera 3. Both the first field of view 11 and the second field of view 12 are formed by the deflected light beam. The two fields of view overlap in an overlapping area 14 that includes the position determination space 13. The camera 3 forms the optical axis 15. In the shown arrangement, the optical deflection elements 1 and 2, and the fields of view 11 and 12 are arranged symmetrically with respect to the optical axis.

[0044] Figure 3 shows a schematic cross-section of the position detector 30, which further comprises a housing 9 surrounding the camera 3, a first optical deflection element 1, a second optical deflection element 2, and a position determination space 13. The housing 9 may have light-shielding walls. Furthermore, in this figure, an object 20 is present in the position determination space. The light beam emanating from this object is deflected by the first and second optical deflection elements and incident on the camera lens 5. This light beam is indicated by a thin line. Furthermore, in this embodiment, light sources 6,6' are arranged. The first diffuser 7 is positioned opposite the first optical deflection element 1 to the position determination space 13. Light from the light sources is diffusely scattered by the diffuser. The diffuser can be, for example, a glass frame with a rough surface or a porous plastic block. Thus, the object 20 is projected onto the camera's image sensor as a kind of shadow image. Symmetrically, a second diffuser 8 is present, which plays a corresponding role with respect to the second optical deflection element. The aperture shields the camera lens from direct light from the light source or diffusing element. The overall arrangement is essentially symmetrical with respect to the camera's optical axis.

[0045] Figure 4 shows a schematic diagram of a system 40 for determining (measuring) the spatial coordinates of an object 20. The system 40 comprises a position detector 30 and an evaluation unit 41. The double lines symbolically represent the active connection for transmitting image data from the position detector to the evaluation unit. The image data can be transmitted, for example, via a USB cable, or via a cable for a series or parallel interface. The active connection for transmitting the image data can also be a wireless connection, such as a WiFi connection. In the diagram shown, the position detector has an access opening to a position determination space 13 into which the object 20 can be inserted at least partially. The system is configured to perform, for example, the method according to the invention shown in the flowchart in Figure 5. The evaluation unit includes, for example, a microprocessor and working memory loaded with software for performing the above method. The evaluation unit can also be directly mounted on a camera. In this case, the camera can simply pass the already determined data to a control computer of the entire system to which the position detector can be mounted. Such a complete system can be, for example, a robotic system, particularly a pipette operation robotic system.

[0046] Figure 5 shows a flowchart of the inventive method 100 for determining (measuring) the spatial coordinates of an object. This method is a) Positioning at least a part of the object relative to the camera (101), b) Generate at least two images of the object on the camera's image sensor (102), where at least one image is generated by reflection from the object, and at least two images reproduce the object from different viewing directions. c) Determine the spatial coordinates of the object from at least two images (103), The process includes the following steps. In one embodiment, step 103 is a substep: c1) Evaluate the first image from the image obtained in step b) (104), where the first horizontal-vertical position (H1, V1) of the object in the image is determined. c2) Evaluate the second image from the image recording obtained in step b) (105), where the second horizontal-vertical position (H2, V2) of the object in the image is determined. c3) Calculate the spatial coordinates of the object as functions of the first and second horizontal-vertical positions (106), It is equipped with.

[0047] In Cartesian coordinates x, y, z, the final calculation step is: x = Fx(H1, H2, V1, V2), y = Fy(H1, H2, V1, V2), z = Fz(H1, H2, V1, V2) It can be expressed as follows.

[0048] The mathematical functions Fx, Fy, and Fz depend on the imaging characteristics of the camera, as well as the relative arrangement of the camera and the optical deflection element. For example, in an arrangement where the X-axis is parallel to the horizontal axis of the image sensor, the Z-axis is parallel to the vertical axis of the image sensor, and the Y-axis is parallel to the optical axis of the camera, the coordinates are approximately as follows: x = ax(H1 + H2) / 2, y=b(H1-H2), z = az(V1 + V2) / 2, It can be calculated. Here, ax, az, and b are coefficients that take into account the conversion of units used (e.g., millimeters / pixel) and imaging scale. In the simple case, the imaging scale is the same in the horizontal and vertical directions, and ax=az=a applies. Other terms can account for correcting the zero point of the coordinates or for the imaging scale that changes depending on the distance from the camera to the object. Subsequent mathematical rotations of the coordinate axes or conversions of the coordinates to, for example, cylindrical or spherical coordinate systems are also possible.

[0049] Figure 6 shows image recordings of the pipette tip at different positions within the position detection area of ​​the position detector, from Figure 6a) to Figure 6c). Figures 6d) to Figure 6f) show schematic cross-sections for each case to illustrate the respective positions of the pipette tip in the adjacent figures on the right; that is, Figure 6d), etc., shows the position relative to the figure in Figure 6a), etc. In the image recordings from Figure 6a) to Figure 6c), the first and second optical deflection elements, illuminated by a diffuser in a plane mirror, are visible in white in the left and right halves of the image. The pipette tip, visible in the two halves under different viewing directions, is shown as a black shaded image with a sharp edge. These are the first and second images, each generated by the deflected light beam. A third, blurry image of the pipette tip can be seen in the dark zone in the center of the image recordings in Figure 6a) and Figure 6b), and offset to the right in Figure 6c). This is the directly imaged pipette tip and is not in focus, as the camera lens is focused at a distance corresponding to the distance to the virtual image in the mirror. At the edge of Figure 6a), the horizontal H and vertical V directions of the image recording are drawn, which also apply to the other two image recordings. In all the figures from Figure 6a) to Figure 6c), the horizontal position of the pipette tip is indicated by a vertical dotted line in the first and second images. At the lower edge of the image, one center of each of the two horizontal positions is indicated by a black circle. Furthermore, the distance between the two horizontal positions is indicated by two arrows. Figures 6a) and 6d) show the pipette tip at the reference position. Figures 6b) and 6e) show the pipette tip after it has moved from the reference position toward the camera. That is, in Figure 6e), the reference position is indicated by a dotted circle. The pipette tips that can be seen in the two images have a greater distance with respect to their horizontal positions. This distance contains information about a third spatial dimension, which corresponds to the distance from the camera and corresponds to the y-direction in the coordinate system according to Figure 6d). Figures 6c) and 6f) show the pipette tip after it has shifted further to the right and in the x-direction, respectively. This displacement is indicated by a rightward shift of the center of the horizontal position of the pipette tip, as shown by a simple arrow at the bottom of Figure 6c). The blurry direct image of the pipette tip is also shifted to the right. The z-direction of the third coordinate direction, along with the x- and y-directions, forms a clockwise Cartesian coordinate system.In the configuration shown here, the z-direction is oriented parallel to the vertical direction of the image sensor and can be read directly from the V-position in the image recording. In the image recording sequence shown here, the z-position of the pipette tip remained unchanged. In all image recordings, it was approximately half the height of the vertical V.

[0050] Figure 7 shows a perspective view of an embodiment of the position detector 30. In this case, the position detector 30 comprises a substantially cubic housing 9 having an access opening 19 in the upper wall. The first optical deflection element 1 in the form of a planar mirror is visible through the access opening. Through the access opening 19, an object can be introduced into the position determination space of the position detector. The shown position detector is suitable for measuring the position of the tip of a rectangular object. Cables 18 led through the wall to the housing 9 include power lines and data transmission lines for the position detector.

[0051] Figure 8 shows a robotic system 50 having a position detector 30. The shown robotic system is a pipette operating robotic system that can move a pipette in three coordinate directions x, y, and z, represented by the coordinate system arrows in the upper left corner of the figure. The object 20 whose position can be determined by the position detector 30 is the pipette of the pipette operating robot. The pipette operating robot includes a linear axis for moving and positioning the pipette tip. The pipette is connected to the pump unit (not shown here) of the pipette operating robot via a flexible hose. In the shown figure, the pipette tip is positioned directly above the access opening of the position detector and can then be lowered into the position determination space by moving the pipette, for example, in the z-direction. The well plate 51 is positioned on the work surface of the pipette operating robotic system within the range of motion of the linear axis. According to the present invention, the accuracy with which individual wells can be approached in the well plate 51 can be significantly increased based on 3D positional information about the pipette tip. For illustrative purposes only, only a well plate with 4 x 6 wells is shown here. The precision in approaching the well position is particularly important when using well plates of the same size with a very large number of wells on the same surface, such as 8×12 wells, 16×24 wells, or 32×48 wells.

[0052] Figure 9 shows an embodiment of the position detector 30 in a perspective view. The position detector has the elements already described in the embodiment shown in Figure 3. The housing 9 is shown transparent only with respect to this figure, so that the arrangement of each element within the housing can be seen. The light source in this embodiment is mounted directly in front of the camera and is not directly visible in Figure 9. The optical deflection elements 1 and 2 here are planar mirrors. The two planar mirrors form first and second planes, respectively, which intersect in a straight line perpendicular to the optical axis of the camera. The two mirrors each essentially fill half of the imaging area of ​​the camera 3. An access opening 19 in the housing allows an object or at least a part of an object to be inserted into the position determination area of ​​the position detector 30. The camera has cable connections for power supply and image data transmission via cable 18. The base plate supports the optical deflection elements 1 and 2, the diffusion elements 7 and 8, and the shielding element 17, which determine their precise positions within the position detector. The means 31 for adjusting the orientation of the camera within the position detector is formed as a socket head screw that screws into an upper projection on the base plate, with its end pressing against the base of the camera 3. The socket head screw, visible in the lower right corner of the figure, and the counteracting screw on the opposite side, allow for precise adjustment of the camera's position relative to the base plate of the position detector. In other words, means 31 allows for fine adjustment of the camera's orientation relative to other elements of the position detector.

[0053] Figure 10 shows a cross-section of an embodiment of the position detector 30 according to Figure 9. The cross-sectional plane extends horizontally and is located directly below the cover plate of the housing. As a result, the four outer surfaces of the housing 9 intersect and can be used to form a plan view of each element of the position detector located inside the housing. Means 31 for adjusting the orientation of the camera, acting from two sides at the base of the camera 3, are clearly visible. The optical deflection elements 1, 2, the diffusion elements 7, 8, and the shielding element 17 are arranged mirror-symmetrically with respect to an unillustrated central plane of the position detector, where the central plane is perpendicular to the cross-sectional plane. The optical deflection elements 1, 2 are planar mirrors here and form a spatial portion with an aperture angle α. The position determination space and the camera are also located in this spatial portion. In the case of α = 133°, i.e., the normals of the mirrors are inclined at 23.5° with respect to the central plane (α = 180° - 2 * 23.5°). The approximate location of the positioning space 13 in the spatial portion with an aperture angle is indicated by a dotted circle. The precise location of the positioning space 13 in this spatial portion is determined in the embodiment shown by the access aperture 19 (shown only in Figure 9). In the configuration shown here, there are first and second fields of view corresponding to the state shown in Figure 2, and beam paths corresponding to the state shown in Figure 3.

[0054] Returning to the characteristics and elements of the position detector, the following features may be implemented in the position detector according to the invention. The focal length of the camera lens can be adjustable. For example, the camera's image sensor can have a resolution of 600 kilopixels or more. Subpixel resolution can be achieved by using image processing software to evaluate blurry areas at the captured edges of an object. For example, the image sensor can be designed for monochrome images. For example, the image sensor can also be designed to produce red-green-blue (RGB) images. The position detector can be assembled, for example, on a surface area of ​​about 5 cm × 21 cm, thus requiring very little space within the robot system. At these dimensions, the field of view (FOV) of the position determination area is about 2 cm. A resolution of 10 micrometers in the determined spatial coordinates can be achieved with a camera resolution of 600 kilopixels or more. The position detector can have means for adjusting the orientation of the position detector relative to externally specified coordinates, particularly for fine adjustment. The position detector may optionally or additionally have means for adjusting the orientation of the camera relative to other elements of the position detector, particularly for fine adjustments, such as means for adjusting the orientation relative to the optical deflection element or the housing. [Explanation of Symbols]

[0055] 1 first optical deflection element, 2 Second optical deflection element 3 Cameras, 4 Image sensors, 5 lenses, 6, 6' light source, 7. First diffusion element, 8. Second diffusion element, 9 Housing, 10 Imaging area, 11. First field of view, 12. Second field of vision, 13 positioning area, 14. Overlapping regions, 15 Optical axis, 16. Light beam (of a light source), 17 Shielding elements, 18 cables, 19 access openings, 20 Objects, 30 position detectors, 31 Orientation setting means, 40. A system for determining spatial coordinates. 41 evaluation units, 50 robot systems, 51 well plate, 100 A method for determining the spatial coordinates of an object in the position determination space of a position detector. 101 Steps for positioning an object, 102 Steps for generating image recordings, 103 Steps for determining the spatial coordinates of an object, 104 Steps for evaluating the first image, 105 Steps for evaluating the second image, 106 Steps for calculating the spatial coordinates of an object, α Aperture angle (of the spatial portion including the position determination space), H Horizontal direction (in image recording), V Vertical direction (in image recording), x x-direction (in Cartesian coordinate system), y y-direction (in Cartesian coordinate system), z-direction (Cartesian coordinate system).

Claims

1. a position detector (30) for generating 3D position information of the object (20) in a position determination space (13) for the object (20), the position detector comprising: a camera (3) having a lens (5) and an image sensor (4) to form an imaging area (10); At least one first optical deflection element (1) disposed in the imaging region (10); Equipped with the camera (3) and the at least one optical deflection element (1) are configured to simultaneously generate at least two images of the positioning space (13) on the image sensor (4), the first image being generated by a light beam deflected by the first optical deflection element (1), and the at least two images differing in terms of the viewing direction of the positioning space; Position detector.

2. 2. The position detector according to claim 1, wherein the positioning space (13) is in the imaging area (10) of the camera (3).

3. 3. A position detector (30) according to claim 1 or 2, wherein the first light deflecting element (1) is configured to deflect light at a first flat surface forming a first plane.

4. A position detector (30) according to any one of claims 1 to 3, further comprising a second optical deflection element (2), wherein the camera (3) and the second optical deflection element (2) are configured to generate a second image in the at least two images by the light beam deflected by the second optical deflection element (2).

5. 5. A position detector (30) according to claim 4, wherein the second light deflecting element (2) is configured to deflect light at a second flat surface forming a second plane.

6. A position detector (30) according to claims 3 and 5.

7. 7. The position detector (30) of claim 6, wherein the first plane and the second plane intersect at an imaginary line that is perpendicular to the optical axis (15) of the camera (3), and in particular the imaginary line and the optical axis form a plane of symmetry in the position detector (30).

8. 8. The position detector (30) according to claim 6 or 7, wherein the first plane and the second plane form four spatial portions, the camera (3) and the positioning space (13) are arranged in a first spatial portion of the four spatial portions, and the opening angle (α) of the first spatial portion is in the range of 100° to 140°, in particular the opening angle (α) of the first spatial portion is in the range of 130.0° to 135.0°.

9. 9. A position detector (30) according to any one of claims 6 to 8, wherein the camera (3), the first optical deflection element (1), and the second optical deflection element (2) are arranged with respect to a center point of the position determination space such that a first beam path from the center point via the first optical deflection element through the lens to the image sensor and a second beam path from the center point via the second optical deflection element through the lens to the image sensor extend at right angles to each other at the center point.

10. 10. A position detector (30) according to any of the preceding claims, further comprising at least one light source (6, 6') arranged to illuminate the position determination space (13).

11. 11. The position detector (30) according to claim 10, further comprising a first diffusing element (7) for diffusingly scattering light, the first diffusing element (7) and the first light deflecting element (1) being arranged on opposite sides of the position determination space (13), and the at least one light source (6, 6') being arranged to indirectly illuminate the position determination space via the first diffusing element (7), in particular a second diffusing element (8), the second diffusing element (8) and the second light deflecting element (2) being arranged on opposite sides of the position determination space (13), and the at least one light source (6, 6') being arranged to indirectly illuminate the position determination space via the second diffusing element (8).

12. 12. A position detector (30) according to any one of claims 1 to 11, comprising a housing (9) enclosing a camera (3), a first optical deflection element (1), a position determination space (13) and optionally a second optical deflection element (2), the housing (9) having an access opening (19) for introducing at least one end of an object (20) into the position determination space (13).

13. A robotic system (50), in particular a pipetting robotic system, comprising a position detector (30) according to any one of claims 1 to 12.

14. A method (100) for determining spatial coordinates of an object, comprising: a) positioning at least a portion of an object relative to a camera (101); b) generating at least two images of the object on an image sensor of the camera (102), where at least one image is generated by reflection of the object, and the at least two images reproduce the object from different viewing directions; c) determining the spatial coordinates of the object from at least two images (103); 1. A method for determining spatial coordinates, comprising:

15. 15. A method (100) according to claim 14 for determining spatial coordinates of an object in a position determination space of a position detector (30) according to any one of claims 1 to 12, comprising: The generation (102) of the image record in step b) is performed by the camera (3) of the position detector (30), wherein the image record comprises a first and a second image, the first image being generated by a beam path deflected by the first optical deflection element (1), The determination of the spatial coordinates of the object (103) in step c) is c1) evaluating (104) a first image in the image record generated in step b), wherein a first horizontal-vertical position of an object in the image is determined; c2) evaluating (105) a second image in the image record generated in step b), wherein a second horizontal-vertical position of the object in the image is determined; c3) calculating the spatial coordinates of the object as a function of the first and second horizontal-vertical positions (106); A method comprising:

16. 16. The method of claim 15, wherein the calculation of the spatial coordinates of the object in step c3) is performed as a function of the average value of the first and second horizontal-vertical positions and the distance between the first and second horizontal-vertical positions.

17. A method of operating a robotic system (50) according to claim 13, comprising: - performing the steps of the method (100) according to claim 15 or 16 to determine a first spatial coordinate of a reference point of the first object while the first object is held by the robot system in the positioning space, - determining a first parameter set for a first movement sequence of the robotic system, whereby the robotic system moves a reference point of the first object from the positioning space to a target position; - performing the steps of the method according to claim 15 or 16 to determine a second spatial coordinate of a reference point of the second object while the second object is held by the robot system in the positioning space, - modifying the first set of parameters for the first movement sequence as a function of the difference between the first and second spatial coordinates to obtain a second set of parameters for a second movement sequence of the robotic system; - performing a second sequence of operations based on a second set of parameters by a second object; A method comprising:

18. 20. Use of the method according to claim 17, wherein the robotic system is a pipetting robotic system, and wherein the first object is a pipette tip and the second object is a pipette tip.

19. A system for determining the spatial coordinates of an object in a position determination space, the system comprising a position detector and an evaluation unit according to any of claims 1 to 12, wherein the image sensor and the evaluation unit are operatively connected for the transmission of image data, and the system is suitable for performing a method according to any of claims 14 to 16. system.

Images