Methods for evaluating objects, methods for manipulating objects, optical systems and manipulator systems

Abstract

The present invention relates to a method for optically acquiring the features of an object, wherein the object is located at an object location and includes physical features in multiple dimensions, the object is optically accessible by an optical system, the optical system includes at least a first optical device with a first adjustable spatial filter for acquiring optical information along a first optical axis, a second optical device with a second adjustable spatial filter for acquiring optical information along a second optical axis, and an adjustment system capable of adjusting the acquisition angle between the first and second optical axes. The present invention further includes an operating device, an optical system, and operating the object with the operating system.

Description

Technical Field

[0001] The present invention relates to a method for optically acquiring features of an object, the object being located at an object position and including physical features in a plurality of dimensions, the object being optically accessible by an optical system, the optical system including at least a first optical device having a first adjustable spatial filter for acquiring optical information along a first optical axis, a second optical device having a second adjustable spatial filter for acquiring optical information along a second optical axis, and an adjustment system capable of adjusting an acquisition angle between the first optical axis and the second optical axis. The present invention further relates to a method for operating an object with an operating device according to object features, the object being located at an object position and including physical features in a plurality of dimensions. Furthermore, the present invention relates to an operating system for operating an object, characterized by an optical system for acquiring features of the object and an operating device physically engaging with the object.

Background Art

[0002] Known methods for optically acquiring features of an object, or technical vision systems, generally rely on a process of supplying features of the object, also referred to as "teaching" the features of the object to the system, or to a software or artificial intelligence system. Based on this "teaching", the system can identify patterns and thus determine whether the viewed object actually belongs to a specific object class as previously "taught".

[0003] For example, an optical system equipped with a camera can be taught using multiple different photographs of a technical item from various viewpoints, for example, under various lighting conditions, so that the system can identify a specific pattern and later identify the technical object itself in a different context. It should be noted that this example does not use classical optical techniques, but can be moved on to methods using similar physical effects in light and / or sound, such as ultrasonic recognition. With ultrasonic recognition, distance and / or derived shape can usually be obtained, but recognition can only be achieved by teaching patterns and features to the respective systems.

[0004] In known methods for manipulating items, such as selection solutions for selecting technical items for an assembly process, the specific geometric characteristics of each object must be known in advance. For example, when selecting a specific bolt from a box of bolts, the bolt's characteristics, such as length, primarily cylindrical shape, and surface color, must be taught to the system to identify the specific bolt. This leads to problems in optically identifying and physically separating a single specific bolt from a large number of bolts in a box, for example, especially for glossy objects or objects with non-discriminative surface characteristics. Even considering the use of different technologies such as ultrasonic ranging and / or other wave-based detection systems like radar and lidar, the problem of actually recognizing the single item itself remains.

[0005] All these methods share the common requirement that a particular object must be at least partially understood and determined before identification and selection can take place. This leads to inflexible automated or handling processes, such as manufacturing processes. Therefore, identifying any object individually and independently from any given situation is impossible with known techniques.

[0006] Other known methods for optical acquisition use specific, predetermined settings to obtain at least some of the necessary information. Therefore, this information is usually inferred from 2D photographs or other evaluations of the scene, and can only provide a partial understanding. In particular, this "depth perception" is derived from primary information, for example. Monocular cues, such as motion parallax or perspective, require at least partially accurate background information about the setting or scene. Other techniques acquire only one piece of information, which does not allow for a complete understanding of depth. Radar can capture moving or changing distances with relatively high accuracy. Lidar can capture distance information that, under certain circumstances, may help in understanding depth for specific points within the field of view. Cameras can capture color because pixel arrays are used to represent specific points within the field of view, for example, color and brightness.

[0007] To understand the shape, position, and further characteristics of objects within the field of view, and to gain a true understanding of "depth" from that information, data stitching is necessary, which increases the need for massive computing power, especially when required for short periods or continuously. Furthermore, reliability can be low.

[0008] Furthermore, technologies with additional projection systems can be used to acquire at least shape information as part of the required depth information. Pattern projection of known geometric shapes and / or patterns can be projected onto an object and viewed by a camera, from which shape information can be calculated from the resulting elongated or flexed pattern. Typically, a stereo camera configuration with known distances between at least two lenses is required to obtain distance information and derive the shape and size of an object. Stereo cameras can also be used without pattern projection, but in that case they lack the ability to acquire shape information.

[0009] U.S. Patent Application Publication 2021 / 0262789 A1 discloses a stereo camera system that can collect 3D information by deriving data from captured pattern projections. Thus, a fixed distance, position, and angle between the two cameras are established.

[0010] Japanese Patent Publication No. 2021-190972A discloses a 3D measurement device equipped with a stereoscopic camera system with convergence angle detection. The convergence angle is part of a closed-loop control system equipped with a stereoscopic camera. [Overview of the project] [Problems that the invention aims to solve]

[0011] The objective of this invention is to improve upon the known state of the art. [Means for solving the problem]

[0012] This problem is solved by a method for optically acquiring the features of an object, wherein the object is located at an object position and includes physical features in multiple dimensions, the object is optically accessible by an optical system, the optical system including at least a first optical device with a first adjustable spatial filter for acquiring optical information along a first optical axis, a second optical device with a second adjustable spatial filter for acquiring optical information along a second optical axis, and an adjustment system capable of adjusting the acquisition angle between the first and second optical axes, the method being as follows: - The steps of aligning the optical system toward the object position so that the first optical device and the second optical device can gain optical access to the object position, -The steps of evaluating an object from a first viewpoint using a first optical device and a second optical device, and ensuring that each optical device engages with the object along its respective optical axis from the first viewpoint, - A step of adjusting the adjustable spatial filters of a first optical device and / or a second optical device to a defined spatial adjustment having defined spatial adjustment parameters, wherein the defined spatial adjustment provides a defined optical representation of an object and / or an object. - A step of deriving the distance between the optical system and the object from defined spatial adjustment parameters, so that first distance information between the optical system and the object can be derived, Includes, The first distance information between the optical system and the object, as well as / or the object's location, color, shape, and / or size, are obtained as the respective physical characteristics of the object.

[0013] In particular, by adjusting the adjustable spatial filter, the first distance to an object can be easily derived, for example, by adjusting the adjustable spatial filter to find a sharp representation or a particularly desired optical representation for determining the distance from defined spatial adjustment parameters.

[0014] One core idea of ​​the present invention is to adjust the focus of at least one camera of an optical device, for example, to find a specific distance to an edge or related structure or other feature of an object, by associating focus adjustment with a certain distance. It should be noted that the present invention, including the alternatives and embodiments described below, enables the evaluation of an object and the deriving of its properties, geometric shape, or other desirable knowledge about the object, despite dynamic situations and / or unstructured environments. Furthermore, it is not particularly necessary to pre-train any of the elements of the optical system with respect to the object being evaluated and / or the scene around the object. The present invention enables the evaluation of an object, as well as the construction of depth of the object and its surroundings, without the need for pre-training. Furthermore, spatiotemporal imaging or event imaging can be used to enhance the capabilities of this method, in particular enabling the ability of information-gathering-based focusing.

[0015] Another core idea of ​​the present invention relating to any of the embodiments presented is to provide a method for dynamically acquiring the manipulable aspects and features of an object, whether known or unknown, in an optically non-contact manner, where “unknown” means an object whose features have not been taught to and / or programmed into the system. This enables the “spontaneous manipulation” of such an unknown object, independent of variations in the signal source, such as illumination, even when the object is presented with dynamic variations in its orientation and / or position.

[0016] For example, an object is located at an object location, includes physical characteristics in multiple dimensions, and signals from the object are accessible to a sensing system.

[0017] Advantageously, the present invention enables the ability to observe multiple dimensions of information about an object using a single sensor, such as a fused sensor, thereby eliminating the need for sensor fusion in particular.

[0018] When used in accordance with this disclosure, unless otherwise indicated, the following terms shall be understood to have the following meanings:

[0019] First, it should be noted that the terms “optics,” “optical,” and “optically,” and similar terms, are presented to facilitate understanding of the present invention and to illustrate a system for acquiring visible light as a good example for carrying out the present invention. However, the scope of the present invention also covers any system that can acquire an energy medium non-contact, and transmitted electromagnetic and / or mechanical waves, in particular acoustic waves, can be used similarly without falling outside the scope of the present invention. This applies to all embodiments and aspects of the present invention as described below.

[0020] A “method for optical acquisition” is a method for optically evaluating an object to determine its so-called “characteristics.” This specifically refers to acquiring these characteristics non-contact by using electromagnetic waves and / or mechanical waves, more specifically acoustic waves, propagation, interference, or equivalent effects, and these characteristics may represent any relevant properties of the object, such as the object’s position, shape, orientation, one or more surface parameters, brightness or reflectivity, and mechanical characteristics, such as the stiffness and / or flexibility of one or various parts of the object, particularly under pressure or deformation. Thus, the optical acquisition itself can be performed by any of the following: visible light, ultraviolet light, infrared light, interferential laser light, radar waves, radio frequencies, or other electromagnetic waves in an appropriate frequency band. Furthermore, non-contact systems can also be presented via sound waves, such as ultrasound transmitted toward an object and used to determine any characteristics of that object. It should also be noted that in any embodiment of the present invention, methods may enable the detection and / or separation of so-called “mirrored objects,” such as objects with glossy surfaces, mirror-coated objects, and objects that are at least partially recognizable by reflection. This is mainly true in any lighting conditions and dynamically changing environments, because, in this example, reflections on a mirrored object are recognizable as changing color patterns, etc. This also applies appropriately to sound or other energy transmission. For example, solid, rigid objects include surfaces that can reflect sound. In certain embodiments of the present invention, even this apparent disadvantage may be overcome or used to collect more features of an object. As a general term for either optical systems and / or acoustic-based systems, “imaging of an energy medium” may be used.

[0021] In this regard, “object” can mean any physical part or component located in an “object location,” for example, in an identified box, on a table, on a plate, or any other location or place accessible by a non-contact system, particularly an optical system, for carrying out the method. Specifically, the object may be in an unknown orientation and position of the object that is not predetermined. The aforementioned features of the object are specifically defined as “physical features in multiple dimensions,” where the multiple dimensions include, but may exceed, the six physically known spatial dimensions, including the rotational dimension, and also include “dimensions” that cannot be determined by geometric material, such as optical features and / or mechanical features as stiffness or surface roughness, which are usually only accessible by touching the object.

[0022] "Optically accessible" is defined by an unbiased viewpoint of an object by an "optical system," which can be represented by any projecting or receiving optical structure represented by a technical part, apparatus, or mechanism. For sound-based systems as described above, this is defined by similarly unbiased contact between the system and the object. Generally, an optical system can be broadly interpreted and incorporates any non-contact sensing system, including ultrasonic, radar, lidar, and other similar systems. Thus, an optical system includes "optical devices," such as projector and / or camera systems for optically interacting with an object by transmitting and / or receiving light, where light is the energy being acquired. For ultrasonic systems, each "device" is a sound-based sensing device, for example, by using a sound source and receiver, similar to the light and camera of an optical system, and the same applies to each of the other systems.

[0023] Each "adjustable spatial filter" is a system that can interfere with light or any other form of energy related to the aforementioned other options along the optical axis, either in accordance with the projection or the depth of the field of view, or in accordance with the distance between each optical device and the adjustment point in space, in a way that enables adjustment of one or various optical paths. Therefore, for example, an adjustable spatial filter can be a focusing lens system that is adjusted to focus on an optical path at a certain distance from each optical device. This can be either a focusing system for projecting onto an object at a certain distance from each optical device and / or a focusing system for focusing on an optical image from an object for a camera. Comparing this approach with the aforementioned acoustic system, this can be achieved, for example, with a condenser microphone, a directable microphone, and / or a sound lens, together with a focused speaker or resonator system, by means of a structure that is adjustably shaped to reflect sound depending on the adjusted shape.

[0024] The "adjustment system" can adjust the acquisition angle between the first optical axis and the second optical axis. In other words, for example, it refers to a motor or actuator for rotating each optical device in such a way that the angle between the first optical axis and the second optical axis can be adjusted. The "acquisition angle" of such an optical system can usually be set respectively from 0°, which represents a parallel view of both optical devices, to 180°, which represents both optical devices facing each other. Usually, the acquisition angle is set between 0° and 120°, and the angle is based on a full circle of 360°. Other angles are also possible.

[0025] It must be mentioned that by adjusting the acquisition angle, both optical devices can be adjusted relative to each other in such a way that the first optical axis and the second optical axis converge and intersect at a certain distance from the optical system.

[0026] "Aligning" a non-contact, particularly an optical system, towards an object position means positioning the optical system at a position relative to the object such that the optical access to the object is fair, whereby the optical system can "see" the object freely and relatively unobstructed. In this regard, it is clear that optical access to the object is impossible when solid objects block their respective optical axes. Furthermore, a dynamic alignment process can be carried out, whereby the optical access angle or optical access path to the object is purposefully shifted and / or adjusted.

[0027] "Evaluating" an object refers to each specific act of using an optical system to collect information about the object, regardless of the method, and / or to project any information onto the object with the optical system. Thereby, the "first perspective" refers to the starting position from which the optical system evaluates the object. This "first perspective" may be, for example, a default position where the optical system starts evaluating with respect to a roughly known object position, and / or may be randomly or intentionally freely selected in space to obtain a "view" of the object. Once this view is established, each optical device "engages" with the object along its respective optical axis and can thus interact optically directly with the object.

[0028] To “adjust” the adjustable spatial filters of each optical device means to actively change the settings or parameters of each adjustable spatial filter and pursue the adjustment until a defined spatial adjustment results in a defined optical representation. In this regard, “defined spatial adjustment” refers to the settings of the adjustable spatial filters, for example, the electronically readable depth of field. “Defined optical representation” can be any particular optical state of an object, or interaction with an object, for example, a sharp image in a particular depth plane, or a particularly obstructed or shifted image from a projection, as well as a sharp projection onto an object at a certain distance. One key core idea of ​​the present invention is to use the focusing of the adjustable spatial filters to find a defined state of an object, which can be used, for example, to determine the distance to the object depending on the focus.

[0029] Therefore, "deriving" distance refers to the act of using defined spatial adjustment parameters of an adjustable spatial filter to reference the distance to an object. In a simple example, a focus setting for a known distance can be used to determine first distance information for, for example, a focused edge on an object. In a simple embodiment, this can also be accomplished by reading distance-related values ​​on the focus ring of a camera lens when focus is established on a feature of an object. In a more complex embodiment, this can also be done by an electronic sensor, software, or other means.

[0030] In this regard, the “first distance” specifically refers to a distance that involves a certain degree of uncertainty. In the case of a camera system, the first distance information may include the approximate location of an object, the first impression of its color, the first impression of its shape and / or size, or other information related to the object’s respective physical characteristics.

[0031] In one embodiment, the first or second optical device includes a projector, which is positioned to project an optical projection onto an object, and in particular, the optical projection is used to utilize a respective adjustable spatial filter.

[0032] In this regard, “projector” refers, for example, to a pattern projector or similar projector that projects an “optical projection” onto an object. Such an optical projection may be, for example, a stripe pattern, a dot pattern, or any other suitable pattern having varying illumination and darkness, colors, and shapes in different locations, and can establish a basis for evaluating an object based on that pattern. In other words, the optical projection can be “used” to adjust each adjustable spatial filter, for example by finding the focus of any detail in the optical projection, in order to determine the physical characteristics of an object, such as its surface or edge and its respective distance to the optical system. This can be achieved in a similar manner with respect to acoustic systems or other energy medium systems. In acoustics, for example, this can be achieved by sending interfering sound waves so that they are refracted at the edges of an object.

[0033] In another embodiment, the first optical device and / or the second optical device include optical sensors, each optical sensor configured to capture an optical image of an object along its respective optical axis, and each optical image is used to adjust its respective adjustable spatial filter.

[0034] An “optical sensor” is better suited to evaluating the optical features of an object by collecting and analyzing light received from it, compared to a projector. Therefore, an optical sensor can be a camera chip, CCD camera, CMOS camera, or equivalent camera system in the visible or invisible spectrum of light for capturing an “optical image” of an object, which is typically represented by pixels in the optical image. Thus, the optical information of any pixel in the optical image exists for analysis. Therefore, the blurring of pixels or the sharpness of the optical image can be used to adjust the respective adjustable spatial filters to determine edge distances based, for example, on the depth of field in the optical image. This can be done not only on visible light, but also on electromagnetic energy transmission of ultraviolet, infrared, radar, lidar, or any other suitable wavelength, respectively, for the embodiments mentioned above and below. The term “imaging” is understood as producing a multidimensional impression of any type of energy in any form. Any technique described and applied using the word “image” and its derivatives is applicable to any explicit definition above.

[0035] In yet another embodiment, the acquired angle between the first and second optical axes is adjusted according to first distance information, and in particular, this acquired angle and the optical distance between the first and second optical axes in the optical base of each optical device are used to calculate second distance information relating to the distance between the optical system and an object, so that the second distance information is characterized by higher accuracy than the first distance information in particular.

[0036] By adjusting the acquisition angle and matching the optical impressions of both optical devices, the acquisition angle can be used to evaluate a specific point on the object. For example, triangulation can determine a more accurate second distance to the object by using trigonometric calculations based on the distance between the optical devices and the acquisition angle.

[0037] By continuously adjusting the acquisition angle, a simulated impression of motion can be generated, and by analyzing the sequential photographs obtained over a series of adjustments, the impression of the object can be greatly improved.

[0038] Therefore, for any of the given options, the rough adjustment of the acquisition angle may be based on a first distance determined by the adjustment of one or more adjustable spatial filters. This particularly reduces the time it takes to actually "find" the correct acquisition angle that matches the actual distance to the object. The acquisition angle can be found physically, for example, by matching the respective pixels in the respective optical impressions from the first and second optical devices, and thus compensating for the distance between the two optical devices.

[0039] To improve the accuracy of the evaluation and / or to gain access to further physical characteristics of the object, the optical system is moved relative to the object, in particular along a nonlinear path with respect to one and / or both of the optical axes, so that the optical system can evaluate the object from a second viewpoint or any further viewpoint, in particular, each adjustable spatial filter and / or acquisition angle is dynamically adjusted so that the shape, size, reflectivity and / or orientation of the object are acquired as the respective physical characteristics of the object.

[0040] In this embodiment, continuous motion can also enhance the acquired information about an object. For example, continuous motion of an optical system along a known path can generate impressions of an object from various viewpoints. Thus, for any of the various images generated along the known path, for example, texture, color, or contour extraction can be used.

[0041] In addition, or as an alternative, motion cues and / or spatiotemporal events of amplitude in the acquired optical information can be used to assign various properties, such as texture, color, contour, shape, or distance, to parts of an object. This may include, for example, assigning similar or appropriate information about any location in the obtained photograph. Spatiotemporal events of a certain amplitude can also be used for this purpose. They may be caused by internal or external stimuli.

[0042] It should be noted that this method is preferably implemented with the powerful computing power of a computer, for example, which collects and uses data from an optical device, an adjustable spatial filter and / or an adjusted acquisition angle, and other acquired information to compute the physical characteristics of an object. Thus, photo analysis, similarity matching, and / or other algorithms may be used.

[0043] In yet another embodiment, which may apply to any of the embodiments described above, one or more of the physical features of an object are determined, in particular, by matching the equivalent optical impressions of adjacent locations in each optical representation of the object, while adjusting each adjustable filter, the acquisition angle, and / or while the optical system is moved.

[0044] Therefore, changes in optical impressions can be mathematically emphasized, for example, because the distinctive shape of an object can be filtered from the optical impression and from photographs of the object from various viewpoints of two optical devices, and as a result, edges, corners, or any other distinctive geometric features can be precisely determined, for example. This embodiment, in combination with one or more of the preceding or succeeding embodiments, takes advantage of the fact that changes in photographs or different optical impressions are easier to determine than filtering any features from a static image. Therefore, information about the known position of the optical system can also be included in the calculation. In multiple and / or consecutive steps, changes in each adjustable spatial filter, adjustment of the angle between the two optical axes, and movement of the optical system itself induce changes in each optical impression, which makes it possible to match adjacent locations with increased detail. This also makes it possible for the system to "see" or detect movement and / or change in multiple dimensions, and as a result, the system can relate information about an object. In this regard, it must be understood that color distinguishes points, pixels, and / or features, while motion associates these features with a certain position, pixel, or set of features.

[0045] For optical systems, the movement is macroscopic, or for any given option to be adjusted, fine-grained; it also allows us to grasp the vector and / or direction of change as well as the amplitude of change in all areas of the image. In a simple example, the glossy edges of an object provide a good reference point in space, which is observed from various viewpoints and thus positioned by the system at a certain coordinate in object space. Changes in reflection occur gradually while changing focus, angle, or position, and can therefore be matched by photographic analysis. The more movement and changes in optical impression are available, the better the evaluation of the object's features themselves becomes.

[0046] In particular, one or more physical characteristics of an object are determined by combining corresponding equivalent optical impressions from adjacent locations in various optical impressions.

[0047] In yet another embodiment, an object is distinguished from other objects in the vicinity of the object's location by adjusting the respective adjustable spatial filters of the first optical device and / or the second optical device to a defined spatial adjustment, particularly by adjusting the respective adjustable spatial filters from a hypothetical near distance between the optical system and the object to a far distance between the optical system and the object.

[0048] In this embodiment, the depth of the optical impression can be used to distinguish various objects from one another.

[0049] It should be further noted that, in any of the aforementioned options and embodiments, the generally known problem of “distortion” due to optical aberrations of the lens and other parts of the optical system can be avoided or tolerated. In any of the preceding embodiments, tolerances for aberrations can be improved. In particular, the dynamic movement of the optical system relative to an object allows for a wider field of view, and thus makes it possible to focus several captured images or optical impressions on a desired region or a specific part of an object.

[0050] In another embodiment, this problem is solved by a method for manipulating an object with a manipulating device according to the characteristics of the object, wherein the object is located at an object position and includes physical characteristics in multiple dimensions, and this method is as follows: -The steps of obtaining one or more optical features of an object using a method according to one or any of the preceding embodiments, such that distance information between the optical system and the object, the location of the object, color, shape, and / or size are obtained as the respective physical features of the object, - The steps of manipulating an object with an operating device while controlling the operating device according to one or more acquired physical characteristics of the object, The object is manipulated according to its characteristics.

[0051] One important idea of ​​this aspect of the present invention is that the acquired optical features in any of the aforementioned "dimensions" of an object can be used to manipulate the object for a purpose. For example, the position, orientation, and shape of the object can be used to determine the position of the object and the possibility of grasping or pushing the object in a certain direction. In this regard, the high quality and capability of the method for optically acquiring the features of an object described earlier in any of the embodiments can be used to acquire the features of an object necessary for a manipulative process.

[0052] In one embodiment, the operating device features a gripping device capable of physically grasping an object, the gripping operation of the gripping device being controlled according to one or more acquired physical characteristics of the object, in particular, changes in one or more physical characteristics of the object being acquired to determine the physical characteristics of the object.

[0053] This embodiment utilizes the interaction between a manipulating device and the object itself. For example, a gripping device may be used to select and position an object according to acquired optical features. In this regard, a “change in the physical features” of an object refers to, for example, a change in the shape of an object due to gripping it with a certain gripping force. For example, selecting a solid metal object does not result in a change in the shape of the metal object by gripping the metal object with a pair of pliers on a robot, but gripping a cushion almost certainly results in a change in the shape of the cushion, which can be evaluated by the method described above. Therefore, this change itself is acquired to determine the physical features of the object, or a change in its physical features, or to enhance information about one or more physical features of the object. As an example, the shape, orientation, and location of a cushion are determined by first optically evaluating its position and its edges. Therefore, a selection strategy can be calculated in three-dimensional space to control a manipulating device, such as a pair of pliers on a robot, in order to select the cushion. This pair of pliers is equipped with a system that measures pressure and / or gripping force on an object, and is used to provide feedback on the gripping process and gripping force. In parallel, the optical evaluation described above is further performed or repeated. The updated optical information, such as the determined shape change caused by gripping, can be matched with the measured gripping force. Thus, even elasticity can be mathematically determined for the cushion selected by the robotic system and stored as a new "dimension" of the cushion's physical characteristics.

[0054] In yet another aspect of the present invention, this problem is solved by an optical system that acquires the features of an object, the object being located at an object location and including physical features in multiple dimensions, and the optical system is configured to carry out the method according to the embodiments described above.

[0055] In yet another embodiment, this problem is solved by an operating system for manipulating an object, which features an operating device that physically engages with the object and is configured to carry out the methods presented above.

[0056] Further aspects of the present invention will become apparent from the drawings and the description of specific embodiments. [Brief explanation of the drawing]

[0057] [Figure 1] This shows a measurement structure equipped with an optical system for evaluating objects. [Figure 2] Detailed isometric views of the object in Figure 1 at its changing position are shown. [Figure 3] This shows a robot equipped with an optical system that engages with an object, as shown in Figure 1. [Figure 4a] Figure 3 shows a detailed view of an object with a certain shape. [Figure 4b] Figure 3 shows detailed views of objects with different shapes. [Modes for carrying out the invention]

[0058] The measuring device 101 features an optical system 111. The optical system 111 includes a carrier 103 that holds a first camera 121 and a second camera 131. The first camera 121 features a lens 123 and a focusing device 125. The focusing device 125 allows the light entering through the lens 123 of the camera 121 to be focused depending on the distance to the camera 121. The camera 131 features a lens 133 with the focusing device 135 and is constructed similarly to the camera 121. In the illustrated embodiment (compared to Figure 1 as well), the camera 121 can be rotated 141 around a pivot point 143 on the carrier 103. Thus, the angle 171 between the optical axis 153 of the camera 121 and the axis 155 of the camera 131 can be adjusted, respectively.

[0059] Object 161 is positioned at object position 160 in the vicinity of the optical system 111 in such a manner that the optical system 111 can evaluate object 161 without optical disturbance. The object itself features corners 163, 165, 167, and 169. From the viewpoint shown in Figure 1, the optical system 111 can evaluate corners 163, 165, and 169. Corner 167 is located in a position where there is no direct optical access to the optical system 111. It should be noted that, as a substitute for one of the cameras, a projector may be used to project, for example, a stripe pattern onto object 161 as seen by each camera. Hereafter, the system will be described based on the aforementioned apparatus with cameras 121 and 131.

[0060] Object 161 is evaluated by the optical system 101 by adjusting either the focusing device 125 and / or 135 to focus on various planes within the distance of cameras 121 and 131.

[0061] For example, the corner 163 located in plane 183 is in focus, and readings from the focusing devices 125 and / or 135 are used to determine the distance 173 to plane 183. However, due to the rough estimation by the focusing devices 125 and 135, there is uncertainty 175 in the distance 173. Planes 185 that cross corner 165, and plane 189 that cross corner 169, are out of focus and therefore not directly detected.

[0062] Specific optical information, as well as photographs of the object 161 focused on the plane 183, are evaluated by a computer 105 connected to and controlling cameras 121 and 131, for example, to determine the color and position of pixels representing corners 163. Further photographic analysis may be performed if it is appropriate to evaluate further information about the object at this stage of processing.

[0063] Distance information is derived from the settings of the focus adjustment devices 125 and 135 as a first evaluation, and more precisely, from the angle 171 adjusted by rotating camera 121 around pivot point 143 until the optical axes 153 and 155 coincide to intersect the corner 163, thus allowing both photographs from both cameras 121 and 131 to coincide with each other. This coincidence can be performed by comparing the pixel characteristics of both photographs taken by cameras 121 and 131. The exact value of distance 173 is calculated using the distance 151 in the base plane 181 passing through cameras 121 and 131. Using angle 171 and the distance 151 between the two cameras, possibly improved by angle sensors (not shown) for each camera, it becomes possible to triangulate the exact position of the corner 163 in three-dimensional space.

[0064] Similarly, the corners 165 and 169 are also located and evaluated by the optical system 111 in succession of steps.

[0065] It should be noted that the color and position of each pixel are then processed by photoanalysis, which can enable software to calculate reflective grade, gloss, surface roughness, and other properties.

[0066] In the following, as shown in Figure 2, it is assumed that the optical system 111 is temporarily moved in space relative to object 161, and the actual photograph of object 161 changes in the views of cameras 121 and 131. This can be done, for example, by a robot or a linear motion system (not shown). A rotation 147 is assumed with respect to object 161 in relation to the optical system 111, and corner 163 shifts to the positions of corner 163' and the other corners, respectively. The same applies to edge 191 which is shifted to become edge 191', and edge 193 which is shifted to become edge 193'.

[0067] Consequently, changes and movements are induced, and the focusing devices 125 and 135, as well as the angle 171 between the optical axes 153 and 155, are continuously adjusted to obtain more information about the object 161. For example, pixels representing corners 163 as described above can be shifted in space and tracked by comparing, for example, color, brightness, and / or reflection grade to match the "same" corner to the viewed corner 163'. The same applies to other features of the object 161, so that a more detailed information cloud about the object 161 can be determined, corners, edges, and shape can be evaluated, and its position and orientation in space can be found.

[0068] Robot 301 is equipped with the aforementioned optical system 111. The robot features a base 303 and joints 305, each joint connecting the arms 307 together to support the hand 309. Thus, robot 301 is a 6-axis industrial robot. Pliers 321 are mounted on the hand 309. Pliers 321 feature fingers 323 and are equipped with sensors to measure the gripping force of the fingers 323.

[0069] Objects 361 and 363 are positioned on platform 341. Robot 301 is tasked with grasping object 361, which is in an approximately known position, but no further information about the object itself is available. The optical system 111 has a clear view of platform 341 with objects 361 and 363 along axis 351. The objects are evaluated as described above, with object 361 oriented and positioned between planes 381 and 383, and object 363 positioned between planes 385 and 387, at a further distance from robot 301.

[0070] By adjusting the focus adjustment devices 125 and 135 and the angle 171, and by moving the position of the robot hand in space relative to object 361, the distinction between the two objects 361 and 363 is made by evaluating their different distances, and the shape and orientation in space of object 361 are evaluated in the manner described above. Subsequently, the robot 301 is controlled in such a manner that it grasps object 361 with the fingers 325 of the pliers 321 according to the previously evaluated edges and corners.

[0071] Object 361 is assumed to be "soft," for example, made of soft plastic foam, and therefore compressible by force. In its normal shape, object 361 features a contour 362. When gripped by the fingers 323, object 361 deforms into object 361' with contour 362' (see also Figures 4a and 4b), which is caused by the force of the fingers 323. The evaluation of object 361' is further repeated by the optical system 111, so that the deformation is recognized by matching the previous edges of object 361 with the deformed edges by comparing the properties of certain pixels. There is also a correlation between the sensor measuring the gripping force of the pliers 321 and the deformed object 361'. Thus, the stiffness of object 361 can be determined as a further property.

[0072] [Reference number] 101 Measuring device 103 Carriers 111 Optical Systems 121 Camera 123 Lens 125 Focus adjustment device 131 Camera 133 Lens 135 Focus adjustment device 141 rotations 143 Pivot point 147 rotations 151 distance 153 axis 155 axis 160 Object position 161 Object 163 Corner 163' Corner 165 corner 165' Corner 167 Corner 167' Corner 169 Corner 169' Corner 171 angle 173 distance 175 Uncertainty 181 Base plane 183 plane 185 plane 187 plane 189 plane 191 Edge 191' Edge 193 Edge 193' Edge 301 Robots 303 Base 305 joints 307 Arm 309 Hand 321 Pliers 323 Finger 341 units 351 axis 361 Object 361' object 362 Outline 362' Outline 363 Object 381 plane 383 plane 385 plane 387 plane

[0073] [Implementation Method] (1) A method for optically acquiring features of an object (161, 361), wherein the object (161, 361) is located at an object position (160, 341) and includes physical features (163, 165, 167, 169, 191, 193) in multiple dimensions, the object (161, 361) is optically accessible by an optical system (111), the optical system (111) includes at least a first optical device (121) with a first adjustable spatial filter (125) for acquiring optical information along a first optical axis (153), a second optical device (131) with a second adjustable spatial filter (135) for acquiring optical information along a second optical axis (155), and an adjustment system (143) capable of adjusting the acquisition angle (171) between the first optical axis (153) and the second optical axis (155), - The steps of aligning the optical system (111) toward the object position (160, 341) so that the first optical device (121) and the second optical device (131) can obtain optical access to the object position (160, 341), - Using the first optical device (121) and the second optical device (131), evaluate the object (161, 361) from a first viewpoint, and ensure that each of the optical devices (121, 131) engages with the object (161, 361) along its respective optical axis (153, 155) from the first viewpoint; - A step of adjusting the adjustable spatial filters (125, 135) of the first optical device (121) and / or the second optical device (131) to a defined spatial adjustment having defined spatial adjustment parameters, wherein the defined spatial adjustment provides a defined optical representation of the object and / or the object. -The steps of deriving the distance (173, 175) between the optical system (111) and the objects (161, 361) from the defined spatial adjustment parameters, and deriving first distance information (173, 175) between the optical system (111) and the objects (161, 361), Includes, A method by which the first distance information (173, 175) between the optical system (111) and the object (161, 361), and / or the location, color, shape, and / or size of the object (161, 361) are obtained as the respective physical characteristics of the object (161, 361). (2) The method according to Embodiment 1, wherein the first optical device (121) or the second optical device (131) includes a projector, the projector being positioned to project an optical projection onto the object (161, 361), and in particular the optical projection being used to adjust the respective adjustable spatial filters (125, 135). (3) The method according to Embodiment 1 or 2, wherein the first optical device (121) and / or the second optical device (131) include an optical sensor, each of which the optical sensor is configured to capture an optical image of the object along its respective optical axis (153, 155), and each of which the optical image is used to adjust the respective adjustable spatial filters (125, 135). (4) The method according to any one of embodiments 1 to 3, wherein the acquired angle (171) between the first optical axis (153) and the second optical axis (155) is adjusted according to the first distance information (173, 175), and in particular the acquired angle (171) and the optical distance (151) between the first optical axis (153) and the second optical axis (155) at the optical base (181) of each of the optical devices (121, 131) are used to calculate second distance information (173) relating to the distance between the optical system (111) and the object (161, 361), and so the second distance information (173) is characterized in that it is characterized in that it is characterized in that it is characterized in that it is characterized in that it is characterized in that it is characterized in that it is characterized in that it is characterized in that the acquired angle (171) and the optical distance (151) between the first optical axis (153) and the second optical axis (155) at the optical base (181) of each of the optical devices (121, 131), and in particular the second distance information (173) is characterized in that it is characterized in that the second distance information (173) is characterized in that it is characterized in that the second distance information (173) is characterized in that the second distance information (173, 175) is characterized in that the acquired angle (171) between the first optical axis (153) and the second optical axis (155) is adjusted according to the first distance information (173, 175), and in particular the acquired angle (171) and the optical distance (15 (5) The method according to any one of embodiments 1 to 4, wherein the optical system (111) is moved relative to the object (161, 361), in particular along a nonlinear path with respect to one and / or either of the optical axes (153, 155), and the optical system (111) is capable of evaluating the object (161, 361) from a second viewpoint, in particular the respective adjustable spatial filters (125, 135) and / or the acquisition angle (171) are dynamically adjusted, so that the shape, size, reflectivity and / or orientation of the object (161, 361) are acquired as respective physical characteristics of the object.

[0074] (6) The method according to any one of embodiments 1 to 5, characterized in that one or more of the physical features of the object are determined, in particular, by matching equivalent optical impressions (163, 163', 165, 165', 169, 169', 191, 191', 193, 193') of adjacent locations in each of the optical representations of the object, while adjusting the respective adjustable filters (125, 135), the acquisition angle (171), and / or while the optical system (111) is moved. (7) The method according to Embodiment 6, characterized in that one or more of the physical features of the object (161, 361) are determined by combining corresponding equivalent optical impressions (163, 163', 165, 165', 169, 169', 191, 191', 193, 193') from adjacent locations from various viewpoints. (8) The method according to any one of embodiments 1 to 7, characterized in that the object (161, 361) is distinguishable from other objects (363) in the vicinity of the object position (160, 341) by adjusting the respective adjustable spatial filters (125, 135) of the first optical device (121) and / or the second optical device (131) to a defined spatial adjustment, in particular by adjusting the respective adjustable spatial filters (125, 135) from a hypothetical close distance (381) between the optical system (111) and the object (161, 361) to a far distance (387) between the optical system (111) and the object (161, 361) (9) A method for manipulating an object (361) with an operating device (301) according to the characteristics of the object, wherein the object (361) is located at an object position (341) and includes physical characteristics in multiple dimensions, -The steps include: obtaining one or more optical features of the object (361) using the method described in any of Embodiments 1 to 8, and obtaining distance information between the optical system (111) and the object (361), the location (381, 383), color, shape (362), and / or size of the object (361) as the respective physical features of the object; - The steps of operating the object (361) with the operating device (301) while controlling the operating device (301) according to one or more of the respective physical characteristics of the object (361) obtained, Includes, A method characterized in that the object (361) is operated according to the characteristics of the object. (10) The method according to Embodiment 9, wherein the operating device (301) is characterized by a gripping device (323) capable of physically gripping the object (361), the gripping operation of the gripping device (323) is controlled according to one or more acquired physical characteristics of the object (361), in particular a change (362') of the physical characteristics (362) of the object or a change in a plurality of physical characteristics is acquired to determine the physical characteristics of the object (361).

[0075] (11) An optical system (111) for acquiring features of an object (161, 361), wherein the object (161, 361) is located at an object position (160, 341) and includes physical features in multiple dimensions, and the optical system is configured to carry out the method described in embodiments 1 to 8. (12) An operating system (301) for operating an object (361), comprising an operating device (323) that physically engages with the object, configured to carry out the method described in Embodiment 9 or 10.

Claims

1. A method for optically acquiring features of an object (161, 361), wherein the object (161, 361) is located at an object position (160, 341) and includes physical features (163, 165, 167, 169, 191, 193) in multiple dimensions, the object (161, 361) is optically accessible by an optical system (111), the optical system (111) includes at least a first optical device (121) with a first adjustable spatial filter (125) for acquiring optical information along a first optical axis (153), a second optical device (131) with a second adjustable spatial filter (135) for acquiring optical information along a second optical axis (155), and an adjustment system (143) capable of adjusting the acquisition angle (171) between the first optical axis (153) and the second optical axis (155). - The steps of aligning the optical system (111) toward the object position (160, 341) so that the first optical device (121) and the second optical device (131) can obtain optical access to the object position (160, 341), - Using the first optical device (121) and the second optical device (131), evaluate the object (161, 361) from a first viewpoint, and ensure that each of the optical devices (121, 131) engages with the object (161, 361) along its respective optical axis (153, 155) from the first viewpoint; - A step of adjusting the adjustable spatial filters (125, 135) of the first optical device (121) and / or the second optical device (131) to a defined spatial adjustment having defined spatial adjustment parameters, wherein the defined spatial adjustment provides a defined optical representation of the object and / or the object. - A step of deriving the distance (173, 175) between the optical system (111) and the objects (161, 361) from the defined spatial adjustment parameters, so that first distance information (173, 175) between the optical system (111) and the objects (161, 361) can be derived, Includes, A method by which the first distance information (173, 175) between the optical system (111) and the objects (161, 361), and / or the location, color, shape, and / or size of the objects (161, 361) are obtained as the respective physical characteristics of the objects (161, 361).

2. The method according to claim 1, wherein the first optical device (121) or the second optical device (131) includes a projector, the projector being positioned to project an optical projection onto the object (161, 361), and in particular the optical projection being used to adjust the respective adjustable spatial filters (125, 135).

3. The method according to claim 1, wherein the first optical device (121) and / or the second optical device (131) include an optical sensor, each of which the optical sensor is configured to capture an optical image of the object along its respective optical axis (153, 155), and each of which optical images is used to adjust its respective adjustable spatial filter (125, 135).

4. The method according to claim 1, wherein the acquired angle (171) between the first optical axis (153) and the second optical axis (155) is adjusted according to the first distance information (173, 175), and in particular the acquired angle (171) and the optical distance (151) between the first optical axis (153) and the second optical axis (155) at the optical base (181) of each of the optical devices (121, 131) are used to calculate second distance information (173) relating to the distance between the optical system (111) and the object (161, 361), and so the second distance information (173) is characterized by higher accuracy than the first distance information (173, 175).

5. The method according to claim 1, wherein the optical system (111) is moved relative to the object (161, 361), in particular along a nonlinear path with respect to one and / or either of the optical axes (153, 155), and the optical system (111) is capable of evaluating the object (161, 361) from a second viewpoint, in particular the respective adjustable spatial filters (125, 135) and / or the acquisition angle (171) are dynamically adjusted, so that the shape, size, reflectivity and / or orientation of the object (161, 361) are acquired as respective physical characteristics of the object.

6. The method according to claim 1, characterized in that one or more of the physical features of the object are determined, in particular, by matching equivalent optical impressions (163, 163', 165, 165', 169, 169', 191, 191', 193, 193') of adjacent locations in each of the optical representations of the object, while adjusting the respective adjustable filters (125, 135), the acquisition angle (171), and / or while the optical system (111) is moved.

7. The method according to claim 6, characterized in that one or more of the physical features of the object (161, 361) are determined by combining corresponding equivalent optical impressions (163, 163', 165, 165', 169, 169', 191, 191', 193, 193') from adjacent locations from various viewpoints.

8. The method according to claim 1, characterized in that the objects (161, 361) are distinguishable from other objects (363) in the vicinity of the object position (160, 341) by adjusting the respective adjustable spatial filters (125, 135) of the first optical device (121) and / or the second optical device (131) to a defined spatial adjustment, in particular by adjusting the respective adjustable spatial filters (125, 135) from a hypothetical close distance (381) between the optical system (111) and the objects (161, 361) to a far distance (387) between the optical system (111) and the objects (161, 361).

9. A method for manipulating an object (361) with an operating device (301) according to the characteristics of the object, wherein the object (361) is located at an object position (341) and includes physical characteristics in multiple dimensions. - A step of obtaining one or more optical features of the object (361) using the method according to any one of claims 1 to 8, wherein distance information between the optical system (111) and the object (361), the location (381, 383), color, shape (362) and / or size of the object (361) are obtained as respective physical features of the object, - The steps of operating the object (361) with the operating device (301) while controlling the operating device (301) according to one or more of the respective physical characteristics of the object (361) obtained, Includes, A method characterized in that the object (361) is operated according to the characteristics of the object.

10. The method according to claim 9, wherein the operating device (301) is characterized by a gripping device (323) capable of physically gripping the object (361), and the gripping operation of the gripping device (323) is controlled according to one or more acquired physical characteristics of the object (361), in particular a change (362') of the physical characteristics (362) of the object or a change in a plurality of physical characteristics is acquired to determine the physical characteristics of the object (361).

11. An optical system (111) for acquiring features of an object (161, 361), wherein the object (161, 361) is located at an object position (160, 341) and includes physical features in multiple dimensions, and the optical system is configured to carry out the method according to claims 1 to 8.

12. An operating system (301) for operating an object (361), comprising an operating device (323) that physically engages with the object, configured to carry out the method described in claim 9 or 10.

Images