Method for operating a mechanical device and device for measuring the location and / or position of objects

Abstract

Method for operating a mechanical device that has an optical gesture recognition device which can recognize the position and / or orientation of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker or the structure of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker, and which, after recognizing at least one predetermined gesture performed by means of the gesture input object and / or the hand and / or the finger and / or another body part of the gesture speaker and / or several other body parts of the gesture speaker, reacts to this recognition by at least one mechanical operation, a. wherein the gesture recognition device determines at least a distance between the gesture input object and / or the hand and / or finger and / or other body part of the gesture speaker from the device that performs the mechanical process and b. wherein the gesture recognition device additionally determines at least one further parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of the gesture speaker simultaneously and c. wherein a measurement of at least one distance and one further parameter is carried out by compensating measurement of the transit time and the amplitude.

Description

Introduction

[0001] The invention relates to a method for operating a mechanical device that has a gesture recognition device. This gesture recognition device is capable of recognizing the position, orientation, and / or orientation of a gesture input object, in particular the hand, finger, and / or other body part of a gesture speaker. The structure of a gesture input object, i.e., also the hand, finger, and / or other body part of a gesture speaker, can also be recognized. "Structure" here refers, for example, to an open hand as opposed to a closed hand. Such gestures are disclosed, for example, in German patent application DE102012010627A1.After recognizing a predetermined gesture or sequence of gestures performed by the said gesture input object and / or a hand and / or finger and / or another body part of a gesture speaker, the device reacts to this recognition through a mechanical process. Such a mechanical process could be, for example, unlocking a door, opening a car's tailgate, opening and / or closing another lockable opening, or changing the locking state, i.e., for example, partially opening and / or closing said opening.

[0002] Various optical methods for controlling the closed state of an opening, particularly an opening in a motor vehicle, are already known in the prior art. Examples include (EP1901947B1, DE102008029223A1, DE102005005185A1, DE102011115760A1). However, all of these methods have the disadvantage that they are not equally capable of distinguishing different skin tones with respect to skin reflectivity.

[0003] Particularly in the case of dark skin types, which are very common in southern latitudes, recognition difficulties may arise because the reflectivity of such skin types is reduced. Object of the invention

[0004] It is therefore the object of the invention to enable reliable gesture recognition even when the measurement of at least one distance and one further parameter is carried out by compensating measurement of the transit time and the amplitude. This is achieved with a method according to claims 1 and 3 and with a device according to claim 21. Description of the invention

[0005] The problem according to the invention is solved in particular by the gesture recognition device simultaneously determining at least the distance of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker and a further parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of a gesture speaker. The additional parameter is preferably the reflectivity of this gesture input object and / or this hand and / or this finger and / or the other body part of a gesture speaker. Preferably, object classification is carried out with the aid of the device according to the invention, which performs the method according to the invention.

[0006] The measured parameters are preferably the amplitude of a reflected light beam and its transit time. This is preferably used to determine the reflectivity of the gesture input object and / or the hand and / or that finger and / or the other body part of a gesture speaker. The other parameter can then be used to determine the distance between the gesture input object and the sensor.

[0007] If more than one such sensor system is used, the location along the great circle at an equal distance from the at least two sensors can be determined by triangulation within the sensitivity lobe, unless a directional measurement limits the location of the detected object to a very narrow solid angle. In such a case, the term "measuring lobe" is used below, where the measuring lobe is limited to a predefined solid angle. If more than two such sensor systems are used, the exact location can be determined by triangulation within the sensitivity lobes. The surfaces of all three distance spheres and the measuring lobes must overlap at this point.

[0008] This spatial coordinate can change as part of a gesture. Such a gesture recognition device therefore determines not only the two- and / or three-dimensional position relative to the sensor's location, but also typically its simple and / or higher-order temporal derivatives, i.e., movement speeds and / or accelerations.

[0009] In some cases, it is useful to limit the reflectivity measurement to a predefined spectral range and to use a broadband transmitter (H). For example, it may be useful to limit the upper wavelength to, say, 2000 nm and / or 1000 nm and / or 750 nm and / or 600 nm and / or 500 nm and / or 400 nm and / or 300 nm and / or 200 nm and / or a value between these values, and to limit the lower wavelength to, say, 1000 nm and / or 750 nm and / or 600 nm and / or 500 nm and / or 400 nm and / or 300 nm and / or 200 nm and / or 100 nm and / or a value between these values, whereby the lower wavelength should always be shorter than the upper wavelength.

[0010] Furthermore, the reflectivity of the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker can change during a gesture. This typically occurs through a change in the structure of the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker. For example, if the gesture speaker's hand opens, its surface area and thus its reflectivity increase. This change in the reflectivity parameter can be part of a gesture, just like the reflectivity itself.

[0011] According to the invention, an optical transmitter is supplied with a first band-limited signal and emits light into a predetermined spatial region, with the light reflected by an object within this predetermined spatial region being at least partially received by a receiver. A compensation transmitter, supplied with a compensation signal, also emits light into the receiver. The compensation signal is complementary to the transmitted signal, and the light from the compensation transmitter and the transmitter is superimposed in the receiver. The receiver generates a receiver output signal that depends on the superimposed received optical signals from the transmitter and the compensation transmitter.A controller adjusts the amplitude and phase and / or amplitude and delay of the transmitted signal and / or the compensation signal so that the receiver output signal contains no correlating components of the transmitted signal's spectrum, except for control errors and system noise. The controller outputs at least two control signals as measured values ​​for the distance and one other parameter.

[0012] The solution according to the invention consists in an evaluation unit, which is part of the gesture recognition device, that evaluates at least two signals for the recognition of a gesture and / or gesture sequence. The controller adjusts the transmit signal and / or the compensation transmit signal in amplitude and phase and / or in amplitude and delay such that the receiver output signal is not a direct signal. The optical transmitter can be a laser. The transmit beam of the optical transmitter is deflected by a micro-optical element, in particular a micromechanical mirror. The distribution function of the measurement points, which is defined as measurement points per solid angle segment within the measuring range of the device, is not homogeneous. The mechanical device to be operated comprises a car door or tailgate or other opening in a car, machine, object, or building.

[0013] A particularly distinctive feature of this device is that it has at least one mechanical sub-device that can be controlled by such a gesture recognition device. Preferably, the device according to the invention operates a car door or tailgate or other opening in a car, machine, or object.

[0014] If the device is installed, for example, in a rearview mirror integrated into a car door, it can monitor the area in front of that specific car door. The sensitivity beam can be adjusted optically and / or through data selection in an evaluation circuit, for example, so that it lies parallel to the car door at the height of the door handle. It should be noted that the measurement beam is the intersection of the points in space detected by the transmit beam and the points in space detected by the receive beam. The sensitivity range can be limited, for example, by an allowed light travel time along the measurement beam. This defines a distance range that is permissible for gesture recognition.Instead of limiting gesture recognition based on distance, limitations based on the solid angle range and / or an angle and / or a solid angle and / or, more generally, on a spatial area are also conceivable, provided that this limitation lies within the measuring range of the system itself. The spatial area defined in this way, in which gestures can be recognized, can otherwise have a more or less arbitrary shape and structure. Preferably, however, one will limit oneself to as few shape parameters as possible for this spatial area in order to limit the necessary computing power.

[0015] Typically, more than one sensor is used. In this case, the data can also be limited by filtering. Only data whose coordinates lie within a permissible set of points on the measuring beam are then processed. If the measuring beam can be swiveled, only data whose coordinates lie within the union of a permissible set of points on the measuring beams with a permissible orientation are processed.

[0016] Through data selection, a spatial area can now be defined that lies directly in front of the door handle or any other designated area in front of the door. In extreme cases, the car doesn't even need to have a door handle anymore, which is a key aspect of the invention.

[0017] To open the door and / or access opening, in addition to verifying access authorization, it is advisable to require a recognizable first gesture sequence and / or gesture for unlocking and a second gesture sequence and / or gesture for opening the door. This can be transmitted to the gesture recognition device via the aforementioned electronic key, i.e., via a personalized and / or customizable transmitter.

[0018] The gesture recognition device typically evaluates the recognized gesture and / or gesture sequences using a confidence level. This confidence level is used to decide whether to accept or reject a gesture or gesture sequence hypothesis. Only gestures or gesture sequences with a confidence level above a certain threshold are accepted for control. It is possible that gestures or gesture sequences may be rejected even though they are correct. The frequency of such an erroneous rejection of a correct gesture or gesture sequence is referred to as the false rejection rate. Conversely, instances may be accepted as correct gestures or gesture sequences that are not correct. These are referred to as the false acceptance rate.The false rejection rate and the false acceptance rate are critical parameters of the system, which are typically seen as opposing requirements.

[0019] For optimization, it is therefore advisable, for example, to activate the gesture recognition device with an initial gesture and to transmit a command sequence with a second and potentially subsequent gesture and / or gesture sequence, which then, for instance, unlocks and opens the door. It is beneficial to set a higher threshold for the initial gesture and / or gesture sequence to prevent excessively frequent false detections when the first gesture or gesture sequence is absent, thus accepting a higher false rejection rate in favor of a lower false rejection rate. The threshold for the first gesture and / or gesture sequence is set higher than the threshold for the second gesture or gesture sequence. This results in a lower false rejection rate and a higher false acceptance rate for the second gesture or gesture sequence.

[0020] It is known that a suitable system for measuring the distance and the other parameter is possible, for example, according to a method according to EP1913420B1 or EP2631674A1.

[0021] The major advantage of such a gesture recognition device, which performs the method according to the invention, is that even interfering parameters such as rain and / or fog can be reliably detected by the system. An exemplary evaluation is known from DE102012010627A1.

[0022] It is advantageous if the user is informed about the status of the gesture recognition via an acoustic, visual, and / or mechanical feedback signal. For example, it is conceivable that a screen, particularly a curved screen—preferably one whose surface follows the curvature of the door—is located on the outside of the door. This screen displays a feedback symbol, informing the user of the measured value interpreted by the device. Such a feedback symbol should be modifiable by at least one, preferably two, feedback parameters. These feedback parameters should correlate with measured parameters, such as the three spatial coordinates of the gesture input object, i.e., the hand or finger, etc. Such feedback parameters could include, for example, size, color, brightness, sharpness, virtual three-dimensional orientation, and shape (e.g.,...).The shape can be round (or square), etc. The possibilities are manifold. If the input object approaches a point that, unlike in the prior art, does not have to be identical to the sensor's location, then, for example, one of these parameters of the feedback signal, such as its magnitude, is changed. For example, it becomes larger when approaching a point on the door or smaller when moving away.

[0023] From an aesthetic point of view, it is particularly advantageous for a car door, tailgate, or other opening in a car, machine, or object if no directly mechanically operated opening devices interrupt the surface of the object on one side, typically the outside. In particular, from an artistic standpoint, it is desirable that the door, etc., no longer has a door handle, at least on the outside, but can be opened exclusively via gesture recognition from that side. In the case of a vehicle or apartment door, it is advisable to supplement the gesture recognition with at least one additional security measure. In the simplest case, this would be another wirelessly transmitted electromagnetic and / or acoustic signal, for example, transmitted by an electronic key.This electronic key can also be a gesture recognition object that is equipped, for example, with sensors about position and location in space and uses this data and / or its temporal derivatives alongside other data for encryption and transmits it to the device according to the invention.

[0024] It is therefore a special feature of the invention if the gesture recognition device can also detect distances to objects other than the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker, and / or the positions of these other objects. These are calculated and related to the coordinates of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker. In some cases, it is advantageous if even just the presence of such objects within a certain distance range and / or spatial angle range and / or spatial area and / or spatial angle range is detected and, if necessary, signaled to the user.

[0025] This property can then also be used to perform additional functions besides gesture recognition. For example, if the device according to the invention is installed in a rearview mirror, it can also be used to detect objects in the blind spot of a car. The preferred installation location of the device according to the invention, or parts thereof, in a rearview mirror or a similar protruding part of a car body, is therefore an essential part of this disclosure.

[0026] The invention will be explained in more detail below with reference to the attached drawings. Fig. Figure 1 shows the positioning of the sensors (1, 2) on the doors (3, 4) of a vehicle (5) in the rearview mirrors (6, 7). Fig. 2 shows the same situation as in Fig. 1, however, now without the door handles (8, 9). These have been replaced by screens (10, 11) which provide the aforementioned feedback signal. Fig. Figure 3 shows the spatially limited areas (12, 13) of gesture recognition relative to the door. Fig. Figure 4 shows a gesture recognition device according to the invention.

[0027] An optical transmitter (H) is fed with a first band-limited transmit signal (S5). This signal is emitted into the defined spatial region (12, 13). An object in this spatial region reflects the emitted light. This reflected light is received by a receiver (D). A compensation transmitter (K), which is fed by a compensation signal (S3), also emits light directly and / or indirectly into the receiver (D). The compensation signal (S3) is typically complementary to the transmit signal (S3). This means that it compensates for a weaker incoming signal from the reflected signal to the receiver (D) so that the total irradiance of the receiver (D) is always approximately constant, except for a control error and system noise. The light from the compensation transmitter (K) and the transmitter (H) therefore overlaps in the receiver (D).The receiver (D) generates a receiver output signal (S0) correlated with the irradiation from the compensation transmitter (K) and transmitter (H). This signal therefore depends on the superimposed received optical signals from the transmitter (H) and the compensation transmitter (K). A controller (CT) adjusts the amplitude and phase and / or the amplitude and delay of the transmit signal (S5) and / or the compensation transmit signal (S3) against each other so that the receiver output signal (S0) contains no correlating components of the spectrum of the transmit signal (S5), except for a control error and system noise. It is particularly important to emphasize here that a controller (CT) adjusts the amplitude and phase and / or the amplitude and delay of the transmit signal (S3) only in the manner described above. The receiver output signal (S0) does not necessarily have to be a DC signal. A laser is particularly suitable as an optical transmitter (H).To scan the space, it is advantageous to use a micro-optical element, particularly a micromechanical mirror, to deflect the light beam from the transmitter (H). This is achieved primarily through line-by-line scanning. The resolution in the area of ​​a relevant object, such as a hand performing a gesture, is increased by adjusting the line spacing in that area. This results in a non-homogeneous distribution of measurement points within the device's measuring range. Typically, after repositioning the light beam from the transmitter (H) to a new measurement point, a measurement is taken, followed by another repositioning of the light beam from the transmitter (H), after which the next measurement is taken at the next measurement point, and so on.Discrete sampling is therefore performed within the area of ​​interest, whereby the distribution of measurement points within this area need not be homogeneous. Instead, a distribution function can be defined to specify a certain density of measurement points per solid angle segment. This distribution function can depend on the already detected objects and / or gestures, their simple and higher-order temporal and spatial derivatives and integrals, and any weighting function that may need to be specified.

[0028] The controller (CT) outputs at least two control signals as measured values ​​for the distance and another parameter, typically the received amplitude. Since the micromechanical scanning device simultaneously possesses one or two spherical coordinates at the time of measurement, three-dimensional recognition can be performed in this way. An evaluation unit, which is part of the gesture recognition device, evaluates these at least two, preferably four, signals to recognize a gesture and / or gesture sequence. Such a measurement method, in which a control loop adjusts the physical quantity to be measured, here the light intensity, at a sensor, here the receiver (D), is referred to below as compensating control measurement. Specifically, this is a compensating control measurement of the transit time and the amplitude.

Claims

[1] Method for operating a mechanical device which has an optical gesture recognition device which can detect the position and / or orientation of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker or the structure of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker, and which, after detecting at least one predetermined gesture performed by means of the gesture input object and / or the hand and / or the finger and / or another body part of the gesture speaker and / or several other body parts of the gesture speaker, reacts to this detection by at least one mechanical operation, a. wherein the gesture recognition device determines at least a distance between the gesture input object and / or the hand and / or finger and / or other body part of the gesture speaker from the device that performs the mechanical process and b. wherein the gesture recognition device additionally determines at least one further parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of the gesture speaker simultaneously and c. wherein a measurement of at least one distance and one further parameter is carried out by compensating measurement of the transit time and the amplitude. [2] Method according to claim 1, a. wherein only movements within a predetermined distance interval and / or a predetermined spatial angle and / or within a predetermined angle and / or within a predetermined spatial area, the position of which is determined by the device that performs the mechanical process, are used for gesture recognition. [3] Method for operating a mechanical device which has an optical gesture recognition device which can detect the position and / or orientation of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker or the structure of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker and / or several other body parts of a gesture speaker, and which, after detecting at least one predetermined gesture performed by means of the gesture input object and / or the hand and / or the finger and / or the other body part of the gesture speaker and / or several other body parts of the gesture speaker, reacts to this detection by a mechanical process, a. wherein the gesture recognition device determines at least a distance of the gesture input object and / or the hand and / or finger and / or other body part of the gesture speaker relative to a moving object and b. wherein the gesture recognition device additionally determines at least one further parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of the gesture speaker and / or several other body parts of the gesture speaker simultaneously, c. wherein the measurement of at least one distance and one further parameter is carried out by compensating measurement of the transit time and the amplitude. [4] Method according to one or more of the preceding claims, a. where the movable object is a door and / or a car door. [5] Method according to one or more of the preceding claims, a. wherein only movements within the specified distance interval and / or within a predetermined angular segment and / or within a specified solid angle and / or within a specified spatial area, the position of which is determined by the device performing the mechanical process, are used for gesture recognition. [6] Method according to one or more of the preceding claims, a. where the further parameter is the reflectivity of the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker and / or the several other body parts of the gesture speaker and / or another object. [7] Method according to one or more of the preceding claims, a. wherein the further parameter is the reflectivity of the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker and / or the several other body parts of the gesture speaker and / or another object in a spectral range, wherein the spectral range in which the measurement is taken lies between a lower and an upper wavelength. [8] Method according to one or more of claims 2 to 7, a. wherein an optical transmitter (H) is fed with a first band-limited transmit signal (S5), radiates into the specified spatial area and b. wherein the light reflected by an object within this given spatial area is at least partially received by a receiver (D) and c. wherein a compensation transmitter (K), which is fed by a compensation signal (S3), also radiates into the receiver (D) and d. where the compensation signal (S3) is complementary to the transmit signal S5 and e. wherein the light from the compensation transmitter (K) and the transmitter (H) overlaps in the receiver (D) and f. wherein the receiver (D) generates a receiver output signal (S0) which depends on the superimposed received optical signals of the transmitter (H) and the compensation transmitter (K) and g. wherein a controller (CT) adjusts the transmit signal (S5) and / or the compensation signal (S3) in amplitude and phase and / or in amplitude and delay such that the receiver output signal (S0) contains no correlating components of the spectrum of the transmit signal (S5) except for a control error and system noise and h. wherein the controller (CT) outputs at least two control signals as measured values ​​for the distance and a further parameter according to claim 1. [9] Method according to claim 8, a. wherein an evaluation unit, which is part of the gesture recognition device, evaluates at least two signals for the recognition of the gesture and / or gesture sequence. [10] Method according to claim 8 or 9, a. wherein the controller (CT) adjusts the transmit signal (S5) and / or the compensation signal (S3) in amplitude and phase and / or in amplitude and delay such that the receiver output signal (S0) is not a DC signal. [11] Method according to claim 8 and / or 9, a. where the optical transmitter (H) is a laser. [12] Method according to one or more of claims 8 to 11, a. wherein the transmitting beam of the optical transmitter (H) is deflected by a micro-optical element, in particular a micromechanical mirror. [13] Method according to one or more of the preceding claims, a. where the distribution function of the measuring points, defined as measuring points per solid angle segment within the measuring range of the device, is not homogeneous. [14] Method according to one or more of the preceding claims, wherein the mechanical device to be operated comprises a car door or tailgate or other opening in a car or machine or object or building. [15] Method according to claim 11, wherein a car door or tailgate or other cover of an opening in a car or in a machine or in an object is opened on the outside side exclusively by gesture recognition of a sequence of at least one first and one second gesture and / or gesture sequences in combination with a further wirelessly electromagnetically and / or acoustically transmitted signal. [16] Method according to claim 15, wherein, prior to the execution of gesture recognition, a first gesture sequence to be recognized is transmitted to the gesture recognition device by a personalized and / or personalizable sender, in particular an electronic key. [17] Method according to one or more of the preceding claims, wherein the mechanical process is the unlocking and / or opening of an access opening, in particular a door and / or a car door. [18] Method according to one or more of the preceding claims, wherein the device transmits a feedback signal, in particular an optical and / or acoustic feedback, to a user. [19] Method according to claim 18, wherein the feedback signal with at least one of its feedback parameters provides feedback on a measured distance and / or the measured two-dimensional location and / or the measured three-dimensional location of the gesture input object and / or the hand and / or finger and / or a foot and / or the other body part of the gesture speaker and / or several other body parts of the gesture speaker. [20] Method according to claim 19, wherein the device provides feedback on the measured three-dimensional location by changing three parameters, in particular by changing the representation in x and y coordinates and by changing the size of a feedback symbol on a screen, wherein this screen may be curved and / or part of the outer shell of a car door and / or the cover of another lockable opening. [21] Device for measuring the location and / or position of objects, a. wherein the device can perform a method according to one or more of the preceding claims and b. wherein the device can also detect distances to objects other than the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker and / or the several other body parts of the gesture speaker through the gesture recognition device and / or detect positions of these other objects and / or c. wherein the device can also detect and signal the presence of objects other than the gesture input object and / or the hand and / or that finger and / or the other body part of the gesture speaker and / or the several other body parts of the gesture speaker in a predefined spatial area. [22] Device according to claim 21, wherein the device can also be used to detect objects in the blind spot of a car. [23] Device according to claim 21, wherein the device or parts thereof are mounted in a rearview mirror or a protruding part of a car body.

Images