Method for operating a mechanical device

Abstract

The invention relates to a method for operating a mechanical device, in particular a car door, tailgate, or similar cover, using optical gesture recognition. The position, location, or structure of a gesture input object, such as a hand, finger, or other body part, is detected. After determining at least one distance and one further parameter of the gesture input object, a predetermined gesture is recognized, which was previously transmitted to the gesture recognition device via a personalized transmitter, such as an electronic key. The gesture is described by a hypothesis, the acceptance or rejection of which is determined by the device. After acceptance, the mechanical device is opened. This method enables intuitive and personalized operation.

Description

Introduction

[0001] The invention relates to a method for operating a mechanical device having a gesture recognition device. Said gesture recognition device is capable of recognizing the position and / or location and / or orientation of a gesture input object, in particular that of a hand and / or a finger and / or another body part of a gesture speaker. The structure of a gesture input object, i.e. also that of said hand and / or a finger and / or another body part of a gesture speaker, can also be recognized. Structure here is understood to mean, for example, an open hand as opposed to a closed hand. Such gestures are disclosed, for example, in document DE 10 2012 010 627 A1.After recognizing a predetermined gesture or a predetermined gesture sequence that was carried out by means of said gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker, the device reacts to this recognition by means of a mechanical process. Such a mechanical process can be, for example, the unlocking of a door (3, 4), the opening of a tailgate of a motor vehicle (5), the opening and / or closing of another closable opening or changing the closing state, for example the partial opening and / or closing of said opening.

[0002] Various optical methods for implementing such control of the closed state of an opening, in particular an opening in a motor vehicle (5), are already known in the prior art. Examples include EP 1 901 947 B1, DE 10 2008 029 223 A1, DE 10 2005 005 185 A1, and DE 102011 115 760 A1. However, these all have the disadvantage that, with regard to the reflectivity of the skin, they are not equally capable of recognizing different skin colors. Recognition difficulties can arise, particularly with dark skin types, which are very common in southern latitudes, since the reflectivity of such skin types is reduced.

[0003] DE 10 2011 115 760 A1 describes the automatic actuation of a locking element of a vehicle. According to the technical teaching of DE 10 2011 115 760 A1, environmental conditions in the surrounding area of ​​the vehicle and the movement of an object in the surrounding area of ​​the vehicle are detected. According to the technical teaching of DE 10 2011 115 760 A1, the detected movement is compared with a predetermined movement profile. If the detected movement corresponds to the predetermined movement profile, the locking element is automatically actuated according to the technical teaching of DE 10 2011 115 760 A1. According to the technical teaching of DE 10 2011 115 760 A1, the movement of the object in the surrounding area is detected and / or the detected movement is compared with the predetermined movement profile depending on the detected environmental conditions.

[0004] DE 10 2012 010 627 A1 discloses an object recognition system, particularly for detecting gesture parameters of a human-machine interface. According to DE 10 2012 010 627 A1, the technical teaching of DE 10 2012 010 627 A1 is characterized in that the system according to the technical teaching of DE 10 2012 010 627 A1 comprises a light transmitter, a light compensation transmitter, a light receiver for the light signals emitted by the light transmitter and the light compensation transmitter, a controller that generates the signals emitted by the light transmitter and the light compensation transmitter, compares them with a signal received by a receiver, and determines the comparison result, and a unit that performs the steps of feature extraction and emission calculation.

[0005] EP 2 631 674 A1 discloses a method and a sensor system for measuring the transmission characteristics of a first transmission path of a measurement system based on feedback compensation between a first transmitter and a receiver. In the system according to the technical teaching of DE 10 2012 010 627 A1, a compensation signal from a compensation transmitter is received superimposed in the receiver in addition to the transmitted transmission signal of the first transmitter of DE 10 2012 010 627 A1. According to the technical teaching of DE 10 2012 010 627 A1, a feed signal of the device of DE 10 2012 010 627 A1 for the first transmitter of DE 10 2012 010 627 A1 and a receiver output signal of the receiver of DE 10 2012 010 627 A1 each form a vector in a pre-Hilbert space.Using the method of DE 10 2012 010 627 A1, a Hilbert projection is performed between the receiver output signal of the receiver and the feed signal, so that a projection image signal is generated according to the technical teaching of DE 10 2012 010 627 A1. An output signal is formed from the projection image signal of DE 10 2012 010 627 A1. By at least partially inverting the output signal with the feed signal, a pre-signal is generated according to the technical teaching of DE 10 2012 010 627 A1. According to DE 10 2012 010 627 A1, the inverting is preferably performed by multiplication. A compensation signal for feeding the compensation transmitter is generated from the formed pre-signal according to the proposal of DE 10 2012 010 627 A1 in order to achieve a feedback control of the receiver output signal. Object of the invention

[0006] It is therefore the object of the invention to supplement gesture recognition with at least one additional security measure. This is achieved by a method according to claim 1. Description of the invention.

[0007] The object of the invention is achieved in particular in that the gesture recognition device simultaneously determines at least a distance of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker and a further parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of a gesture speaker. The additional parameter is preferably the reflectivity of this gesture input object and / or this hand and / or this finger and / or the other body part of a gesture speaker. Preferably, an object classification is carried out with the aid of the device according to the invention, which carries out the method according to the invention. In some cases, it is useful to limit the measurement of the reflectivity to a predetermined spectral range and to work with a broadband transmitter (H).It may be useful, for example, to limit the upper wavelength to 2000nm and / or 1000nm and / or 750nm and / or 600nm and / or 500nm and / or 400nm and / or 300nm and / or 200nm and / or a value between these values ​​and to limit the lower wavelength to 1000nm and / or 750nm and / or 600nm and / or 500nm and / or 400nm and / or 300nm and / or 200nm and / or 100nm and / or a value between these values, whereby the lower wavelength should always be shorter than the upper wavelength.

[0008] The measured parameters are preferably the amplitude of a reflected light beam and its travel time. This preferably determines the reflectivity of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker. The other parameter can then be used to determine the distance of the gesture input object from the sensor (1, 2).

[0009] If more than one such sensor system is used, the location along the great circle equidistant from the at least two sensors (1, 2) can be determined by triangulation in the area of ​​the sensitivity lobe, unless the location of the detected object is limited to a very narrow solid angle range by a directed measurement. In such a case, the term "measurement lobe" is used below, whereby the measurement lobe is limited to a predetermined solid angle range.

[0010] If more than two such sensor systems are used, the exact location can be determined by triangulation in the area of ​​the sensitivity lobes. The surfaces of all three distance spheres and the measurement lobes must overlap there.

[0011] This spatial coordinate may be subject to change within the context of a gesture. Such a gesture recognition device therefore not only determines the two- and / or three-dimensional location relative to the position of the sensors (1, 2), but also typically determines the simple and / or higher temporal derivatives, i.e., movement speeds and / or accelerations.

[0012] Furthermore, the reflectivity of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker can change within the context of a gesture. This can typically occur due to the structure of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker changing. For example, if the hand of the gesture speaker opens, its surface area increases and thus its reflectivity increases. This change in the reflectivity parameter can be just as much a part of a gesture as the reflectivity itself.

[0013] It is known that a suitable system for measuring the distance and the further parameter is possible, for example, according to a method according to EP 1 913 420 B1 or EP 2 631 674 A1.

[0014] The major advantage of such a gesture recognition device that implements the method according to the invention is that even disturbing parameters such as rain and / or fog can be reliably detected by the system. An exemplary evaluation is known from DE 10 2012 010 627 A1.

[0015] A particularly characteristic feature of this device is that it has at least one mechanical sub-device that can be controlled by such a gesture recognition device. Preferably, the device according to the invention operates a car door (3, 4) or tailgate or other opening in a car, a machine, or an object.

[0016] If the device is installed, for example, in a rear-view mirror (6, 7) housed in a car door (3, 4), it can monitor the area in front of an exemplary car door (3, 4). The sensitivity lobe can be adjusted, for example, by optical means and / or by data selection in an evaluation circuit such that this sensitivity lobe lies parallel to the exemplary car door (3, 4) at the height of the door handle (8, 9). It should be mentioned at this point that the measuring lobe results from the intersection of the set of points in space detected by the transmitting lobe and the set of points in space detected by the receiving lobe. The sensitivity range can be limited, for example, by a permitted light propagation time range along the measuring lobe. This defines a distance range that is permitted for gesture recognition.Instead of limiting gesture recognition with respect to distance, limitations with respect to the solid angle range and / or an angle and / or a solid angle and / or, more generally, with respect to a spatial area are also conceivable, whereby this limitation should only lie within the measurement range of the system itself. The spatial area defined in this way, in which gestures are recognized, can otherwise have a more or less arbitrary shape and structure. However, it is preferable to limit the shape parameters of this spatial area to as few as possible in order to limit the required computing power.

[0017] Typically, more than one sensor (1, 2) is used. In this case, the restriction can also be achieved by filtering the data. In this case, only data whose coordinates lie within a permissible point set of the measurement lobe are processed. If the measurement lobe can be swiveled, only data whose coordinates lie within the union of a permissible point set of the measurement lobes with a permissible orientation are processed.

[0018] Through data selection, a spatial area can now be defined that lies directly in front of the door handle (8, 9) or some other designated area in front of the door (3, 4). In extreme cases, the car no longer even needs to have a door handle (8, 9), which is an essential part of the invention.

[0019] It is advantageous if the user is informed of the status of gesture recognition by an acoustic and / or visual and / or mechanical feedback signal. For example, it is conceivable that a screen (10, 11), in particular a curved screen (10, 11), preferably one whose surface follows the curvature of the door surface, is located on the outside of the door (3, 4). This screen informs the user of the measured value interpreted by the device by displaying a feedback symbol. Such a feedback symbol should be modified in at least one, preferably two, feedback parameters. These feedback parameters should correlate with measured parameters, such as the three spatial coordinates of the gesture input object, i.e., the hand or finger, etc.Such feedback parameters can include size, color, brightness, sharpness, virtual three-dimensional orientation, shape (e.g., round<->square), etc. The possibilities are manifold. If the input object approaches a point, which, unlike the state of the art, does not have to be identical to the location of the sensor, one of these parameters of the feedback signal, such as its size, is changed. For example, it becomes larger when approaching a point on the door (3, 4) or smaller when moving away.

[0020] From an optical point of view, it is particularly advantageous in the case of a car door (3, 4) or a tailgate or any other opening in a car or a machine or an object if, on one side and typically on the outside, no directly mechanically or manually operated opening devices interrupt the surface of the object. In particular, from an artistic point of view, it is desirable that the door (3, 4) etc. no longer has a door handle (8, 9), at least on the outside, but can be opened from the relevant side exclusively via gesture recognition. In the case of a motor vehicle (5) or an apartment door, it is useful if gesture recognition is supplemented by at least one further security measure. In the simplest case, this would be another wireless electromagnetic and / or acoustically transmitted signal, transmitted, for example, by an electronic key.This electronic key can also be a gesture recognition object that is equipped, for example, with sensors for location and position in space and uses this data and / or its temporal derivatives, along with other data, for encryption and transmits it to the device according to the invention.

[0021] In order to open the door (3, 4) and / or access opening, in addition to verifying access authorization, it is useful to require a recognized first gesture sequence and / or gesture for unlocking and a second gesture sequence and / or gesture for opening the door (3, 4). This can be transmitted to the gesture recognition device via the aforementioned electronic key, i.e., via a personalized and / or customizable transmitter.

[0022] The gesture recognition device typically evaluates the recognized gesture and / or gesture sequences with a confidence level. This confidence level is used to decide whether to accept or reject a gesture or gesture sequence hypothesis. Only gestures or gesture sequences with a confidence level above a threshold are accepted for control. It can happen that gestures or gesture sequences are rejected even though they are correct. The frequency of such an erroneous rejection of a correct gesture or gesture sequence is referred to below as the false rejection rate. Conversely, incidents that are not correct gestures or gesture sequences can be accepted as correct. These are referred to below as the false acceptance rate.The false rejection rate and the false acceptance rate are critical parameters of the system, which are typically seen as conflicting requirements.

[0023] For optimization purposes, it is therefore useful, for example, to activate the gesture recognition device with a first gesture and to transmit a command sequence with a second and possibly subsequent gesture and / or gesture sequence, which then unlocks and opens the door (3, 4), for example. It is useful if the first threshold value for recognizing the first gesture and / or gesture sequence is higher in order to prevent too frequent misrecognition in the absence of the first gesture or gesture sequence, i.e. to accept a higher false rejection rate in favor of a lower false acceptance rate. The position of the threshold value of the first gesture and / or gesture sequence refers to a higher position than that of the second threshold value for recognizing the second gesture or gesture sequence. As a result, the recognition of the second gesture or gesture sequence has a lower false rejection rate and a higher false acceptance rate.

[0024] It is therefore a special feature of the invention if distances to objects other than the distance of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker can also be detected by the gesture recognition device and / or positions of these other objects. These are calculated and related to the coordinates of the gesture input object and / or the hand and / or this finger and / or the other body part of a gesture speaker. In some cases, it is expedient if even just the presence of such objects in a certain distance range and / or solid angle range and / or spatial region (12, 13) is detected and, if necessary, signaled to the user.

[0025] This property can then also be used to perform additional functions in addition to gesture recognition. If the device according to the invention is installed, for example, in a rearview mirror (6, 7), it can also be used to detect objects in the blind spot of a car. The preferred installation location of the device according to the invention or parts thereof in a rearview mirror (6, 7) or a similar protruding part of a car body is therefore an essential part of this disclosure.

[0026] The invention is explained in more detail below with reference to the accompanying drawings. Fig. 1 shows the positioning of the sensors (1, 2) on the doors (3, 4) of a motor vehicle (5) in the rearview mirrors (6, 7). Fig. 2 shows the same situation as in Fig. 1, but now without the door handles (8, 9). These are replaced by screens (10, 11) that provide the aforementioned feedback signal. Fig.3 shows the spatially limited areas (12,13) ​​of gesture recognition relative to the door (3, 4)

[0027] An optical transmitter (H) is fed with a first band-limited transmission signal (S5). This signal radiates into the specified spatial region (12, 13). An object in this spatial region (12, 13) reflects the thus radiated light. This reflected light is received by a receiver (D). A compensation transmitter (K), fed with a compensation signal (S3), also radiates directly and / or indirectly into the receiver (D). The compensation signal (S3) is typically complementary to the transmission signal (S3). This means that it compensates for a weaker irradiation of the reflected signal in the receiver (D) such that the total irradiation of the receiver (D) is always approximately constant, except for a control error and system noise. The light from the compensation transmitter (K) and the transmitter (H) therefore overlap in the receiver (D).The receiver (D) generates a receiver output signal (S0) correlated with the irradiation by the compensation transmitter (K) and transmitter (H). This output signal therefore depends on the superimposed received optical signals from the transmitter (H) and the compensation transmitter (K). A controller (CT) adjusts the amplitude and phase and / or amplitude and delay of the transmission signal (S5) and / or the compensation signal (S3) relative to one another such that the receiver output signal (S0) no longer contains any correlating components of the spectrum of the transmission signal (S5) except for a control error and system noise. It is particularly important to emphasize here that a controller (CT) adjusts the amplitude and phase and / or amplitude and delay of the transmission signal (S5) and / or the compensation signal (S3) solely in the manner described above. The receiver output signal (S0) does not necessarily have to be a DC signal. A laser is particularly suitable as an optical transmitter (H).In order to scan the space, it is useful for a micro-optical element, in particular a micromechanical mirror, to deflect the light beam from the transmitter (H). This is achieved primarily by scanning line by line. The resolution in the area of ​​a relevant object, for example, the hand performing a gesture, is increased by setting a smaller line spacing in this area. This is done in such a way that the distribution function of the measurement points within the device's measurement range is not homogeneous. Typically, after repositioning the light beam from the transmitter (H) to a new measurement point, a measurement is taken, followed by another repositioning of the light beam from the transmitter (H), followed by the next measurement at the next measurement point, and so on.Thus, discrete sampling occurs within the solid angle range of interest, whereby the distribution of the measurement points within the solid angle range of interest does not have to be homogeneous. Rather, a distribution function can be defined such that it specifies a certain density of measurement points per solid angle segment. This distribution function can depend on the objects and / or gestures already detected and their temporal and spatial simple and higher derivatives and integrals, as well as a possibly specified weighting function.

[0028] The controller (CT) outputs at least two control signals as measured values ​​for the distance and another parameter, typically the received amplitude. Since the micromechanical scanning device simultaneously has one or two spherical coordinates at the time of measurement, three-dimensional recognition can be performed in this way. An evaluation unit, which is part of the gesture recognition device, evaluates these at least two, preferably four signals to recognize a gesture and / or gesture sequence. Such a measurement method, in which a control loop adjusts the physical quantity to be measured, in this case the light intensity, at a sensor (1, 2), in this case the receiver (D), is referred to below as a compensating control measurement. In concrete terms, this is a compensating control measurement of the propagation time and the amplitude. List of reference symbols 1 sensor 2 sensors 3 Door especially car door 4 Door, especially car door 5 vehicles 6 rearview mirrors 7 rearview mirrors 8 Door handle 9 Door handle 10 screen 11 Screen 12 spatially limited area of ​​gesture recognition relative to the door 13 spatially limited area of ​​gesture recognition relative to the door CT controller (not used in the figures) D Receiver (not used in the figures) H Transmitter (not used in the figures) K Compensation transmitter (not used in the figures) S0 receiver output signal (not used in the figures) S3 compensation signal (not used in the figures) S5 Transmit signal (S5) (not used in the figures)

Claims

[1] A method of operating a mechanical device which is a car door or tailgate or other covering of an opening in a car or in a machine or in an object on the outside side, • which has an optical gesture recognition device that can recognize the position and / or location and / or orientation of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker or the structure of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker and / or several other body parts of a gesture speaker, and • which, after detecting at least one predetermined gesture performed by means of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker and / or several other body parts of a gesture speaker, reacts to this detection by at least one mechanical process, comprising the steps a. determining at least one distance of a gesture input object and / or a hand and / or a finger and / or another body part of a gesture speaker by the optical gesture recognition device of the device performing the mechanical operation; b. simultaneous determination of at least one additional parameter of this gesture input object and / or this hand and / or this finger and / or the other body part of a gesture speaker by the optical gesture recognition device; c. Transmitting a gesture to be recognized by a personalized and / or customizable transmitter, by means of a wireless electromagnetically transmitted signal from an electronic key as a gesture recognition object, to the optical gesture recognition device before performing the gesture recognition; d. Gesture recognition of the gesture with formation of a gesture hypothesis by the gesture recognition device; e. Acceptance or rejection of the gesture hypothesis by the gesture recognition device; f. Opening the mechanical device after acceptance of the gesture hypothesis by the gesture recognition device. [2] Method according to the preceding claim, a. whereby only gestures within a predetermined distance interval and / or a predetermined solid angle and / or within a predetermined angle and / or within a predetermined spatial area, the position of which is determined by the device that carries out the mechanical process, are used for gesture recognition.

Images