Method for implementing automotive vehicle functions through ultrasound-based gesture control - Patent Application 20070122997

Abstract

The present invention relates to a method for performing a vehicle function (5) of a motor vehicle (1) through ultrasound-based gesture control, comprising: providing (S1) ultrasound information (10) describing at least one object in the vicinity of at least one ultrasound sensor (2); applying (S2) action identification criteria (11) to the provided ultrasound information (10) to identify action information (12) describing a sequence of actions of the at least one object; applying (S4) object determination criteria (15) to the provided ultrasound information (10) and the identified action information (12) to determine whether the object is a body part targeted for gesture control of the vehicle function (5); and performing (S6) the vehicle function (5) if the object is the body part targeted for gesture control.

Description

[Technical Field]

[0001] The present invention relates to a method for implementing vehicle functions in a motor vehicle by ultrasound-based gesture control, and further to a motor vehicle, a control device and a computer program product for implementing such a method. [Background technology]

[0002] The automobile may have vehicle functions that can be activated by detecting a gesture made by a user. For example, such a vehicle function may be provided to open or close the trunk of the automobile through gesture control. The vehicle function implemented through gesture control may be based on data from the electrocapacitive system. Alternatively, the gesture may be detected by at least one ultrasonic sensor in the automobile.

[0003] For this purpose, the motor vehicle is equipped with at least one ultrasonic sensor designed to detect at least one object in the surrounding environment of the motor vehicle. The distance between the ultrasonic sensor and the object in the surrounding environment can be determined by evaluating the ultrasonic information provided by the ultrasonic sensor. If the ultrasonic information is provided by multiple ultrasonic sensors, the object may be located further in the surrounding environment. The motor vehicle is typically equipped with multiple ultrasonic sensors, for example, installed in the rear area of ​​the motor vehicle. It is particularly intended that a parking assistance system may be provided in the motor vehicle.

[0004] DE 10 2020 116 716 A1 discloses a method for turning on ultrasound-based gesture control when a person approaches a vehicle. Ultrasonic signals are emitted by a subset of ultrasonic sensors, and at least one echo signal is received by the subset of ultrasonic sensors. Object classification is performed on the echo signals to detect a person. If a person is detected, ultrasound-based gesture control is turned on.

[0005] Furthermore, radar-based gesture-controlled vehicle functions are known. For example, in this context, US 2019 / 0302253 A1 describes a method in which radar data is received and a movement trajectory is determined from the radar data. It is then determined whether the movement trajectory corresponds to a human signature, where the human signature is associated with a specific control of the vehicle.

[0006] Ultrasound-based gesture control is known to have a relatively high false trigger rate, which may also be referred to as a false detection rate, when compared to, for example, radar-based solutions. Summary of the Invention

[0007] It is at least an object of the present invention to provide a solution with low false trigger rates for automotive vehicle functions implemented by ultrasound-based gesture control.

[0008] This object is achieved by the subject matter of the independent claims.

[0009] One aspect of the present invention relates to a method for implementing a vehicle function of a motor vehicle by ultrasound-based gesture control. The vehicle function is designed, for example, to open and close the trunk of the motor vehicle without contact, and thus by gesture control. Alternatively or additionally, the vehicle function may be designed to open and close the doors of the motor vehicle without contact. Alternatively or additionally, the function implemented by ultrasound-based gesture control may be located inside the motor vehicle, i.e., such function may be activated by a gesture performed inside the motor vehicle.

[0010] The method provides ultrasonic information describing at least one object in the vicinity of the at least one ultrasonic sensor. The at least one ultrasonic sensor is preferably located on the vehicle, for example on the rear and / or bumper of the vehicle. The vehicle preferably includes multiple ultrasonic sensors, for example, six ultrasonic sensors in the rear area of ​​the vehicle and / or six ultrasonic sensors in the front area of ​​the vehicle. Alternatively or additionally, the at least one ultrasonic sensor may be located inside the vehicle, for example on the roof area of ​​the vehicle.

[0011] The ultrasonic information may be provided, for example, by first transmitting an ultrasonic signal from at least one ultrasonic sensor, which reflects off an object in the surroundings, and then receiving the reflected ultrasonic signal as an echo from at least one ultrasonic sensor. The provided ultrasonic information describes the object in the surroundings, e.g., describes at least one distance to the object. Thus, the ultrasonic information may be detected by at least one ultrasonic sensor in the vehicle. The ultrasonic information may then be provided, for example, to a control device of the vehicle. The ultrasonic information is preferably described by or comprises ultrasonic data.

[0012] At least one object may be, for example, a user of a vehicle function. The object may in particular be a body part of the user, such as a foot, leg, arm, and / or palm. The object may thus perform a gesture, thereby performing a function. Alternatively or additionally, the object may be a pedestrian, an animal, an object, and / or another road user located in the surroundings. The surroundings may be spatially defined, for example, by a detection area of ​​at least one ultrasonic sensor. The range of the ultrasonic sensor may be limited, for example, to 5 m. Thus, in this example, the boundary of the detection area extends at a distance of 5 m from the ultrasonic sensor.

[0013] The method identifies motion information describing a motion sequence, particularly a motion, of at least one object. The motion information is identified by applying motion identification criteria to the provided ultrasound information. Thus, the ultrasound information is evaluated to determine information about the motion sequence performed by the object. The motion information can describe, for example, how long it takes the object to move, whether there is at least one waiting period during which the object does not move, and / or what distance the object has from the ultrasound sensor during, before, and / or after the motion. The motion sequence may consist solely of the object's motion. The motion sequence is the ultrasound information that is ultimately evaluated. Therefore, the ultrasound information may also be referred to as raw data. And, the motion information may also be referred to as processed data of the respective ultrasound sensors.

[0014] The motion identification criterion is an algorithm and / or at least one rule, the application of which evaluates the provided ultrasound information and identifies and describes the motion sequence of the object. The motion identification criterion is based on machine learning techniques, such as an artificial neural network. The artificial neural network is trained to be able to conclude from the ultrasound information whether and how an object is currently moving around the ultrasound sensor. Thus, it is possible to distinguish at least between approaching, waiting, and performing a specific motion, such as a predetermined gesture. Preferably, various moving objects around the ultrasound sensor, such as a pedestrian passing by or standing in front of a vehicle, an animal, a rolling ball, and / or other road users, such as a bicycle or another vehicle, are taken into account in training the object determination criterion.

[0015] The method checks whether at least one object is a body part targeted for gesture control of a vehicle function. This is performed by applying object determination criteria to the provided ultrasound information and the identified motion information. The object determination criteria may be an algorithm and / or at least one rule based on which it may be determined whether an object is associated with a specific body part of a person. Possible body parts are, for example, the user's feet, legs, arms, and / or palms. In this example, it is sufficient to determine that the object is associated with a person, rather than, for example, an animal or an object.

[0016] If the object is a body part targeted for gesture control, a vehicle function is implemented, such as turning the vehicle function on or off. For example, if a body part intended for contactless trunk opening, for which motion information has been identified, such as a user's foot, is the object and ultrasound information is provided to the object and it is determined that the motion information has been identified, contactless opening or closing of the trunk is automatically performed. Finally, if the body part intended for gesture control is detected by at least one ultrasonic sensor and it is at least determined that the motion of the body part was performed in the detection area of ​​the ultrasonic sensor, the vehicle function is implemented by ultrasound-based gesture control.

[0017] Both the raw data, i.e., the provided ultrasound information, and the processed data, i.e., the identified motion information, are evaluated to determine whether the object is the expected body part. In this way, because the object is determined based on both the raw data and the processed data, the false trigger rate, i.e., the false detection rate, can be kept low. Therefore, vehicle functions for automobiles that are implemented by ultrasound-based gesture control can be provided with a low false trigger rate.

[0018] One advantageous exemplary embodiment provides that the object determination criterion is based on machine learning techniques, particularly on artificial neural networks and / or support vector machines. Thus, an artificial intelligence technique is utilized. The artificial neural network is, for example, a flat neural network, particularly a feedforward neural network. The artificial neural network can include multiple latent or hidden neurons and / or layers of such neurons. A support vector machine is an automated method for pattern recognition provided in the form of software. The object determination criterion is pre-trained to recognize from ultrasound and motion information that an object is located around the ultrasound sensor and that this object is a specific body part of a person. Finally, the object determination criterion reliably determines whether the recognized object is assigned to a specific object class describing a body part intended for gesture control of vehicle functions.

[0019] A further exemplary embodiment provides that user action information describing an expected sequence of body part movements performed by a user, in particular the expected body part movements performed by a user, is identified. The object determination criteria are retrained taking into account the identified user action information. Thus, it is possible to provide information about what actions an actual user of a vehicle will perform with their body parts, thereby providing information about which body parts provide which movements to perform vehicle functions. The pre-trained object determination criteria, in particular an artificial neural network, can be further trained in a learning process to, for example, recognize user-specific gestures, possibly using user-specific selected body parts. Since the user's action sequence is taken into account, it is also possible to take into account, for example, typical approach and waiting times for this user before and after the user's action. In this way, the detection threshold of the object determination criteria is finely set. In this example, it is assumed that the object determination criteria are generally pre-trained to recognize body parts as objects, and are only refined when details about the movement patterns of the user's body parts used in the retraining exist, for example, due to the user action information.

[0020] This allows the object determination criteria to be adapted to the user by the described optimization step. In this way, the object determination criteria can always be very reliably recognized as being a body part intended for the vehicle function, since the object determination criteria has been specifically trained to recognize the body part of the user of the vehicle function. In this way, the false trigger rate, such as the false trunk opening rate, can be further improved.

[0021] Furthermore, exemplary embodiments provide that user motion information is identified by applying motion identification criteria to additional ultrasound information. The additional ultrasound information is detected during a motion sequence, particularly during the execution of a motion, of a body part by the user. For this purpose, at least one identical ultrasonic sensor is used, for example. This provides ultrasound information for performing a vehicle function. Evaluating the additional ultrasound information to identify motion information can be performed similarly to the process described above, i.e., by providing motion identification criteria to the provided additional ultrasound information. Finally, since similar data describing a motion sequence, particularly at least a motion, of at least one object detected with respect to the user's body part is provided to the object determination criteria, actual information about the user's motion is present. In this way, the described retraining becomes highly rational.

[0022] Furthermore, in one exemplary embodiment, it may be provided that additional ultrasound information is further taken into account in the retraining. Thus, in the retraining of the object determination criteria, the determined user motion information and the additional ultrasound information based on which the user motion information is determined may be provided. Thus, the retraining may be performed based on both raw data and processed data. By evaluating the additional ultrasound information, for example, the shape and / or size of the user's body part may be determined and taken into account. In this way, various information obtained from, for example, echo signals detected by the ultrasound sensor is taken into account in the retraining.

[0023] Further exemplary embodiments provide that feature recognition criteria are applied to the provided ultrasound information to identify feature information describing at least one feature of the echo signals received by at least one ultrasound sensor. Object determination criteria are then applied to the identified feature information. It may therefore be provided that the object determination criteria are applied to the feature information in addition to, or in place of, the provided ultrasound information. The feature information describes previously evaluated features that can be obtained from the ultrasound information. Thus, the feature information is, for example, a pre-evaluated version of the raw data. The feature information may, for example, describe the amplitude of the echo signal, particularly in the range of its maxima, and / or the width of the maxima. The feature recognition criteria are algorithms and / or rules, the application of which may retrieve and quantify at least one feature from the raw data provided by the ultrasound sensor, for example, based on mathematical evaluation and / or machine learning techniques. The echo signal features may also be referred to as echo signal features. In this way, the object determination criteria can be simplified in that it is no longer necessary to evaluate the curve course provided as ultrasound information, but rather the feature information is provided, allowing direct access to the features of the ultrasound information. In this way, for example, the computational effort of applying the object determination criteria may be reduced.

[0024] According to a further exemplary embodiment, the feature information is provided to describe at least one of the following features: One possible feature is the amplitude of the global maximum and / or at least one local maximum. The amplitude is determined, for example, in the form of a voltage detected by the ultrasonic sensor. The amplitude may also be referred to as a maximum value. In this case, it is assumed, for example, that the echo signal has multiple maxima. Thus, for example, a main maximum and its surrounding lateral maxima may be distinguished. The main maximum may be a global maximum or a local maximum. Additionally or alternatively, the feature information may describe the width of each maximum, the gradient in the area of ​​each maximum, and / or the distribution of, for example, the surrounding and / or neighboring local maxima of the global maximum and / or other local maxima. Alternatively or additionally, the shape of each maximum may be described as a feature. From the described features, it is possible, for example, to at least partially infer what type of object is located in front of the ultrasonic sensor, and in particular, what type of object is moving in front of it. In other words, the reflectivity of the object, i.e., characteristic values ​​of the reflection pattern, such as, for example, the amplitude and / or the width and / or shape of the maximum, can be provided and taken into account. This information is obtained from the ultrasound information as its features. If no feature information is provided and the object determination criterion is only applied to the ultrasound information and the motion information, this information is also present, but it must first be evaluated, for example, in the context of the application of the object determination criterion. Finally, various types of features of the echo signal can be evaluated if they can be inferred from the provided ultrasound information.

[0025] Furthermore, one exemplary embodiment provides that the determined motion information describes at least one of the following motion characteristics: the approach of an object relative to the at least one ultrasonic sensor, a first waiting time after the approach of the object, a motion of the approaching object, a second waiting time after the motion of the approaching object, and / or a distance of the object after the motion and / or after the second waiting time. Accordingly, multiple phases or stages described by the motion information may be distinguished. These phases are included in a motion sequence. The motion of the approaching object may be, for example, a specific gesture by the object and thus a specific motion pattern. The motion of the approaching object may be, for example, a kicking motion with the user's foot. Alternatively or additionally, the motion may be, for example, a circular motion, a semicircular motion, a polygonal motion, an ascent followed by a descent, and / or another type of motion pattern.

[0026] Additionally or alternatively, the action characteristic may be the change in distance of an approaching object. Thus, for example, the distance covered by an object approaching the ultrasonic sensor can be detected. In this way, it can be concluded, for example, whether a person is approaching the ultrasonic sensor, i.e., whether they are walking towards it. Alternatively or additionally, the action characteristic may be the duration of the respective waiting time. For example, if an object waits in front of the ultrasonic sensor without any further change in distance from the ultrasonic sensor before or after performing an action, this may indicate, for example, that a gesture control is desired, which the user is initially preparing, and / or that, after gesture control, the user is waiting until a vehicle function is performed. Here, a maximum waiting time may be preset, with which the duration of each detected waiting time is compared. For example, if the waiting time is shorter than the maximum waiting time, it can be concluded, for example, that the object is permanently located around the ultrasonic sensor and gesture control is not performed. Alternatively or additionally, the action characteristic may describe the duration of the approaching object's action. For example, a maximum duration for which gesture control can continue may be provided. For example, if a motion continues for longer than a predetermined motion duration, it can be concluded that an intentional gesture for gesture control was not made, but rather that an object moved for another reason, for example, in the detection area of ​​an ultrasonic sensor. The maximum duration and / or maximum wait time can be stored in the object determination criteria, e.g., can be stored therein.

[0027] Alternatively or additionally, the motion feature may describe the position of the object at the start of the motion and / or at the end of the motion. The position of the object at the start of the motion can be referred to as, for example, the start position, and the position of the object at the end of the motion can be referred to as the end position. In particular, the change in position between the start position and the end position, i.e., the change in position between the start position and the end position, can be considered as a motion feature. In this way, it can be determined whether the object returns to the start position of the motion after the motion. From this, it can be concluded that the motion envisioned by the gesture control, such as a kicking motion, was actually performed and that the object did not simply change its position by, for example, passing a car. For this purpose, for example, it is necessary to store the position of the object at the start of the motion and the position at the end of the motion, respectively, so that the described position change can be determined.

[0028] Alternatively or additionally, the motion characteristics may describe the direction of the approaching object's motion, i.e., whether it is moving further toward, for example, an ultrasonic sensor. In particular, to evaluate the object's direction of motion and its position at the start and / or end for each of the described motion characteristics, it is necessary to evaluate data from multiple ultrasonic sensors, and thus, for example, multiple provided ultrasonic information items. Data from multiple ultrasonic sensors is preferably always compared with each other. This allows not only the object's distance or distance from each ultrasonic sensor to be determined, but also the object's location around the sensor. Finally, a comprehensive analysis of the object's motion in the detection area of ​​at least one ultrasonic sensor can be performed. This provides more detailed information about the object's motion for the object determination criterion. This criterion can therefore reliably determine whether the moving object is actually a body part targeted for gesture control.

[0029] Furthermore, one exemplary embodiment provides that the provided ultrasonic information is constantly detected by multiple ultrasonic sensors spatially spaced apart from one another. These sensors are preferably located on the bumper in the rear area of ​​the vehicle, and thus around the trunk. For example, two, four, six, or eight individual ultrasonic sensors may be arranged in the rear area. This allows the entire rear area of ​​the vehicle to be monitored by ultrasonic sensors. Similarly, ultrasonic sensors may be provided on the bumper in the front area of ​​the vehicle. Alternatively or additionally, at least one ultrasonic sensor may be arranged on each door of the vehicle. In this way, reliable monitoring of the surroundings of the vehicle related to each function is guaranteed.

[0030] It is also provided that in one exemplary embodiment, the object is a user's foot. Therefore, the described method is intentionally designed for, for example, a foot kick or other type of movement in the detection area of ​​an ultrasonic sensor in a car, for contactlessly opening or closing a trunk. Alternatively or additionally, the object may be a user's leg, arm, and / or palm. Depending on the desired object, different placements of the ultrasonic sensor may be envisioned, such that the ultrasonic sensor is located in a typical area where each of the user's body parts typically moves.

[0031] In a further exemplary embodiment, if the object is a body part targeted for gesture control, a movement gesture of the object is identified. A vehicle function is performed according to the identified movement gesture. The movement gesture is identified by evaluating at least ultrasound information and / or movement information. In identifying the movement gesture, feature information is preferably taken into account in addition to or instead of ultrasound information. The movement gesture can be described, for example, by movement gesture information identified upon evaluation of ultrasound information, movement information, and / or feature information. The identified movement gesture can describe, for example, a specific movement of the user's foot as the object, such as a kicking movement, or one of the other movement patterns described above. The movement gesture is provided, for example, in the form of a identified movement trajectory of the object. Thus, not only can it be detected whether the body part targeted for gesture control has moved, but the specific movement gesture performed by this body part is also detected and evaluated.

[0032] In this context, it can be recognized that an object, such as a user's leg or foot, in front of the vehicle has moved relative to the rest of the body. This can be recognized, for example, because the position of the maxima in the echo signal changes over time in the direction of decreasing distance to the ultrasonic sensor, while the remaining maxima as stationary components of the echo signal remain at a constant distance. This allows the conclusion that only part of the object, and thus the body part, has intentionally moved towards the vehicle. Finally, the operation of vehicle functions can be reliably performed in this way.

[0033] The method steps of the method are in particular performed by a control unit of a motor vehicle, to which the ultrasound information is provided, and which can then perform a vehicle function, for example in terms of switching on a trunk opening unit.

[0034] The interval between the object and the at least one ultrasonic sensor and the distance between the object and the at least one ultrasonic sensor should be understood to be synonymous. Therefore, the above-mentioned interval change may also be referred to as a distance change.

[0035] A further aspect of the present invention relates to a vehicle. The vehicle is designed to implement the above-described method. The vehicle implements the method. Thus, the vehicle is designed to provide ultrasound information, identify motion information by applying motion identification criteria to the provided ultrasound information, determine whether the object is a body part targeted for gesture control of a vehicle function by applying object determination criteria to the provided ultrasound information and the identified motion information, and perform the vehicle function if the object is a body part targeted for gesture control. The vehicle may be, for example, a car, a truck, a bus, and / or a motorcycle.

[0036] One advantageous exemplary embodiment of the vehicle according to the present invention provides that the vehicle comprises at least one ultrasonic sensor in the rear area of ​​the vehicle. The vehicle function is designed to automatically open and close the trunk of the vehicle by ultrasound-based gesture control. In other words, the vehicle function is designed to open and close the trunk of the vehicle without contact. The vehicle preferably comprises a plurality of sensors in the rear area, which are arranged, for example, in the bumper. At least six ultrasonic sensors are preferably arranged therein, spatially spaced apart from one another.

[0037] A further aspect of the present invention relates to a control device for a motor vehicle. The control device is designed to implement the described method. The control device implements the described method, in particular an exemplary embodiment or a combination of exemplary embodiments of the method. The control device comprises a processor unit. The processor unit may comprise at least one microprocessor, and / or at least one microcontroller, and / or at least one FPGA (Field Programmable Gate Array), and / or at least one DSP (Digital Signal Processor). Furthermore, the processor unit may comprise program code, which may also be referred to as a computer program product. The program code may be stored in a data memory of the processor unit. A motor vehicle according to the present invention may comprise the described control device.

[0038] A further aspect of the invention relates to a computer program product, which is a computer program comprising instructions which, when the program is executed by a computer, e.g. by a controller, cause the computer to carry out the method according to the invention.

[0039] The exemplary embodiments described in conjunction with the method according to the invention apply, respectively individually and in combination with one another, to the motor vehicle according to the invention, to the control device according to the invention, and to the computer program product according to the invention, as appropriate. The invention includes combinations of the exemplary embodiments described. [Brief explanation of the drawings]

[0040] [Figure 1] FIG. 1 shows a schematic diagram of a car having multiple ultrasonic sensors. [Figure 2] FIG. 2 shows a signal flow graph schematic of a method for implementing vehicle functions in an automobile through ultrasound-based gesture control. [Figure 3] FIG. 3 shows a schematic signal flow graph of further method steps of the method of FIG. [Figure 4]FIG. 4 shows a schematic diagram of an echo signal course detected by an ultrasonic sensor. [Figure 5] FIG. 5 shows a schematic diagram of the movement curve of an object around an ultrasonic sensor. DETAILED DESCRIPTION OF THE INVENTION

[0041] 1 shows a vehicle 1 equipped with a plurality of ultrasonic sensors 2. Six ultrasonic sensors 2, arranged spatially spaced apart from one another, are exemplarily positioned in a rear area 3 of the vehicle 1 in this example. They are able to monitor the surroundings of the vehicle 1 in the rear area 3. The surroundings of the vehicle 1 are spatially defined by the boundaries of the respective detection areas of the respective ultrasonic sensors 2. Each of the detection areas has a range of, for example, up to 5 m.

[0042] The motor vehicle 1 is equipped with a control device 4 in which vehicle functions 5 are stored. The vehicle functions 5 can be implemented by the control device 4. For example, the vehicle functions 5 are configured to open and close a trunk 6 of the motor vehicle 1 contactlessly, i.e., to open and close the trunk 6 automatically. The vehicle functions 5 can be implemented by ultrasound-based gesture control. For this purpose, the measurement data of the ultrasonic sensors 2 in the rear area 3 are evaluated to recognize a gesture. Based on this recognition, the trunk 6 is opened automatically, for example. The evaluation of the measurement data of the individual ultrasonic sensors 2 and further steps for implementing the vehicle functions 5 are implemented by the control device 4 of the motor vehicle 1.

[0043] Here, a user 7 is positioned around the automobile 1. The user 7 makes a gesture with his / her foot 8 in the detection area of ​​the plurality of ultrasonic sensors 2 in the rear area 3, or in the detection area of ​​at least each of the plurality of ultrasonic sensors 2. This gesture may be, for example, a kicking motion with the foot 8 of the user 7.

[0044] 1 also depicts ultrasonic sensors 2 in a front area 9 of the vehicle 1. Furthermore, ultrasonic sensors 2 (not shown here) may be placed on the doors of the vehicle 1 so as to monitor each side area of ​​the vehicle 1. Thus, further vehicle functions 5 by ultrasound-based gesture control can be provided, for example automatic opening and closing of at least one door of the vehicle 1.

[0045] 2 shows method steps of a method for implementing vehicle functions 5 of a motor vehicle 1 by ultrasound-based gesture control. In method step S1, ultrasound information 10 is provided that describes at least one object in the vicinity of at least one of the ultrasonic sensors 2. The ultrasound information 10 preferably describes an object located in the vicinity of multiple ultrasonic sensors 2 in a rear area 3 of the motor vehicle 1. The object may be, for example, a user 7, in particular a foot 8 of the user 7. Before the ultrasound information 10 is provided, it may be detected by at least one of the ultrasonic sensors 2, preferably by multiple ultrasonic sensors 2. The ultrasound information 10 is then provided, for example, to a control device 4 of the motor vehicle 1.

[0046] In method step S2, motion identification criteria 11 are applied to the provided ultrasound information 10 to identify motion information 12. The motion information 12 describes a motion sequence of at least one object, for example a user 7 and its feet 8.

[0047] Furthermore, it may be provided that in method step S3, feature recognition criteria 13 are applied to the provided ultrasound information 10 to determine feature information 14. The feature information 14 describes at least one feature of an echo signal received by at least one ultrasound sensor 2. In this example, it is provided that at least one ultrasound sensor 2 emits an ultrasound pulse. The ultrasound pulse is reflected by objects in the surroundings. The reflected ultrasound pulse is received as an echo signal by the ultrasound sensor 2, in particular by several of the ultrasound sensors 2.

[0048] In method step S4, it is determined whether the object is a body part targeted for gesture control of the vehicle function 5. This is performed by applying object determination criteria 15 to the provided ultrasound information 10 and the identified motion information 12. Alternatively or in addition to the ultrasound information 10, the object determination criteria 15 may be applied to the identified feature information 14. By applying the object determination criteria 15, it may be determined whether the object is a foot 8 of the user 7 targeted for gesture control of the vehicle function 5, where the foot 8 is the intended body part.

[0049] The object determination criterion 15 is based, for example, on machine learning techniques, in particular artificial neural networks and / or support vector machines, and is pre-trained to at least recognize whether an object is a specific body part of the subject of the vehicle function 5. That is, the object determination criterion 15 can, for example, recognize that an object in the detection area is a user 7 moving their foot 8 for gesture control.

[0050] In method step S5, it may be provided that if the object is a body part targeted for gesture control, a movement gesture 16 of the object is identified. The movement gesture 16 is identified by evaluating the ultrasound information 10, the movement information 12, and / or the feature information 14. The movement gesture 16 is provided such that in method step S6, a vehicle function 5 is performed in accordance with the identified movement gesture 16. Regardless of the identification of the movement gesture 16, it is provided in method step S5 that the vehicle function 5 is performed if the object is a body part targeted for gesture control.

[0051] If in method step S4 it is determined that the object is not a body part that is the subject of gesture control of the vehicle function 5, method steps S1, and optionally S2 and S3, may be performed again.

[0052] 3 shows a schematic representation of the method steps for retraining the object determination criterion 15. In method step S7, at least one movement 29 (see reference number 29 in FIG. 5 ) of a body part that is the subject of gesture control of a vehicle function 5 can be performed by the user 7, which can then be detected by the at least one ultrasonic sensor 2. Preferably, a movement sequence of the user 7 is detected that includes phases without any body part movements in addition to the movement 29. In this way, further ultrasound information 17 is detected and provided. The further ultrasound information 17 describes the performance of the movement 29 of a body part of the user 7, for example a foot 8.

[0053] In method step S8, action identification criteria 11 can be applied to the further ultrasound information 17. In this way, user action information 18 is identified. The user action information 18 describes a body part action 29 performed by the user that is the subject of gesture control of the vehicle function 5.

[0054] In method step S9, the object determination criteria 15 may be retrained taking into account the identified user action information 18. In this case, further ultrasound information 17 may also be taken into account.

[0055] FIG. 4 shows a schematic representation of an example of an echo signal course 19, which may be described by ultrasound information 10. On the one hand, amplitude 20, e.g., as a voltage specification, and on the other hand, time 21, e.g., specified in seconds, are plotted on two axes. Feature information 14 may describe individual features of the echo signal course 19, such as the amplitude 20 of a global maximum 22 and / or at least one local maximum 23 of the echo signal course 19. Furthermore, the feature may describe the width 24 of each maximum, as shown here for the global maximum 22. Alternatively or additionally, feature information 14 may describe the gradient in the area of ​​each maximum 22, 23, in particular the distribution of the local maximums 23 around the global maximum 22 or another local maximum 23, and / or the morphology of each maximum 22, 23. Finally, feature information 14 describes specific details that can be inferred from the echo signal course 19.

[0056] FIG. 5 shows an exemplary motion curve 25 for an object. The motion curve 25 can be described by the motion information 12. The distance 26 between the object and each ultrasonic sensor 2 and the time 21 are plotted on the axes. It is clear that the motion curve 25, and therefore the motion information 12, can describe various phases: the object's approach 27 to at least one ultrasonic sensor 2; the motion 29 of the approaching object, e.g., a kicking motion; a first waiting time 28 after the object's approach 27; a second waiting time 30 after the motion 29 of the approaching object; and / or the object's distance 31 from the ultrasonic sensor 2 after the motion 29 and / or the second waiting time 30. Typical distance changes are shown for each of these phases. The identified motion information 12 can describe one of these phases, i.e., the course of the approach 27, the first waiting time 28, the motion 29, the second waiting time 30, and / or the distance 31, as a respective motion characteristic.

[0057] Alternatively or additionally, the motion information 12 may describe the change in distance of the object during approach 27, and thus the change in spacing, the duration of the respective waiting times 28, 30, the duration of the motion 29 of the approaching object, the position and / or the direction of the motion 29 of the approaching object. Alternatively or additionally, the position 32 of the object at the start of the motion 29 and / or the position 33 at the end of the motion may be considered as motion features. In this case, the position change between the position 32 at the start of the motion 29 and the position 33 at the end of the motion 29 is particularly observed.

[0058] Overall, the embodiment demonstrates feature-based foot recognition for an ultrasonic sensor system. It is important not only to recognize the foot 8 and identify a movement pattern based solely on the foot's distance, but also to intentionally extract various features from the ultrasound data, i.e., ultrasound information 10. These features can be, for example, features from the movement pattern, such as the change in distance during the foot's movement or the movement time (duration of the movement 29). These movement features can be included in the movement information 12. Also, features from the ultrasound signal itself can be considered. Typical features are the foot's reflection value (e.g., the amplitude 20 of the main maximum) or characteristic values ​​of the reflection pattern (e.g., the width 24 of the echo and thus of each maximum 22, 23). That is, the ultrasound information 10 itself can be evaluated and considered. Furthermore, features from multiple ultrasonic sensors 2, such as the direction of the foot's movement or the start and end positions 32, 33, can be calculated as a two-dimensional projection. This information is conventionally referred to as an expected movement feature.

[0059] The described features can be evaluated using routine machine learning techniques, i.e., by applying object determination criteria 15. Conventional approaches such as flat neural networks or support vector machines are envisioned for this purpose. Ultimately, a binary approach, in which a foot 8, i.e., a given body part, is identified as either present or absent, is sufficient. The performance of foot recognition can be significantly improved by evaluating various data-level features and classifying the data using machine learning. The false positive rate, i.e., the inadvertent opening of the trunk, is particularly important here. In particular, raw-data-level features, i.e., taking into account the ultrasound information 10 with correspondingly provided hardware, significantly improve detection and, in ideal cases, even object classification. This allows for object determination in method step S4.

[0060] A further advantage is that the described machine learning approach also makes it possible to adapt the system to the user 7 himself. Thus, with a pre-trained network, after a simple learning process, a fine-tuning of the detection threshold can be carried out again. The final customer, i.e., user 7, only needs to perform a few foot movements himself for this purpose. These can be recorded in an optimization step and can be retrained, as explained in the context of method steps S7-S9.

[0061] FIG. 5 shows an example of a typical process for opening a trunk using a foot 8. This process shows at least four steps (see references 27-30) that can be referred to as approach, wait, kick, and wait. Features from this sequence can be, for example, the duration of the kick, i.e., movement 29, as well as the kick amplitude 20 and thus the distance change, e.g., the position change described above. Features can also be, for example, the wait time, i.e., the duration of each wait time 28, 30, or the maximum distance change during the wait process. Features can also be extracted from the ultrasound signal itself, i.e., the ultrasound information 10 can be considered. For this purpose, FIG. 4 clearly shows local maxima 22, 23 that may originate from foot reflections. Various small echoes can be seen before and after the actual reflection. Not only the echo distribution but also the morphology of the main reflection can be evaluated and considered as features.

[0062] Features from these data can be used by a support vector machine or a simple feedforward neural network with few layers to classify foot movements. For example, a latent layer with multiple latent neurons can be provided. Typically, many classifications are used; that is, it is reasonable to use multiple latent neurons and incorporate multiple latent layers with selectively complete links. Using the above approach, the false positive rate can be significantly reduced. Therefore, this method allows for contactless opening of the trunk 6 using the ultrasonic sensor 2 already present in the automobile 1.

Claims

1. A method for implementing a vehicle function (5) of a motor vehicle (1) by ultrasound-based gesture control, comprising: - providing (S1) ultrasound information (10) describing at least one object in the vicinity of at least one ultrasound sensor (2); - identifying (S2) motion information (12) describing a motion sequence of said at least one object by applying motion identification criteria (11) to said provided ultrasound information (10); - determining (S4) whether the object is a body part targeted for the gesture control of the vehicle function (5) by applying object determination criteria (15) to the provided ultrasound information (10) and the identified motion information (12); - if the object is the body part targeted by the gesture control, perform the vehicle function (5) (S6); method.

2. the object determination criterion (15) is based on machine learning techniques, in particular artificial neural networks and / or support vector machines; The method of claim 1.

3. User action information (18) describing the intended sequence of body part actions to be performed by the user (7) is identified, and the object determination criteria (15) are retrained (S9) taking into account the identified user action information (18). The method of claim 2.

4. The user action information (18) is identified (S8) by applying the action identification criteria (11) to further ultrasound information (17), the further ultrasound information (17) being detected (S7) during the performance of the action sequence by the user (7); The method of claim 3.

5. The additional ultrasound information (17) is further taken into account in the retraining. The method of claim 4.

6. feature recognition criteria (13) are applied to the provided ultrasound information (10) to identify (S3) feature information (14) describing at least one feature of the echo signals received from the at least one ultrasound sensor (2), and the object determination criteria (15) are applied to the identified feature information (14); The method according to any one of claims 1 to 5.

7. The characteristic information (14) describes at least one of the following characteristics: the amplitude (20) of the global maximum (22) and / or of at least one local maximum (22), the width (24) of each of said maxima (22, 23), the gradient in the area of ​​each of the maxima (22, 23), the distribution of said local maxima (23) in particular around said global maxima (22) and / or other local maxima (23), and / or the form of each of said maxima (22, 23), The method of claim 6.

8. The identified operational information (12) describes at least one of the following operational characteristics: the proximity (27) of said object to said at least one ultrasonic sensor (2), a first waiting time (28) after said approach of said object, - the movement of said object approaching (29), a second waiting time (30) after said movement (29) of said approaching object, the spacing (31) of said objects after said action (29) and / or said second waiting time (30), - change in distance of said object during said approach (27), the duration of each of said waiting periods (28, 30), the duration of said movement (29) of said object approaching, the position (32, 33) of the object at the beginning of the movement (29) and / or at the end of the movement (29), in particular the change in position between the position (32) at the beginning and the position (33) at the end, and / or the direction of the movement (29) of the approaching object, The method according to any one of claims 1 to 7.

9. The provided ultrasonic information (10) is detected by a plurality of ultrasonic sensors (2) spatially spaced apart from one another. The method according to any one of claims 1 to 8.

10. The object is a foot (8) of a user (7). The method according to any one of claims 1 to 9.

11. If the object is the body part targeted for the gesture control, a movement gesture (16) of the object is identified (S5), and the vehicle function (5) is performed according to the identified movement gesture (16), the movement gesture (16) being identified by evaluating at least the ultrasound information (10) and / or the movement information (12). The method according to any one of claims 1 to 10.

12. A motor vehicle (1) designed to implement the method according to any one of claims 1 to 11.

13. The at least one ultrasonic sensor (2) is arranged in a rear area (3) of the vehicle (1), and the vehicle function (5) is designed to automatically open and close a trunk (6) of the vehicle (1) by the ultrasonic-based gesture control. A motor vehicle (1) according to claim 12.

14. A control device (4) for a motor vehicle (1), the control device (4) being designed to implement the method according to any one of claims 1 to 11.

15. A computer program product comprising instructions that, when said program is executed by a computer, cause it to perform the method of any one of claims 1 to 11.

Images