Method for operating vehicle function of motor vehicle by means of ultrasonic-based gesture control

Abstract

The invention relates to a method for operating a vehicle function (5) of a motor vehicle (1) by means of ultrasonic-based gesture control. The method comprises: providing (S1) ultrasonic information (10) describing at least one object around at least one ultrasonic sensor (2); determining (S2) motion information (12) describing a sequence of motions of the at least one object by applying a motion determination criterion (11) to the provided ultrasound information (10); evaluating (S4) whether the object is a body part intended for gesture control of the vehicle function (5) by applying object evaluation criteria (15) to the provided ultrasonic information (10) and the determined motion information (12); and operating (S6) the vehicle function (5) if the object is a body part intended for gesture control.

Description

Field of the Invention

[0001] The present invention relates to a method for operating a vehicle function of a motor vehicle by ultrasonic-based gesture control. Furthermore, the present invention relates to a motor vehicle, a control device, and a computer program product for performing such a method. Background Art

[0002] A motor vehicle can have a vehicle function that can be activated by a detected gesture performed by a user. For example, such a vehicle function can be provided for opening or closing the gesture control of the luggage compartment of the motor vehicle. The vehicle function operated by gesture control can be based on data of a capacitive system. Alternatively, a gesture can be detected by at least one ultrasonic sensor of the motor vehicle.

[0003] For this purpose, the motor vehicle includes at least one ultrasonic sensor, which is designed to detect at least one object around the motor vehicle. The distance between the ultrasonic sensor and the surrounding object can be determined by evaluating the ultrasonic information provided by the ultrasonic sensor. If the ultrasonic information is provided by a plurality of ultrasonic sensors, the object can be additionally located in the surrounding environment. A motor vehicle typically includes a plurality of ultrasonic sensors, which are, for example, installed in the rear region of the motor vehicle, and these ultrasonic sensors are particularly used to provide a parking assistance system for the motor vehicle.

[0004] DE 10 2020 116 716 A1 discloses a method for activating ultrasonic-based gesture control when a person approaches a vehicle. Ultrasonic signals are emitted by a subset of ultrasonic sensors, and at least one echo signal is received by the subset of ultrasonic sensors. Therefore, object classification is performed in the echo signal to detect a person. When a person is positively detected, the ultrasonic-based gesture control is activated.

[0005] Furthermore, vehicle functions of radar-based gesture control are known. For example, in this case, US2019 / 0302253A1 describes a method in which radar data is received and a motion trajectory is determined from these radar data. It is thus determined whether the motion trajectory corresponds to a human feature, where the human feature is related to a specific control of the vehicle.

[0006] It is well known that in combination with ultrasonic-based gesture control, it has a relatively high incorrect triggering rate compared to, for example, radar-based solutions. The incorrect triggering rate can be referred to as the false positive rate. Summary of the Invention

[0007] At least one object of the present invention is to provide a solution by which a vehicle function with a low incorrect triggering rate is provided for a motor vehicle, and the vehicle function is operated by ultrasonic-based gesture control.

[0008] This object is achieved by the subject matter of the independent claims.

[0009] One aspect of the invention relates to a method for operating a vehicle function of a motor vehicle by ultrasound-based gesture control. For example, the vehicle function is designed to open or close the luggage compartment of the motor vehicle contactlessly and thus gesture-controlled. Alternatively or additionally, the vehicle function can be designed to open or close the doors of the motor vehicle contactlessly. Alternatively or additionally, the function operated by ultrasound-based gesture control can be located inside the motor vehicle, i.e., it can be activated by a gesture performed inside the motor vehicle.

[0010] The method includes providing ultrasound information. The ultrasound information describes at least one object around at least one ultrasound sensor. The at least one ultrasound sensor is preferably arranged in the motor vehicle, for example, behind and / or in the bumper of the motor vehicle. The motor vehicle preferably includes a plurality of ultrasound sensors, for example, six ultrasound sensors in the rear region of the motor vehicle and / or six ultrasound sensors in the front region of the motor vehicle. Alternatively or additionally, the at least one ultrasound sensor can be arranged inside the motor vehicle, for example, in the roof region of the motor vehicle.

[0011] The ultrasound information can be provided, for example, because initially at least one ultrasound sensor emits an ultrasound signal, which is reflected at an object in the environment, and then the at least one ultrasound sensor receives the reflected ultrasound signal as an echo. The provided ultrasound information describes the objects in the surroundings such that it describes, for example, at least one distance to the object. The ultrasound information can thus be detected by at least one ultrasound sensor of the motor vehicle. Thus, it can be provided, for example, to the control device of the motor vehicle. The ultrasound information is preferably described by or includes ultrasound data.

[0012] The at least one object is, for example, a user of the vehicle function. The object is in particular a body part of the user, such as a foot, a leg, an arm, and / or a hand. Thus, the object can perform a gesture by which the function is operated. Alternatively or additionally, the object can be a pedestrian, an animal, an article, and / or another road user located in the surroundings. The surroundings are spatially delimited, for example, by the detection area of at least one ultrasound sensor. The range of the ultrasound sensor can be limited, for example, to 5 m, such that in this example, the boundary of the detection area extends at a distance of 5 m from the ultrasound sensor.

[0013] The method includes determining motion information that describes a sequence of motions of at least one object, in particular the motion. The motion information is determined by applying a motion determination criterion to the provided ultrasonic information. The ultrasonic information is thus evaluated so as to present information about the sequence of motions performed by the object. The motion information can describe, for example, how long the object has moved, whether there has been at least one waiting time during which the object has not moved, and / or the distance of the object to the ultrasonic sensor during, before, and / or after the motion. The sequence of motions can include only the motion of the object. The motion information is ultimately evaluated as ultrasonic information. Thus, the ultrasonic information can alternatively be referred to as raw data, and the motion information can alternatively be referred to as processed data of the corresponding ultrasonic sensor.

[0014] The motion determination criterion is an algorithm and / or at least one rule, and when the algorithm and / or rule is applied, the provided ultrasonic information is evaluated so as to determine and describe the sequence of motions of the object. The motion determination criterion is based on, for example, machine learning methods, such as artificial neural networks. The artificial neural network has been trained, for example, to be able to infer from the ultrasonic information whether and how the object is currently moving around the ultrasonic sensor. Thus, it is possible, for example, to distinguish at least between approaching, waiting, and / or performing a specific movement (such as a predefined pose). Preferably, various types of moving objects around the ultrasonic sensor are considered in the training of the object evaluation criterion, such as pedestrians, animals, rolling balls, and / or another road user, such as a cyclist or another motor vehicle, running past a motor vehicle or standing in front of a motor vehicle.

[0015] The method includes checking whether at least one object is a body part intended for pose control of a vehicle function. This is performed by applying an object evaluation criterion to the provided ultrasonic information and the determined motion information. The object evaluation criterion is an algorithm and / or at least one rule, based on which it can be determined whether the object is related to a specific body part of a person. The provided body parts are, for example, the user's feet, legs, arms, and / or hands. In this case, it is sufficient to determine in the evaluation that the object is related to a person rather than, for example, an animal or an item.

[0016] If the object is a body part provided for pose control, the vehicle function is operated. For example, the vehicle function can thus be activated or deactivated. For example, if it is determined that the body part provided for contactless opening of the luggage compartment (such as the user's foot) is the object for which the ultrasonic information is provided and the motion information is determined, the contactless opening or closing of the luggage compartment can be performed automatically. Ultimately, if at least the body part provided for pose control is detected by at least one ultrasonic sensor and the motion of the body part is performed in the detection area of the ultrasonic sensor, the vehicle function is operated by ultrasonic-based pose control.

[0017] Both the raw data (i.e., the provided ultrasonic information) and the processed data (i.e., the determined motion information) are evaluated to assess whether the object is the provided body part. In this way, the incorrect triggering rate, i.e., the false positive rate, can be kept low because the object is evaluated based on both the raw data and the processed data. Thus, a vehicle function for a motor vehicle can be provided that operates by ultrasonic-based gesture control and has a low incorrect triggering rate.

[0018] An advantageous exemplary embodiment provides an object evaluation criterion based on machine learning methods. It is particularly based on artificial neural networks and / or support vector machines. Thus, methods using artificial intelligence are used. The artificial neural network is, for example, a flat neural network, in particular a feedforward neural network. The artificial neural network may include a plurality of hidden or concealed neurons and / or layers of these neurons. The support vector machine is an automatic pattern recognition method provided in software form. Starting from the ultrasonic information and the motion information, the object evaluation criterion is pre-trained to recognize that the object is located around the ultrasonic sensor and that the object is a specific body part of a person. Finally, it is reliably determined by the object evaluation criterion whether the recognized object is to be assigned to a predetermined object class, where the object class describes the body part provided for the gesture control of the vehicle function.

[0019] Another exemplary embodiment provides that user motion information is determined, which describes the motion sequence of the provided body part performed by the user, in particular the motion of the provided body part performed by the user. Taking into account the determined user motion information, the object evaluation criterion is re-trained. Thus, the actual user of the motor vehicle may perform a gesture with his body part and thus provide information about which body part provides which motion to operate the vehicle function. The already pre-trained object evaluation criterion, in particular the artificial neural network, can thus be trained by a further learning process to recognize, for example, user-specific gestures with a possibly user-specific selected body part. Since the motion sequence of the user is taken into account, for example, the typical approach and waiting times of the user before or after the body part motion can also be considered. Thus, a fine adjustment of the detection threshold of the object evaluation criterion is carried out. In this case, it is assumed that the object evaluation criterion was previously generally trained in recognizing the body part as an object, and now it is only improved, for example, with respect to the details of the motion pattern of the user's body part, which will be used for re-training and presented by the user motion information.

[0020] This enables the object evaluation criteria to adapt to the user through the described optimization steps. In this way, it is always possible to particularly reliably identify whether an object is a body part provided for a vehicle function, since the object evaluation criteria are specifically trained to identify the body parts of the user of the vehicle function. In this way, the incorrect triggering rate, for example, the incorrect opening rate of the luggage compartment, can be further improved.

[0021] Furthermore, an exemplary embodiment provides that user movement information is determined by applying movement determination criteria to additional ultrasonic information. During the execution of a movement sequence by the user, in particular during the movement of a body part, additional ultrasonic information is detected. For this purpose, for example, the same or even the same at least one ultrasonic sensor is used, through which ultrasonic information for operating a vehicle function is provided. The evaluation of the additional ultrasonic information to determine the movement information can be carried out analogously to the above process, i.e., by applying the movement determination criteria to the provided additional ultrasonic information. Finally, simulation data is provided to the object evaluation criteria, which describes the movement sequence of at least one object, in particular at least the movement, which is detected for the body part of the user, thereby presenting actual information about the user's movement. In this way, the described retraining becomes particularly reasonable.

[0022] Furthermore, in an exemplary embodiment, it can be provided that additional ultrasonic information is also considered in the retraining. Thus, in the retraining of the object evaluation criteria, the determined user movement information and the additional ultrasonic information can be provided to it, based on which the user movement information is determined. Thus, the retraining can also be carried out based on the original data and the processed data. By evaluating the additional ultrasonic information, for example, the shape and / or size of the user's body part can be determined and considered. In this way, various possible information is considered in the retraining, which can be obtained, for example, from the echo signals detected by the ultrasonic sensor.

[0023] Another exemplary embodiment provides that the characteristic information that describes at least one characteristic of the echo signal received by at least one ultrasonic sensor is determined by applying a characteristic recognition criterion to the provided ultrasonic information. An object evaluation criterion is then applied to the determined characteristic information. Thus, it can be provided that, in addition to the provided ultrasonic information, and in particular alternatively, the object evaluation criterion is applied to the characteristic information. The characteristic information describes the characteristics that have been evaluated and that can be obtained from the ultrasonic information. Thus, the characteristic information is, for example, a pre-evaluated version of the raw data. The characteristic information describes, for example, the amplitude of the echo signal, in particular in the range of the maximum value of the echo signal, and / or the width of the maximum value. The characteristic recognition criterion is an algorithm and / or a rule, and when applying this algorithm and / or rule, for example, based on a mathematical evaluation method and / or a machine learning method, at least one characteristic can be read out and quantified from the raw data provided by the ultrasonic sensor. The characteristic of the echo signal can alternatively be referred to as the feature of the echo signal. In this way, the object evaluation criterion can be simplified because it no longer has to evaluate, for example, the curve trend provided as ultrasonic information, but can directly access the characteristics of the ultrasonic information because the characteristic information is provided to it. In this way, for example, the computational effort for applying the object evaluation criterion can be reduced.

[0024] According to another exemplary embodiment, it is provided that the characteristic information describes at least one of the following characteristics: One possible characteristic is the amplitude of the global maximum and / or at least one local maximum. The amplitude is specified, for example, in the form of the voltage detected by the ultrasonic sensor. The amplitude can alternatively be referred to as the maximum value. For example, in this case, it is assumed that the echo signal includes a plurality of maximum values. Thus, for example, a distinction can be made between the main maximum and the lateral maximums around it. The main maximum can be a global maximum or a local maximum. Additionally or alternatively, the characteristic information can describe the width of the corresponding maximum value, the slope in the region of the corresponding maximum value, and / or the distribution of the local maximums, for example, around and / or adjacent to the global and / or another local maximum. Alternatively or additionally, the form of the corresponding maximum value can be described as a characteristic. It can be inferred at least partially from the mentioned characteristics what type of object is located in front of the ultrasonic sensor, in particular moving in front of it. In other words, the reflectivity of the object can be provided and considered, that is, for example, the amplitude and / or the characteristic value of the reflection pattern, such as the width and / or the form of the maximum value. This information can be extracted from the ultrasonic information as its characteristic. If the characteristic information is not provided and only the object evaluation criterion is applied to the ultrasonic information and the motion information, this information also exists, but initially it must be evaluated, for example, in the case of applying the object evaluation criterion. Ultimately, if various types of characteristics of the echo signal can be inferred from the provided ultrasonic information, these characteristics can be evaluated.

[0025] In addition, an exemplary embodiment provides that the determined motion information describes at least one of the following motion characteristics: the approach of an object to at least one ultrasonic sensor, a first waiting time after the object approaches, the motion of the approaching object, a second waiting time after the motion of the approaching object, and / or the distance of the object after the motion and / or after the second waiting time. Thus, multiple phases or periods described by the motion information can be distinguished. These phases consist of a motion sequence. The motion of the approaching object is, for example, the manifestation of a specific posture and thus a specific motion pattern of the object. The motion of the approaching object can be, for example, a kicking motion of a user's foot. Alternatively or additionally, the motion can be, for example, a circular motion, a semi-circular motion, a polygonal motion, a rising and subsequent falling, and / or another type of motion pattern.

[0026] The motion characteristic can be additionally or alternatively the change in the distance of the object during the approach. Thus, for example, the distance traveled by the object during the approach to the ultrasonic sensor can be detected. In this way, for example, it can be inferred whether a person is approaching the ultrasonic sensor, i.e., whether, for example, walking towards the ultrasonic sensor. Alternatively or additionally, the motion characteristic can be the duration of the corresponding waiting time. For example, if the object waits in front of the ultrasonic sensor before or after the executed motion without further change in the distance from the ultrasonic sensor, this can indicate a desired posture control, for example, the user initially prepares for this and / or waits thereafter until the vehicle function has been executed. In this case, a maximum waiting duration can be predetermined and the detected duration of the corresponding waiting time is compared with this maximum waiting duration. For example, if the waiting time is below the maximum waiting time, it can be inferred that the object is permanently located around the ultrasonic sensor and, for example, no posture control is performed. Alternatively or additionally, the motion characteristic can describe the duration of the motion of the approaching object. For example, a maximum duration for which the posture control can last can be provided. For example, if the movement lasts longer than a predetermined movement duration, it can be concluded that no intentional posture for the posture control has occurred, but rather, for example, the object is moving in the detection area of the ultrasonic sensor for another reason. The maximum duration and / or the maximum waiting time can be stored in the object evaluation criteria, i.e., for example, stored therein.

[0027] Alternatively or additionally, the motion characteristics may describe the position of the object at the start and / or end of the motion. The position of the object at the start of the motion may be referred to as, for example, the starting position, while the position at the end of the motion is referred to as the ending position. In particular, the change in position between the starting position and the ending position may be regarded as a motion characteristic, i.e., the change in position between the starting position and the ending position. In this way, it can be determined whether the object returns to its starting position after the motion, from which it can be inferred that the motion provided for postural control (e.g., a kicking motion) has actually occurred, and the object has not merely changed its position, for example, by walking past a motor vehicle. For this purpose, it is necessary, for example, to store the position of the object at the start of the motion and at the end of the motion, so that the described change in position can be determined.

[0028] Alternatively or additionally, the motion characteristics may describe the direction of motion of an approaching object, i.e., for example, whether it is moving further towards the ultrasonic sensor. In particular, for each of the described motion characteristics, it is necessary to evaluate the data from multiple ultrasonic sensors, and thus evaluate, for example, multiple provided items of ultrasonic information, such as evaluating the direction of motion and the position of the object at the start and / or end of the motion. The data from multiple ultrasonic sensors are preferably always compared with each other, because this can not only determine the spacing or distance of the object from the respective ultrasonic sensor, but also perform the positioning of the object around the sensor. Ultimately, a comprehensive analysis of the motion of the object in the detection area of at least one ultrasonic sensor can be carried out, so as to provide more detailed information about the object motion for the object evaluation criteria. Thus, the criteria can reliably determine whether the object that has been moved can actually be a body part intended for postural control.

[0029] Furthermore, an exemplary embodiment provides that the provided ultrasonic information is always detected by a plurality of ultrasonic sensors spatially separated from each other. These sensors are preferably located in the bumper in the rear area of the motor vehicle and thus around the luggage compartment. For example, two, four, six, or eight individual ultrasonic sensors may be arranged in the rear area, so that the entire rear area of the vehicle can be monitored by means of the ultrasonic sensors. Similarly, ultrasonic sensors may be provided in the bumper in the front area of the vehicle. Alternatively or additionally, at least one ultrasonic sensor may be arranged in the respective door of the motor vehicle. In this way, it is ensured that the environment of the motor vehicle related to the respective function can be reliably monitored.

[0030] Furthermore, in an exemplary embodiment, it is stipulated that the object is the user's foot. Thus, the described method is deliberately designed, for example, for detecting kicking movements or other types of movements of the foot in the detection area of an ultrasonic sensor of a motor vehicle, such as for opening or closing the luggage compartment contactlessly. Alternatively or additionally, the object can be the user's leg, arm, and / or hand. Depending on the desired object, different arrangements of the ultrasonic sensors can be provided such that they are located in typical areas where the corresponding body part of the user usually moves.

[0031] In another exemplary embodiment, it is stipulated that if the object is a body part for posture control, the movement posture of the object is determined. The vehicle function is operated according to the determined movement posture. The movement posture is determined at least by evaluating ultrasonic information and / or movement information. Preferably, when determining the movement posture, feature information is also considered in addition to or as an alternative to ultrasonic information. The movement posture can be described, for example, by movement posture information, which is determined based on the evaluation of ultrasonic information, movement information, and / or feature information. The determined movement posture can describe, for example, a specific movement of the user's foot as the object, such as a kicking movement or one of the other movement patterns mentioned above. For example, the movement posture is provided in the form of a determined movement trajectory of the object. Thus, it is possible not only to detect whether the body part for posture control moves, but also to detect and evaluate the specific movement posture performed by this body part.

[0032] In this case, it can be recognized that, for example, a part of the object arranged in front of the motor vehicle, such as the user's leg or foot, moves relative to the rest of their body. For example, this can be recognized as the position of the maximum value in the echo signal changing in the direction of a shorter distance to the ultrasonic sensor over time, while the remaining maximum values as the fixed components of the echo signal remain at a constant distance, and thus it can be concluded in this way that only a part of the object and thus the body part is deliberately moved towards the motor vehicle. The operation of the vehicle function can ultimately be carried out reliably in this way.

[0033] The method steps of the method are particularly executed by means of a control device of the motor vehicle. The ultrasonic information is provided to the control device, and then it can operate the vehicle function, for example, because it activates the opening unit of the luggage compartment.

[0034] The distance and the spacing between the object and at least one ultrasonic sensor should be understood synonymously. Thus, the above-mentioned change in spacing can also be alternatively referred to as a change in distance.

[0035] Another aspect of the present invention relates to a motor vehicle. The motor vehicle is designed to perform the above method. The motor vehicle performs the method. The motor vehicle is thus designed to provide ultrasonic information, determine motion information by applying a motion determination criterion to the provided ultrasonic information, evaluate whether an object is a body part for posture control intended for a vehicle function by applying an object evaluation criterion to the provided ultrasonic information and the determined motion information, and operate the vehicle function if the object is a body part for posture control. The motor vehicle is, for example, a passenger car, a truck, a bus, and / or a motorcycle.

[0036] According to an advantageous exemplary embodiment of the motor vehicle according to the present invention, the motor vehicle includes at least one ultrasonic sensor in a rear region of the motor vehicle. The vehicle function is designed to automatically open and / or close a luggage compartment of the motor vehicle by ultrasonic-based posture control. In other words, the vehicle function is designed to open or close the luggage compartment of the motor vehicle without contact. The motor vehicle preferably includes a plurality of ultrasonic sensors in the rear region, which are arranged there, for example, in the bumper. Preferably, at least six ultrasonic sensors are arranged there spatially separated from each other.

[0037] Another aspect of the present invention relates to a control device for a motor vehicle. The control device is designed to perform the described method. The control device performs the described method, in particular an exemplary embodiment or combination of exemplary embodiments of the method. The control device includes a processor unit. The processor unit may include at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (field programmable gate array) and / or at least one DSP (digital signal processor). In addition, the processor unit may include program code, which may alternatively be referred to as a computer program product. The program code may be stored in a data memory of the processor unit. The motor vehicle according to the present invention may include the described control device.

[0038] Another aspect of the present invention relates to a computer program product. The computer program product is a computer program. The computer program product includes commands that, when the program is executed by a computer, for example, by the control device, cause the computer to perform the steps of the method according to the present invention.

[0039] The exemplary embodiments described in connection with the method according to the present invention, each individually and in combination with each other, correspondingly apply to the motor vehicle according to the present invention, the control device according to the present invention, and the computer program product according to the present invention. The present invention includes combinations of the described exemplary embodiments. Description of the Drawings

[0040] In the drawings:

[0041] Figure 1 Shows a schematic view of a motor vehicle having a plurality of ultrasonic sensors;

[0042] Figure 2 Shows a schematic view of a signal flow diagram of a method for operating a vehicle function of a motor vehicle by ultrasonic-based posture control;

[0043] Figure 3 Shows according to Figure 2 A schematic representation of a signal flow diagram of further method steps of the method;

[0044] Figure 4 Shows a schematic view of an echo signal process detected by an ultrasonic sensor; and

[0045] Figure 5 Shows a schematic view of a motion curve of an object around an ultrasonic sensor. Detailed Description

[0046] Figure 1 Shows a motor vehicle 1 including a plurality of ultrasonic sensors 2. Here, six ultrasonic sensors 2 spatially separated from each other are located, for example, in a rear region 3 of the motor vehicle 1. These can monitor the surroundings of the motor vehicle 1 in the rear region 3. The surroundings of the motor vehicle 1 are spatially bounded by the boundaries of the respective detection regions of the respective ultrasonic sensors 2, each detection region having a range of, for example, up to 5 m.

[0047] The motor vehicle 1 includes a control device 4 in which a vehicle function 5 is stored. The vehicle function 5 can be executed by the control device 4. It is configured to, for example, open or close the luggage compartment 6 of the motor vehicle 1 without contact, that is, automatically open and close the luggage compartment 6. The vehicle function 5 can be operated by ultrasonic-based posture control. For this purpose, the measurement data of the ultrasonic sensors 2 in the rear region 3 are evaluated to identify a posture, based on the identification of which, for example, the luggage compartment 6 is automatically opened. The evaluation of the measurement data of the individual ultrasonic sensors 2 and other steps for providing the vehicle function 5 are performed by the control device 4 of the motor vehicle 1.

[0048] Here, a user 7 performing a posture with a foot 8 is located around the motor vehicle 1 in the detection region of the ultrasonic sensor 2 or at least the individual ultrasonic sensors 2 in the rear region 3. The posture can be, for example, a kicking motion of the foot 8 of the user 7.

[0049] Furthermore, in Figure 1In it, an ultrasonic sensor 2 is drawn in the front region 9 of the motor vehicle 1. In addition, ultrasonic sensors 2 (not shown here) can be arranged in the doors of the motor vehicle 1 in order to monitor the respective side regions of the motor vehicle 1. Thus, additional vehicle functions 5, such as automatically opening or closing at least one door of the motor vehicle 1, can be provided by ultrasonic-based gesture control.

[0050] Figure 2 Method steps of a method for operating a vehicle function 5 of a motor vehicle 1 by ultrasonic-based gesture control are shown. In method step S1, ultrasonic information 10 is provided, which describes at least one object around at least one ultrasonic sensor 2. The ultrasonic information 10 preferably describes objects around a plurality of ultrasonic sensors 2 located in the rear region 3 of the motor vehicle 1. The object can be, for example, a user 7, in particular the feet 8 of the user 7. Before the ultrasonic information 10 is provided, the ultrasonic information 10 can be detected by at least one ultrasonic sensor 2, preferably by a plurality of ultrasonic sensors 2. Thus, it is provided, for example, to the control device 4 of the motor vehicle 1.

[0051] In method step S2, a motion determination criterion 11 is applied to the provided ultrasonic information 10 to determine motion information 12. The motion information 12 describes the motion sequence of at least one object, such as the user 7 and their feet 8.

[0052] Furthermore, it can be provided that in method step S3, a feature recognition criterion 13 is applied to the provided ultrasonic information 10 to determine feature information 14. The feature information 14 describes at least one feature of the echo signal received by at least one ultrasonic sensor 2. In this case, it is provided that at least one ultrasonic sensor 2 emits an ultrasonic pulse, which is reflected at an object in the surrounding environment, and the reflected ultrasonic pulse is received by the ultrasonic sensor 2 as an echo signal, in particular by a plurality of ultrasonic sensors 2.

[0053] In method step S4, it is evaluated whether the object is a body part for gesture control of the vehicle function 5. This is performed by applying an object evaluation criterion 15 to the provided ultrasonic information 10 and the determined motion information 12. Instead of or in addition to the ultrasonic information 10, it can be applied to the determined feature information 14. It can be determined by applying the object evaluation criterion 15 whether the object is the foot 8 of the user 7, which is intended for gesture control of the vehicle function 5. The foot 8 is then the expected body part.

[0054] The object evaluation criterion 15 is, for example, based on a machine learning method, in particular based on an artificial neural network and / or a support vector machine. The object evaluation criterion 15 is pre-trained to at least recognize whether the object is a specific body part for the vehicle function 5. That is, the object evaluation criterion 15 can recognize, for example, that the object in the detection area is the user 7, and the user 7 moves his foot 8 for posture control.

[0055] In method step S5, it can be stipulated that if the object is a body part for posture control, the movement posture 16 of the object is determined. The movement posture 16 is determined by evaluating the ultrasonic information 10, the movement information 12, and / or the feature information 14. Then the movement posture 16 is provided such that in method step S6, the vehicle function 5 is operated according to the determined movement posture 16. Independently of the determination of the movement posture 16, in method step S5 it is stipulated that if the object is a body part for posture control, the vehicle function 5 is operated.

[0056] If it is determined in method step S4 that the object is not a body part for posture control of the vehicle function 5, method step S1 and optionally S2 and S3 can be executed again.

[0057] Figure 3 The method steps for re-training the object evaluation criterion 15 are outlined. In method step S7, at least one movement 29 of the body part that is intended for posture control of the vehicle function 5 (see reference numeral 29 in Figure 5 the drawings) can be performed by the user 7 and this can be detected by at least one ultrasonic sensor 2. Preferably, the movement sequence of the user 7 is detected, which movement sequence includes the movement 29 but also includes phases in which the body part is not moving. In this way, additional ultrasonic information 17 is detected and provided. This describes the manifestation of the movement 29 of the body part (e.g., the foot 8) of the user 7.

[0058] In method step S8, the movement determination criterion 11 can be applied to the other ultrasonic information 17. In this way, the user movement information 18 is determined. The user movement information 18 describes the movement 29 of the body part that is performed by the user 7 and is intended for posture control of the vehicle function 5.

[0059] In method step S9, the determined user movement information 18 can be considered to re-train the object evaluation criterion 15. In this case, the additional ultrasonic information 17 can also be considered.

[0060] Figure 4An example of the echo signal path 19 is outlined. This can be described by the ultrasonic information 10. On the one hand, the amplitude 20, for example as a voltage specification, and on the other hand, the time 21, for example in seconds, are plotted on the two axes shown. The characteristic information 14 can describe the individual characteristics of the echo signal curve 19, such as the amplitude 20 of the global maximum 22 and / or at least one local maximum 23 of the echo signal curve 19. In addition, the characteristic can describe the width 24 of the corresponding maximum, where the global maximum 22 is shown here. Alternatively or additionally, the characteristic information 14 can describe the slope in the region of the corresponding maxima 22, 23, the distribution of the local maxima 23, especially around the global maximum 22 or another local maximum 23, and / or the form of the corresponding maxima 22, 23. Finally, the characteristic information 14 describes the specific details that can be inferred from the echo signal curve 19.

[0061] Figure 5 An exemplary motion curve 25 of an object is shown. The motion curve 25 can be described by the motion information 12. The distance 26 between the object and the corresponding ultrasonic sensor 2 and the time 21 are plotted on the axes. It is obvious that the motion curve 25 and the motion information 12 can describe different phases. These are the approach 27 of the object to at least one ultrasonic sensor 2, the first waiting time 28 after the approach 27 of the object, the motion 29 of the approaching object (such as a kicking motion), the second waiting time 30 after the motion 29 of the approaching object, and / or the distance 31 of the object from the ultrasonic sensor 2 after the motion 29 and / or the second waiting time 30. Typical distance changes are shown in these individual phases. The determined motion information 12 can describe one of these phases as the corresponding motion characteristic, namely the approach 27 path, the first waiting time 28, the motion 29, the second waiting time 30, and / or the distance 31.

[0062] Alternatively or additionally, the motion information 12 can describe the distance change and thus the spacing change of the object during the approach 27, the duration of the corresponding waiting times 28, 30, the duration of the motion 29 of the approaching object, the position of the approaching object, and / or the direction of the motion 29. Alternatively or additionally, the position 32 of the object at the start of the motion 29 and / or the position 33 at the end of the motion can be considered as motion characteristics. In this case, in particular, the position change between the position 32 at the start of the motion 29 and the position 33 at the end of the motion 29 is observed.

[0063] Generally speaking, the example shows feature-based foot recognition for an ultrasonic sensor system. It is important to not only recognize the foot 8 and determine the movement pattern based solely on the foot distance, but also to intentionally extract various features from the ultrasonic data, i.e., the ultrasonic information 10. These features can be, for example, those from the movement pattern, such as the distance change during foot movement or the movement time (the duration of movement 29). These movement features can be included in the movement information 12. Additionally, features from the ultrasonic signal itself can be considered. Then, typical features are the reflection values of the foot (e.g., the amplitude 20 of the main maximum) or the characteristic values of the reflection pattern (e.g., the width 24 of the echo and the corresponding maxima 22, 23). That is to say, the ultrasonic information 10 itself can be evaluated and considered. Furthermore, features from multiple ultrasonic sensors 2 can be calculated, such as the foot movement direction or position 32, 33 at the start time and end time, for example, as a two-dimensional projection. This information has previously been referred to as possible movement features.

[0064] The mentioned features can be evaluated by means of conventional methods of machine learning, i.e., by applying the object evaluation criterion 15. The possibilities for this purpose are a planar neural network or traditional methods, such as support vector machines. Ultimately, a binary method for determining the presence of the foot 8, i.e., the predetermined body part, is sufficient. By evaluating the features at various data levels and classifying the data through machine learning, the performance of foot recognition can be significantly improved. In particular, the false positive rate, i.e., accidentally opening the luggage compartment, is very important here. Specifically, due to the features at the raw data level, i.e., by considering the ultrasonic information 10, with the correspondingly provided hardware, the detection can be significantly improved, and in an ideal case, even the classification of the object can be performed, i.e., the object evaluation in method step S4 is possible.

[0065] Another advantage is that the system can also be adapted to the user 7 himself by the described machine learning method. With a pre-trained network, after a short learning process, the fine-tuning of the detection threshold can be performed again. The end customer, i.e., the user 7, only needs to perform a few foot movements himself for this purpose. As described in the context of method steps S7 to S9, these can then be recorded and re-trained in the optimization step.

[0066] In Figure 5An example of a typical process of opening a luggage compartment using the foot 8 can be seen. This process shows at least four steps, which can be called approach, wait, kick, and wait (see reference signs 27 to 30). The characteristics of this sequence are, for example, the duration of the kick, i.e., movement 29, and the amplitude 20 and thus the distance variation of the kick, such as the position variation described. In addition, the characteristics can be, for example, the waiting time, i.e., the duration of the respective waiting times 28, 30, or can also be the maximum distance variation during the waiting process. Furthermore, characteristics can be extracted from the ultrasonic signal itself, i.e., ultrasonic information 10 can be considered. For this purpose, in Figure 4 the maximum values 22, 23 can be clearly seen, which may originate from the foot reflection. Various smaller echoes can be seen before and after the actual reflection. The distribution of the echoes and the form of the main reflection can be evaluated and considered as characteristics.

[0067] The characteristics from these data can be used to classify the foot movement by a support vector machine or a simple feedforward neural network with several layers. For example, a hidden layer with a plurality of hidden neurons can be provided. Usually, many classifications are used, i.e., it is reasonable to use a plurality of hidden neurons and optionally combine a plurality of fully connected hidden layers. Using the described method can significantly reduce the false positive rate. Thus, by means of this method, the ultrasonic sensor 2 already included in the motor vehicle 1 can be used to open the luggage compartment 6 without contact.

Claims

1. A method for operating a vehicle function (5) of a motor vehicle (1) by ultrasonic-based posture control, comprising: - providing (S1) ultrasonic information (10) that describes at least one object around at least one ultrasonic sensor (2); - determining (S2) motion information (12) that describes a motion sequence of at least one object by applying a motion determination criterion (11) to the provided ultrasonic information (10); - evaluating (S4) whether the object is a body part intended for posture control of the vehicle function (5) by applying an object evaluation criterion (15) to the provided ultrasonic information (10) and the determined motion information (12); and - operating (S6) the vehicle function (5) if the object is a body part intended for posture control.

2. The method according to claim 1, wherein, The object evaluation criterion (15) is based on a machine learning method, in particular on an artificial neural network and / or a support vector machine.

3. The method according to claim 2, wherein Determining user motion information (18) that describes a motion sequence of an intended body part performed by a user (7), and retraining (S9) the object evaluation criterion (15) taking into account the determined user motion information (18).

4. The method according to claim 3, wherein, The user motion information (18) is determined (S8) by applying the motion determination criterion (11) to additional ultrasonic information (17) detected (S7) during the execution of the motion sequence by the user (7).

5. The method according to claim 4, wherein, The additional ultrasonic information (17) is also taken into account in the retraining.

6. The method according to any one of the preceding claims, wherein, Determining (S3) feature information (14) that describes at least one feature of an echo signal received from the at least one ultrasonic sensor (2) by applying a feature recognition criterion (13) to the provided ultrasonic information (10), and applying the object evaluation criterion (15) to the determined feature information (14).

7. The method according to claim 6, wherein, The feature information (14) describes at least one of the following features: - the amplitude (20) of a global maximum (22) and / or at least one local maximum (22); - the width (24) of a corresponding maximum (22, 23); - the slope in the region of a corresponding maximum (22, 23); - the distribution of local maxima (23), in particular the distribution around the global maximum (22) and / or another local maximum (23); and / or - the form of a corresponding maximum (22, 23).

8. The method according to any one of the preceding claims, wherein, The determined motion information (12) describes at least one of the following motion characteristics: - an object approaching (27) the at least one ultrasonic sensor (2); - a first waiting time (28) after the object approaches; - the motion (29) of the approaching object; - a second waiting time (30) after the motion (29) of the approaching object; - the spacing (31) of the object after the motion (29) and / or the second waiting time (30); - the distance change of the object during the approach (27); - the duration of a corresponding waiting time (28, 30); - the duration of the motion (29) of the approaching object; - The position (32, 33) of the object at the start and / or end of the movement (29), in particular the change in position between the position (32) at the start and the position (33) at the end; and / or - The direction of movement (29) of the approaching object.

9. The method according to any one of the preceding claims, wherein, The provided ultrasonic information (10) is detected by a plurality of ultrasonic sensors (2) that are spatially separated from each other.

10. The method according to any one of the preceding claims, wherein The object is the foot (8) of the user (7).

11. The method according to any one of the preceding claims, wherein, If the object is a body part intended for postural control, the movement posture (16) of the object is determined (S5), and the vehicle function (5) is operated based on the determined movement posture (16), wherein the movement posture (16) is determined at least by evaluating the ultrasonic information (10) and / or the movement information (12).

12. A motor vehicle (1) designed to perform the method according to any one of the preceding claims.

13. The motor vehicle (1) according to claim 12, wherein, The at least one ultrasonic sensor (2) is arranged in the rear region (3) of the motor vehicle (1), and the vehicle function (5) is designed to automatically open and / or close the luggage compartment (6) of the motor vehicle (1) by ultrasonic-based postural control.

14. A control device (4) for a motor vehicle (1), wherein, The control device (4) is designed to perform the method according to any one of claims 1 to 11.

15. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

Images