System and method for agile, intuitive control of vehicle functions

Abstract

System (100) for the agile, intuitive control of vehicle functions of a vehicle (110), comprising: a wearable device (120) that is configured to collect sensor data, wherein the sensor data includes motion data and optical data of a user of the wearable device (120); a computing unit (114, 124) that is set up, - to process the captured sensor data, wherein the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device (120); - to determine a functional relationship between the processed sensor data and the vehicle (110), wherein determining the functional relationship includes a time limitation for capturing the predefined movement and / or gesture, wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined time period; and - to determine an assignment to a predefinable vehicle function, taking into account the determined functional relationship; wherein the vehicle (110) includes a control unit (112) which is configured to control or regulate the predefinable vehicle function according to the determined assignment.

Description

[0001] The present invention relates to a system and a method for the agile, intuitive control of vehicle functions.

[0002] Comfort features related to the user's ability to operate vehicle functions are well-known. For example, so-called comfort access systems are known that allow a vehicle to be unlocked and / or started without actively using the vehicle key. This is usually achieved using a radio-based system to locate the key relative to the vehicle. Another example of a comfort vehicle feature is automatic or electric tailgate operation. This enables contactless opening and closing of the tailgate by using sensors that detect a predetermined movement on the vehicle and, upon detection, trigger the opening or closing of the tailgate. Additionally or alternatively, one or more control elements can be installed on the vehicle and / or the vehicle key, which, when activated, open or close the tailgate.Furthermore, it is known to enable additional vehicle functions, such as opening or closing the sunroof, activating or deactivating lighting elements, etc., using appropriate controls within the vehicle. Gesture control in the vehicle is another example of a comfort vehicle function. For this, an interior camera in the vehicle, for example, recognizes a predefined hand movement of a vehicle occupant. Depending on the recognized hand movement, a vehicle function can be triggered, which, in relative time, corresponds to one or more predefined vehicle functions, such as changing radio stations, accepting or rejecting an incoming call, adjusting the volume, etc. A disadvantage of this is that the aforementioned vehicle functions require the costly integration of the necessary sensors and / or controls within the vehicle.Retrofitting the aforementioned comfort features is also only possible with considerable effort, as the corresponding sensors and / or controls must be retrofitted into the vehicle. Built-in sensors have the additional disadvantage of requiring additional hardware in the vehicle, such as a control unit with a suitable housing and a corresponding processing unit, which leads to increased costs and, for example, environmental pollution during manufacturing and disposal. Furthermore, it necessitates energy consumption in the vehicle.

[0003] German patent application DE 10 2018 207 656 A1 discloses a system for positioning vehicle components comprising a mobile device, a vehicle, and a network server, as well as the methods performed by these system components for positioning vehicle components. The methods are characterized by the fact that body measurements of a user, captured by a smartphone and linked to a user ID, are stored in the memory of the network server and can be retrieved by multiple vehicles to determine the settings of the respective vehicle components.

[0004] Publication US 2017 / 0151918 A1 discloses a vehicle comprising a vehicle network and a wearable device that is communicatively connected to the vehicle network. The vehicle is configured to determine vehicle settings from data received by the wearable device and to derive corresponding user settings. User data is received from the wearable device on the vehicle, based on which the vehicle determines one or more user settings. Based on these determined user settings, one or more vehicle properties are automatically adjusted.

[0005] Publication DE 11 2015 003 882 T5 discloses a method for operating a vehicle, comprising: receiving a first input from a user, indicating a vehicle function to be performed on the vehicle, on a portable article; receiving a second input, indicating a gesture by the user for authentication and generating a control signal on the portable article to execute the vehicle function on the vehicle based on successful authentication of the user.

[0006] Publication DE 10 2016 215 251 A1 discloses a method for operating a navigation device of a motor vehicle comprising the steps of: providing a route of the motor vehicle described by a travel plan; receiving user-specific state data that correlates with a physical state of the user, and determining an index value from the user-specific state data, wherein the index value describes a comparative value of the user's physical well-being.A processing unit provides and operates a digital model that, based on the index value or multiple index values, predicts a future index value of the user depending on the course of the route and the environmental data, which describe at least one current property of the vehicle's environment and / or at least one expected property of a future environment of the vehicle along the route, and, depending on the predicted future index value, changes the travel plan and controls the navigation device according to the changed travel plan.

[0007] German patent application DE 10 2016 005 066 A1 discloses a method for determining and outputting situation-specific information relating to the physiological state of a vehicle occupant via an output device in a vehicle, in which data provided by the vehicle's own sensor devices are evaluated to determine the physiological state of the vehicle occupant. To provide such a method, in which the output information is particularly situation-specific and individually adapted to the respective vehicle occupant, it is provided that data provided by an electronic device carried by the vehicle occupant are also taken into account to determine the physiological state of the vehicle occupant.

[0008] Publication WO 2017 / 118 509 A1 discloses a method, a computer program, and devices for remotely controlling a vehicle using a mobile device. The method comprises determining a suggestion to trigger a function of the vehicle. The method further comprises outputting the suggestion via an output device. The method further comprises detecting a response from a user of the mobile device, wherein the response consists of an input via an input device or the absence of an input. The method further comprises transmitting a trigger signal for the function from the mobile device to the vehicle if the input is absent within a certain time period or if the input does not correspond to a cancellation of the suggestion.

[0009] The object of the invention is to provide a solution that enables improved, cost-effective, agile, and intuitive delivery of vehicle functions within the vehicle.

[0010] This problem is solved according to the invention by the features of the independent claims. Preferred embodiments are the subject of the dependent claims.

[0011] The aforementioned task is solved by a system for the agile, intuitive control of vehicle functions, comprising: a wearable device that is set up to collect sensor data, wherein the sensor data includes motion data and optical data of a user of the wearable device; a computing unit that is set up, - to process the captured sensor data, whereby the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device; - to determine a functional relationship between the processed sensor data and the vehicle, wherein determining the functional relationship includes a time limitation for capturing the predefined movement and / or gesture, and wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined time period; and - to determine an assignment to a predefinable vehicle function, taking into account the determined functional relationship; wherein the vehicle includes a control unit which is set up to control or regulate the predefinable vehicle function according to the determined assignment.

[0012] The system includes at least one wearable device. Within the scope of this document, the term "wearable device" or "wearable" or "portable computer system" refers in particular to modern wearable computer systems that are worn on the user's body during use, especially smartwatches, but also smart glasses, smart bands, etc., which have a variety of sensors capable of capturing movement and optical data from the wearer and transmitting it wirelessly via a communication unit – for example, via an air or radio interface such as Bluetooth Low Energy (BLE) or the mobile network.

[0013] The wearable device includes a sensor unit configured to collect sensor data. Specifically, this sensor data can include movement data of the wearer. To collect this movement data, the sensor unit can acquire data from one or more of the following sensors: - an accelerometer that determines acceleration by measuring an inertial force acting on a mass or test mass, so that it can determine the acceleration, an increase or decrease in velocity and / or a direction of movement of the wearable device; and / or - A position detection sensor or unit for acquiring or determining the geographic position or current position data using a navigation satellite system. The navigation satellite system can be any current or future global navigation satellite system (GNSS) for position determination and navigation by receiving signals from navigation satellites and / or pseudolites. Examples include the Global Positioning System (GPS), GLONASS, Galileo, and / or BeiDou Navigation Satellite System. In the case of GPS, the position detection sensor or unit can include a GPS module configured to determine the current GPS position data of the wearable device; and / or - a gyroscope, which is an acceleration and / or position sensor configured to detect minute accelerations, rotational movements, and / or changes in position of a mass or test mass. Gyroscope data can be combined with position data from a navigation module, whereby the combination of gyroscope and position data allows, for example, changes in direction to be determined very precisely; and / or - a magnetic field sensor configured to detect the current orientation or direction of movement of the wearable device; and / or - a proximity sensor to activate or deactivate the display of the wearable device; and / or - a UWB sensor system that sends a signal from the wearable device to several anchors or UWB receivers in the vehicle and is received at the vehicle at different times depending on the distance to the anchors, from which the processing unit can determine the position of the wearable device relative to the vehicle; and / or - at least one additional sensor that is set up to capture movement data from the wearable device.

[0014] The sensor data also includes optical data from the wearer of the wearable device. To acquire this optical data, the sensor unit can acquire data from an optical sensor. The optical sensor emits light, for example in the green wavelength range, into the tissue, such as on the wrist, and measures the reflected light. The reflected light intensity fluctuates with the pulsation of the blood vessels. This allows the wearer's heart rate to be determined. In other words, the sensor unit can include an optical sensor for acquiring heart rate data from the wearer. For example, a function is known in smartwatches that can distinguish between two hand gestures on the hand wearing the smartwatch. The first hand gesture involves bringing the thumb and forefinger together, and the second hand gesture involves making a fist.

[0015] The system comprises at least one vehicle. For the purposes of this document, the term "vehicle" encompasses mobile means of transport used for the transport of persons (passenger transport), goods (freight transport), or tools (machinery or equipment). In particular, the term "vehicle" includes motor vehicles as well as motor vehicles that may be at least partially electrically powered (electric cars, hybrid vehicles).

[0016] The vehicle can be controlled by a driver. Alternatively, or in addition, the vehicle can be at least partially automated. Within the context of this document, the terms "automated vehicle" and "automated driving" can refer to driving with automated longitudinal or lateral control, or autonomous driving with automated longitudinal and lateral control. Automated driving can, for example, involve extended periods of driving on the highway or limited-time driving during parking or maneuvering. The term "automated driving" encompasses automated driving at any level of automation. Examples of automation levels include assisted, partially automated, highly automated, and fully automated driving.These levels of automation were defined by the Federal Highway Research Institute (BASt) (see BASt publication "Research Compact", issue 11 / 2012). In assisted driving, the driver continuously performs longitudinal or lateral control, while the system takes over the other function within certain limits. In partially automated driving, the system takes over longitudinal and lateral control for a certain period and / or in specific situations, whereby the driver must continuously monitor the system, as with assisted driving. In highly automated driving, the system takes over longitudinal and lateral control for a certain period without the driver having to continuously monitor the system; however, the driver must be able to take over vehicle control within a certain timeframe.In fully automated driving, the system can automatically handle driving in all situations for a specific use case; no driver is required for this use case. The four automation levels mentioned above correspond to SAE Levels 1 to 4 of the SAE J3016 standard (SAE - Society of Automotive Engineering). Furthermore, SAE J3016 also includes SAE Level 5 as the highest level of automation, which is not included in the BASt definition. SAE Level 5 corresponds to driverless driving, in which the system can automatically handle all situations like a human driver throughout the entire journey.

[0017] The system includes at least one processing unit. This processing unit is configured to process the acquired sensor data. Processing the sensor data may include preprocessing the raw sensor data.

[0018] Furthermore, processing the sensor data can include determining a predefined movement of the wearable device user. This data is also referred to as movement data in this document. Examples of a predefined movement as movement data of the wearable device user include a step to the side, a jump, a wave, or any other predefined movement detectable by the sensors.

[0019] The determination of the predefinable or predefined movement of the user of the wearable device can be carried out using suitable machine learning algorithms, for example models created using machine learning methods - e.g. through supervised learning or unsupervised learning.

[0020] Furthermore, or alternatively, processing the sensor data can include determining predefined or predefinable gestures of the user or wearer of the wearable device. Determining these predefined gestures can be achieved using the aforementioned motion data, which can be captured by one or more of the aforementioned motion sensors, as well as the optical data from the optical sensor or optical heart rate sensor, with the aid of suitable machine learning algorithms. In particular, the aforementioned sensors can detect subtle differences in muscle movement and tendon activity of the wearer of the wearable device. Such a procedure is known from the prior art, as described above. For example, two different hand gestures can be analogous to Morse code.The Morse code alphabet, in which letters are formed by the appropriate sequence of short and long tone signals, can be used to represent any number of different vehicle functions through any combination of the aforementioned gestures "bringing thumb and forefinger together" and "making a fist." The described combinations with the movement gesture or functional reference, the spatial reference, the temporal reference, the location on the vehicle, and the temporal aspect offer an enormous range of possibilities.

[0021] Advantageously, this allows the user of the wearable device to perform a predefined or predefinable movement with minimal effort using subtle gestures - in the example of the smart watch - with the hand wearing the smart watch, for example by pressing the index finger and thumb together, by closing a fist, etc.

[0022] Determining the functional relationship involves a time limit for recording the movement and / or gesture and / or any combination thereof. The functional relationship requires that the gesture / movement / any combination thereof be completed within a predetermined time period; otherwise, no functional relationship is recognized. For example, it can be predefined that a gesture combination of "bringing thumb and forefinger together" combined with "making a fist" must be performed within 2 seconds to trigger the vehicle function "open trunk."

[0023] The predefined movement or gesture thus determined—that is, the determined functional relationship—can be assigned to a predefined vehicle function. Furthermore, any combination of a predefined movement or gesture can be assigned to a predefined vehicle function. For example, the vehicle user might be standing behind the vehicle. The vehicle user might be wearing a smartwatch on one hand. The vehicle user might raise this hand to greet a neighbor. This hand movement alone would not be assigned to any predefined vehicle function. However, if the vehicle user first performs a predefined hand gesture with this hand and then raises it, this combination of gesture and movement can be assigned to the vehicle function "open trunk."

[0024] Furthermore, the computing unit is designed to determine a functional relationship between the processed sensor data and the vehicle.

[0025] The functional relationship can arise in particular from one or more of the following conditions: - detected movement or sequence of movements of the user of the wearable device, as explained above; - detected gesture or sequence of gestures of the user of the wearable device, as explained above.

[0026] Furthermore, the processing unit is configured – upon detection of a functional relationship – to determine an assignment to a predefined vehicle function, taking the identified functional relationship into account. This can be done, for example, using an assignment stored in a suitable memory unit that can be continuously updated.

[0027] The predefined or predefinable vehicle function can include one or more of the following vehicle functions: A. Exterior area or external space of the vehicle: - Opening / closing a loading flap or fuel filler flap of the vehicle; - Unlocking a charging plug from a vehicle's charging socket; - Opening / closing the vehicle's tailgate; - Opening / closing any flap of the vehicle; - Opening / closing one or more automatic doors of the vehicle; - Controlling an entry aid for the vehicle; - Opening / closing a garage / entrance gate via a radio interface of the vehicle; - Opening / closing one or more windows of the vehicle; - Opening / closing one or more sunshades and / or shading systems of the vehicle; - Opening / closing a sliding / tilting sunroof of the vehicle; - Extending / retracting a trailer hitch on the vehicle; - Locking / unlocking or securing the vehicle; - Switching the vehicle's auxiliary heater and / or auxiliary ventilation and / or auxiliary cooling on / off; - Switching the vehicle's parking lights and / or sidelights and / or hazard warning lights on / off; - Activate / deactivate any lighting effect of the vehicle; - Triggering a camera recording (photo or video); - Activating / deactivating a sound of the vehicle; - Activating the vehicle's horn; - Adjustment of the vehicle's seating position and / or steering column with reference to a personalized setting of the wearer's device; - Controlling the vehicle's level control system; - Triggering a parking / unparking maneuver of the vehicle; - Controlling a garage parking system for automatic parking and unparking of the vehicle; - Command to stop the vehicle for garage parkers (automatic parking and exiting); - etc. B. Interior or interior of the vehicle: - Scrolling, swiping, selecting and / or sliding in the vehicle interior, for example as a replacement for the gesture camera in the vehicle interior; - Accept, reject, or cancel a telephone call; - Changing the volume of audio output in the vehicle; - Activating / deactivating a display on a display unit in the vehicle; C. Head-up display - Activating / deactivating a function of the head-up display; - Enabling / disabling content displayed via the head-up display; - etc. D. Status queries relating to the vehicle: - Querying the state of charge of an energy storage device, for example a battery of a vehicle that is at least partially electrically powered; - Querying the tire condition of at least one tire of the vehicle; - Querying the general condition of the vehicle; - Querying the status of the vehicle's central locking system; - Checking the status of the vehicle's anti-theft alarm system; - Checking the fuel level of a vehicle's tank; - Querying the range using the current fuel level and / or current charge status of the vehicle; - etc.

[0028] The aforementioned predefined or predefinable vehicle functions are listed only as examples; in principle, any vehicle functions can be controlled or regulated and any vehicle status can be retrieved using the procedure described in this document.

[0029] The vehicle includes a control unit. The control unit is configured to control or regulate the predefined vehicle function according to the determined assignment.

[0030] Advantageously, this allows for particularly agile, flexible, and intuitive control and / or regulation of a multitude of vehicle functions without the need for expensive sensors and / or controls in the vehicle. This enables improved usability of functions in and around the vehicle by utilizing the latest technologies in the field of wearable devices, which already provide the necessary sensors. Another significant advantage is that the control and regulation of vehicle functions can be hands-free, i.e., without requiring input from the wearable device user via the device itself. A gesture, such as a hand gesture from a smartwatch user, can trigger the aforementioned vehicle function(s) without requiring any interaction with the smartwatch's display. This is particularly convenient for the wearable device user. New vehicle functions can be added or modified as needed.They can be adapted or changed without requiring complex and expensive modifications or installations in the vehicle.

[0031] Preferably, the vehicle and the wearable device each comprise a communication unit, wherein the vehicle and the wearable device are configured to establish a Bluetooth Low Energy (BLE) connection with each other.

[0032] BLE is a wireless technology that enables communication between two devices. Compared to traditional Bluetooth, BLE has very low power consumption.

[0033] For example, the wearable device can already be configured as a digital key or digital vehicle key for the vehicle using state-of-the-art technology. Using BLE technology, the wearer of the device can therefore control or regulate the aforementioned vehicle function(s) hands-free by approaching the vehicle with a gesture, without any initial input or request for an initial connection between the vehicle and the device. At the same time, the security of the communication between the vehicle and the wearable device is ensured by the digital key security requirements.

[0034] For example, using a smartwatch as a wearable device, it is sufficient for the smartwatch user to perform a hand gesture within BLE communication range of the vehicle to trigger the aforementioned vehicle function(s). For instance, the user can make a fist with the hand on which the smartwatch is worn to unlock and / or lock the vehicle. Alternatively, bringing the thumb and forefinger of the hand wearing the smartwatch together can display the charge level of the vehicle's energy storage system on the smartwatch's screen.

[0035] Preferably, the processing unit is also configured to determine the spatial relationship of the processed sensor data to the vehicle, whereby the determined spatial relationship is taken into account when determining the assignment to the predefined vehicle function, taking into account the determined functional relationship. The spatial relationship can be established, for example, using ultra-wideband technology (UWB, see below).

[0036] Data transmission from the wearable device to the vehicle can be achieved, for example, using ultra-wideband (UWB) technology. This is a short-range radio communication technology that utilizes extremely large frequency ranges with a bandwidth of at least 500 MHz or at least 20% of the arithmetic mean of the lower and upper cutoff frequencies of the frequency band used. A key advantage of UWB technology is the ability to determine the wearable device's position relative to the vehicle with high precision. The data can be transmitted locally from the wearable device to the vehicle via a suitable radio interface, such as Bluetooth Low Energy (BLE) or the mobile network. This local reference is derived from the highly precise determination of the wearable device's position relative to the vehicle, or the position of the wearer of the wearable device.Depending on the system design, the position of the wearable device relative to the vehicle can be defined in zones. For example, a rear zone, a front zone, and side zones are conceivable on the exterior of the vehicle. Furthermore, determining the location is also possible by precisely positioning the wearable device relative to the vehicle, for example, 1 meter (m) in front of the driver's door or near the charging port. Determining the location within the vehicle interior is also possible, either by precisely positioning the device within the vehicle. Alternatively, the vehicle interior can be divided into zones, e.g., driver's seat, passenger seat, rear right area, etc. To determine the location, the wearable device can include a UWB sensor, which can be connected to multiple anchors or...The vehicle sends a signal to UWB receivers, which are received at different times depending on the distance to the respective anchors, from which the processing unit determines the position of the wearable device relative to the vehicle.

[0037] By incorporating the local reference, the agility and flexibility of controlling vehicle functions is further increased, since, for example, with a smartwatch as a wearable device, the wearer can trigger the opening of the driver's door while standing on the driver's side of the vehicle with a predefined hand gesture, the opening of the passenger door while standing on the passenger side of the vehicle, the opening of the trunk while standing in the rear of the vehicle, etc.

[0038] Preferably, the computing unit is also configured to determine a temporal relationship between the processed sensor data and the vehicle, whereby the determined temporal relationship is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional relationship.

[0039] The temporal reference can include, for example, determining the duration of a gesture by the wearer of the wearable device and / or a temporal parameter, such as a start and stop time. In the smartwatch example, the wearer can initiate the opening of a vehicle window with a hand gesture, with the process being canceled when the hand gesture is released. This further increases the agility and flexibility of controlling vehicle functions.

[0040] Furthermore, or alternatively, the temporal reference can also serve to achieve functional safety objectives. A functional safety objective may require that the correct functionality of a vehicle function be ensured along its entire chain of effects. In classic vehicle network communication, this can be an "alive" signal. The "alive" signal is a signal from the wearable device that cyclically changes its value to signal to the vehicle that the wearable device is still "alive." This allows the vehicle to ensure that a signal status, such as that of a hand gesture from the wearable device, is not "frozen" in the chain of effects.

[0041] Using the smartwatch as an example, the wearer can initiate an action, such as an automated parking maneuver, by making a fist. The functional safety objective might then require that, during the automated parking maneuver, the smartwatch wearer cyclically bring their thumb and forefinger together and then release them again to signal that the automated parking maneuver should continue. If this functional safety objective is not met, the automated parking maneuver would be terminated for safety reasons. By making a fist again, the smartwatch wearer can trigger the end of the action, in this case, the automated parking maneuver. Such an approach demonstrates a combinatorial control of vehicle functions.

[0042] Advantageously, this further increases the agility and flexibility of controlling vehicle functions, while simultaneously ensuring safety in controlling vehicle functions.

[0043] Preferably, the local reference also or alternatively includes the recognition of a direction of the movement data of the user of the wearable device, whereby the recognized direction is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional reference.

[0044] The aforementioned UWB technology can be used for this purpose. In addition to the highly precise determination of the wearable device's position relative to the vehicle, UWB technology also enables the determination of the direction of the wearable device's movement relative to the vehicle. Determining or recognizing the direction of the wearable device user's movement data can thus be achieved, for example, by employing UWB technology in a manner known from the prior art. Using the smartwatch as a wearable device, the wearer can, for instance, initiate the opening of the rear driver's side door window by making a hand gesture and pointing towards the rear door. With the same hand gesture and pointing towards the front driver's side door, the wearer can initiate the opening of the front driver's side door window.

[0045] This advantageously further increases the agility and flexibility in controlling vehicle functions.

[0046] The functional, temporal, and spatial (position and / or direction) approaches described above can be combined in any way. Furthermore, the gestures mentioned above for controlling vehicle functions can be combined with any movements of the wearer. For example, a wearer can point at the driver's door, perform a gesture, and then move upwards. This will initiate the closing of the driver's window. Using the same procedure—pointing at the driver's door, performing the same gesture—but with a downward movement, the opening of the driver's window can be initiated.

[0047] Furthermore, or alternatively, the functional, temporal, and / or spatial references can also be linked to other technologies as desired. For example, a link can be established with speech processing technology known from the prior art. A microphone can be used in the wearable device for this purpose. Thus, the aforementioned methods can be combined with voice commands to trigger vehicle functions. In the smartwatch example, the wearer can activate a speech processing module in the smartwatch by making a fist near the vehicle. Subsequently, the wearer can ask a question, such as "What is the vehicle's charge level?" and / or "How much longer will charging take?", whereupon the vehicle function can output the corresponding vehicle data.Furthermore, or alternatively, linking it to voice input allows for further detail regarding vehicle functions. In the example above, a gesture to open the windows combined with the voice command "open driver's window only" can trigger the opening of the driver's window, whereas with the gesture alone, all vehicle windows would be opened.

[0048] According to a second aspect, the underlying task is solved by a method for the agile, intuitive control of vehicle functions, encompassing: Acquisition of sensor data on a wearable device, wherein the sensor data includes motion data and optical data of a user of the wearable device; Processing, by means of a computing unit, the captured sensor data, wherein the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device; Determining, by the computing unit, a functional relationship between the processed sensor data and the vehicle, wherein the determination of the functional relationship includes a time limitation for capturing the predefined movement and / or the predefined gesture, wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined period of time; Determining, by means of the computing unit, an assignment to a predefinable vehicle function, taking into account the determined functional reference; Control or regulate, by a control unit of the vehicle, the determined predefinable vehicle function according to the determined assignment.

[0049] Preferably, the vehicle and the wearable device each comprise a communication unit, wherein the vehicle and the wearable device are configured to establish a Bluetooth Low Energy (BLE) connection with each other.

[0050] Preferably, the computing unit is also configured to determine a local reference of the processed sensor data to the vehicle, whereby the determined local reference is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional reference.

[0051] Preferably, the computing unit is also configured to determine a temporal relationship between the processed sensor data and the vehicle, whereby the determined temporal relationship is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional relationship.

[0052] Preferably, the local reference includes the recognition of a direction of the captured movement data of the user of the wearable device, wherein the recognized direction is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional reference.

[0053] These and other problems, features, and advantages of the present invention will become clear from studying the following detailed description of preferred embodiments and the accompanying figures. It is evident that—although embodiments are described separately—individual features can be combined to form additional embodiments. Fig. Figure 1 schematically shows a system for the agile, intuitive control of vehicle functions of a vehicle; Fig. Figure 2 shows an exemplary method for the agile, intuitive control of vehicle functions.

[0054] Fig. Figure 1 schematically shows a system 100 for the agile, intuitive control of vehicle functions of a vehicle 110.

[0055] System 100 comprises at least one wearable device 120. Within the scope of this document, the term wearable device, wearable, or portable computer system 120 encompasses, in particular, modern wearable computer systems that are worn on the body of the user of the wearable device 120 during use, especially smartwatches, but also smart glasses, smart bands, etc., which have a variety of sensors capable of capturing movement and optical data of the wearer of the wearable device 120 and transmitting this data wirelessly via a communication unit – for example, via an air or radio interface 130 such as Bluetooth Low Energy (BLE) and / or the mobile network.

[0056] The wearable device 120 includes a sensor unit 122, which is configured to collect sensor data. In particular, the sensor data can include movement data of the wearer of the wearable device 120. To collect the movement data, the sensor unit 122 can collect sensor data from one or more of the following sensors and thus include one or more of the following sensors: - an accelerometer that determines acceleration by measuring an inertial force acting on a mass or test mass, so that it can determine the acceleration, an increase or decrease in velocity and / or a direction of movement of the Wearable Device 120; and / or - a position detection sensor or position detection unit for acquiring or determining the geographic position or current position data using a navigation satellite system. The navigation satellite system can be any current or future global navigation satellite system (GNSS) for position determination and navigation by receiving signals from navigation satellites and / or pseudolites. Examples include the Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Galileo positioning system, and / or BeiDou Navigation Satellite System. In the case of GPS, the position detection sensor or position detection unit can include a GPS module configured to determine current GPS position data of the Wearable Device 120; and / or - a gyroscope, which is an acceleration and / or position sensor configured to detect minute accelerations, rotational movements, and / or changes in position of a mass or test mass. Gyroscope data can be combined with position data from a navigation module, whereby the combination of gyroscope and position data allows, for example, changes in direction to be determined very precisely; and / or - a magnetic field sensor configured to detect the current orientation or direction of movement of the Wearable Device 120; and / or - a proximity sensor for activating or deactivating the display of the wearable device 120; and / or - a UWB sensor 128 on the wearable device 120, which sends a signal from the wearable device 120 to several UWB anchors or UWB receivers 118 A ... 118 N in the vehicle 110 and is received at different times at the vehicle 110 depending on the distance to the anchors 118 A ... 118 N, from which the processing unit 114, 124 can determine the position of the wearable device 120 relative to the vehicle 110; and / or; - at least one additional sensor that is set up to capture movement data from the wearable device.

[0057] The sensor data also includes optical data from the wearer of the Wearable Device 120. To acquire the wearer's optical data, the sensor unit can acquire data from an optical sensor and thus include an optical sensor. The optical sensor emits light, for example in the green wavelength range, into the tissue, for example on the wrist, and measures the reflected light. The reflected light intensity fluctuates with the pulsation of the blood vessels. This allows, for example, the determination of the wearer's heart rate. In other words, the sensor unit 122 includes a heart rate sensor that can be configured to record the wearer's heart rate.

[0058] System 100 includes at least one vehicle 110. For the purposes of this document, the term vehicle 110 encompasses mobile means of transport used for the transport of persons (passenger transport), goods (freight transport), or tools (machinery or equipment). In particular, the term vehicle 110 includes motor vehicles as well as motor vehicles that may be at least partially electrically powered (electric cars, hybrid vehicles).

[0059] The system includes at least one computing unit 114, 124. This computing unit is configured to process the acquired sensor data. Processing the sensor data may include preprocessing the raw sensor data.

[0060] Furthermore, processing the sensor data can include determining a predefined movement of the user of the Wearable Device 120, or a predefined movement of the user's arm, with the Wearable Device 120 attached to the user's arm. This data is also referred to as movement data in this document. Examples of a predefined movement as movement data of the user of the Wearable Device 120 could be a step to the side, a jump, a sequence of jumps, or any other predefined movement detectable by the sensors.

[0061] The determination of the predefinable or predefined movement of the user of the Wearable Device 120 can be carried out using suitable machine learning algorithms, for example models created using machine learning methods - e.g. through supervised learning or unsupervised learning.

[0062] Furthermore, or alternatively, processing the sensor data can include determining predefined or predefinable gestures of the user or wearer of the Wearable Device 120. Determining these predefined gestures can be achieved using the aforementioned motion data, which can be captured by one or more of the aforementioned motion sensors, as well as the optical data from the optical sensor or optical heart rate sensor, with the aid of suitable machine learning algorithms. In particular, the aforementioned sensors can be used to detect subtle differences in muscle movement and tendon activity of the wearer of the Wearable Device.

[0063] Advantageously, this enables the user of the Wearable Device 120 to perform a predefined or predefinable movement with minimal effort using subtle gestures - in the example of the Smart Watch - of the hand wearing the Smart Watch, for example by pressing the index finger and thumb together, by closing a fist, etc.

[0064] The predefined movement or gesture thus determined can be assigned to a predefined vehicle function. Furthermore, any combination of a predefined movement or gesture can be assigned to a predefined vehicle function. For example, the user of vehicle 110 might be standing behind the vehicle. The user of vehicle 110 might be wearing a smartwatch 120 on one hand. The user of vehicle 110 might raise this hand to greet a neighbor. This hand movement would not be assigned to any predefined vehicle function. However, if the user of vehicle 110 performs a predefined hand gesture with this hand and then raises it, this combination of gesture and movement can be assigned to the vehicle function "open trunk."

[0065] Furthermore, the computing unit 114, 124 is set up to determine a functional relationship between the processed sensor data and the vehicle 110.

[0066] The functional relationship can arise in particular from one or more of the following conditions: - detected movement or sequence of movements of the user of the Wearable Device 120, as explained above; - detected gesture or sequence of gestures of the user of the Wearable Device 120, as explained above.

[0067] Furthermore, the processing unit 114, 124 is configured – upon detection of a functional relationship – to determine an assignment to a predefined vehicle function. This can be done, for example, using an assignment stored in a suitable memory unit that can be continuously updated.

[0068] The predefined or predefinable vehicle function can include one or more of the following vehicle functions: B. Exterior area or external space of vehicle 110: - Opening / closing a loading flap or a fuel filler flap of vehicle 110; - Unlocking a charging plug from a vehicle charging socket 110; - Opening / closing the tailgate of vehicle 110; - Opening / closing any flap of vehicle 110; - Opening / closing one or more automatic doors of vehicle 110; - Controlling an entry aid for vehicle 110; - Opening / closing a garage via a radio interface of the vehicle 110; - Opening / closing one or more windows of the vehicle 110; - Opening / closing one or more sunshades and / or shading systems of the vehicle 110; - Opening / closing a sliding / tilting sunroof of vehicle 110; - Extending / retracting a trailer hitch of vehicle 110; - Locking / unlocking or securing the vehicle 110; - Switching on / off a parking heater and / or parking ventilation and / or parking cooling of the vehicle 110; - Switching on / off of a parking light and / or sidelight and / or hazard warning light of the vehicle 110; - Activate / deactivate any lighting effect of the vehicle 110; - Triggering a camera recording (photo or video); - Activating / deactivating a sound or audio output of the vehicle 110; - Activating a fanfare on vehicle 110; - Adjustment of a seat position and / or steering column of the vehicle 110 with reference to a personalized setting of the wearer of the wearable device 120; - Controlling a level control system of the vehicle 110; - Triggering a parking / unparking maneuver of vehicle 110; - Control of a garage parking system for automatic parking and unparking of the vehicle 110; - Command to stop the vehicle 110 for garage parkers (automatic parking and exiting); - etc. B. Interior or interior of vehicle 110: - Scrolling, swiping, selecting and / or sliding in the vehicle interior, for example as a replacement for the gesture camera in the vehicle interior; - Accept, reject, or cancel a telephone call; - Changing the volume of audio output in the vehicle 110; - Activating / deactivating a display on a display unit in the vehicle 110; C. Head-up display of vehicle 110: - Activating / deactivating a function of the head-up display; - Enabling / disabling content that is displayed via the head-up display; - etc. D. Status queries relating to vehicle 110: - Querying the state of charge of an energy storage device, for example a battery of a vehicle that is at least partially electrically powered 110; - Querying the tire condition of at least one tire of vehicle 110; - Querying the general condition of vehicle 110; - Querying the status of the central locking system of vehicle 110; - Querying the status of the vehicle's anti-theft alarm system 110; - Querying the fuel level of a tank of vehicle 110; - Querying the range using the current fuel level and / or current charge status of the vehicle 110; - etc.

[0069] The aforementioned predefined or predefinable vehicle functions are listed only as examples; in principle, any vehicle functions can be controlled or regulated and / or any statuses of the vehicle 110 - as listed above - can be retrieved using the procedure described in this document.

[0070] The vehicle 110 includes a control unit 112. The control unit 112 is configured to control or regulate the predefined vehicle function according to the determined assignment.

[0071] Advantageously, this allows for particularly agile, flexible, and intuitive control and / or regulation of a multitude of vehicle functions without requiring expensive sensors and / or controls in the vehicle itself. This enables improved usability of functions in and around the vehicle by utilizing the latest technologies in the field of wearable devices, which already provide the necessary sensors. Another significant advantage is that the control and regulation of vehicle functions can be hands-free, i.e., without requiring input from the user of the wearable device. A gesture, such as a hand gesture from a smartwatch user acting as a wearable device, can trigger the aforementioned vehicle function(s) without requiring any interaction with the smartwatch's display. This is particularly convenient for the user of the wearable device.New vehicle functions can be added, adapted, or changed as desired without requiring complex and expensive modifications or installations in the vehicle 110.

[0072] Preferably, the vehicle 110 and the wearable device 120 each comprise a communication unit 116, 126, wherein the vehicle 110 and the wearable device 120 are configured to establish a Bluetooth Low Energy (BLE) connection with each other.

[0073] BLE is a wireless technology that enables communication between two devices. Compared to traditional Bluetooth, BLE has very low power consumption.

[0074] For example, the wearable device 120 can already be configured as a digital key or digital vehicle key for the vehicle 110 in a manner known from the prior art. Using BLE technology, the wearer of the wearable device 1220 can thus control or regulate the aforementioned vehicle function(s) hands-free by approaching the vehicle 110 with a gesture, without any initial input or request for an initial connection between the vehicle 110 and the wearable device 120, while simultaneously ensuring the security of the communication between the vehicle 110 and the wearable device 120 through the digital key security requirements.

[0075] For the example of a smartwatch as a wearable device 120, it is therefore sufficient for the smartwatch user to perform a hand gesture within BLE communication range of the vehicle 110 to trigger the aforementioned vehicle function(s). For example, the user can make a fist with the hand on which the smartwatch is worn to unlock and / or lock the vehicle 110. Alternatively, bringing the thumb and forefinger together with the hand wearing the smartwatch can display the charge level of an energy storage device in the vehicle 110 on the smartwatch's screen.

[0076] Preferably, the computing unit 114, 124 is also configured to determine a local reference of the processed sensor data to the vehicle 110, wherein the determined local reference is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional reference.

[0077] The transmission of data from the wearable device 120 to the vehicle 110 can be achieved, for example, using ultra-wideband (UWB) technology. This is a short-range radio communication technology that utilizes extremely large frequency ranges with a bandwidth of at least 500 MHz or at least 20% of the arithmetic mean of the lower and upper cutoff frequencies of the frequency band used. Advantageously, the use of UWB technology allows for highly precise determination of the positions of the wearable device 120 relative to the vehicle 110. The data can be transmitted locally from the wearable device 120 to the vehicle 110 via a suitable radio interface, such as Bluetooth Low Energy (BLE). The local reference can be derived from the highly precise determination of the position of the wearable device 120, or the wearer of the wearable device 120, relative to the vehicle 110.Depending on the system design, the position of the wearable device 120 relative to the vehicle 110 can be defined in zones. For example, a rear zone, a front zone, and side zones can be defined on the exterior of the vehicle 110. Furthermore, the spatial reference can also be determined by precisely positioning the wearable device 120 relative to the vehicle 110, for example, 1 meter (m) in front of the driver's door of the vehicle 110. Determining the spatial reference within the vehicle interior is also possible, either by precisely positioning the device within the vehicle interior or by dividing the vehicle interior into zones, such as the driver's seat, front passenger seat, rear right-hand area, etc.

[0078] By incorporating the local reference, the agility and flexibility of the control of vehicle functions is further increased, since in the example of the smartwatch as a wearable device 120, the wearer can, for example, trigger the opening of the driver's door with a predefined hand gesture while standing on the driver's side of the vehicle 110, the opening of the passenger door while standing on the passenger side of the vehicle 110, the opening of the trunk while standing in the rear area of ​​the vehicle 110, etc.

[0079] Preferably, the computing unit 114, 124 is also configured to determine a temporal relationship between the processed sensor data and the vehicle 110, whereby the determined temporal relationship is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional relationship.

[0080] The temporal reference can, for example, include determining the duration of a gesture by the wearer of the wearable device 120 and / or a temporal parameter, such as a start and stop time. In the smartwatch example, the wearer can initiate the opening of a vehicle window with a hand gesture, whereby the process is canceled when the hand gesture is released. Thus, the opening of the vehicle window is aborted when the hand gesture is released, and the window is only opened to a state desired by the smartwatch user. This further increases the agility and flexibility of controlling vehicle functions.

[0081] Furthermore, or alternatively, the temporal reference can also serve to achieve functional safety objectives. A functional safety objective may require that the correct functionality of a vehicle function be ensured along its entire chain of effects. In classic vehicle network communication, this can be an "alive" signal. The "alive" signal is a signal from the wearable device 120 that cyclically changes its value to signal to the vehicle 110 that the wearable device 120 is still "alive." This allows the vehicle 110 to ensure that a signal status, for example, of a hand gesture from the wearable device 120, is not "frozen" in the chain of effects.

[0082] Using the smartwatch as an example, the wearer can initiate an action, such as an automated parking maneuver for vehicle 110, by making a fist. The functional safety objective might then require that, during the automated parking maneuver, the smartwatch wearer cyclically brings their thumb and forefinger together and then releases them again to signal that the automated parking maneuver should continue. If this functional safety objective (cyclically bringing the thumb and forefinger together and releasing them) is not fulfilled, the automated parking maneuver would be terminated for safety reasons. By making a fist again, the smartwatch wearer can trigger the end of the action, in this case, the automated parking maneuver. Such an approach demonstrates agile, combinatorial control of vehicle functions.

[0083] Advantageously, this further increases the agility and flexibility of controlling vehicle functions, while simultaneously ensuring safety in controlling vehicle functions.

[0084] Preferably, the local reference also or alternatively includes the recognition of a direction of the movement data of the user of the wearable device 120, wherein the recognized direction is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional reference.

[0085] The aforementioned UWB technology can be used for this purpose. In addition to the highly precise determination of the position of the wearable device 120 relative to the vehicle 110, UWB technology also enables the determination of the direction of movement of the wearable device 120 relative to the vehicle 110. The determination or recognition of the direction of the movement data of the user of the wearable device 120 can thus be carried out, for example, by employing UWB technology in a manner known from the prior art. In the example of the smartwatch as wearable device 120, the wearer of the smartwatch can, for instance, initiate the opening of the window in the rear driver's side door of the vehicle 110 by making a hand gesture and pointing towards the rear door of the driver's side of the vehicle 110. By making the same hand gesture and pointing towards the front driver's side door of the vehicle 110, the wearer can initiate the opening of the window in the front driver's side door of the vehicle 110.

[0086] This advantageously further increases the agility and flexibility in controlling vehicle functions.

[0087] The functional, temporal, and spatial (position and / or direction) approaches described above can be combined in any way. Furthermore, the gestures mentioned above for controlling vehicle functions can be combined with any movements of the wearer of the Wearable Device 120. For example, a wearer of the Wearable Device can point to the driver's door, perform a gesture, and then move upwards. This can, for instance, initiate the closing of the driver's door window. Using the same procedure—pointing at the driver's door, performing the same gesture—but with a downward movement, the opening of the driver's door window can be initiated.

[0088] Furthermore, or alternatively, the functional, temporal, and / or spatial references can also be linked to other technologies as desired. For example, a link can be established with speech processing technology known from the prior art. A microphone can be used in the wearable device 120 for this purpose. Thus, the aforementioned methods can be combined with voice commands to trigger vehicle functions. In the smartwatch example, the wearer can activate a speech processing module in the smartwatch by making a fist near the vehicle 110. Subsequently, the wearer can ask a question, such as "What is the vehicle's charge level?" and / or "How much longer will charging take?", whereupon the vehicle function can output the corresponding vehicle data.Furthermore, or alternatively, linking it to voice input allows for further detail regarding vehicle functions. In the example above, a gesture to open the windows combined with the voice command "open driver's window only" can trigger the opening of the driver's window, whereas with the gesture alone, all vehicle windows would be opened.

[0089] Fig. Figure 2 shows a method 200 for the agile, intuitive control of vehicle functions of a vehicle, which is derived from a system 100 as with reference to Fig. 1 can be described and executed.

[0090] Procedure 200 includes: Capture 210, on a Wearable Device 120, of sensor data, wherein the sensor data includes motion data and optical data of a user of the Wearable Device 120; Processing 220, by means of a computing unit 114, 124 of the captured sensor data, wherein the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device 120; Determine 230, by the computing unit 114, 124 a functional relationship of the processed sensor data to the vehicle 110, wherein the determination of the functional relationship includes a time limitation for capturing the predefined movement and / or the predefined gesture, wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined period of time; Determine 240 by the computing unit 114, 124, an assignment to a predefinable vehicle function taking into account the determined functional reference; Control or regulate 250, by a control unit 112 of the vehicle 110, the determined predefinable vehicle function according to the determined assignment.

[0091] The vehicle 110 and the wearable device 120 can each include a communication unit 116, 126, wherein the vehicle 110 and the wearable device 120 are configured to establish a Bluetooth Low Energy, BLE, connection with each other.

[0092] The computing unit 114, 124 can also be configured to determine a local reference of the processed sensor data to the vehicle 110, whereby the determined local reference is taken into account when determining the assignment of the processed sensor data to the predefinable vehicle function.

[0093] The computing unit 114, 124 can also be configured to determine a temporal relationship between the processed sensor data and the vehicle 110, whereby the determined temporal relationship is taken into account when determining the assignment of the processed sensor data to the predefinable vehicle function.

[0094] The local reference can include the recognition of a direction of the captured movement data of the user of the Wearable Device 120, whereby the recognized direction is taken into account when determining the assignment of the processed sensor data to the predefinable vehicle function.

Claims

[1] System (100) for agile, intuitive control of vehicle functions of a vehicle (110), comprising: a wearable device (120) that is configured to collect sensor data, wherein the sensor data includes motion data and optical data of a user of the wearable device (120); a computing unit (114, 124) that is set up, - to process the captured sensor data, wherein the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device (120); - to determine a functional relationship between the processed sensor data and the vehicle (110), wherein determining the functional relationship includes a time limitation for capturing the predefined movement and / or gesture, wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined time period; and - to determine an assignment to a predefinable vehicle function, taking into account the determined functional relationship; wherein the vehicle (110) includes a control unit (112) which is configured to control or regulate the predefinable vehicle function according to the determined assignment. [2] System (100) according to claim 1, wherein the vehicle (110) and the wearable device (120) each comprise a communication unit (116, 126), and wherein the vehicle (110) and the wearable device (120) are configured to establish a Bluetooth Low Energy, BLE, connection between each other. [3] System (100) according to one of the preceding claims, wherein the computing unit (112, 122) is also configured to determine a local reference of the processed sensor data to the vehicle (110), wherein the determined local reference is taken into account when determining the assignment to the predefinable vehicle function taking into account the determined functional reference. [4] System (100) according to one of the preceding claims, wherein the computing unit (112, 122) is also configured to determine a temporal relationship of the processed sensor data to the vehicle (110), wherein the determined temporal relationship is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional relationship. [5] System (100) according to one of the preceding claims, wherein the local reference comprises the recognition of a direction of the movement data of the user of the wearable device (120), wherein the recognized direction is taken into account when determining the assignment to the predefinable vehicle function taking into account the determined functional reference. [6] Method (200) for agile, intuitive control of vehicle functions of a vehicle (110), comprising: Acquisition (210) of sensor data on a wearable device (120), wherein the sensor data includes motion data and optical data of a user of the wearable device (120); Processing (220) the acquired sensor data by means of a computing unit (114, 124), wherein the processing of the sensor data includes determining a predefined movement and / or a predefined gesture of the user of the wearable device (120); Determining (230) by the computing unit (114, 124) a functional relationship of the processed sensor data to the vehicle (110), wherein the determination of the functional relationship includes a time limitation for capturing the predefined movement and / or the predefined gesture, wherein the functional relationship requires that the predefined gesture and / or movement must be completed within a predetermined period of time; Determine (240) by the computing unit (114, 124) an assignment to a predefinable vehicle function taking into account the determined functional reference; Control or regulate (250) by a control unit (112) of the vehicle (110) according to the determined predefinable vehicle function in accordance with the determined assignment. [7] Method (200) according to claim 6, wherein the vehicle (110) and the wearable device (120) each comprise a communication unit (116, 126), and wherein the vehicle (110) and the wearable device (120) are configured to establish a Bluetooth Low Energy, BLE, connection with each other. [8] Method (200) according to claim 6 or 7, wherein the computing unit (112, 122) is also configured to determine a local reference of the processed sensor data to the vehicle (110), wherein the determined local reference is taken into account when determining the assignment to the predefinable vehicle function taking into account the determined functional reference. [9] Method (200) according to one of claims 6 to 8, wherein the computing unit (112, 122) is also configured to determine a temporal relationship of the processed sensor data to the vehicle (110), wherein the determined temporal relationship is taken into account when determining the assignment to the predefinable vehicle function, taking into account the determined functional relationship. [10] Method (200) according to one of claims 6 to 9, wherein the local reference comprises the recognition of a direction of the detected movement data of the user of the wearable device (120), wherein the detected direction is taken into account when determining the assignment taking into account the determined functional reference to the predefinable vehicle function.

Images