Method for taking into account a state of mind of at least one or more users

Abstract

The invention relates to a method (100) for taking into account a state of mind of at least one or more users (20) in a vehicle (50) using a movement- and preferably gesture-sensor (40) of the vehicle (50), comprising: determining (101) movement data which result from a detection of at least one movement by the movement- and preferably gesture-sensor (40) of the vehicle (50) in the stationary state of the vehicle (50); evaluating (102) the determined movement data with respect to at least one feature of the at least one gesture in order to examine a state of mind of the user or at least one of the users (20) on the basis of the at least one feature; comparing (103) a result of the examination with at least one specification in order to determine at least one action which is carried out on the basis of the result.

Description

[0001] Description

[0002] Methods for taking into account the emotional state of at least one or more users

[0003] The present invention relates to a method for taking into account the emotional state of at least one or more users. The invention further relates to a device, a motion sensor, and a computer program.

[0004] State of the art

[0005] It is known from the state of the art that taking into account the emotions of drivers and passengers in vehicles plays an important role in increasing safety and comfort.

[0006] However, current systems and methods for detecting and responding to emotional states are often inadequate, either because they are not precise enough or because they have a limited range of functions. This means that potential adjustments to driver assistance systems or recommendations for improving the driving situation cannot be optimally utilized.

[0007] It is therefore an object of the present invention to at least partially overcome the disadvantages described above. In particular, it is an object of the present invention to enable improved consideration of a vehicle's emotional state.

[0008] Disclosure of the invention

[0009] The invention relates to a method with the features of claim 1, a device with the features of claim 11, a motion sensor with the features of claim 12, and a computer program with the features of claim 15. Further features and details of the invention will become apparent from the respective dependent claims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the device, the motion sensor, and the computer program according to the invention, and vice versa, so that mutual reference is always possible with regard to the disclosure of the invention.The invention relates in particular to a method for taking into account the emotional state of at least one or more users in a vehicle using a motion and preferably gesture sensor of the vehicle.

[0010] The vehicle can be configured as a motor vehicle and / or passenger vehicle and / or truck. The vehicle can also be configured as a vehicle capable of at least partial autonomous driving.

[0011] The method can comprehensively: Determine motion data, which advantageously results from the detection of at least one movement by the motion and preferably gesture sensor of the vehicle, preferably when the vehicle is stationary. The detection of the at least one movement or gesture can be carried out, for example, by radar detection, in which the motion or gesture data results from radar signals and preferably comprises at least one radar image – for example, at least two-dimensional.

[0012] The detected movement or gesture data can result, for example, from an analog-to-digital conversion of received radar signals. The received radar signals may have been reflected by the movement / gesture or by the user performing the movement / gesture.

[0013] Furthermore, it may be possible for the process to be executed in real time, thus enabling a rapid response to the detected movements.

[0014] Furthermore, the procedure can include an evaluation of the determined movement data with regard to at least one feature or several features of the at least one movement and in particular gesture, in order to examine a state of mind of the user or at least one of the users based on the at least one feature.

[0015] An example of a mood state could be an emotional state such as joy, stress, anger, or sadness of a user. A mood state can also encompass a range of emotions from which inferences can be made about a user's physical condition—such as irritability due to tiredness or exhaustion. The emotional state can be deduced during the evaluation of at least one gesture through features such as fast or slow movements, smiling, or frowning. Therefore, by evaluating the acquired movement data with respect to this at least one feature, the method can reliably detect the emotional state of at least one user.

[0016] Furthermore, the procedure can include comparing a result of the investigation (e.g., a classification result for mood) with at least one target, in particular a digital target, in order to determine at least one action to be carried out based on the result. It is then possible to initiate the at least one identified action.

[0017] The method according to the invention makes it possible to collect and analyze movement and, in particular, gesture information – preferably in real time – to determine the emotional state of the users. By evaluating this information, an action can then be determined that is based on the emotional state, allowing the vehicle to be adapted to the needs and emotions of the users. The method thus enables improved interaction between the vehicle and users, contributing to a better experience and safer and more efficient ferry operations.

[0018] By combining motion and, in particular, gesture sensors with appropriate algorithms, the system can react to subtle changes in mood, such as stress, joy, or boredom. This enables adaptive vehicle control that meets the emotional needs of the users.

[0019] The system can also contribute to improved driving safety and efficiency by warning users in stressful situations or other emotionally charged states, or by offering special assistance functions. Furthermore, the analysis of movement and gesture information enables further adjustments, such as personalized vehicle navigation tailored to the individual needs and preferences of the users.

[0020] A gesture can be understood as a movement or series of movements performed, for example, with the hands or arms, but possibly also with other body parts such as the leg or foot, and which communicates a specific meaning or intention. In the specific vehicle-related context, this could be, for example, the activation of a vehicle action such as unlocking or opening. Features and details described below in connection with one or more "gestures" also apply equally to one or more "movements." The gesture sensor described below can therefore also be designed more generally as a motion sensor. Conversely, the motion sensor can also be implemented as a gesture or radar sensor. When a sensor is referred to generally below, the corresponding features and details also apply to the motion, gesture, and radar sensors.

[0021] Furthermore, within the scope of the invention, it is conceivable that the evaluation also includes: detecting at least one movement in the form of a gesture based on the determined movement data. The evaluation of the determined movement data can be adapted in such a way that the accuracy of gesture detection is optimized as the primary goal.

[0022] Furthermore, within the scope of the invention, it is conceivable that the evaluation also includes: identifying meanings or intentions based on the determined movement data and the detected gestures, in order to enable better interaction between the vehicle and a user. In other words, analyses of the emotional state can be carried out based on the detected gesture and the movement data. Thus, emotional state recognition can be used as a secondary application of a gesture sensor.

[0023] Furthermore, the method may include: assigning the at least one detected gesture to the user or at least one of the users, in particular by identifying the respective user based on at least one characteristic of the at least one detected gesture. It may also include: recognizing the emotional state of the respective user to whom the gesture has been assigned, in particular the identified user. This may also have the advantage that the method makes it possible to understand the needs and preferences of the users and to offer them personalized controls and settings.

[0024] The procedure can further include selecting at least one action that corresponds to the respective user to whom the gesture has been assigned, in particular the identified user, and to the recognized emotional state. In other words, the result of this evaluation can be used to select a suitable action for the respective user that corresponds to their emotional state and the assigned gesture. This allows for even better adaptation of the vehicle's operation to human behavior. Furthermore, the procedure can also make it possible to recognize a user's operating request more precisely by relating the gesture to the recognized emotional state. Conversely, gesture recognition can also be improved (alternatively or additionally) by relating the recognized emotional state to the gesture.

[0025] Furthermore, an embodiment of the invention is conceivable in which the determined movement data and gestures are combined in real time with the vehicle behavior and the output values ​​of associated sensor and / or actuator specification data. This combination makes it possible to adapt the operation of the vehicle even more precisely to the individual needs and the current situation of the users. For example, the speed or direction of the vehicle can be adjusted to the emotional state of a specific user in order to achieve optimal adaptation of the vehicle to human behavior.

[0026] Furthermore, the procedure can be extended to examine the emotional state and, if necessary, the dynamics of groups of users. This can be particularly useful when several people are in the vehicle and a collective adjustment to human behavior is desired. Such a function is especially advantageous for passenger transport vehicles, including potentially autonomous and / or driverless vehicles, such as buses or taxis.

[0027] Advantageously, the invention provides that the motion data results from the detection of at least one movement by the motion sensor, preferably a gesture sensor, and optionally further motion sensors, preferably gesture sensors, of the vehicle. The motion data can be implemented as radar data. Preferably, the respective motion sensor, preferably a gesture sensor, can be configured as a radar sensor that detects the at least one movement in the environment outside the vehicle, preferably only when the vehicle is stationary. Thus, the radar data resulting from the detection of movements by the vehicle's radar sensor(s) can be analyzed directly and particularly efficiently, or even in real time, to examine the emotional state of one or more users. For example, this could be used to...The system focuses on the speed, direction, and intensity of the detected movements to provide an indication of the mood and arousal level of the user, or at least one of the users, or even of passersby. This method also allows for immediate monitoring of the user's reactions to the vehicle's surroundings, enabling a comprehensive assessment of the current driving situation and the vehicle's environment. This assessment can then be used, for example, for autonomous vehicle control.

[0028] It is further possible to extend the method by perceiving additional aspects, in particular the behavior of users or passers-by. For example, the evaluation of the acquired movement data can take into account further aspects, such as the way in which users / passers-by interact with and / or perceive their environment. This can be achieved, for example, by analyzing parallels between the recorded movements and the perceived environmental features. In other words, the perception of the environment and the reactions of users / passers-by to it can be considered. Furthermore, within the scope of the invention, it is optionally possible to provide fusion, preferably sensor fusion, for examining the emotional state and / or for identifying at least one user.in which, in addition to movement data, further data acquisition and preferably sensor data are taken into account. This further data acquisition and preferably sensor data can result from at least one of the following acquisition methods: acquisition of a password and / or a cryptographic key and / or a biometric characteristic and / or an electronic characteristic; sensory acquisition at a mobile ID transmitter, preferably an electronic key, carried by the user; sensory acquisition from a smartphone; acquisition of data via a cloud; acquisition of a force exerted on the vehicle, preferably a door handle; acquisition of at least one gesture and / or facial expression of the user at a later or earlier time while the vehicle is in motion; acquisition of a driving movement and / or characteristic of the vehicle at a later or earlier time while the vehicle is in motion.a capture of image and / or video and / or audio information, preferably of the at least one user and / or the vehicle's surroundings, a capture of a voice signature, a capture of the user's face.

[0029] In this way, sensor fusion can consider a multitude of data sources to determine a comprehensive and accurate emotional state of the user, or at least one user. By fusing this data, a highly accurate impression of the user's condition can be gained, providing a better basis for executing actions and controlling the vehicle. Furthermore, sensor fusion can also contribute to increasing the reliability and accuracy of the investigation results by combining multiple sources of information. This can be particularly beneficial when the acquisition and sensor data provide conflicting information, as sensor fusion offers improved plausibility and / or reduces the error rate.Thus, the method for considering the emotional state of at least one or more users in a vehicle offers the possibility of taking into account a multitude of aspects and factors to gain a comprehensive and accurate impression of the users' condition. By combining motion and gesture sensor data with other data collection results, such as biometric characteristics, a higher degree of accuracy and reliability can be achieved.

[0030] In particular, the process can also include providing feedback to the user(s) regarding their emotional state. This can help users better respond to their needs and feelings by providing more information about their emotional state and allowing them to adjust their driving behavior or interactions accordingly. This can be especially beneficial on long journeys or in stressful situations.

[0031] A further advantage of the invention can be achieved if the at least one initiated action is a vehicle action and includes at least one of the following vehicle actions:

[0032] Opening or closing a front or rear hatch or door of the vehicle,

[0033] Unlocking a front or rear hatch or door of the vehicle,

[0034] Setting a display device in the vehicle,

[0035] Setting the vehicle's lighting configuration,

[0036] Adjusting the vehicle's air conditioning, and preferably switching the vehicle's heating or cooling systems on or off,

[0037] Setting the vehicle's comfort functions,

[0038] Setting a user-specific pre-configuration of the vehicle, preferably by adjusting seat and / or rear seat settings,

[0039] Adjusting the vehicle's driving behavior,

[0040] Control of an input and / or output device, preferably a display, of the vehicle, which is preferably located on the outside of the vehicle and / or is intended for operation from the environment outside the vehicle,

[0041] User settings selections for the vehicle,

[0042] Setting a navigation system of the vehicle, preferably a route selection, an output, preferably an acoustic and / or written output to the at least one user, a documentation of the emotional state, a sending of a message to an external data processing device.

[0043] The respective vehicle action, and in particular the respective setting, can preferably be determined based on a preset user preference of the user to whom the movement is assigned. Therefore, a variety of vehicle actions can be initiated to offer the user(s) an adaptive and user-specific driving experience.

[0044] These vehicle actions can also be adapted to the user's mood to ensure optimal vehicle operation. By using a motion sensor, and preferably a gesture sensor, the vehicle enables continuous information exchange between the user and the vehicle, resulting in improved operation and an adaptive driving experience. This leads to increased user-friendliness and enhanced comfort while driving.

[0045] Thus, the method of considering emotional state offers a multitude of possibilities for providing users with a personalized and adaptive driving experience. By analyzing movement data, the system can recognize the individual preferences and needs of the users and then initiate appropriate vehicle actions.

[0046] These can range from simple changes to the seat and rear seat settings to more complex functions such as controls for a navigation system or settings for the climate and heating system.

[0047] The method can also be used to establish better communication between the vehicle and its users. By sending messages to the user or documenting their mood, users can be informed about how the system has addressed their needs and preferences.

[0048] Another potential of the method lies in its ability to be combined with other functions and systems. By analyzing movement data, the system can recognize when a user is approaching a specific point and can then initiate appropriate vehicle actions to assist the user in reaching their destination. This could, for example, include activating lighting based on a (predicted) movement path of the user.

[0049] Furthermore, the method can be used to achieve better integration of vehicles and other devices such as smartphones or wearables. By analyzing movement data, the system can recognize when a user performs a specific action and then initiate appropriate vehicle actions to support the user in achieving their goal.

[0050] Furthermore, within the scope of the invention, it is conceivable that, based on the assignment of at least one movement to the respective user and / or based on a classification of the at least one movement, an automated decision is made as to which of several actions, preferably vehicle actions, are initiated. This can be done depending on a predefined assignment of various predefined movements, in particular gestures, to associated actions, preferably vehicle actions. The classification can preferably include a classification of a movement pattern. In this context, at least one of the predefined movements can particularly preferably be provided for opening a front or rear hatch or a door of the vehicle, and at least one other of the predefined movements can be provided for unlocking, locking, or closing the hatch / door.Thus, by assigning at least one movement to the respective user and / or by classifying at least one movement, an automated decision can be made that initiates one of several actions, preferably vehicle actions. Classifying a movement pattern allows for more accurate and objective recognition of the movement. This enables precise triggering of the respective action, such as opening or closing a front or rear hatch or a vehicle door. The automated decision can therefore improve the user experience and facilitate vehicle operation.

[0051] Furthermore, a classification of the emotional state during the examination may also be provided by a classification algorithm in order to indicate the recognized emotional state as the result.

[0052] The process also offers new possibilities for creating individual user profiles based on recognized movements and gestures. This profile information can then be used to trigger personalized actions, such as adjusting the volume or temperature level in the vehicle to the user's needs. A user can also preset their preferred settings for different moods in their user profile.

[0053] Thus, a user profile can provide an individual assignment of gestures and / or moods and / or user identities to settings and / or controls on the vehicle. The initiation of the action can therefore depend on the user profile and / or the recognized user and / or the recognized gesture and / or the recognized mood, which may have been at least partially determined during the evaluation based on the movement data.

[0054] Furthermore, the system can be used to make vehicle operation more convenient and safer. For example, a specific movement or gesture can be detected indicating that the user intends to enter or exit the vehicle. In this case, the system can automatically unlock the doors and / or shut off the engine to ensure safe exit.

[0055] Optionally, the evaluation process can take into account a temporal sequence of movements, particularly gestures, and / or a sequence of multiple movements in order to recognize a signature specific to the emotional state and / or a signature individual to the respective user. Preferably, the respective signature can include a movement signature. Furthermore, the evaluation process can preferably include a learning procedure to learn a user-specific association between the at least one feature of the at least one movement and the user's emotional state. This makes it possible to understand the meaning of gestures and movements for the emotional state of one or more users in a vehicle and to trigger an appropriate response to these signals.

[0056] Another advantage of this method is its ability to detect users' emotional states and trigger appropriate responses. For example, if a user is feeling anxious or nervous, the system can start a relaxing ambient music program or adjust the vehicle's heating to calm them. Furthermore, the system can assist in configuring vehicle settings and parameters. For instance, it can detect when a user is particularly focused or distracted and adjust the vehicle's settings to provide a more comfortable driving experience. This can also include, as an initiated action, deactivating vehicle features such as the multimedia system.

[0057] According to an advantageous embodiment of the invention, the respective movement can be recognized as a hand / arm and / or leg / foot gesture. Therefore, the recognized hand / arm and / or leg / foot gesture can be used as an indicator of the emotional state of the user, or at least one of the users. Through this type of gesture, the invention can infer the user's emotions, such as fear, anger, joy, or surprise, and then initiate a suitable action. This approach makes it possible to capture the user's reactions to specific situations in the vehicle and thus create a better environment for the user.

[0058] Another advantage of this method is the ability to monitor users' moods in real time and use this information to take appropriate action. For example, the system can detect if a user is bored or restless and then take a suitable measure, such as changing the music in the car or initiating a conversation.

[0059] Furthermore, the system can be used to monitor user reactions to specific events in the vehicle. For example, it can detect if a user feels anxious, is inattentive, needs to brake suddenly, or if another vehicle has nearly collided. The system can then initiate an appropriate action, such as initiating or suggesting emergency braking or alerting emergency services.

[0060] It is also possible that the comparison process includes the following steps:

[0061] Retrieving the specification from a data source, where the specification may include several predefined signatures,

[0062] Checking for a match between a signature recognized on the basis of at least one feature of at least one movement and specific to the emotional state with the specified signatures, in order to assign the recognized signature to at least or exactly one of the emotional states, and thereby to determine the emotional state probably present in the respective user.

[0063] This allows, for example, an appropriate response to the emotional state of the user, or at least one user, to be initiated in order to carry out the driving activity safely and efficiently.

[0064] According to an advantageous embodiment of the invention, the detection, evaluation, comparison, and / or initiation can be carried out automatically by the motion and, preferably, gesture sensor. This allows users to operate the vehicle more easily and efficiently, as the sensor perceives their movements and gestures and reacts accordingly.

[0065] The invention also relates to a data processing device configured to execute the method according to the invention, preferably comprising means for carrying out the steps of the method according to the invention. Thus, the data processing device according to the invention offers the same advantages as those described in detail with reference to a method according to the invention.

[0066] The invention also relates to a computer program, in particular a computer program product, comprising instructions that, when executed by a computer, cause the computer to execute the method according to the invention. The computer program according to the invention thus offers the same advantages as those described in detail with reference to a method according to the invention. Furthermore, the computer program can be at least partially non-volatile and / or available as downloadable software and / or as a cloud service and / or as an executable program and / or as a configuration file and / or as a program library and / or as source code and / or in compiled and / or encrypted and / or compressed form and / or in a combination thereof.

[0067] The computer can be a data processing device, preferably the data processing device according to the invention.

[0068] The device for data processing according to the invention, and preferably the computer, can be configured to execute the computer program according to the invention.

[0069] For this purpose, the device according to the invention can have at least one processor for data processing. A non-volatile data storage medium can also be provided in which the computer program is stored and from which the computer program can be read by the processor for execution.

[0070] It is also conceivable that the data processing device according to the invention comprises at least one integrated circuit such as a microprocessor, an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a digital signal processor (DSP), a field-programmable gate array (FPGA), a microcontroller, or the like. The data processing device according to the invention can further comprise at least one interface for data exchange, e.g., an Ethernet interface, an interface for LAN (Local Area Network), WLAN (Wireless Local Area Network), a system-on-a-chip (SoC), or another radio interface such as for Bluetooth or near-field communication (NFC). Furthermore, the data processing device according to the invention can be implemented as one or more control units, i.e., also as a system of control units.The data processing device according to the invention can be implemented wholly or partially in a cloud and / or as a server in order to provide data processing for a local application via the interface. Accordingly, the data processing device according to the invention can also be designed as a distributed system. It is also possible for the data processing device according to the invention to be implemented as a mobile device, such as a smartphone.

[0071] The invention may also include a computer-readable storage medium comprising the computer program according to the invention. The storage medium is, for example, designed as a data storage device such as a hard drive and / or non-volatile memory and / or a memory card. The storage medium can, for example, be integrated into the computer and / or into the data processing device according to the invention.

[0072] Furthermore, the method according to the invention can also be implemented as a computer-implemented method. Alternatively or additionally, each or all of the disclosed method steps can optionally be computer-implemented and / or carried out automatically.

[0073] The invention also relates to a motion sensor, preferably a gesture sensor, for a vehicle. The motion sensor serves in particular to take into account the emotional state of at least one or more users of the vehicle. For this purpose, the motion sensor can include a data processing device according to the invention. Thus, the motion sensor according to the invention offers the same advantages as those described in detail with reference to a method according to the invention.

[0074] Furthermore, within the scope of the invention, it is optionally possible that the motion and preferably gesture sensor according to the invention is designed for integration into at least one of the following vehicle components:

[0075] A door handle,

[0076] At least one vehicle pillar, preferably an A- and / or B- and / or C-pillar,

[0077] A headlight,

[0078] On both sides or on several sides of the vehicle

[0079] At least one flap, preferably the front and / or rear flap,

[0080] Located on the outside of the vehicle,

[0081] Located inside the vehicle,

[0082] Sills,

[0083] Bumpers

[0084] Exterior cladding, vehicle pillar cladding.

[0085] Furthermore, it is conceivable that the motion and preferably gesture sensor is designed as a radar sensor. Using a radar sensor as a gesture sensor allows for high accuracy in gesture recognition even under challenging environmental conditions, thus increasing the system's reliability and safety. Moreover, using a radar sensor reduces power consumption compared to other sensor systems, thereby improving the vehicle's battery life. As a result, the solution according to the invention can offer the user a simple and convenient operating method without compromising safety and reliability.

[0086] The motion and preferably gesture and / or radar sensor can be arranged inside or on the vehicle, particularly in the area around the doors and / or tailgate, to detect movements / gestures such as hand, arm, foot or leg movements.

[0087] The sensor housing can preferably be made at least partially of radar-signal-transparent materials such as polycarbonate or acrylic glass to effectively transmit the radar signals. The geometric arrangement of the antennas of one or more sensors on the vehicle can be, for example, in a line or a circle, to detect a wide range of gestures at different distances and angles.

[0088] The housing may include fastening elements. These may be selected to enable stable and permanent mounting of the sensor to the vehicle, e.g., using screws or clamps.

[0089] A gesture recognition technology can utilize advanced algorithms, such as pattern recognition, to interpret radar data and translate it into usable input for the vehicle system. These algorithms can be tailored to identify specific movement patterns typically performed by users when they want to trigger certain commands or actions. For example, a hand gesture in front of the tailgate can be interpreted as a command to open it. The algorithms can also analyze features such as the speed, direction, and / or duration of the gestures to differentiate between various gestures and minimize misinterpretations.Additionally, the sensor / algorithm can be able to differentiate between the gestures of different users by analyzing features such as the shape and dynamics of the gestures, enabling personalized interaction and / or authentication by associating the gesture with the person. Examples of suitable algorithms include Support Vector Machines (SVMs) or decision trees.

[0090] It may be possible to determine motion and, in particular, gesture data resulting from the detection of at least one movement / gesture by the vehicle's (at least one or exactly one) motion or gesture sensor. The (respective) motion or gesture sensor may include one or more radar sensors.

[0091] It is conceivable that the respective sensor is integrated into a side mirror and / or the front bumper and / or the rear bumper and / or the bumper cover and / or the door handle and / or the sill and / or the vehicle pillar such as the A-, B- or C-pillar. This arrangement allows for optimal detection of movements or gestures in the immediate vicinity of the vehicle.

[0092] For example, at least one sensor in the door handle or side mirrors can detect hand gestures used to open or close the doors. Furthermore, at least one sensor in the door handle or vehicle pillar can detect locking or unlocking movements.

[0093] The sensor's mechanical design may include a robust and / or weatherproof housing that protects the sensor from dust, water, and mechanical impacts. The housing may be made of durable materials such as polycarbonate or metal and preferably has a protection rating of at least IP67. The sensor may be mounted on the vehicle in such a way that it is easily accessible for maintenance and replacement.

[0094] The radar sensor may operate in the frequency range of 76 GHz to 81 GHz, enabling high resolution and accuracy in motion and gesture detection. This frequency range allows for the use of smaller antennas while still ensuring precise motion detection. At least one radar sensor utilizes Frequency Modulated Continuous Wave (FMCW) technology, which emits continuous signals with varying frequencies and uses the reflected signals to measure distance and / or speed. Furthermore, the radar sensor's Doppler function can be used to precisely measure the relative speed of movements / gestures and / or objects.By analyzing the frequency shift between the emitted and reflected signals, the sensor can detect and differentiate movements, which is particularly advantageous for the accurate recognition of hand and finger movements and other gesture characteristics. Various algorithms can be used for gesture recognition. These include, for example, machine learning algorithms such as neural networks and support vector machines (SVMs), which are trained to recognize and classify specific gestures. These algorithms analyze the gesture data captured by the at least one radar sensor and identify patterns characteristic of certain gestures. Clustering algorithms can also be used to group the captured gesture data and recognize movement patterns.

[0095] It is possible that the processing of gesture data is further carried out by special algorithms that include noise reduction, signal amplification, and the extraction of relevant features. Fourier transforms can be used to analyze the frequency components of the received signals. Wavelet transforms can be used to analyze transient and non-stationary signals in order to extract precise information about the distance, speed, and direction of movement of gestures / objects.

[0096] The electronics of the respective sensor may include at least one high-frequency module and / or at least one signal processing unit and / or at least one communication interface. The high-frequency modules can generate and transmit radar waves, while the signal processing units can analyze the received echo signals. Communication interfaces such as CAN bus or Ethernet enable the integration of the sensor into the vehicle network and data exchange with other vehicle systems.

[0097] It is conceivable that the sensor design includes transmitting and receiving antennas integrated on a single circuit board. The antennas can be designed to offer high directivity and sensitivity. The use of multi-antenna systems (MIMO technology) enables improved spatial resolution and the simultaneous detection of multiple gestures, users, or objects. Adaptive algorithms can dynamically adjust the transmit power and receive sensitivity to achieve optimal results under varying environmental conditions. This comprehensive design and integration of radar sensors allows the vehicle to precisely detect gestures and movements and react accordingly, significantly improving safety and user-friendliness.

[0098] It is possible to design the respective sensor and its associated electronics to operate in an energy-efficient manner. This can be achieved through the use of low-power components and energy management algorithms that optimize energy consumption depending on the operating conditions. For example, the sensors could be put into an energy-saving mode when the vehicle is parked or in a low-traffic environment.

[0099] It is conceivable that the sensor in question has an automatic calibration function that ensures it always operates in an optimized state. This calibration can also be performed while driving or at regular intervals to compensate for deviations caused by aging or mechanical influences. Furthermore, the sensor may be equipped with self-diagnostic functions that continuously monitor its own condition and report any errors or maintenance requirements at an early stage.

[0100] The sensor may be designed to be modular and easily adaptable to different vehicle types and models. This can be achieved, for example, through various mounting options, adapters, and software configurations that allow for easy integration into different vehicle designs.

[0101] It is conceivable that the respective sensor is specifically designed for operation under extreme environmental conditions. This could be achieved through the use of temperature-resistant materials and special coatings that protect the sensors from corrosion and wear.

[0102] It is also possible that the sensor / gesture sensor is equipped with an interface, such as a wireless interface, to connect to a data source like a database. The data can be retrieved and optionally synchronized with the sensor's local data. This wireless interface can be implemented, for example, via cellular networks, Wi-Fi, or Bluetooth. The connection to the data source allows for the direct transfer of current software updates, calibration data, predefined signatures, and / or new gesture recognition profiles to the sensor. Additionally, sensor data can be sent in real time to an external data processing device for gesture and / or signature recognition, analysis, and / or optimization. Secure encryption of the data transmission can further ensure that communication is protected against unauthorized access.

[0103] It is conceivable that the respective sensor could be combined with other devices, such as a mobile ID transmitter (electronic vehicle key) or a smartphone, and / or with other sensor types such as cameras, ultrasonic sensors, or lidar sensors to perform sensor fusion. This data fusion enables more accurate and reliable gesture and object recognition by leveraging the strengths of the different sensor types. For example, radar and camera data can be combined to determine both the distance and the visual appearance of a user / object, which improves the accuracy and robustness of recognition, particularly the recognition of the signature and / or features of the gesture.

[0104] It can be implemented that, depending on the detected movement or gesture and / or emotional state, a specific vehicle action is selected and executed as the initiated action. The movement and gesture recognition can identify specific movement patterns as signatures that are assigned to predefined vehicle actions. This enables intuitive control of various vehicle functions, for example, through simple hand gestures.

[0105] It is conceivable that the action, particularly a vehicle action, could be the selection of a specific route or journey in the navigation system. For example, a hand movement to the left or right could display or select an alternative route in the navigation system, while an up or down movement could change the zoom level of the map.

[0106] It's possible that the action / vehicle action could also be a different vehicle setting. Examples include adjusting the climate control, adjusting the seat position, controlling the audio system, or opening and closing windows and doors. Gestures like circling the hand could adjust the audio system volume, while an upward movement could increase the air conditioning temperature.

[0107] Further advantages, features, and details of the invention will become apparent from the following description, in which exemplary embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description can each be essential to the invention individually or in any combination. The drawings show:

[0108] Fig. 1 is a schematic representation to visualize embodiment variants of the invention.

[0109] Fig. 2 shows a further schematic representation for visualizing embodiment variants of the invention.

[0110] Fig. 3 is a visualization of a method according to embodiment variants of the invention.

[0111] In the following figures, identical reference numerals are used for the same technical features even for different embodiments. Figure 1 shows the vehicle 50 with several motion and gesture sensors 40, arranged by way of example in the side and rear areas. These can detect movements and gestures in the area 5 surrounding the vehicle 50 and thus identify the user performing the gesture. A central control unit 55, to which the sensors 40 are connected, is also shown.

[0112] A vehicle 50 is stationary, for example, in a parking lot. Various sensors 40 are provided around the vehicle 50 to monitor its surroundings 5 ​​and detect the user 20. A control unit 55 can be provided for central control and data processing tasks on the vehicle 50. The evaluation of movement data can be performed at least partially by the control unit 55 and / or, if applicable, also by the respective sensor 40. At least part of the data processing device 10 can therefore optionally be provided in the sensor 40 and, for example, by a computer program 15 (see Fig. 2).

[0113] In addition to motion and preferably gesture sensors 40, further sensors 40 can also be provided and mounted in various positions on the vehicle 50 to detect the environment 5 and the user 20. These include, for example, sensors 40 for voice signature or facial recognition, and others.

[0114] Figure 2 shows a device 10 for data processing with further details. In the example shown, the device 10 is fully or partially part of the gesture sensor 40. In alternative versions, the device 10 can also be part of the control unit 55 or be located externally from the vehicle 50, e.g., in a cloud server.

[0115] Fig. 3 illustrates, according to exemplary embodiments of the invention, a method 100 for taking into account a state of mind of at least one or more users 20 in a vehicle 50 using at least one motion and preferably gesture sensor 40 of the vehicle 50.

[0116] According to a first process step 101, movement data is determined, which results from the detection of at least one movement by the motion and preferably gesture sensor 40 of the vehicle 50 in the stationary state of the vehicle 50.

[0117] According to a second procedural step 102, the determined movement data is evaluated with regard to at least one characteristic of the at least one movement in order to examine the emotional state of the user or at least one of the users 20 based on this at least one characteristic. According to a third procedural step 103, a result of the examination is compared with at least one specification in order to determine at least one action to be carried out based on the result. The specification can, for example, be retrieved from a data source 16. The result can, for example, be a classification result for the emotional state and thus indicate the emotional state.

[0118] According to a fourth procedural step 104, at least one identified action is initiated. This can, for example, involve unlocking or opening a vehicle door or tailgate 30.

[0119] The preceding explanation of the embodiments describes the present invention solely by way of examples. Naturally, individual features of the embodiments can be freely combined with one another, provided this is technically feasible, without departing from the scope of the present invention.

[0120] Reference symbol list

[0121] 5 Environment

[0122] 10 Device

[0123] 15 Computer program

[0124] 16 Data source

[0125] 20 users

[0126] 30 Door, flap

[0127] 40 Sensor, gesture sensor

[0128] 50 vehicles

[0129] 55 Control unit

[0130] 100 procedures

[0131] 101 First procedural step

[0132] 102 second procedural step

[0133] 103 third procedural step

[0134] 104 fourth procedural step

Claims

Claims 1. Method (100) for taking into account the emotional state of at least one or more users (20) in a vehicle (50) using a motion and preferably gesture sensor (40) of the vehicle (50), comprising: Determining (101) motion data resulting from the detection of at least one movement by the motion and preferably gesture sensor (40) of the vehicle (50) while the vehicle (50) is stationary, - Evaluating (102) the determined movement data with regard to at least one feature of the at least one movement in order to examine, on the basis of the at least one feature, a state of mind of the user or at least one of the users (20), Comparing (103) a result of the investigation with at least one target in order to identify at least one action to be carried out on the basis of the result, Initiating (104) the at least one identified action.

2. Method (100) according to claim 1 , characterized in that the evaluation (102) further comprises: Detecting at least one movement in the form of a gesture based on the determined movement data, Assigning the at least one detected gesture to the user or at least one of the users (20), in particular identifying the respective user (20) based on the at least one feature of the at least one detected gesture, Recognizing the emotional state of the respective user (20) to whom the gesture has been attributed, in particular the identified user (20), including comparing (103): - Selections of at least one action which is assigned to the respective user (20) to whom the gesture has been assigned, in particular the identified user (20), as well as to the recognized emotional state.

3. Method (100) according to one of the preceding claims, characterized in that the motion data results from the detection of at least one movement by the motion and preferably gesture sensor (40) and optionally further motion and preferably gesture sensors (40) of the vehicle (50), wherein the motion data is implemented as radar data, and wherein preferably the respective motion and preferably gesture sensor (40) is designed as a radar sensor which detects the at least one movement in the environment (5) outside the vehicle (50) and preferably only when the vehicle (50) is stationary. Method (100) according to one of the preceding claims, characterized in that, for the purpose of examining the emotional state and / or identifying the at least one user (20), a fusion, preferably sensor fusion, is provided in which, in addition to the motion data, further acquisition and preferably sensor data are taken into account, wherein the further acquisition and preferably sensor data result from at least one of the following acquisitions: acquisition of a password and / or a cryptographic key and / or a biometric feature and / or an electronic feature, sensory acquisition at a mobile ID transmitter, preferably an electronic key, which is carried by the user, sensory acquisition of a smartphone, acquisition of data via a cloud, acquisition of a force which is exerted on the vehicle (50), preferably a door handle,a recording of at least one gesture and / or facial expression of the user (20) at a later time while the vehicle (50) is moving, a recording of a driving movement and / or characteristic of the vehicle (50) at a later time while the vehicle (50) is moving, a recording of image and / or video and / or audio information, preferably of the at least one user (20) and / or the surroundings of the vehicle (50), a recording of a voice signature, a recording of a face of the user. Method (100) according to one of the preceding claims, characterized in that the at least one initiated action is a vehicle action and comprises at least one of the following vehicle actions: Opening or closing a front or rear hatch (30) or door (30) of the vehicle (50), Unlocking a front or rear hatch (30) or door (30) of the vehicle (50), Setting a display device in the vehicle (50), Setting a vehicle lighting configuration (50), Setting the vehicle's air conditioning (50), and preferably switching the vehicle's heating or cooling devices on or off (50), Setting vehicle comfort functions (50), Setting a user-specific pre-configuration of the vehicle (50), preferably by adjusting seat and / or rear seat settings, Adjusting the vehicle's driving behavior (50), Control of an input and / or output device, preferably a display, of the vehicle (50), which is preferably located on the outside of the vehicle (50) and / or is intended for operation from the environment outside the vehicle (50), - Selections of user settings for the vehicle (50), Setting a navigation system of the vehicle (50), preferably a route selection, an output, preferably an acoustic and / or written output to the at least one user (20), a documentation of the emotional state, a sending of a message to an external data processing device, wherein the respective vehicle action and in particular the respective setting preferably takes place depending on a preset user preference of the user (20) to whom the movement is assigned.

6. Method (100) according to one of the preceding claims, characterized in that, based on the assignment of the at least one movement to the respective user (20) and / or based on a classification of the at least one movement, an automated decision is made as to which of the several actions, preferably vehicle actions, are initiated, depending on a predetermined assignment of various predetermined movements, in particular gestures, to associated actions, preferably vehicle actions, wherein the classification preferably comprises a classification of a movement pattern, wherein at least one of the predetermined movements is provided for opening a front or rear hatch or a door of the vehicle (50), and at least one other of the predetermined movements is provided for unlocking or locking or closing the hatch / door.

7. Method (100) according to one of the preceding claims, characterized in that the evaluation (102) takes into account a temporal sequence of movements, in particular gestures, and / or a sequence of several movements in order to recognize a signature specific to the emotional state and / or a signature individual to the respective user, wherein preferably the respective signature comprises a movement signature, wherein preferably the evaluation has a learning procedure in order to learn a user-specific association between the at least one feature of the at least one movement and the emotional state of the user.

8. Method (100) according to one of the preceding claims, characterized in that the respective movement is recognized as a hand / arm and / or leg / foot gesture.

9. Method (100) according to one of the preceding claims, characterized in that the comparison (103) further comprises the following steps: - Retrieving the specification from a data source, where the specification includes several predefined signatures, Checking for a match between a signature identified by the at least one feature of the at least one movement and specific to the emotional state with the specified signatures, in order to assign the identified signature to at least or exactly one of the emotional states, and thereby to determine the emotional state likely to be present in the respective user (20).

10. Method (100) according to one of the preceding claims, characterized in that the detection (101) and / or evaluation (102) and / or comparison (103) and / or initiation (104) are carried out automatically by the motion and preferably gesture sensor (40).

11. Device (10) for data processing, which is configured to perform the method (100) according to one of the preceding claims.

12. Motion sensor (40) for a vehicle (50), for taking into account a mood state of at least one or more users (20) in the vehicle (50), comprising a device (10) for data processing according to claim 11.

13. Motion sensor (40) according to claim 12, characterized in that the motion and preferably gesture sensor (40) is designed for integration into at least one of the following vehicle components: A door handle, At least one vehicle pillar, preferably an A- and / or B- and / or C-pillar, A headlight, On both sides or on several sides of the vehicle (50), At least one flap, preferably the front and / or rear flap, - Located on the outside of the vehicle, Located inside the vehicle, Sills, Bumpers - Exterior cladding, Vehicle pillar trim.

14. Motion sensor (40) according to claim 12 or 13, characterized in that the motion and preferably gesture sensor (40) is designed as a radar sensor.

15. Computer program (15) comprising instructions which, when the computer program (15) is executed by a computer, cause it to execute the method (100) according to any one of claims 1 to 10.

Images