Method and device for automated device control

Abstract

The invention relates to a method for automated device control for use in a vehicle (400) comprising steps (S100 to 180): - Identification and localization of a person by means of at least one sensor (200) for detecting the vehicle's surroundings (400), - Determining parameters for assessing a person's emotional state, - Determining the person's emotional state based on the parameters, - Establishing a response learned by an algorithm from previously recorded emotional states and associated interactions of the person with the vehicle (400) and / or at least one vehicle system, and - automated control of the vehicle (400) and / or the vehicle systems according to the specified response, wherein - the parameters for recording the person's emotional state are determined when the person moves towards the vehicle (400) and an intention to interact is detected by means of a posture and / or at least one gesture, - and data on the person's posture and / or gait pattern and / or movement speed are used as parameters for recording the person's emotional state and - the person's emotional state is determined based on data relating to posture and / or gait and / or speed of movement.

Description

[0001] The invention relates to a method for automated device control according to the preamble of claim 1 and to a device for automated device control for use in a vehicle according to claim 5.

[0002] GB 2528083A discloses an automated device control system for use in a vehicle, with - a user identification module, - a monitoring module configured to determine the value of a parameter indicating the user's emotional state, - an input for receiving control instructions from the user and - a training module, wherein the training module is configured to use user control instructions and parameter values ​​indicating the user's emotional state to train a training algorithm which is configured, based on the training, to provide a control instruction in response to the measured parameters indicating the user's emotional state.

[0003] The invention is based on the objective of providing an improved method and an improved device for automated device control for use in a vehicle.

[0004] The problem is solved according to the invention by a method which has the features of claim 1 and a device which has the features of claim 6.

[0005] Advantageous embodiments of the invention are the subject of the dependent claims.

[0006] The invention relates to a method for automated device control for use in a vehicle comprising the following steps: - Identification and localization of a person using at least one sensor to detect the vehicle's surroundings, - Determining parameters for assessing a person's emotional state, - Determining the person's emotional state based on the parameters, - Establishing a response that was learned by algorithms from previously recorded emotional states and associated interactions of the person with the vehicle and / or at least one vehicle system, and - automated control of the vehicle and / or vehicle systems according to the specified response.

[0007] According to the invention, it is provided that - the parameters for recording the person's emotional state are determined when the person moves towards the vehicle and an intention to interact is recognized by means of a posture and / or at least one gesture, - where parameters used to assess the person's emotional state include data on the person's posture and / or facial expressions and / or data on the person's gait and / or movement speed and - the person's emotional state is determined based on data relating to posture and / or gait and / or speed of movement.

[0008] The method allows the recording of a person's emotional state outside the vehicle, particularly in the immediate vicinity of the vehicle, by means of gait pattern and / or posture and / or pace of movement.

[0009] This allows the vehicle's emotional state to be determined even before the person enters, and based on this, settings for the vehicle and / or vehicle systems can be automated, such as seat position, climate control temperature, and / or radio station. The vehicle and / or vehicle systems are then already configured when the person enters the vehicle.

[0010] Furthermore, the use of an algorithm to learn and determine emotional states makes it possible to adjust settings automatically without any further action required from the person.

[0011] Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

[0012] This shows: Fig. 1. Schematic representation of a possible embodiment of a method for automated device control for use in a vehicle as a flowchart. Fig. 2 schematically a possible placement of sensors on the vehicle, Fig. 3 schematically a possible placement of cameras on the vehicle according to Fig. 2, Fig. 4 schematically shows a possible embodiment of an image of a body with detection points, Fig. 5 schematically different examples of individual images of different gait patterns, which are represented by means of fictitious skeletons and Fig. 6 schematically different examples of individual images of different gait patterns, which are represented by means of fictitious skeletons.

[0013] Corresponding parts are marked with the same reference symbols in all figures.

[0014] Fig. Figure 1 schematically shows a possible embodiment of the method as a flowchart.

[0015] In a first step, S100 locates a person who is in the immediate vicinity of a vehicle 400, which is in Fig. 2 is shown. Localization can be achieved, for example, by means of at least one sensor 200 of the vehicle 400, as shown in Fig. 2 shown, via CCC-compliant ultra-wideband and / or channel sounding.

[0016] In step S110, the person is identified. This is done, for example, using at least one camera 300 of the vehicle 400. In Fig. Figure 3 shows several cameras 300 distributed around the vehicle 400, which are used for person identification via facial recognition. Alternatively or additionally, the person can be identified using a smartphone or transponder paired with the vehicle 400.

[0017] In step S120, it is determined whether the person intends to interact with vehicle 400. This can be achieved, for example, by determining the person's direction of movement using a transponder assigned to vehicle 400 or by analyzing a sequence of images from cameras 300, thus determining whether the person is moving towards or past vehicle 400. Additionally, to determine the intention to interact, a gesture made by the person, in conjunction with seat occupancy detection within vehicle 400, can be interpreted as a gesture directed at a vehicle occupant.

[0018] If no interaction intention is detected, no further action will be taken by vehicle 400.

[0019] If an intention to interact is detected, in step S130 a determination of the person's gait pattern and / or posture and / or gestures and / or facial expressions and / or movement speed is carried out using the sensors 200 and / or the cameras 300 of the vehicle 400.

[0020] In one step S140, as in the Fig. Figures 4 to 6 illustrate and explain in more detail how the data on the person's gait, posture, gestures, facial expressions, and / or movement speed are interpreted and assigned to a person's emotional state. These emotional states are based on previous data on the person's gait, posture, gestures, facial expressions, and / or movement speed, which were assigned to an emotional state and thus learned using an algorithm, as described in steps S170 and S180.

[0021] In step S150, regardless of the person's mood, the vehicle 400 is unlocked, the engine of the vehicle 400 is optionally started and / or at least one driver assistance system of the vehicle 400 is optionally started.

[0022] Once the mood, determined in step S140, is known, step S160 proceeds to a personalized vehicle setting. This personalized vehicle setting can include, for example, the following: - Personal driving settings: seat position, mirror adjustment, climate control and / or infotainment, - Switching on a welcome light of the vehicle 400 on the outside and / or inside, - Activate speech recognition, - Pre-conditioning: temperature settings, fresh air supply, interior fragrance, - Navigation system: Suggest known destinations, especially in conjunction with a smartphone connected to the vehicle 400 and a calendar contained therein, where the calendar contains an appointment with a location, - System check to ensure all vehicle systems are functioning correctly and - Setting up an entertainment system: podcasts, radio stations or playlists.

[0023] The following are examples of mood states with corresponding settings for some vehicle functions: - Cheerful - Lighting: Bright and welcoming interior lighting, - Music: Lively and energetic playlist, - Air conditioning: Pleasant and slightly warm temperature, - Seating position: Slightly more upright for a more alert and engaged driving posture and - Infotainment: Displays entertaining and engaging content or apps. - Sad - Lighting: Soft and calming interior lighting, - Music: Calm and soothing playlist, - Air conditioning: Neutral temperature, neither too warm nor too cold, - Seating position: Comfortable and slightly reclined for a relaxed posture and - Infotainment: Display calming images or content. - Stressed - Lighting: Subdued and relaxing interior lighting, - Music: Relaxing and stress-relieving playlist, - Air conditioning: Cool temperature to calm the nerves, - Seating position: Comfortable and supportive posture and - Infotainment: Display of mindfulness or relaxation apps. - Angry - Lighting: Neutral lighting to avoid further excitement, - Music: Calming playlist, - Air conditioning: Cool temperature for cooling, - Seating position: Comfortable and supportive posture and - Infotainment: Display calming images or content. - Stimulated - Lighting: Bright and dynamic interior lighting, - Music: Energetic and vibrant playlist, - Air conditioning: Slightly cool temperature to keep the driver alert, - Seating position: Upright and engaged posture and - Infotainment: display engaging and interactive content - Tired - Lighting: Soft and gentle interior lighting, - Music: Relaxing and calming playlist, - Air conditioning: Pleasant and slightly warm temperature, - Seating position: Comfortable and slightly reclined to relax and - Infotainment: Display of calming images or content. - concentrated - Lighting: Neutral and unobtrusive interior lighting, - Music: Instrumental or ambient playlist to maintain concentration, - Air conditioning: Neutral temperature to maintain comfort, - Seating position: Upright and supportive posture and - Infotainment: Display of productivity or navigation apps.

[0024] If the emotional state determined in step S140 is unknown, step S170 follows, in which the algorithm creates a new emotional state in a database and stores it with the data of the gait pattern and / or gestures and / or movement speed of the person, which were determined in step S140.

[0025] In step S180, the person's interaction with the vehicle 400 and the settings made by the person to the vehicle 400 and / or its systems are assigned to the mood state newly created in step S170. For example, the algorithm can request confirmation of the new mood state and the associated settings of the vehicle 400 and / or its systems, and, if confirmed, use this information for future determinations of the person's mood state. Alternatively, the algorithm can be configured to be self-learning, for example, as artificial intelligence, and can independently use the mood state for future determinations after several reassignments.

[0026] Furthermore, the algorithm can be designed in such a way that, in addition to data on gait, gestures, and / or movement speed, other factors are taken into account when determining a person's emotional state. For example, this could include the day of the week, recurring calendar events, and / or weather conditions.

[0027] The following are case studies: Thursday - After a stressful workday, the person approaches the Vehicle 400. The Vehicle 400 recognizes the person's stressed gait and facial expression. As the person approaches the Vehicle 400, it automatically adjusts the interior lighting to a dimmed and relaxing setting, provides a stress-reduction playlist, sets the interior temperature to a cool setting, and adjusts the seat to a comfortable and supportive position. The infotainment system displays mindfulness apps to help the person relax. Friday - The person approaches the vehicle after a long workday. The system detects their gait and posture and determines that they are tired and stressed. The system automatically unlocks the doors, starts the engine, adjusts the seat to a comfortable position, activates calming interior lighting, and plays relaxing music. The interior temperature is set to a cool level to help the person unwind. Saturday The person approaches the vehicle on a sunny weekend morning, feeling energized and happy. The system recognizes their cheerful gait and posture, thus determining their mood. The system unlocks the doors, starts the engine, adjusts the seat to a sporty position, activates bright and welcoming interior lighting, and compiles an upbeat playlist. The interior temperature is set to a warm setting, reflecting the person's mood.

[0028] Fig. Figure 2 schematically shows a possible placement of sensors 200 on the vehicle 400.

[0029] The sensors 200 are arranged on the outside of the vehicle 400 and inside the vehicle 400 in such a way that the sensors 200 monitor the immediate vicinity of the vehicle 400 and its interior.

[0030] The sensors 200 are, for example, ultra-wideband sensors and / or channel-sounding sensors, or a camera 300. The sensors 200 locate and identify, for example, a transponder assigned to the vehicle 400 or a smartphone that is wirelessly connected to the vehicle 400. The sensors 200 are further configured to determine the direction of movement using ultra-wideband and / or channel sounding after localization.

[0031] Fig. Figure 3 schematically shows a possible placement of cameras 300 on the vehicle 400 according to Fig. 2.

[0032] The cameras 300 are arranged on the outside of the vehicle 400 and inside the vehicle 400 in such a way that the cameras 300 monitor the immediate vicinity of the vehicle 400 and its interior.

[0033] For example, the cameras 300 can be used to monitor parking operations and / or as part of driver assistance systems.

[0034] Furthermore, the cameras 300 capture images of the person located by the sensors 200. In a further configuration, the person can be located directly using the cameras 300.

[0035] Furthermore, identification of the person can be carried out, for example, by means of facial recognition.

[0036] Fig. Figure 4 schematically shows a possible embodiment of an image 500 of a body 501 with detection points 510.

[0037] The images 500 from the cameras 300 are evaluated using the algorithm. For this purpose, on one of the images 500, a body 501 of a person, which was detected in the image 500 by the algorithm, is assigned several detection points 510 at a plurality of points of the body 501 relevant for movement. Movement-relevant points can, for example, joints such as ankles and / or knees and / or hips and / or shoulders and / or elbows. The assigned detection points 510 can then be transferred to a fictitious skeleton 520 by means of connecting lines. Based on this fictitious skeleton 520, the algorithm can, as described in the Fig. 5, Fig. 6 is shown, for example, the person's gait pattern can be determined.

[0038] Furthermore, relevant points on the head and / or face can be identified and assigned for facial recognition and / or facial expression recognition. The data obtained in this way can also be used to determine a person's emotional state.

[0039] The Fig. 5 and Fig. Figure 6 schematically shows various examples of individual images of different gait patterns which are represented by means of fictitious skeletons 520.

[0040] In both Fig. 5 and Fig. In 6, on the left, is a fictional skeleton 520 depicted in a gait pattern, which corresponds, for example, to an angry state of mind.

[0041] In both Fig. 5 and Fig. In 6, a fictional skeleton 520 is depicted in the center, showing a gait pattern that corresponds, for example, to a sad state of mind.

[0042] In both Fig. 5 and Fig.In 6, on the right, a fictitious skeleton 520 is depicted in a gait pattern, which corresponds, for example, to a happy state of mind. Reference symbol list S100 to S180 step 200 Sensor 300 camera 400 vehicles 500 images 501 bodies 510 Detection point 520 fictional skeleton QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] GB 2528083A

[0002]

Claims

Method for automated device control for use in a vehicle (400) with the steps (S100 to 180): - Identification and localization of a person by means of at least one sensor (200) for detecting an environment of the vehicle (400), - Determination of parameters for detecting a state of mind of the person, - Determination of the state of mind of the person based on the parameters, - Determination of a reaction which was learned by an algorithm from previously detected states of mind and associated interactions of the person with the vehicle (400) and / or at least one vehicle system, and - Automated control of the vehicle (400) and / or the vehicle systems according to the defined reaction, characterized in that - the parameters for detecting the state of mind of the person are then determined,when the person moves towards the vehicle (400) and an intention to interact is detected by means of a posture and / or at least one gesture, - wherein data of the person's posture and / or data of a gait pattern of the person and / or a speed of movement of the person are used as parameters for detecting the person's state of mind, and - the person's state of mind is determined based on the data of the posture and / or the data of the gait pattern and / or the speed of movement., Method according to claim 1, characterized in that the identification of the person takes place via an external device which the person carries with him. Method according to claim 1 or 2, characterized in that during the determination of the position, the interaction intention of the person with the vehicle (400) is recognized on the basis of a direction of movement of the person and via the posture and / or the at least one gesture of the person. Method according to one of the preceding claims, characterized in that a plurality of different emotional states of the person are learned and stored by means of the algorithm and these emotional states are assigned to certain settings of the vehicle (400) and / or vehicle systems and the use of the vehicle (400) and / or vehicle systems. Device for automated device control for use in a vehicle (400), comprising:- a control unit,- at least one sensor (200) for detecting an environment of the vehicle (400),- an evaluation unit,characterized in that the device is designed to carry out the method according to claim 1. Device according to claim 5, characterized in that the at least one sensor (200) uses a CCC-compliant ultra-wideband and / or channel sounding to locate the person.

Images