Controlling a function of a motor vehicle
The control system uses hand position and gesture detection with electromyographic sensors and machine learning to allow intuitive, hands-free operation of motor vehicle functions, addressing the challenges of complex control systems and enhancing safety and usability.
Patent Information
- Application Number
- PCT/DE2024/100148
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-16
- Filing Date
- 2024-02-23
- Publication Date
- 2024-11-21
AI Technical Summary
Modern motor vehicles with numerous operable elements present challenges in control, particularly for drivers who must navigate complex menus or learn new gestures, risking distraction and safety.
A control system utilizing a detection device, electromyographic sensors, and a processing facility to determine hand position and gestures, allowing intuitive control of vehicle functions without physical contact or visual attention, using a combination of sensors and machine learning for accurate recognition.
Enables safe and efficient operation of vehicle controls by allowing hands-free operation, reducing the need to look at controls, and simplifying the learning of gestures, enhancing safety and usability.
Smart Images

Figure DE2024100148_21112024_PF_FP_ABST
Abstract
Description
[0001] Controlling a function of a motor vehicle
[0002] The present invention relates to the control of a motor vehicle. In particular, the invention relates to the control of a function of the motor vehicle.
[0003] A motor vehicle comprises numerous elements that can be controlled by a person on board the vehicle. Some of these elements affect the driving of the vehicle or are necessary for its operation. This may include, for example, activating or deactivating a direction indicator or a windshield wiper. Other elements primarily affect the interior of the vehicle and may, for example, relate to a navigation system, an entertainment system, or seat adjustment. Still other elements may include the use or configuration of an assistance system, such as an automatic distance control or an automatic braking system.
[0004] With the increasing number of operable elements in modern motor vehicles, the number of assigned controls is also increasing. It has been proposed to provide a central control element that can be dynamically controlled haptically, for example, by moving, rotating, or pressing to select and control predetermined functions. This allows a variety of controls to be controlled purely haptically, without the operator having to look at the control element. However, it can be time-consuming for the operator to learn a predetermined menu structure. Locating a function that the person is unaware of can be difficult.
[0005] Another operating concept provides an interactive graphical user interface. This typically involves a touch-sensitive screen on which selectable or controllable elements are displayed. By touching, dragging, rotating, etc., the operator can select an operable element and control a predetermined function. However, this typically requires the operator to look at the screen, so menu navigation can be risky for a driver operating the vehicle. One object underlying the present invention is to provide an improved technology for controlling a function of a motor vehicle. The invention solves this problem by means of the subject matter of the independent claims. Subclaims specify preferred embodiments.
[0006] A proposed control system on board a motor vehicle comprises a detection device for determining the position of a person's hand on board the motor vehicle; an electromyographic sensor attached to the person for detecting stimulation of a muscle for moving the person's hand; and a processing device. The processing device is configured to determine a controllable element of the motor vehicle in the area of the hand and to control the element depending on a posture or gesture of the hand.
[0007] The control system can be provided as an alternative to or in addition to a conventional control system. The person can bring their hand close to an element to be operated and trigger a predetermined function using a predetermined hand position or hand gesture, allowing intuitive operation of the element. To select or control the function, it is also possible to evaluate how close the hand is to the element, the direction in which the hand is pointing relative to the element, the orientation of the hand, and / or whether the person is pointing at something or at what they are pointing.
[0008] The person does not have to look in the direction of the element or their hand and can devote their attention to other things. If the person is a driver of the motor vehicle, they can, in particular, keep their gaze on the surroundings of the motor vehicle while operating the element. A variety of hand positions or hand gestures can be predefined, allowing a variety of functions of an element to be controlled.
[0009] This can replace a variety of conventional switches, levers, wheels, buttons, or other input elements. A gesture can be easy to learn, allowing for simple and safe operation. It is preferable that the same gestures or actions are supported for comparable functions of different elements. This can reduce the effort required to learn postures or gestures. The control system does not require physical contact between the person's hand and an input element. In particular, the person does not have to touch or mechanically move any control element with their hand. The electromyographic sensor can operate very sensitively and determine different levels of stimulation or activation of one or more of the person's muscles. This enables sensitive and safe control of the element's function.
[0010] The electromyographic sensor may comprise an electrode for application to the person's skin. More preferably, at least two electrodes are provided, between which an electrical voltage can be determined. Preferably, a plurality of electrodes is used, which can be applied to a predetermined area of the skin. Voltages, each indicative of stimulation of a muscle, can be determined between pairs of electrodes. An electrode may comprise a surface electrode that can rest loosely on the skin. In another embodiment, the sensor may be embedded in a device or a piece of clothing worn by the person. Operation of the control element can be easy and elegant for the person. Wearing the electromyographic sensor can be very comfortable. The person's skin does not have to be injured.
[0011] When a muscle is stimulated, it contracts and exerts a force on a joint or limb. Different types of contraction can be distinguished. An isometric ("equal measure") contraction or isometric muscle contraction occurs when a muscle only undergoes a change in tension but no change in length. In an isotonic ("equally tensioned") contraction, the muscle shortens without a change in force. This can happen, for example, when a weight or object is lifted a short distance very slowly. An auxotonic ("differently tensioned") contraction involves a change in both force and length. This is the most common type of contraction and accompanies a normal movement.
[0012] Based on a determination of the stimulation, different types of contraction can be distinguished. This allows not only the position or posture of a limb connected to the muscle to be determined, but also its tension. For example, the hand can grasp an object and hold the object with varying degrees of strength without any significant change in posture. Similarly, the hand can be pressed against an object, for example, the surface of an object, with varying degrees of strength. Two sections of the hand can be pressed against each other with varying degrees of strength, for example, two fingers, one of which may encompass the thumb. The person can also stimulate two muscles with different degrees of strength that counteract each other at the same limb or joint. For example, one muscle may comprise a flexor and the other an extensor of a finger of the hand.
[0013] Different types or intensities of stimulation can be considered when determining a posture or gesture. These parameters can be evaluated alone or in combination with, for example, a position, an orientation, or a nearby object. Additional sensor signals, such as those indicating a position, acceleration, or orientation, can also be considered. This allows selection from a wide range of parameters that are characteristic for determining a predetermined action or posture.
[0014] In particular, when many signals are collected, for example when many electrodes are used, when different types or strengths of stimulation or contraction of a muscle are recorded, when signals from another sensor or another scanning device are taken into account, or when a sequence of observations is to be evaluated, the recognition of the gesture or posture can be carried out using a learning technique.
[0015] For example, an artificial neural network (ANN) can be trained to recognize a predetermined gesture or posture based on a large number of such observations. The training can take place outside the vehicle. For training, different test subjects can perform a number of gestures or postures to be recognized, while the aforementioned observations are recorded. During training, the ANN can be configured, for example using back propagation, to recognize a posture or gesture in different people. Training usually requires a large number of measurements, each with associated gestures or postures to be recognized. These must be presented to the ANN very frequently in order to achieve good recognition results.
[0016] The required recognition performance of the ANN can depend on a controlled function. Thresholds can be selected for a false positive and a correct positive rate. If the first rate is exceeded and the second rate is simultaneously exceeded, training can be terminated, and the ANN can be used for an evaluation on board a motor vehicle as described herein. Optionally, the ANN can also be further trained while it is being used to recognize gestures or postures on board the motor vehicle. This allows the ANN's function to be adapted to a specific person.
[0017] The electromyographic sensor can detect even very small stimulation of the muscle. The stimulation can be so slight that there is barely any perceptible movement of the hand. It can also be determined that part of the hand is merely tensed without being moved. This allows two muscles with opposing functions to be stimulated simultaneously. Even if the hand is resting on an object and increased stimulation causes a section of the hand to be pressed more firmly against the object, any movement is practically undetectable from the outside. It is therefore possible to control the element or a function of the element very unobtrusively, so that from the outside the impression of purely mental control can arise.
[0018] The electromyographic (EMG) sensor can be configured for attachment to a person's wrist or forearm. Muscles required to move a finger or the entire hand may be located above a wrist, allowing particularly insightful data about hand posture or movement to be collected in this area.
[0019] The electromyographic sensor may comprise a wristband for attachment to the person's wrist. The person may wear the wristband on either the left or right wrist. The effect of a gesture may depend on whether it is performed with the left or right hand. The selection of the left or right wrist may be made depending on the person's preference or the person's seating position in the motor vehicle.
[0020] The bracelet can be aesthetically pleasing so that it can be perceived as jewelry by the person or another person. In a further embodiment, the sensor can be integrated into a device worn on a bracelet, for example a smartwatch or a fitness tracker. The bracelet is preferably configured to hold one or more electrodes of the electromyographic sensor on the surface of the person's skin. For this purpose, the bracelet can be elastic or comprise an elastic element. Alternatively, an active device can be provided to press an electrode against the skin, for example a pneumatic or thermal actuator. The bracelet can be aesthetically pleasing and can also serve as jewelry. The bracelet can preferably be worn on a right or left extremity of the person. Several people on board the motor vehicle can each wear one or more electromyographic sensors.
[0021] In a first variant, the detection device for determining the position of the hand is attached to the motor vehicle. The detection device can, for example, comprise a camera, an infrared camera, a ToF camera, a stereo camera, a LiDAR sensor, a radar sensor, or an ultrasonic sensor. Other sensors are also possible. In one embodiment, the wristband provides a marker whose position can be determined by the detection device. The marker can be passive and, for example, comprise a reflector that reflects a signal from the vehicle-based detection device. For optical scanning, an optical retroreflector or a so-called cat's eye can be used. For other scanning types, corresponding reflectors can be used. The marker can also be active, in which case a predetermined signal can be emitted from the wristband. The signal can, for example, be optical or acoustic.The vehicle-based detection device can easily determine the position of the marker or an element emitting the marker on the wristband. In another variant, the detection device is attached to the wristband. For example, an inertial platform (inertial measurement unit, IMU) can be used to determine the position of the wristband. In another embodiment, one or more markers attached to the motor vehicle can be detected from the wristband, so that the position of the wristband can be determined based on the detected markers. The processing device can be configured to determine a position, an orientation, a gesture, or a posture of the hand depending on acceleration data from the acceleration sensor.
[0022] It should be noted that several variants or embodiments described herein can be combined with one another, for example by determining the position of the bracelet or hand on the basis of camera data from a camera attached to the motor vehicle and additionally on the basis of acceleration data from an acceleration sensor attached to the bracelet or hand.
[0023] A process of selecting an element or a function of the element and controlling the function can be supported by providing appropriate feedback to the person. Any modality of the motor vehicle, the element, or the wristband can be used to provide feedback. In a simple embodiment, instrument or accent lighting can be used to highlight a selected element. A controllable element can generally be comprised of a graphical user interface (GUI) for controlling a function.
[0024] Examples of controllable elements may include an entertainment system, a climate control system (heating, ventilation, air conditioning, HVAC), a navigation system, or a communication system. Optionally, gestures can be determined that can be assigned to a selectable function. For example, approximately eight predetermined gestures can be assigned functions that the user can select from a predetermined list. A function can also include a sequence of control interventions. The control system can include an output device for providing the person with an indication regarding the control of the element. The display device can be attached to a predetermined location on the motor vehicle and is preferably designed as a visual display device. The display device can be viewed by one or more people.In another embodiment, a customized display device can be used, which can be worn by the person on their head, for example, VR glasses. The display device can provide feedback regarding the selection of an element or function; moreover, an indication of other possible elements or functions can be given. This can facilitate learning locations within the motor vehicle where controllable elements are located, or hand positions or hand gestures for controlling such elements. In other embodiments, the output device can also be configured to output acoustic or haptic feedback to the person.
[0025] In one embodiment, the processing device is configured to provide an indication of the controllable element. For example, a visual display can be generated indicating controllable elements. The highlighting can also be provided in other ways, for example, acoustically or haptically; an airflow, a temperature, or an odor can also be provided as an indication to the person. This feedback can be different from the specific controlled function.
[0026] In a further embodiment, the processing device is configured to provide an indication of a controllable function of the element. In addition, an indication of a hand posture or a hand gesture can be provided, with which the function can be controlled. The indication can be provided graphically, for example by graphically displaying a corresponding hand posture or gesture. In the case of a gesture, an animation can be shown. The posture or gesture can generally correspond to a GUI display of an active feature or function. For example, the GUI element corresponding to the hand position or hand orientation can be highlighted. The person can also be at a greater distance from the controllable element. In particular, the element can be located outside the person's usual reach. For example, a person in the second row of seats can control an element that is located in or in front of the first row of seats.To select such an element, the processing device can be configured to determine the element based on the direction in which the person points their hand. This allows for intuitive and accurate selection of the element. Certain vehicle functions can also be used or controlled from a greater distance from different positions or people located there.
[0027] The control system may comprise a detection device for determining a person's line of sight. The processing device may be configured to additionally determine the element depending on the line of sight.
[0028] The detection device for determining the line of sight can, for example, comprise an indoor camera. The line of sight can be determined based on an optical scan of the person, their head, and / or at least one of their eyes. Optionally, the camera operates in the infrared range and can include infrared illumination of the person. In another embodiment, the detection device for determining the line of sight is attached to the person, for example, as a head-worn device.
[0029] The detection device for determining the gaze axis can be integrated with the detection device for determining the position of the hand. For example, a camera or an infrared camera can be used to determine the gaze axis. An element can be determined as selected if the gaze axis intersects the element or a predetermined area associated with the element. A similar determination can be made regarding the direction in which the person is pointing their hand. In a preferred embodiment, the gaze axis and the pointed direction are combined to determine a selected element.
[0030] The processing device can be configured to improve user guidance based on machine learning methods. The processing device can adapt the assignment of a hand position to a controllable element or a posture or gesture of the hand to a function using machine learning. The adaptation can be specific to the individual person. This allows different adaptations to be learned for different people. This can significantly improve the accuracy of recognizing an element or controlling a function of the element.
[0031] A motor vehicle comprises a control system described herein. The motor vehicle is preferably configured to transport one or more persons. The motor vehicle may, in particular, comprise a motorcycle, a passenger car, a truck, or a bus.
[0032] A method for controlling a motor vehicle comprises steps of determining a position of a hand of a person on board the motor vehicle; detecting stimulation of a muscle for moving the person's hand by means of an electromyographic sensor attached to the person; determining a controllable element of the motor vehicle in the region of the hand; and controlling the element depending on a posture or gesture of the hand.
[0033] The method can be carried out, in particular, by means of a control system described herein. For this purpose, the control system can comprise an electronic processing device, for example, a programmable microcomputer or microcontroller. The method can be in the form of a computer program product with program code means. Features or advantages of the method can be transferred to the control system, or vice versa.
[0034] The invention will now be described in more detail with reference to the accompanying drawings, in which:
[0035] Figure 1 shows a control system on board a motor vehicle;
[0036] Figure 2 shows a wristband with an electromyographic sensor;
[0037] Figure 3 is a flowchart of a method;
[0038] Figure 4 shows exemplary controllable elements on board a motor vehicle; and
[0039] Figure 5 illustrates exemplary gestures for controlling functions.
[0040] Figure 1 shows a control system 100 on board a motor vehicle 105. The control system 100 is configured to facilitate or enable the operation of a predetermined element 115 for a person 110 on board the motor vehicle 105. The element 115 is illustrated here as an example as a ventilation grille and symbolizes a ventilation, heating, or air conditioning system. The person 110 can in particular comprise a driver or a passenger. As a passenger, the person 110 can be located in a first, second, or further row of seats in the motor vehicle 105. The element 115 can serve to control the motor vehicle 105, for example a drive, a lighting system, or a door system, or to perform an additional function, for example a navigation function, air conditioning, heating, or an entertainment system.
[0041] The control system 100 comprises a detection device 120, a processing device 125 and a sensor 130 which is mounted in the area of a hand 135 of a person 110 on board the motor vehicle 105.
[0042] The detection device 120 is configured to determine a position of the hand 135 on board the motor vehicle 105. Preferably, an orientation of the hand 135 is also determined, so that an overall pose is present. The detection device 120 can, for example, comprise a camera with which the hand 135 can be recognized and positioned. In another embodiment, the detection device 120 can position the sensor 130 or an element connected thereto instead of the hand. The sensor 130 comprises an electromyographic sensor attached to a surface of the skin of the person 110. By means of the sensor 130, a stimulation of a muscle that can move or hold the hand 135 can be determined. In this case, even small changes in a state of tension can be resolved. The sensor 130 can be configured to determine stimulations of multiple muscles, each of which can control different parts of the hand 135.In this case, the sensor 130 is attached to a wristband 140, which the person 110 can wear on either the right or left wrist. In the illustration of Figure 1, both persons 110 are each wearing a wristband 140; the function of the control system 100 is explained by way of example with reference to the person shown on the right.
[0043] The detection device 120 or another detection device can be configured to determine a gaze axis 145 of the person 110. For this purpose, a position and / or an orientation of the head of the person 110 can be determined. A position of one or both eyes of the person 110 can also be determined. The gaze axis 145 follows a gaze of the person 110 and originates from their head.
[0044] In a corresponding manner, the detection device 120 or another detection device can be configured to determine a pointing direction 150 of the person 110. The person 110 can point with their hand 135 at an element 115 in the motor vehicle 105. In doing so, they typically move the hand 135 away from their body. The pointing direction 150 typically runs through the hand 135 and, in various embodiments, can originate in the region of an elbow joint, a shoulder, or the head of the person 110. In general, the pointing direction 150 runs through an element 115 to which the person 110 is pointing.
[0045] The processing device 125 is configured to receive information from the detection device 120 and the sensor 130 and optionally to determine the viewing axis 145 and / or the pointing direction 150. Based on this information, the processing device 125 determines an element 115 near which the hand 135 of the person 110 is located, which element is intersected by the viewing axis 145 and / or the pointing direction 150. Furthermore, the processing device 125 determines a posture or gesture of the hand 135 based on information from the sensor 130. The processing device 125 can receive corresponding sensor information via a preferably wireless communication device 155. An output device 160 can assist in selecting an element 115, selecting a function of the element 115, and / or controlling this function by outputting feedback to the person 110.The output device 160 can be optical and comprise a display device. In the illustrated embodiment, the output device 160 is shown as a panoramic head-up display (BMW Panoramic Vision). In other embodiments, any other elements on board the motor vehicle 105 can be used to output feedback to the person 110. This includes, for example, a lighting element, an acoustic element, or a haptic element of the motor vehicle 105. An output device 160 can also be provided on the wristband 140.
[0046] The processing device 125 can control the element 115 of the motor vehicle 105 via an interface 165. Optionally, feedback can be provided via the interface 165, which may, for example, concern a state or an option of the element 115. The feedback can be further processed to operate the element 115 or to inform the person 110.
[0047] Figure 2 shows an exemplary wristband 140 that a person 110 can wear on board the motor vehicle 105. The wristband 140 can be attached to the hand 135, an adjoining wrist, or the forearm of the person 110. The wristband 140 includes the electromyographic sensor 130 and preferably a power supply 205 and a preferably wireless communication device 210. Furthermore, an actuator 215 can be provided. Furthermore, an acceleration sensor 220 can be provided, which is configured to determine an acceleration of the hand 130. Based on the acceleration, a movement or a position of the hand 130 can be derived.
[0048] The energy supply 205 preferably comprises a rechargeable battery, which can be charged, for example, via an on-board electrical system of the motor vehicle 105. In one embodiment, wireless charging can occur, for example, using a charging coil near which the wristband 140 can be held or placed. The communication device 210 is configured to communicate with the communication device 155 of the processing device 125. The communication is preferably bidirectional and can be carried out using conventional methods, such as Bluetooth, WLAN, or ZigBee. The actuator 215 is configured to provide feedback to the person 110. The actuator 215 can, in particular, be haptic, electrical, optical, or acoustic. Electrical feedback can occur by applying a predetermined, low stimulus current to the skin of the person 110.Figure 3 shows a flowchart of an exemplary method 300 for controlling a function of an element 115 on board a motor vehicle 105. The method 300 can be carried out in particular by means of a control system 100.
[0049] In a step 305, a position and / or orientation of the hand 135 of a person 110 on board the motor vehicle 105 can be determined. Optionally, the pointing direction 150 can also be determined in a step 310.
[0050] Also optionally, the gaze axis 145 can be determined in a step 315. In a step 318, stimulation of a muscle that controls the hand 135 can be detected. For this purpose, one or more sensor values from the electromyographic sensor 130 can be evaluated. Sensor values over a predetermined period of time can be taken into account in the evaluation.
[0051] In a step 320, a controllable element 115 on board the motor vehicle 105 can be determined based on the collected information. In one embodiment, the element 115 is determined based on the proximity of the hand 135 or the wristband 140. In other words, an element 115 can be considered selected if its distance from the hand 135 or the wristband 140 is below a predetermined threshold. In particular, if this determination is ambiguous or the element 115 is located outside a reachable distance from the person 110, the element 115 can also be determined based on the viewing axis 145 and / or the pointing direction 150.
[0052] In a step 325, feedback can be provided to the person 110. The feedback can, for example, relate to an available or selected element 115. Furthermore, the feedback can represent one or more functions of the element 115. For an available function, a hand posture or a gesture can be represented, with which the function can be triggered or controlled. It should be noted that the feedback is not necessarily provided via a dedicated output device 160, but can be provided via virtually any actuator that can be controlled on board the motor vehicle 105. Furthermore, feedback can be provided via the actuator 215 on the wristband 140. In a step 330, a posture or gesture performed with the hand 135 can be determined. Based on the hand posture or hand gesture, a selected function of the selected element 115 can be controlled in a step 335.
[0053] Determinations of steps 320 to 330 can be determined or associated with a vehicle state, which can be determined in a step 340. For example, different functions can be available in different vehicle states. A function can also be controllably modified depending, for example, on a driving speed. Other dependencies are also conceivable.
[0054] Figure 4 shows exemplary controllable elements 115 on board a motor vehicle 105. For improved clarity, no persons 110 are shown. However, the hand 135 with the wristband 140 is indicated symbolically at various locations in the motor vehicle 105.
[0055] A first element 405 relates to a rearview mirror, in particular an interior mirror. Available functions include, for example, dimming, adjusting the interior mirror or a right or left exterior mirror, or interior lighting. Another element 410 is designed as a panoramic head-up display of the type shown in the output device 160 of Figure 1.
[0056] Another element may be a steering wheel 415, which a driver can use for lateral control of the motor vehicle 105. A variety of functions can be controlled on the steering wheel 415, which may relate to various elements on board the motor vehicle 105, for example, an entertainment system or a navigation system.
[0057] A central information display (CID) 420 can be provided in the area of a dashboard. Another element can be a ventilation or air conditioning system (HVAC) 425. For example, an interior temperature of the motor vehicle 105, a distribution of air flows, the choice between fresh air and recirculated air, or a distribution of air flows (dual) in the interior can be controlled. A footwell 430 is shown as an example as a further element. With respect to the footwell 430, for example, lighting or air conditioning can be controlled.
[0058] Yet another element is formed by an armrest 435, which can be located between adjacent seats for a driver and a passenger.
[0059] Figure 5 shows exemplary gestures 500 for controlling a function on board a motor vehicle 105. An exemplary right hand 135 of a person 110 comprises five fingers, of which the thumb is designated as the first finger, the index finger as the second finger, the middle finger as the third finger, the ring finger as the fourth finger, and the little finger as the fifth finger. Various exemplary hand positions or gestures will be explained using the first finger as an example.
[0060] The first finger can be spread away from the palm (abduction) or drawn toward the palm (adduction). The finger can also be bent (flexion) or stretched (extension). Other possible movements include touching the opposite fifth finger (opposition) or moving in a circular motion (circumduction).
[0061] In a first gesture 505, hand 135 is depicted in a basic position, with fingers 2 through 4 parallel and the thumb slightly spread out. In a second gesture 510, an opening or closing motion can be performed with the thumb. This gesture 510 can generally be associated with a rejection, as may be familiar from operating a smartphone when rejecting an incoming call.
[0062] A third gesture 515 can comprise a bending of the second finger, in particular a flexion / extension, wherein the third gesture 515 can be associated with an acceptance, such as can be used on a smartphone to accept an incoming call. The gestures 505 to 515 can generally be used for a neutral, negative, or positive input. Depending on an existing context, in particular a selected element 115, a selected function of this element 115, and possibly a vehicle state, the respective function can be controlled with these gestures 505 to 515. Further gestures can be available for function control.
[0063] Reference symbol
[0064] 100 Tax system
[0065] 105 Motor vehicle
[0066] 110 people
[0067] 115 elements
[0068] 120 recording device
[0069] 125 processing facility
[0070] 130 electromyographic sensor
[0071] 135 Hand
[0072] 140 Bracelet
[0073] 145 Line of sight
[0074] 150 Pointing direction
[0075] 155 Communication device
[0076] 160 Output device
[0077] 165 Interface
[0078] 205 Energy supply
[0079] 210 Communication device
[0080] 215 Actuator
[0081] 220 acceleration sensor
[0082] 300 procedures
[0083] 305 Determine hand position
[0084] 310 determine the direction indicated
[0085] 315 Determine the line of sight
[0086] 318 Capture stimulation
[0087] 320 determine controllable element
[0088] 325 Provide feedback
[0089] 330 Determine hand position / gesture
[0090] 335 Control the function of the element
[0091] 340 Determine vehicle condition 405 Rearview mirror
[0092] 410 BMW Panoramic Vision
[0093] 415 Steering wheel 420 Central display (CID)
[0094] 425 HVAC
[0095] 430 foot space
[0096] 435 Armrest 500 Gesture
[0097] 505 first gesture, basic position
[0098] 510 second gesture, rejection
[0099] 515 third gesture, acceptance
Claims
Claims 1. A control system (100) on board a motor vehicle (105), the control system (100) comprising the following elements (115): a detection device (120) for determining a position of a hand (135) of a person (110) on board the motor vehicle (105); an electromyographic sensor (130) attached to the person (110) for detecting stimulation of a muscle for moving the hand (135) of the person (110); a processing device (125) configured to determine a controllable element (115) of the motor vehicle (105) in the region of the hand (135); and to control the element (115) depending on a posture or gesture of the hand (135).
2. The control system (100) of claim 1, wherein the electromyographic sensor (130) comprises an electrode for application to the skin of the person (110).
3. Control system (100) according to claim 1 or 2, wherein the electromyographic sensor (130) is adapted to be attached to a wrist (135) or a forearm of the person (110).
4. The control system (100) of any preceding claim, wherein the electromyographic sensor (130) comprises a wristband (140) for attachment to a wrist (135) of the person (110).
5. The control system (100) of claim 4, wherein the detection device (120) is mounted on the motor vehicle (105); wherein the wristband (140) provides a marker whose position can be determined by the detection device (120).
6. The control system (100) of claim 5, wherein the detection device (120) is attached to the wristband (140).
7. The control system (100) according to any one of claims 4 to 6, wherein an acceleration sensor (220) is attached to the wristband (140); wherein the processing device (125) is configured to determine a position, an orientation, a gesture, or a posture of the hand (135) depending on acceleration data from the acceleration sensor (220).
8. Control system (100) according to one of the preceding claims, further comprising an output device (160) for providing an indication regarding the control of the element (115) to the person (110).
9. Control system (100) according to one of the preceding claims, wherein the processing device (125) is arranged to provide an indication of the controllable element (115).
10. Control system (100) according to one of the preceding claims, wherein the processing device (125) is arranged to provide an indication of a controllable function of the element (115).
11. Control system (100) according to one of the preceding claims; wherein the processing device (125) is configured to determine the element (115) depending on a direction (150) in which the person (110) points with the hand (135).
12. The control system (100) according to any one of the preceding claims, further comprising a detection device (120) for determining a viewing axis of the person (110); wherein the processing device (125) is configured to additionally determine the element (115) as a function of the viewing axis (145).
13. Control system (100) according to one of the preceding claims, wherein the processing device (125) is configured to adapt an assignment of a position of the hand (135) to a controllable element (115) or a posture or gesture of the hand (135) to a function by means of machine learning.
14. Motor vehicle (105) comprising a control system (100) according to one of the preceding claims.
15. Method (500) for controlling a motor vehicle (105), the method comprising the following steps: Determining (305) a position of a hand (135) of a person (110) on board the motor vehicle (105); Detecting (318) a stimulation of a muscle to move the hand (135) the person (110) by means of an electromyographic sensor (130) attached to the person (110); Determining (320) a controllable element (115) of the motor vehicle (105) in the area of the hand (135); and Controlling (335) the element (115) depending on a posture or gesture of the hand (135).
Citation Information
Patent Citations
Neuromuscular control of physical objects in an environment
US20200097083A1
Methods and apparatuses for low latency body state prediction based on neuromuscular data
US20220269346A1
Neuromuscular electrical stimulation controlled by computer vision
US20230068682A1
