Control of a motor vehicle

The motor vehicle control system uses hand-based interaction with electromyographic sensors and haptic feedback to simplify and enhance accessibility of vehicle controls, addressing complexity and accessibility issues, and providing a comfortable and private user experience.

WO2024235375A1PCT designated stage expired Publication Date: 2024-11-21BAYERISCHE MOTOREN WERKE AG
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/DE2024/100155
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-16
Filing Date
2024-02-26
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Motor vehicle control systems are complex and prone to user confusion due to numerous controls, leading to increased susceptibility to malfunctions, and existing touch-sensitive screens are not accessible from all locations within the vehicle, making it difficult for passengers in rear seats to interact with central controls.

Method used

A control system using a detection device for the first hand, an electromyographic sensor for the second hand, and a processing device to determine posture or gestures, allowing users to interact with vehicle functions using their hands as a screen and providing haptic feedback, with optional integration into clothing or accessories like smartwatches.

Benefits of technology

Enables intuitive and accessible control of vehicle functions from various locations, reducing user confusion and enhancing comfort with high sensitivity and aesthetic appeal, while maintaining skin integrity and providing privacy through localized feedback.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure DE2024100155_21112024_PF_FP_ABST
    Figure DE2024100155_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The invention relates to a control system (100) on board a motor vehicle (105), which control system comprises: a detection device (135) for detecting a first hand (115) of a person (110) on board the motor vehicle (105); an output device (130) for displaying a predetermined graphic element (405-415, 505) on the first hand (115); an electromyographic sensor (210) for sensing a stimulation of a muscle of a second hand (120) of the person (110); and a processing device (125) which is designed to determine a pose or gesture of the second hand (120) relative to the element (405-415, 505).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Control of a motor vehicle

[0002] The invention relates to the control of a motor vehicle. In particular, the invention relates to a contactless control of a function of the motor vehicle.

[0003] A motor vehicle comprises a multitude of subsystems that can be controlled by a person on board. Examples of such systems include an air conditioning system, a power seat adjustment system, or an entertainment system. The subsystems may be equipped with dedicated controls to control their respective functions. However, the resulting need for multiple controls can confuse a user and increase the vehicle's vulnerability to malfunctions or failure of a control.

[0004] Some motor vehicles include a central control element, which may be located, for example, between a driver's seat and a passenger seat.

[0005] Such a control element can be used to navigate through a predefined menu and select a function that can then be controlled directly. The menu can only reach a predetermined level of complexity to enable operation without searching or thinking.

[0006] Another approach involves a touch-sensitive screen displaying a graphical user interface. A person can touch a displayed element to trigger an associated control function. Such a screen is often not equally accessible from all seats in the vehicle. A passenger in a second or third row of seats may have difficulty or even not be able to reach a screen mounted at the front between the driver and front passenger.

[0007] An object underlying the present invention is to provide an improved operating concept for providing an input on board a motor vehicle. The present invention solves this problem by means of the subject matter of the independent claims. Subclaims specify preferred embodiments. A control system on board a motor vehicle comprises a detection device for detecting a first hand of a person on board the motor vehicle; an output device for displaying a predetermined graphic element on the first hand; an electromyographic sensor for detecting stimulation of a muscle of a second hand of the person; and a processing device configured to determine a posture or gesture of the second hand relative to the element.

[0008] The control system can allow the person to use their first hand as a kind of screen and interact with an object or element displayed on it using their second hand. This allows the person to make a simple or complex input, which can be evaluated, in particular, to select or control a function of the vehicle. Optionally, the concept can also be extended to other parts of the person's body. For example, a person's thigh can be used as a projection surface instead of the first hand. The posture or gesture of the second hand can then be determined relative to this body part.

[0009] The person's interaction with the hand projection can provide real, physical, or tactile haptics in one or both hands. Optionally, this sensation can be supported by haptic feedback. Other forms of sensory feedback, such as audio, visual, etc., are also possible.

[0010] The electromyographic sensor may comprise an electrode for application to the person's skin. More preferably, at least two electrodes are provided, between which an electrical voltage can be determined. Preferably, a plurality of electrodes is used, which can be applied to a predetermined area of ​​the skin. Voltages, each indicative of stimulation of a muscle, can be determined between pairs of electrodes. An electrode may comprise a surface electrode that can rest loosely on the skin. In another embodiment, the sensor may be embedded in a device or a piece of clothing worn by the person. Operation of the control element can be easy and elegant for the person to perform. The wearing comfort of the electromyographic

[0011] The sensor's sensitivity can be high. The person's skin doesn't have to be injured.

[0012] The electromyographic sensor can be configured for attachment to a person's wrist or forearm. The sensor is preferably configured to determine electrical stimulation of a person's muscle. For this purpose, the sensor can be placed in the area of ​​the muscle. In the area of ​​the wrist or forearm, several muscles can be accessible, with which even complex gestures or movements can be controlled by the person. This can make it possible to detect and differentiate between a large number of different muscle activations in this area. In a further embodiment, the electromyographic sensor can also be configured for attachment, for example, in the area of ​​an ankle or on a calf of the person. Activations of different muscles, which may be responsible for different movements of the person's foot, can be detected.The person can wear the wristband on either the left or right wrist. The effect of a gesture can depend on whether it is performed with the left or right hand. The choice of the left or right wrist can be made depending on the person's preference or the person's seat in the vehicle.

[0013] In a particularly preferred embodiment, the electromyographic sensor comprises a wristband. The wristband can be aesthetically pleasing so that it can be perceived as jewelry by the person or another person. In a further embodiment, the sensor can be integrated into a device worn on a wristband, for example a smartwatch or a fitness tracker. The wristband is preferably configured to hold one or more electrodes of the electromyographic sensor on the surface of the person's skin. For this purpose, the wristband can be elastic or comprise an elastic element. Alternatively, an active device can be provided to press an electrode against the skin, for example a pneumatic or thermal actuator. The wristband can be aesthetically pleasing and can also serve as jewelry. The wristband can preferably be worn on a right or left extremity of the person.Several persons on board the motor vehicle can each wear one or more electromyographic sensors.

[0014] The electromyographic sensor is preferably connected wirelessly to the control system. The control device can be located in the area of ​​the sensor or in the area of ​​the motor vehicle. The control system further preferably comprises a conventional control element, which can be used as an alternative to the electromyographic sensor to signal an actuation or release to the control system. Such a control element can, for example, comprise a button, a touch-sensitive surface of a display, or an operating lever.

[0015] Further preferably, the electromyographic sensor is configured to detect the stimulation of a muscle that moves the person's hand. In particular, the sensor can detect activation of a muscle that moves a predetermined portion of the hand, for example, a finger or phalanx.

[0016] Further preferably, the electromyographic sensor is configured to detect a predetermined combination of stimuli from predetermined muscles of the person. The muscles can be located on the same or different extremities of the person.

[0017] When a muscle is stimulated, it contracts and exerts a force on a joint or limb. Different types of contraction can be distinguished. An isometric ("equal measure") contraction or isometric muscle contraction occurs when a muscle only undergoes a change in tension but no change in length. In an isotonic ("equally tensed") contraction, the muscle shortens without a change in force. This can happen, for example, when a weight or object is lifted very slowly over a short distance. An auxotonic ("differently tensed") contraction involves a change in both force and length. This is the most common type of contraction and accompanies a normal movement. Different types of contraction can be distinguished based on the level of stimulation.In this way, not only the position or posture of a limb connected to the muscle can be determined, but also its tension. For example, the hand can grasp an object and hold the object with varying degrees of strength without any significant change in posture. Similarly, the hand can be pressed against an object, such as the surface of an object, with varying degrees of strength. Two sections of the hand can be pressed against each other with varying degrees of strength, for example, two fingers, one of which may enclose the thumb. The person can also stimulate two muscles that oppose each other at the same limb or joint with varying degrees of strength. For example, one muscle may be a flexor and the other an extensor of a finger of the hand.

[0018] Different types or intensities of stimulation can be considered when determining a posture or gesture. These parameters can be evaluated alone or in combination with, for example, a position, an orientation, or a nearby object. Additional sensor signals, such as those indicating a position, acceleration, or orientation, can also be considered. This allows selection from a wide range of parameters that are characteristic for determining a predetermined action or posture.

[0019] In particular, when many signals are collected, for example when many electrodes are used, when different types or strengths of stimulation or contraction of a muscle are recorded, when signals from another sensor or another scanning device are taken into account, or when a sequence of observations is to be evaluated, the recognition of the gesture or posture can be carried out using a learning technique.

[0020] For example, an artificial neural network (ANN) can be trained to recognize a predetermined gesture or posture based on a large number of such observations. The training can take place outside the vehicle. For training, different test subjects can perform a number of gestures or postures to be recognized, while the aforementioned observations are recorded. During training, the ANN can be configured, for example using back propagation, so that it recognizes a posture or gesture in different people. Training usually requires a large number of measurements, each with associated gestures or postures to be recognized. These must be presented to the ANN very frequently in order to achieve good recognition results.

[0021] The required recognition performance of the ANN can depend on a controlled function. Thresholds can be selected for a false positive and a correct positive rate. If the first rate is exceeded and the second rate is simultaneously exceeded, training can be terminated, and the ANN can be used for an evaluation on board a motor vehicle as described herein. Optionally, the ANN can also be further trained while it is being used to recognize gestures or postures on board the motor vehicle. This allows the ANN's function to be adapted to a specific person.

[0022] A person's movement caused by muscular control can be very small. For example, the movement may not be perceptible or barely perceptible to the naked eye. In another embodiment, the movement may be only a few millimeters. This can create the impression that the person is operating the control system solely through thought. The person's immersion in the vehicle's operating concept can be enhanced.

[0023] The control device can be configured to detect the muscular activation of a predetermined gesture. A temporal progression of the muscular activation or activation can be evaluated. For example, movements of different speeds or strengths, or forces exerted by different magnitudes, can be distinguished from one another.

[0024] In one variant, the detection device comprises an additional electromyographic sensor attached to the person's first hand. This allows a force acting between the hands to be determined from two sides, thus increasing the accuracy of the determination. In another embodiment, an electromyographic sensor can be provided only on the first hand; the second hand cannot be scanned for muscle tension.

[0025] In another variant, the detection device comprises a camera or another non-contact imaging sensor. The camera is preferably permanently installed on board the motor vehicle and can comprise an interior camera. A simple camera can be used, or an infrared camera, a stereo camera, a ToF camera, or a depth camera. Alternatively, another imaging sensor can be used, such as a LiDAR sensor, a radar sensor, or an ultrasonic sensor. Various sensors can also be used in combination to determine a position and / or orientation of the first hand.

[0026] Optionally, an acceleration sensor is provided for determining a movement of the second hand. The processing device can be configured to determine the posture or gesture based on an acceleration of the second hand. The acceleration sensor on the second hand can be used to determine an impact of the second hand on the first hand or a manipulation of the second hand relative to the first hand, for example a pulling movement. Furthermore, a movement of the hand in space or with respect to the element can be determined in an improved manner. The acceleration sensor is further preferably embodied micromechanically (MEMS) and can be easily integrated as a semiconductor. In a further development, an acceleration sensor can also be provided on the first hand.

[0027] The input can include a force acting between the first and second hand. The force is preferably determined using the electromyographic sensor. The same posture or gesture can trigger different inputs depending on the respective forces acting. For example, a function can be controlled depending on the force. Furthermore, the function can be controlled depending on a movement.

[0028] In one variant, the output device comprises a projector configured to project the image onto the first hand. This can make it easier or more intuitive for the person to operate the control system. Preferably, the control system is configured to track the display of the element as closely as possible to the movement of the first hand relative to the motor vehicle, making it easier for the person to precisely target or manipulate the element with the second hand.

[0029] The control system can be configured to be used by people at different locations in the vehicle. One or more projectors can be provided to project onto the people's hands.

[0030] In another variant, the output device comprises data glasses assigned to the person. The data glasses can provide the person with a visual representation of the element, allowing the person to experience an impression similar to that of the projection. The visual representation can overlay or replace a natural visual perception of the interior of the motor vehicle. In this variant, another person cannot see the representation on the hand, so that the other person cannot be disturbed, for example if it is dark in the motor vehicle. Conversely, an input made by the person on their hand cannot easily be reproduced by another person on board, because the element is only visible to them. The person's privacy can thus be better protected.

[0031] According to a further aspect, a motor vehicle comprises a control system described herein. The motor vehicle may, for example, comprise a passenger car, a motorcycle, a truck, or a bus.

[0032] A method for controlling a motor vehicle comprises steps of detecting a first hand of a person on board the motor vehicle; displaying a predetermined graphic element on the first hand; detecting stimulation of a muscle of a second hand of the person; and determining a posture or gesture of the second hand relative to the element.

[0033] The method can be carried out using a control system described herein on board a motor vehicle. A control device of the control system can comprise an electronic processing device, for example, a programmable microcomputer or microcontroller. The method can be in the form of a computer program product with program code means. The computer program product can also be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the control system, or vice versa.

[0034] In a preferred embodiment, a function associated with the posture or gesture is determined and preferably also controlled. The assignment of a function to a posture or gesture can be predetermined. Alternatively, the assignment can be created or changed by the person—or another responsible person. Control can involve triggering or executing the function. For example, a gesture can be used to activate or deactivate sound playback, or to adjust the volume of the sound playback louder or quieter.

[0035] The controllable function preferably relates to the motor vehicle. The motor vehicle can have one or more subsystems that can be controlled by the control function. Examples of controllable systems of the motor vehicle include a navigation system, a seat adjustment system, a seat heater, an air conditioning system, lighting, or an entertainment system. However, other systems are also possible. It is also possible to use the control system to control a movement of the motor vehicle. This allows a driver to directly influence the control of the motor vehicle.

[0036] The posture may involve an orientation of the second hand and / or one or more fingers. An example posture may include the thumb and index finger touching; another example posture may include an extended index finger with the middle, ring, and little fingers curled.

[0037] The gesture further preferably comprises one of a swipe, a tap, or a slide of the element. For this purpose, a contact of the second hand with the first, a movement of the second hand relative to the first, and / or a release of the second hand from the first can be detected and evaluated. The invention will now be described in more detail with reference to the accompanying drawings, in which:

[0038] Figure 1 a system;

[0039] Figure 2 shows a flow diagram of a method;

[0040] Figure 3 a bracelet with sensors;

[0041] Figure 4 shows a first exemplary input;

[0042] Figure 5 shows a second exemplary input;

[0043] Figure 6 shows a third exemplary input; and

[0044] Figure 7 illustrates a fourth example input.

[0045] Figure 1 shows a control system 100 on board a motor vehicle 105. A person 110 on board the motor vehicle is typically located at a designated location, for example, a driver's or passenger's seat. The former is typically located in a first row of seats, while the latter can also be located in a second or further row of seats. The control system 100 is preferably configured to control a predetermined control function on board the motor vehicle. Example control functions relate to route selection of the motor vehicle, adjustment of a ventilation system (HVAC: heating, ventilation, air conditioning), opening or closing a window, door, skylight, or convertible top, or operation of an auditory or audiovisual entertainment system.In a further embodiment, the control system 100 can also be used to control another device on board the motor vehicle, for example a device associated with the person 110, such as a smartphone.

[0046] Person 110 has a first hand 115, which is exemplary a left hand, and a second hand 120, which is here a right hand. The control system 100 comprises a processing device 125 connected to an output device 130 and a detection device 135.

[0047] The output device 130 is configured to display a predetermined graphic element on the first hand 115. In the illustrated embodiment, the output device 130 comprises a projector that can be mounted on the motor vehicle 105. The projector 130 can provide a monochrome or color image. A projection area of ​​the projector 130 can be fixedly predetermined, and a location at which the element is displayed by means of the image can be suitably selected within the projection area. In another embodiment, the projector 130 can be moved, for example, pivoted or adjusted in its elevation, so that the location can be changed by changing or aligning the projector 130.

[0048] In another embodiment, the output device 130 comprises data glasses or a data helmet and is typically worn on the head of the person 110. In this case, a representation is only visible to the person 110; another person 110 does not perceive the represented element. The representation can be superimposed on a usual visual impression of the person 110 (augmented reality, AR) or replace it (virtual reality, VR). A location of a representation perceivable by the person 110 by means of the data glasses 125 is typically compensated for a position and / or orientation of their head, so that the representation appears to be related to a surrounding coordinate system, here preferably the coordinate system of the motor vehicle 105.

[0049] In the present embodiment, the detection device 135 comprises a camera or another, preferably contactless, imaging sensor. Using appropriate processing, a position and / or orientation of the first hand 115 can be determined based on the provided sensor data. The output device 130 can be controlled based on the determined pose of the first hand 115 such that the displayed element remains largely stationary relative to the first hand 115 when the first hand is moved relative to the motor vehicle 105.

[0050] Wristbands 140, each of which can carry one or more sensors, can be provided on one or both hands 115, 120. The processing device 125 can comprise a wireless communication device 145 for wirelessly communicating with a wristband 140. A movement, an impulse, and / or a stimulation of a muscle of the respective hand 115, 120 can be determined using a wristband 140. Based on this, the processing device 125 can determine a posture and / or gesture of the hand 115, 120.

[0051] A posture or gesture can be assigned a function, which can then be controlled by the processing device 125. Selecting, starting, ending, or executing the function can be controlled depending on the posture or gesture. Control can be performed via an interface 150, which can be connected, for example, via a communications bus to another system, device, or appliance on board the motor vehicle 105. The bus can comprise, for example, a CAN or LIN bus.

[0052] Figure 2 shows an exemplary embodiment of a wristband 140. It should be noted that the devices described here can also be attached to the person 110 in a manner other than a wristband 140, for example, using some type of glove or a finger ring.

[0053] The wristband 140 comprises a local energy storage device 205, which can be formed, for example, by a rechargeable battery, as well as an electromyographic first sensor 210 and an optional second sensor 215, which is designed as an acceleration sensor. A wireless communication device 220 is also preferably provided for communication with the processing device 125.

[0054] The energy storage device 205 can be charged via a wired connection or wireless charging technology. In one embodiment, a corresponding charging coil can be provided at a predetermined position on the motor vehicle 105. Charging can occur while the person 110 is wearing the EMG sensor 210. For example, the charging coil can be attached in the area of ​​a left or right hand rest or armrest for the person 110. In another embodiment, a charging device can be provided on board the motor vehicle 105, which can be used after the person 110 has removed the EMG sensor 210 and placed it there. The electromyographic (EMG) sensor 210 is configured to electrically determine activation of a muscle that controls a movement or posture of a hand 115, 120 of the person 110.For this purpose, the EMG sensor 210 comprises at least one electrode that is attached to a skin surface in the area of ​​the hand 115, 120 and that detects electrical potentials associated with stimulation of the muscle. The EMG sensor 210 can provide a sensor value that indicates a force transmitted by the muscle of the hand. An electrode can be kept in contact with the skin, for example by mechanical pressure, such as by means of the wristband 140. Preferably, several electrodes are provided that can detect stimulations of several muscles that control different parts of the hand 115, 120, for example, different fingers in different directions. By tracking one or more stimulation levels over a certain time, a gesture that the person 110 performs with their hand 115, 120 can be determined.

[0055] Using the acceleration sensor 215, an acceleration can be determined, and based on this, a movement of the wristband 140 or a hand 115, 120. Gravity can also be determined using the acceleration sensor 215, so that the orientation of the wristband 140 can also be determined. Based on the acceleration, the impact of one hand 115 on or off the other hand 120 can be determined.

[0056] Figure 3 shows a flowchart of a method 300 for controlling a function on board a motor vehicle 105. In a step 305, a position of the first hand 115 of the person 110 on board the motor vehicle is determined. Preferably, a pose is determined, which, in addition to the position, also includes an orientation. The determination can be made based on sensor data from the detection device 135, an electromyographic sensor 210, and / or an acceleration sensor 215, wherein the sensors 210, 215 are attached in the region of the hand 115, 120 in question, for example, by means of a wristband 140.

[0057] In a step 310, a graphic element can be displayed on the first hand 115. For this purpose, the element can be projected onto the hand 115 by means of the output device 130, preferably onto an inside or outside of the hand 115.

[0058] In a step 315, stimulation of one or more muscles of the second hand 120 can be determined. Optionally, stimulation of one or more muscles of the first hand 115 can also be determined. The stimulations preferably indicate forces acting on associated parts of the respective hand 115, 120.

[0059] In parallel, in a step 320, an acceleration acting on the second hand 120 can be determined. Optionally, an acceleration acting on the first hand 115 can also be determined.

[0060] In a step 325, a posture or gesture of the second hand 120 relative to the displayed element can be determined based on electromyographic and / or acceleration-indicating sensor values. The element is displayed on the first hand 115, and the posture or gesture of the second hand 120 relative to the first hand 115 can also be determined in the manner described.

[0061] In a step 330, an input from person 110 can be detected based on the posture or gesture. The input can take into account the context in which the posture or gesture was detected. The context can include an operating context and consider which inputs were previously made, or a context of a controlled device or subsystem of motor vehicle 105. A driving context of motor vehicle 105, i.e., a driving state or a vehicle state, can also be taken into account.

[0062] In a step 335, a function associated with the specific input can be determined. The function can also be determined in a context-sensitive manner. In a step 340, the specific function can be controlled. Controlling can include selecting, activating, influencing, or terminating the function. Figure 4 shows an exemplary first input 400 performed by a person 110 on board a motor vehicle 105. By way of example, the first hand 115 is again the left hand and the second hand 120 the right hand of the person 110. A question 405 is symbolically represented on the first hand 115, which the person 110 can answer haptically. The question can, for example, relate to a system question, such as a security question as to whether the person 110 really wants to perform a previously selected function, even if doing so could entail a disadvantage.Alternatively, the question may also concern a vehicle function or a currently available function, such as accepting or rejecting an incoming call. Such queries are known, for example, in response to a request to delete data.

[0063] Further by way of example, two input areas 410 and 415 are provided below question 405, of which the first input area 410 comprises a confirmation (positive answer) and the second input area 415 comprises a refusal (negative answer). For example, with a finger of the second hand 120, the person can touch one of the input areas 410, 415 and thus make the associated input. The input can be triggered when the finger has fallen below a predetermined minimum distance from an input area 410, 415, when the finger touches an input area 410, 415, when, in addition to a touch, a predetermined minimum force acts between the hands 115, 120, when contact is made with a predetermined minimum speed, and / or when a predetermined acceleration is determined upon contact. Sensor values ​​underlying these facts can be recorded using one or more of the sensors 135, 205, 210.

[0064] Figure 5 shows an exemplary second input 500 performed by a person 110 on board a motor vehicle 105. By way of example, unlike the first input 400 of Figure 4, the first hand 115 is the right hand and the second hand 120 is the left hand of the person 110. In this example, the person 110 may be left-handed, and their left hand 120 may be the dominant hand.

[0065] With the second hand 120, the person 110 can describe an input to the first hand 115. The input can include characters, symbols, or graphic information 505, which can be displayed on the first hand 115 like a writing or drawing surface. The person 110 can use their second hand 120 or a finger thereof like a writing or drawing pen. Where the finger touches the first hand 115, a point or a line can be displayed on the first hand 115. If the finger is drawn along the first hand 115, a line 505 can be displayed on the first hand 115. In this way, the person 110 can write or draw with their one hand 120 on their other hand 115.

[0066] Preferably, a gesture is also predetermined with which a drawn or written input can be manipulated. A first gesture can send the input to the processing device 125 for processing. A second gesture can move the representation 505 of the input on the first hand 115. Content that is moved upwards beyond a boundary of the first hand 115 can no longer be displayed. Thus, the input can be continued further down, where space may have been freed up by the movement.

[0067] Figure 6 shows a third exemplary input 600. By way of example, a keyboard 605 is shown on the first hand 115, in this case in the form of a numeric keypad, such as can be used to dial a telephone number. The keyboard 605 comprises a number of buttons, each of which can be actuated using the second hand 120. Additional input areas, for example for dialing an entered number or for canceling the input, can also be shown on the first hand 115. A number that has already been entered can be shown in a further field on the first hand 115. In a related embodiment, another keyboard 605 can also be shown, for example an alphanumeric keyboard, a keyboard, or a game controller.

[0068] Figure 7 shows a fourth exemplary input 700. On the first hand 115, a list 705 is shown here, which includes a number of entries 710, one or more of which can be selected with the second hand 120. On one side, a vertical scroll bar 715 is shown, with which the displayed list 705 can be moved so that other entries 710 can become visible. In another embodiment, alternatively or additionally, a horizontal scroll bar can be provided at the top or bottom. One or more further buttons, for example for accepting selected entries 710, can also be provided. Accepting an input can generally be controlled by touching a button or by means of a predetermined gesture, for example forming the OK sign, in which the thumb and index finger of the second hand 120 touch each other.Aborting an input can be controlled by another predetermined gesture, for example a throwing motion or by forming a fist with the second hand 120.

[0069] Reference symbol

[0070] 100 Tax system

[0071] 105 Motor vehicle

[0072] 110 people

[0073] 115 first hand

[0074] 120 second hand

[0075] 125 processing facility

[0076] 130 Output device

[0077] 135 Recording device

[0078] 140 Bracelet

[0079] 145 Communication device

[0080] 150 interface

[0081] 205 energy storage

[0082] 210 first sensor, electromyographic sensor

[0083] 215 second sensor, acceleration sensor

[0084] 220 Communication device

[0085] 300 procedures

[0086] 305 Capturing a First Hand Pose

[0087] 310 Project element onto the first hand

[0088] 315 Capture stimulation of a muscle of a second hand

[0089] 320 Determine acceleration on the second hand

[0090] 325 Detect the posture or gesture of the second hand towards the element

[0091] 330 Enter input

[0092] 335 determine assigned function

[0093] 340 Control function

[0094] 400 first entry

[0095] 405 Question

[0096] 410 first input area

[0097] 415 second input area

[0098] 500 second input

[0099] 505 characters, symbols, graphic information 600 third input

[0100] 605 Keyboard

[0101] 700 fourth input

[0102] 705 List

[0103] 710 Roll bars

Claims

Claims 1. A control system (100) on board a motor vehicle (105), the control device comprising the following elements (405-415, 505): a detection device (135) for detecting a first hand (115) of a person (110) on board the motor vehicle (105); an output device (130) for displaying a predetermined graphic element (405-415, 505) on the first hand (115); an electromyographic sensor (210) for detecting stimulation of a muscle of a second hand (120) of the person (110); and a processing device (125) configured to determine a posture or gesture of the second hand (120) relative to the element (405-415, 505).

2. The control system (100) of claim 1, wherein the electromyographic sensor (210) comprises an electrode for application to the skin of the person (110).

3. The control system (100) of claim 1 or 2, wherein the electromyographic sensor (210) is configured to be mounted on a wrist or a forearm of the person (110).

4. The control system (100) of any preceding claim, wherein the electromyographic sensor (210) comprises a wristband.

5. Control system (100) according to one of the preceding claims, wherein the detection device (135) comprises a further electromyographic sensor (210) attached to the first hand (115) of the person (110).

6. Control system (100) according to one of claims 1 to 4, wherein the detection device (135) comprises a camera.

7. Control system (100) according to one of the preceding claims, further comprising an acceleration sensor (215) for determining a movement movement of the second hand (120); wherein the processing device (125) is configured to determine the posture or gesture based on an acceleration of the second hand (120).

8. The control system (100) of any preceding claim, wherein the input comprises a force acting between the first and second hands (120).

9. Control system (100) according to one of the preceding claims, wherein the output device (130) comprises a projector configured to project the image onto the first hand (115).

10. Control system (100) according to one of the preceding claims, wherein the output device (130) comprises data glasses assigned to the person (110).

11. Motor vehicle (105) comprising a control system (100) according to one of the preceding claims.

12. A method (300) for controlling a motor vehicle (105), the method (300) comprising the following steps: Detecting (305) a first hand (115) of a person (110) on board the motor vehicle (105); Displaying (310) a predetermined graphic element (405-415, 505) on the first hand (115); detecting (315) a stimulation of a muscle of a second hand (120) of the person (110); Determining (325) a posture or gesture of the second hand (120) towards the element (405-415, 505).

13. The method (300) according to claim 12, wherein a function associated with the posture or gesture is determined (335) and / or controlled (340).

14. The method (300) according to claim 13, wherein the function relates to the motor vehicle (105).

15. The method (300) of any one of claims 12 to 14, wherein the gesture comprises one of a swipe, a tap, or a slide of the element (405-415, 505).

Citation Information

Patent Citations

  • Method for contactless interaction with a module, computer program product, module and motor vehicle

    DE102019206606A1

  • Mobile terminal and vehicle control

    EP2945137A1

  • System and methods for on-body gestural interfaces and projection displays

    US20170123487A1

  • Universal input / output human user interface

    WO2011055326A1