Steering wheel interaction device and vehicle

Abstract

The utility model discloses a steering wheel interaction device and a vehicle. The steering wheel interaction device comprises a steering wheel body, a processing unit and at least one event camera. At least one event camera is connected with the processing unit and is arranged on a spoke and / or an air bag cover of the steering wheel body. Therefore, non-contact human-vehicle interaction between the user and the vehicle is realized.

Description

Technical Field

[0001] The present application relates to the technical field of automobile steering wheels, and in particular to a steering wheel interaction device and a vehicle. Background Art

[0002] With the development of intelligent driving technology, the human-computer interaction function of the steering wheel in the vehicle has gradually been upgraded from traditional button-type and gear-shift operation to contactless operation to achieve more intelligent and convenient human-computer interaction.

[0003] To enhance the user interaction experience, steering wheels are equipped with cameras that capture and recognize gesture images containing user interaction gestures to implement various control functions. In high-light environments, cameras often struggle to capture clear gesture images, reducing gesture recognition accuracy. Furthermore, to better capture user interaction gestures, cameras often capture the user's face, which poses certain privacy and security concerns. Utility Model Content

[0004] The present application provides a steering wheel interaction device and a vehicle.

[0005] A technical solution adopted by the present application is to provide a steering wheel interaction device; the steering wheel interaction device includes:

[0006] a steering wheel body, a processing unit, and at least one event camera;

[0007] At least one event camera is connected to the processing unit and is arranged on the spokes of the steering wheel body and / or the airbag cover.

[0008] Optionally, the steering wheel interaction device further includes at least one ultrasonic sensor array, which is connected to the processing unit and is arranged on the spokes and / or the airbag cover.

[0009] Optionally, the steering wheel interaction device further includes at least one millimeter wave sensor, which is connected to the processing unit and is disposed on the spokes and / or the airbag cover.

[0010] Optionally, the steering wheel body further comprises a rim, and the airbag cover is connected to the rim via spokes;

[0011] The steering wheel interaction device includes a plurality of capacitive sensors, the plurality of capacitive sensors are connected to the processing unit, and the plurality of capacitive sensors are arranged on the wheel rim.

[0012] Optionally, the capacitive sensor is in sheet form and is attached to the surface of the rim.

[0013] Optionally, a plurality of capacitive sensors are arranged at intervals along the circumference of the rim.

[0014] Optionally, the steering wheel interaction device includes a vibration motor, which is connected to the processing unit and is arranged on the spoke.

[0015] Optionally, the steering wheel interaction device includes a light bar, which is connected to the processing unit and is arranged on the spokes and / or the rim.

[0016] Another technical solution adopted in the present application is to provide a vehicle, which includes the steering wheel interaction system as described above.

[0017] Optionally, the vehicle further includes an intelligent cockpit system, and the steering wheel interaction device is coupled to the intelligent cockpit system via a CAN bus.

[0018] The beneficial effects of this application are: Unlike traditional cameras, event cameras installed on spokes and / or airbag covers generate images based solely on brightness changes of pixels within their field of view. Therefore, clear gesture images can be captured even in high-illuminance scenarios, improving the recognition accuracy of the steering wheel interaction device. Furthermore, because event cameras do not capture specific facial images, user privacy is better protected. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 This is a schematic diagram of the signal output of the event camera provided by this application and a conventional CCD / CMOS camera under a rotating disk;

[0021] Figure 2 This is a schematic diagram of the signal output of the event camera provided by this application and a conventional CCD / CMOS camera when the disk changes from stationary to rotating;

[0022] Figure 3 These are non-event images obtained by a conventional CCD / CMOS camera and an event camera provided in this application in one embodiment;

[0023] Figure 4 These are event images obtained by a conventional CCD / CMOS camera and an event camera provided in this application in one embodiment;

[0024] Figure 5 This is a structural diagram of an embodiment of a steering wheel interaction device provided by the present application;

[0025] Figure 6This is a structural diagram of an embodiment of a capacitive sensor provided by the present application being arranged on a wheel rim;

[0026] Figure 7 It is a structural schematic diagram of an embodiment of a vehicle provided by this application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] To facilitate driver-vehicle interaction, the steering wheel is equipped with several mechanical buttons, levers, and knobs. Drivers control in-vehicle functions such as lighting, audio, video, and phone calls by pressing buttons, turning levers, or rotating knobs. Due to the limited space on the steering wheel, the number of physical buttons that can be installed is also limited. As a result, the user's interactive options through this type of steering wheel are relatively limited and lack flexibility.

[0029] To address these shortcomings, related technologies incorporate cameras on the steering wheel. These cameras recognize user gestures and enable remote, contactless control of vehicle functions, making driver-vehicle interaction more natural and convenient. Furthermore, the ability to configure associations between gestures and relevant vehicle functions significantly increases the flexibility of driver-vehicle interaction.

[0030] In the aforementioned steering wheel, a camera on the steering wheel captures image frames containing user gestures at regular intervals and sends them to a processing unit. This unit uses image segmentation and other methods to extract foreground frames containing only the user gestures. It then recognizes the foreground frames, determines the user gestures, and, based on the mapping between the user gestures and control commands, sends corresponding control signals to the vehicle controller.

[0031] Research has found that cameras (camcorders) using CCD / CMOS sensors require continuous image frame capture, resulting in relatively high power consumption. Furthermore, gesture recognition accuracy using CCD / CMOS sensors is significantly affected by lighting. It should be noted that strong and low-light conditions are common in driving environments. For example, when a vehicle exits a tunnel, the brightness varies greatly, significantly reducing gesture recognition accuracy in such scenarios. Furthermore, to better capture the user's gestures, CCD / CMOS sensors mounted on the steering wheel are often aimed at the user's upper body. This setup inevitably captures the user's face, posing privacy concerns.

[0032] If the user's face needs to be processed by algorithms such as mosaic, image occlusion, and image blur, it not only increases the processing performance requirements of the processing unit, but also increases the overall power consumption of the system.

[0033] An event camera is a biomimetic vision sensor that contains a dynamic vision sensor that is completely different from a CCD / CMOS sensor. Optionally, the dynamic vision sensor can include one or more of an event vision sensor (DVS), an asynchronous time-based image sensor (ATIS), and a dynamic and active pixel vision sensor (DAVIS), without limitation.

[0034] Unlike conventional CCD / CMOS cameras, which capture complete image frames at fixed intervals, event cameras independently capture the moment each pixel's brightness changes. When the brightness change at a pixel exceeds a set threshold, the pixel triggers an event and outputs the event's timestamp, spatial coordinates, and the polarity of the brightness change (increase or decrease). Therefore, event cameras output not conventional image frames but a series of asynchronous event streams.

[0035] Specifically, the event camera has the following features:

[0036] (1) Low latency: Since event cameras only record events with brightness changes, they have extremely low latency and can respond to changes in dynamic scenes in real time.

[0037] (2) Low power consumption: Event cameras only transmit information when the brightness changes, so the power consumption is extremely low and suitable for application scenarios that require long-term operation.

[0038] Please read together Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the signal output of a conventional CCD / CMOS camera and an event camera under a rotating disk. Figure 3 This is a schematic diagram of the signal output from a conventional CCD / CMOS camera and an event camera when the disk changes from stationary to rotating.

[0039] Figure 1 and Figure 2 On the left side is a disk with black spots, which rotates counterclockwise around the center of the disk. Figure 1 and Figure 2 The upper right corner represents several visible light images acquired by conventional CCD / CMOS cameras. Figure 1 and Figure 2 The lower right corners represent the images captured by the event camera.

[0040] like Figure 1 As shown in the figure, the conventional CCD / CMOS camera periodically acquires several visible light images, including information such as the center point of the disk, the black spot, and the disk outline, while the event sensing image acquired by the event camera only contains the motion information of the black spot.

[0041] like Figure 2 As shown in the figure, for the same disk, the conventional CCD / CMOS camera captures several visible light images at regular intervals, including information such as the disk's center, dark spots, and disk outline when the disk is not rotating. However, the event camera captures no image information at this time. After the disk begins to rotate, the event camera begins to capture event-sensing images that only contain the motion of the dark spots.

[0042] Furthermore, the event camera outputs visual signals in the form of a sparse event stream, which does not contain any information about the static background, thus filtering out a large amount of redundant data. Figure 3 and Figure 4 .

[0043] like Figure 3 and Figure 4 As shown in the figure, compared to images captured by conventional cameras, images captured by event cameras do not contain a large amount of image information. If conventional CCD / CMOS cameras need to hide sensitive information after imaging, they must use additional processing algorithms such as mosaics, image occlusion, and image blurring. This not only increases the processor performance requirements of the camera system but also increases the overall system power consumption. Images captured by event cameras do not require secondary algorithm processing, while effectively protecting user privacy.

[0044] like Figure 5As shown, the steering wheel interaction device 10 provided in the present application includes a steering wheel body 11 , a processing unit 12 and at least one event camera 13 .

[0045] The event camera 13 is connected to the processing unit 12 , and the event camera 13 is disposed on the spokes 111 of the steering wheel body 11 and / or the airbag cover 112 .

[0046] In some possible embodiments, the event camera 13 is communicatively connected to the processing unit 12 .

[0047] In some possible embodiments, the processing unit 12 is disposed in the spoke 111 or the airbag cover 112 .

[0048] In some embodiments, the processing unit 12 is configured to process event images captured by the event camera 13. Exemplarily, the processing unit 12 parses the event images captured by the event camera 13 to obtain a target image, and then performs gesture recognition on the target image. For example, the processing unit 12 sets the background of the target image to a first color and sets the pixels corresponding to the dynamic event to a second color, where the first color and the second color are different.

[0049] Optionally, the event image includes row and column information of pixel points generated by a series of events.

[0050] In some possible embodiments, the processing unit 12 processes the event image to obtain the target image, including the following steps:

[0051] Set the background of the target image to the first color. For example, the first color is black.

[0052] The pixel corresponding to the dynamic event is set to a second color, where the first color and the second color are different. Optionally, the second color is the foreground color of the first color. For example, when the first color is black, the second color is white.

[0053] Optionally, the format type of the target image can be YUV, RGB, etc., which is not limited here.

[0054] In some possible embodiments, the airbag cover 112 is relatively small. Considering that in the event of a vehicle collision, the user may collide with the event camera 13 mounted thereon, causing secondary injury to the user and / or damage to the event camera. In this embodiment, the event camera 13 is mounted on the spoke 111 to prevent the user's body from colliding with the event camera 13. More specifically, the event camera 13 is mounted on the first target surface of the spoke 111, facing the user.

[0055] In some possible embodiments, the airbag cover 112 is relatively large, and the event camera 13 may be disposed in a target area of ​​the second target surface of the airbag cover 112 facing the user. When the vehicle collides, the user does not collide with the target area.

[0056] In some embodiments, the event camera 13 is configured to capture user gestures.

[0057] In some possible embodiments, the event camera 13 may also be configured to recognize the user's head movements (e.g., at least one of raising the head, lowering the head, turning the head, tilting the head sideways, making circular motions, and lateral bending motions), thereby further increasing the flexibility of human-vehicle interaction.

[0058] In some embodiments, the processing unit 12 may be an integrated controller.

[0059] Optionally, the steering wheel interaction device 10 further includes at least one ultrasonic sensor array (not shown). The at least one ultrasonic sensor array is connected to the processing unit 12 and is disposed on the spokes and / or the airbag cover.

[0060] In some possible embodiments, the ultrasonic sensor array is communicatively connected to the processing unit 12 .

[0061] It can be understood that the ultrasonic sensor array is composed of a plurality of ultrasonic sensors, and the ultrasonic sensor array determines the specific gesture of the user by acquiring the distance and angle between the user's gesture and each ultrasonic sensor.

[0062] In some application scenarios, the illumination of the vehicle interior is low, and the event camera 13 may not be able to effectively capture event images containing user gestures. By providing an ultrasonic sensor array in the steering wheel interaction device 10, the gesture recognition capability of the steering wheel interaction device 10 in low-illumination environments can be effectively improved.

[0063] In some possible application scenarios, the illumination of the vehicle interior fluctuates significantly (for example, when the vehicle enters or exits a tunnel), and the event camera 13 may not be able to effectively capture event images containing user gestures. By incorporating an ultrasonic sensor array into the steering wheel interaction device 10, the steering wheel interaction device 10 can effectively improve its gesture recognition capabilities within a high dynamic range.

[0064] Optionally, the steering wheel interaction device 10 further includes at least one millimeter wave sensor (not shown), which is connected to the processing unit 12 and is disposed on the spoke 111 and / or the airbag cover 112 .

[0065] In some possible embodiments, the millimeter wave sensor is communicatively connected to the processing unit 12 .

[0066] Similarly, in some application scenarios, the illumination of the vehicle interior is low, and the event camera 13 may not be able to effectively capture event images containing user gestures. By providing an ultrasonic sensor array in the steering wheel interaction device 10, the gesture recognition capability of the steering wheel interaction device 10 in low-illumination environments can be effectively improved.

[0067] In some potential application scenarios, the illumination of the vehicle interior fluctuates significantly (for example, when the vehicle enters or exits a tunnel), and the event camera 13 may not be able to effectively capture event images containing user gestures. By incorporating a millimeter wave sensor into the steering wheel interaction device 10, the steering wheel interaction device 10 can effectively improve its gesture recognition capabilities within a high dynamic range.

[0068] Optionally, the steering wheel body further includes a rim 113 , and the airbag cover 112 is connected to the rim 113 through spokes 111 .

[0069] The steering wheel interaction device 10 includes a plurality of capacitive sensors 14 (see Figure 6 ), multiple capacitive sensors 14 are connected to the processing unit 12, and multiple capacitive sensors 14 are set on the wheel rim 113.

[0070] In some possible embodiments, the capacitive sensor 14 is communicatively connected to the processing unit 12 .

[0071] Specifically, capacitive sensor 14 is a static capacitance sensor, which is used to detect the capacitance value when a user is in proximity or in contact with capacitive sensor 14. Furthermore, because the capacitance value can be used to represent the distance between the user's gesture and capacitive sensor 14, capacitive sensor 14 can be used to obtain the distance between the user's non-contact gesture and the capacitive sensor 14. By providing multiple capacitive sensors 14 on rim 113, this embodiment can accurately obtain the user's non-contact gesture.

[0072] When the user's hand approaches the rim 113 and slides non-contact, the capacitance value of the capacitive sensor 14 changes. The capacitance value changes of different capacitive sensors 14 are different, so the processing unit 12 can recognize different non-contact gestures of the user. The processing unit 12 generates different signals based on the recognized non-contact gestures, thereby realizing non-contact human-vehicle interaction.

[0073] In addition, the capacitive sensor 14 can also be used to detect the user's grip on the rim 113. In other words, the capacitive sensor 14 can also be configured to detect the user's contact gestures on the rim 113. For example, the contact gestures can include at least one of a press gesture, a slide gesture, and a handwriting gesture, which are not limited here.

[0074] In some possible embodiments, the capacitive sensor 14 may also be disposed on the spoke 111 and / or the airbag cover 112. By changing the installation area of ​​the capacitive sensor 14, the detection space of the capacitive sensor 14 may be increased.

[0075] Optionally, the capacitive sensor 14 is in sheet form and attached to the surface of the rim 113 .

[0076] In this embodiment, since the capacitive sensor 14 is sheet-shaped and can be attached to the surface of the wheel rim 113, the installation difficulty of the capacitive sensor 14 can be reduced. In addition, when the capacitive sensor 14 fails, the failed capacitive sensor 14 can be replaced without complicated operations.

[0077] Alternatively, as Figure 6 As shown, a plurality of capacitive sensors 14 are arranged at intervals along the circumferential direction of the rim 113 .

[0078] It should be noted that the detection space formed by a single capacitive sensor 14 is limited. By arranging multiple capacitive sensors at intervals along the circumferential direction of the rim 113, the formation of a detection blind area can be avoided.

[0079] like Figure 5 As shown, the steering wheel interaction device 10 includes a vibration motor 15 , which is connected to the processing unit 12 and is disposed on the spoke 111 .

[0080] In some possible embodiments, the vibration motor 15 is communicatively connected to the processing unit 12 .

[0081] In some application scenarios, the vibration motor 15 is configured to provide vibration feedback to the user when the user interacts with the steering wheel interaction device 10 in a contact manner.

[0082] Exemplarily, the vibration feedback may be at least one of short press interaction vibration feedback, long press interaction vibration feedback, invalid interaction vibration feedback, sliding interaction vibration feedback, special gesture interaction vibration feedback, and handwriting vibration feedback, which is not limited here.

[0083] Optionally, the steering wheel interaction device 10 includes a light bar 16 , which is connected to the processing unit 12 and is disposed on the spokes 111 and / or the rim 113 .

[0084] In some possible embodiments, the light bar 16 is communicatively connected to the processing unit 12 .

[0085] In some application scenarios, the light bar 16 is configured to provide lighting feedback to the user when the user performs contactless interaction or contact interaction with the steering wheel interaction device 10 .

[0086] In some possible application scenarios, the light bar 16 is disposed on the first target surface of the spoke 111 facing the user, and / or on the third target surface of the rim 113 facing the user. The light bar 16 can serve as a light source for the event camera 13, providing fill light for the user in low-light conditions, thereby improving the accuracy of gesture recognition using the event camera 13.

[0087] In some possible application scenarios, the user first registers various gestures and head postures into the vehicle system, mapping each gesture to an operable function in the vehicle system. The user's gestures are then recognized by sensors on the steering wheel interaction device 10 (e.g., at least one of the aforementioned event camera 13, ultrasonic sensor array, millimeter wave sensor, and capacitive sensor 14). The sensor signals are then sent to the processing unit 12, which interprets the signals and generates control signals, which are then sent to the light bar 16 and vibration motor 15. Furthermore, the processing unit 12 sends the control signals to the smart cockpit system.

[0088] In music mode, the smart cockpit system controls the speakers to produce different sound effects and / or displays the corresponding control content on the vehicle instrument panel. The light bar 16, the interior ambient light, the vibration motor 15, and the smart cockpit screen work together to perform interactive music performances.

[0089] In other modes, the smart cockpit system controls the vehicle computer to adjust the left and right turn signals, switch to the previous / next song, open the sunroof, switch screens, answer calls, open and close the trunk, and other functions.

[0090] The following is a detailed introduction to user gesture interaction:

[0091] (1) Music mode: multiple sensors are arranged on the wheel rim 113 and the airbag cover 112 (the sensor position is not limited, and the sensor can be at least one of the event camera 13, ultrasonic sensor array, millimeter wave sensor, and capacitive sensor 14 mentioned above). The user can place one hand on the steering wheel body 11 and slide the other hand above the sensor detection area. Different signals can be triggered by gestures. For example, sliding from left to right, sliding from right to left, sliding from top to bottom, sliding from bottom to top, sliding on the left side, sliding on the right side, etc. Different gestures can correspond to different percussion sound effects, such as drum beats / shakers / triangle bells / sliding sound effects and other musical sounds. The above musical sounds can be mixed with the currently playing music, and the sound effect curve and effect diagram can be displayed on the in-car display screen. In some possible embodiments, the vibration of the vibration motor 15, the rhythm of the light bar 16 and / or the in-car ambient light, etc., can achieve the effect of improving the driving experience.

[0092] (2) Game mode: Different percussion instrument icons pop up on the screen in the car and are arranged in numerical order from left to right. Different sensor areas on the wheel rim 113 correspond to numbers 1-10. By placing your hands on different sensor areas, non-contact percussion sound effects and screen display effects are achieved, corresponding to different instrument icons on the screen in the car, and game interaction is carried out.

[0093] (3) The left area and the right area of ​​the steering wheel can be set, and the detection area above the sensor on the rim 113 can be set. By sliding the left or right area, for example, a left turn signal / right turn signal can be sent to the smart cockpit system, a previous song / next song signal can be sent when playing music, and the volume can be increased or decreased, etc.

[0094] (4) In non-gaming mode, swipe from left to right, swipe from right to left, swipe from top to bottom, swipe from bottom to top, swipe on the left side, swipe on the right side, swipe multiple times, etc. Gesture control is achieved by setting the corresponding relationship between each gesture and functions such as the sunroof, switching screens, answering calls, opening and closing the trunk, etc.

[0095] In this solution, the event camera, located on the spokes and / or airbag cover, generates images based solely on brightness changes in pixels within its field of view. Therefore, it can capture clear gesture images even in high-illuminance scenarios, improving the recognition accuracy of the steering wheel interaction device. Furthermore, since the event camera does not capture specific facial images, it better protects user privacy.

[0096] See also Figure 7 , Figure 7 It is a structural schematic diagram of an embodiment of a vehicle provided by this application.

[0097] In this embodiment, the vehicle 20 includes a steering wheel interaction device 21 and a smart cockpit system 22 .

[0098] The steering wheel interaction device 21 is coupled to the intelligent cockpit system 22 via a CAN bus.

[0099] In some possible embodiments, the steering wheel interaction device 21 may also be coupled to the smart cockpit system 22 via a LIN bus.

[0100] Optionally, the steering wheel interaction device 21 may be the steering wheel interaction device 10 described above.

[0101] In some possible embodiments, the smart cockpit system 22 may be a vehicle domain controller.

[0102] In this solution, the event camera, located on the spokes and / or airbag cover, generates images based solely on brightness changes in pixels within its field of view. Therefore, it can capture clear gesture images even in high-illuminance scenarios, improving the recognition accuracy of the steering wheel interaction device. Furthermore, since the event camera does not capture specific facial images, it better protects user privacy.

[0103] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Equivalent structures or equivalent process changes made by utilizing the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A steering wheel interaction device, characterized in that: The steering wheel interaction device includes: a steering wheel body, a processing unit, and at least one event camera; The at least one event camera is connected to the processing unit and is arranged on the spokes of the steering wheel body and / or the airbag cover.

2. The steering wheel interaction device according to claim 1, characterized in that: The steering wheel interaction device further includes at least one ultrasonic sensor array, which is connected to the processing unit and is arranged on the spoke and / or the airbag cover.

3. The steering wheel interaction device according to claim 1, characterized in that: The steering wheel interaction device further includes at least one millimeter wave sensor, which is connected to the processing unit and is disposed on the spoke and / or the airbag cover.

4. The steering wheel interaction device according to any one of claims 1 to 3, characterized in that: The steering wheel body further comprises a rim, and the airbag cover is connected to the rim via the spokes; The steering wheel interaction device includes a plurality of capacitive sensors, which are connected to the processing unit and are arranged on the wheel rim.

5. The steering wheel interaction device according to claim 4, characterized in that: The capacitive sensor is in sheet form and is attached to the surface of the rim.

6. The steering wheel interaction device according to claim 4, characterized in that: The plurality of capacitive sensors are arranged at intervals along the circumferential direction of the rim.

7. The steering wheel interaction device according to claim 4, characterized in that: The steering wheel interaction device includes a vibration motor, which is connected to the processing unit and is arranged on the spoke.

8. The steering wheel interaction device according to claim 4, characterized in that: The steering wheel interaction device includes a light bar, which is connected to the processing unit and is arranged on the spokes and / or the rim.

9. A vehicle, characterized in that: The vehicle comprises the steering wheel interaction device according to any one of claims 1-8.

10. The vehicle according to claim 9, characterized in that The vehicle also includes an intelligent cockpit system, and the steering wheel interaction device is coupled to the intelligent cockpit system via a CAN bus.

Images