Control method and apparatus for in-vehicle system

Abstract

The embodiment of the present application provides a kind of control method and device of vehicle-mounted system, the method comprises: in the process that driver holds steering wheel, the trajectory of current gesture of driver is determined by the sensor installed in steering wheel;According to the trajectory of current gesture, the kind of current gesture is determined from pre-set gesture category;Vehicle-mounted system is controlled to execute the operation corresponding to the kind of current gesture.The embodiment of the present application identifies the kind of gesture executed by driver by the sensor installed in steering wheel, and according to different gesture category, vehicle-mounted system is executed the operation corresponding to the kind of gesture, greatly facilitates some functional operations of driver in the process of driving car, guarantees the safety of driving, so that the interaction between car and people is more close.

Description

Technical Field

[0001] The present invention relates to the field of automotive electronics, and more specifically, to a control method and apparatus for an in-vehicle system. Background Technology

[0002] With the continuous improvement of automotive safety standards and the level of automotive electronics, as well as the increasing demand for driving safety, the automotive electronics market has seen steady growth.

[0003] In traditional cars, to achieve different functions, such as switching songs, playing or pausing music, or controlling the map size on the in-car navigation display, one needs to use function buttons located on the steering wheel or center console. However, because these function buttons are fixed in position, not adjustable, and numerous, and some are far from the steering wheel, drivers need to be distracted to perform these operations, posing a risk of traffic accidents. Therefore, automotive electronics require a safer, more reliable, and simpler operating method in terms of human-machine interaction. Summary of the Invention

[0004] This invention provides a control method and device for an in-vehicle system. By using sensors installed in the steering wheel to identify the types of gestures made by the driver, the system performs operations corresponding to the different types of gestures. This greatly satisfies the driver's needs for some functions while driving, ensures driving safety, and makes the interaction between the car and the driver closer.

[0005] On the one hand, a control method for an in-vehicle system is provided. The method includes: determining the trajectory of the driver's current gesture by means of a sensor installed in the steering wheel while the driver is holding the steering wheel; determining the type of the current gesture from a preset set of gesture types based on the trajectory of the current gesture; and controlling the in-vehicle system to perform an operation corresponding to the type of the current gesture.

[0006] In this embodiment of the invention, when the driver holds the steering wheel, the sensors installed in the steering wheel can identify the types of gestures made by the driver, and perform the corresponding operations on the vehicle system according to the different types of gestures. This greatly satisfies the driver's needs for some functions during the driving process, ensures driving safety, and makes the interaction between the car and the person closer.

[0007] In conjunction with the first aspect, in a first implementation of the first aspect, the steering wheel is equipped with multiple sensors, which are multiple ultrasonic sensors. Determining the trajectory of the driver's current gesture using the sensors installed in the steering wheel includes: acquiring ultrasonic signals collected by the multiple ultrasonic sensors over a period of time; selecting reflected signals from the collected ultrasonic signals; and locating the position of the driver's hand or fingers over a period of time based on the selected reflected signals to determine the trajectory of the current gesture.

[0008] In this embodiment of the invention, the trajectory of the current gesture can be accurately determined by determining the reflected signal in the ultrasonic signal.

[0009] In conjunction with the first aspect and the first implementation of the first aspect, in the second implementation of the first aspect, before determining the trajectory of the driver's current gesture using the sensor installed in the steering wheel, the method further includes: determining whether the steering wheel is in a rotating state; determining the trajectory of the driver's current gesture using the sensor installed in the steering wheel includes: determining the trajectory of the driver's current gesture using the sensor installed in the steering wheel when the steering wheel is not in a rotating state.

[0010] In conjunction with the first aspect, and in the first and second implementations of the first aspect, in the third implementation of the first aspect, a gyroscope is installed in the steering wheel, and determining whether the steering wheel is in a rotating state includes: determining whether the steering wheel is in a rotating state based on the gyroscope.

[0011] In this embodiment of the invention, by determining whether the steering wheel is in a rotating state, it is possible to avoid recognizing the driver's gestures when the steering wheel is in a rotating state, thus ensuring the accuracy of gesture recognition.

[0012] In conjunction with the first aspect and the first to third implementations of the first aspect, in the fourth implementation of the first aspect, the preset gesture types include at least one of the following gesture types: sliding two fingers inward or outward; sliding two fingers clockwise or counterclockwise; sliding one finger up, down, left, or right; double-tapping one finger; waving one hand; touching and sliding one finger along the inner circumference of the steering wheel; double-tapping two fingers.

[0013] In conjunction with the first aspect, the first to fourth implementations of the first aspect, and the fifth implementation of the first aspect, a plurality of sound holes are provided on the circumference of the steering wheel, the plurality of sound holes correspond one-to-one with the plurality of ultrasonic sensors, and the direction of the sound holes is determined by the area of ​​the driver's hand or fingers and the coverage of the ultrasonic signal.

[0014] In conjunction with the fifth implementation of the first aspect, in the sixth implementation of the first aspect, a waterproof and breathable membrane is provided between the ultrasonic sensor and the sound outlet, the waterproof and breathable membrane being used to prevent water vapor from entering the ultrasonic sensor through the sound outlet.

[0015] In this embodiment of the invention, the waterproof and breathable membrane can prevent water vapor from entering the ultrasonic sensor, ensuring the accuracy of gesture recognition.

[0016] Secondly, a control device for an in-vehicle system is provided, the device comprising: a first determining module, configured to determine the trajectory of the driver's current gesture by means of a sensor installed in the steering wheel while the driver is holding the steering wheel; a second determining module, configured to determine the type of the current gesture from a preset set of gesture types based on the trajectory of the current gesture; and a control module, configured to control the in-vehicle system to perform an operation corresponding to the type of the current gesture.

[0017] In conjunction with the second aspect, in the first implementation of the second aspect, the steering wheel is equipped with multiple sensors, which are multiple ultrasonic sensors. The first determining module is specifically used to: acquire ultrasonic signals collected by the multiple ultrasonic sensors over a period of time; select reflected signals from the collected ultrasonic signals; and locate the position of the driver's hand or fingers over a period of time based on the selected reflected signals, so as to determine the trajectory of the current gesture.

[0018] In conjunction with the second aspect, the first implementation of the second aspect, in the second implementation of the second aspect, the device includes a third determining module for determining whether the steering wheel is in a rotating state. Specifically, the third determining module is used to determine the driver's current gesture trajectory by means of a sensor installed in the steering wheel when the steering wheel is in a non-rotating state.

[0019] In conjunction with the second aspect, the first and second implementations of the second aspect, in the third implementation of the second aspect, a gyroscope is installed in the steering wheel, and the third determining module is specifically used to determine whether the steering wheel is in a rotating state based on the gyroscope.

[0020] In conjunction with the second aspect, the first to third implementations of the second aspect, and in the fourth implementation of the second aspect, the preset gesture types include at least one of the following gesture types: sliding two fingers inward or outward; sliding two fingers clockwise or counterclockwise; sliding one finger up, down, left, or right; double-tapping one finger; waving one hand; sliding one finger along the inner circumference of the steering wheel; double-tapping two fingers.

[0021] In conjunction with the second aspect, and the first to fourth implementations of the second aspect, in the fifth implementation of the second aspect, a plurality of sound holes are provided on the circumference of the steering wheel, and the plurality of sound holes correspond one-to-one with the plurality of ultrasonic sensors. The direction of the sound holes is determined by the area of ​​the driver's hand or fingers and the coverage range of the ultrasonic signal.

[0022] In conjunction with the fifth implementation of the second aspect, in the sixth implementation of the second aspect, a waterproof and breathable membrane is provided between the ultrasonic sensor and the sound outlet, and the waterproof and breathable membrane is used to prevent water vapor from entering the ultrasonic sensor through the sound outlet.

[0023] Thirdly, a control device for an in-vehicle system is provided, including a memory and a processor, wherein the memory is used to store program code, and the processor is used to call the program code to implement the methods in the first aspect and various implementations thereof.

[0024] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0025] This invention can identify the types of gestures made by the driver through sensors installed in the steering wheel, and perform the corresponding operations on the vehicle system according to the different types of gestures, thereby enabling the driver to perform different operations on the vehicle system, greatly facilitating the driver and ensuring driving safety. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of the control method of the vehicle system according to an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram illustrating the application of the first gesture in an embodiment of the present invention.

[0028] Figure 3 This is a schematic diagram illustrating the application of the second gesture in an embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram illustrating the application of the third gesture in an embodiment of the present invention.

[0030] Figure 5 This is a schematic diagram illustrating the application of the fourth gesture in an embodiment of the present invention.

[0031] Figure 6 This is a schematic diagram illustrating the application of the fifth gesture in an embodiment of the present invention.

[0032] Figure 7 This is a schematic diagram illustrating the application of the sixth gesture in an embodiment of the present invention.

[0033] Figure 8 This is a schematic diagram illustrating the application of the seventh gesture in an embodiment of the present invention.

[0034] Figure 9 This is a schematic diagram of the sound outlet hole provided on the circumference of the steering wheel according to an embodiment of the present invention.

[0035] Figure 10 This is a schematic diagram of an ultrasonic sensor installed in a steering wheel according to an embodiment of the present invention.

[0036] Figure 11 This is a schematic diagram of the layout of a flexible printed circuit board (FPC) according to an embodiment of the present invention.

[0037] Figure 12 This is another schematic flowchart of the control method of the vehicle system according to an embodiment of the present invention.

[0038] Figure 13 This is a schematic block diagram of the control device of the vehicle system according to an embodiment of the present invention.

[0039] Figure 14 This is another schematic block diagram of the control device of the vehicle system according to an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions in the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0041] In traditional cars, drivers use various function buttons to operate different functions. For example, buttons on the steering wheel adjust the volume for multimedia playback, and in cars with hands-free calling, they can answer calls. Buttons on the center console control navigation, temperature control, and other functions. However, due to the large number of function buttons, their fixed locations, and the fact that many are far from the driver, operating certain functions while driving can be distracting and potentially lead to traffic accidents.

[0042] Based on the above problems, this invention proposes a control method for an in-vehicle system. By recognizing the driver's gestures through sensors installed in the steering wheel, the system can conveniently and quickly perform some functional operations required by the driver while driving the car, making the interaction between the car and the person closer.

[0043] Figure 1 The diagram shown is a schematic flowchart of a control method 100 for an in-vehicle system according to an embodiment of the present invention. Figure 1 The method 100 shown includes:

[0044] 110. During the process of the driver holding the steering wheel, the trajectory of the driver's current gesture is determined by a sensor installed in the steering wheel;

[0045] 120. Based on the trajectory of the current gesture, determine the type of the current gesture from a preset set of gesture types;

[0046] 130. Control the vehicle system to perform the operation corresponding to the type of the current gesture.

[0047] Specifically, when the driver holds the steering wheel, sensors installed in the steering wheel can identify the type of gestures the driver makes and execute the corresponding operation on the vehicle system based on the different gesture types. Throughout the process, the driver's hands can remain on the steering wheel, and different functions can be performed by using only some of the driver's fingers or hands to perform some gestures. This greatly satisfies the driver's needs for some functions while driving, ensures driving safety, and makes the interaction between the car and the driver closer.

[0048] It should be understood that the in-vehicle system in the embodiments of the present invention may be an in-vehicle head-up display (HUD), an in-vehicle navigation system, an in-vehicle menu system, an in-vehicle multimedia system, or other in-vehicle systems. The embodiments of the present invention do not limit this.

[0049] Optionally, in some embodiments, the sensor may be mounted in the circumference of the steering wheel.

[0050] Optionally, in some embodiments, there may be multiple sensors, each of which is connected in series to form a sensor array.

[0051] It should be understood that the specific number of sensors can be determined by the size of the steering wheel or by the required accuracy of gesture recognition. For example, when it is necessary to accurately recognize the gestures performed by the driver, the number of sensors can be increased appropriately.

[0052] Optionally, in some embodiments, the sensor may be an ultrasonic sensor.

[0053] Optionally, in some embodiments, there may be multiple ultrasonic sensors. The specific number of these multiple ultrasonic sensors may be determined by the size of the steering wheel or by the accuracy of the hand gesture recognition. These multiple ultrasonic sensors are connected in series to form an ultrasonic sensor array.

[0054] It should be understood that, since the embodiments of the present invention use the ultrasonic multi-point positioning principle to locate the position of the driver's gesture and then recognize the gesture, the number of ultrasonic sensors is at least two.

[0055] It should be understood that when the sensor is an ultrasonic sensor, the ultrasonic sensor itself can act as both a transmitter and a receiver.

[0056] Optionally, in some embodiments, when the sensor is an ultrasonic sensor, the ultrasonic signal received by the ultrasonic sensor may include the signal of the driver's gesture, or other interference signals, such as the noise signal of a stationary engine or other mechanical structures, as well as signals of other frequencies emitted by non-ultrasonic sensors.

[0057] Optionally, in some embodiments, the ultrasonic signal received by the ultrasonic sensor receiver can be an ultrasonic signal over a period of time, such as an ultrasonic signal within a sampling period. This avoids the problem of large amounts of data that are difficult to process.

[0058] Optionally, in some embodiments, the sampling time period can be preset in the ultrasonic sensor. Users can freely set the length of the sampling time period, or preset the length of the sampling time period in the ultrasonic sensor according to the speed of signal propagation.

[0059] It should be understood that the sampling time periods can be continuous or intermittent.

[0060] When the sampling time intervals are continuous, when the time of the ultrasonic signal received by the ultrasonic sensor reaches a sampling time interval, the data storage unit in the ultrasonic sensor can automatically transmit the ultrasonic signal collected in this sampling time interval to the processor and clear the ultrasonic signal in the storage unit. Alternatively, when it is not necessary to process the currently collected data signal, the ultrasonic signal received in this sampling time interval can be directly deleted to prepare for storing the ultrasonic signal collected in the next sampling time interval.

[0061] When there are intervals between sampling periods, this can be achieved by setting an interrupt in the ultrasonic sensor. When the current sampling period is reached, the data storage unit in the ultrasonic sensor can automatically transmit the ultrasonic signal of this sampling period to the processor and clear the ultrasonic signal in the storage unit. During the interval between two sampling periods, the ultrasonic sensor does not send ultrasonic signals. When the next sampling period arrives, the interrupt in the ultrasonic sensor is activated, and the ultrasonic sensor starts sending and receiving ultrasonic signals. This method can avoid the ultrasonic sensor being in a working state for a long time, saving the energy consumption of the ultrasonic sensor.

[0062] The interruption time interval can be set by the user or preset in the ultrasonic sensor; this embodiment of the invention does not limit this.

[0063] Optionally, in some embodiments, after the ultrasonic sensor receives the ultrasonic signal, the reflected signal in the ultrasonic signal can be determined. For example, the propagation distance of the ultrasonic signal can be determined based on the phase difference and delay parameters of the ultrasonic signal. Based on the propagation distance of different signals in the ultrasonic signal, the reflected signal in the ultrasonic signal can be finally determined. The reflected signal is a useful signal for recognizing the gestures performed by the driver.

[0064] Optionally, in some embodiments, some of the reflected signals may be signals outside the active area of ​​the driver's hand or fingers. Signals outside the active area of ​​the driver's hand or fingers may interfere with the driver's gesture recognition. Therefore, the reflected signals within the active area of ​​the driver's hand or fingers can be determined based on the phase difference and delay parameters of the reflected signals. The reflected signals within the active area of ​​the driver's hand or fingers are the signals required for the final gesture recognition.

[0065] Optionally, in some embodiments, before determining the reflected signal from the ultrasonic signal, the method 100 further includes: when the ultrasonic sensor receives the ultrasonic signal, it can first perform a pre-judgment on the received ultrasonic signal. If the time length between the ultrasonic signal being emitted and received meets a preset condition, the received ultrasonic signal can be saved. If the preset condition is not met, the ultrasonic sensor does not save the current ultrasonic signal and re-receives the ultrasonic signal for the next time period and performs a pre-judgment. The preset condition can be a time threshold preset by the user. When the time length between the ultrasonic signal being emitted and received is greater than the preset time threshold, it can be determined that the ultrasonic signal meets the preset condition.

[0066] In this embodiment of the invention, the pre-judgment step can remove some ultrasonic signals that enter the ultrasonic receiver directly without reflection from the received ultrasonic signals in advance, saving the time for gesture recognition and improving the efficiency of gesture recognition.

[0067] It should be understood that the pre-judgment step can remove a portion of the ultrasonic signal, but it cannot remove all the unreflected signals in the ultrasonic signal. This step is only a pre-judgment. If it is determined that the ultrasonic signal does not meet the preset conditions, no further processing will be performed on the received ultrasonic signal, thus improving the efficiency of gesture recognition.

[0068] Optionally, in some embodiments, before making a preliminary judgment on the ultrasonic waves received by the ultrasonic sensor receiver, the method 100 further includes: determining whether the steering wheel is in a rotating state; if the steering wheel is in a non-rotating state, then recognizing the driver's gestures through a sensor installed in the steering wheel.

[0069] It should be understood that, under normal circumstances, the steering wheel is in a rotating state because the driver is operating the steering wheel, such as when turning or changing lanes. At this time, the probability of the driver operating the vehicle system is very low. Moreover, when the driver is operating the steering wheel, the position of the driver's hands may affect the sensor's recognition of gestures. Therefore, when the steering wheel is in a rotating state, the driver's gestures are not recognized.

[0070] Optionally, in some embodiments, the determination of whether the steering wheel is rotating can be performed by a gyroscope installed in the steering wheel. Specifically, the determination of whether the steering wheel is rotating can be based on the data information obtained by the gyroscope.

[0071] Optionally, in some embodiments, since the ultrasonic signal received by the ultrasonic sensor also includes other interference signals, in order to more efficiently and accurately identify the driver's gestures, it is also necessary to remove the interference signals in the ultrasonic signal before determining the reflected signals in the ultrasonic signal.

[0072] Optionally, in some embodiments, the ultrasonic signal received by the ultrasonic sensor may include some interference signals with inherent frequencies, such as engine noise or noise from other mechanical structures. Since the noise signal is a signal with inherent frequencies, these noise signals with inherent frequencies can be transformed in the time domain and frequency domain, and then removed by a bandpass filter.

[0073] Optionally, in some embodiments, after removing noise signals with inherent frequencies, the resulting ultrasonic signal may also include some other signals that are not emitted by the ultrasonic sensor. Since the signal emitted by the ultrasonic sensor is a signal with inherent frequencies, the frequency of the other signals is different from the frequency of the signal emitted by the ultrasonic sensor. The frequency of the other signals can be transformed in the time domain and frequency domain, and the other frequency signals can be filtered out by a bandpass filter.

[0074] It should be understood that there is no necessary order between the steps of removing inherent noise signals and removing signals of other frequencies. In the embodiments of the present invention, only one of the two steps may be performed, or both steps may be performed. Since performing both steps can remove all interference signals in the received ultrasonic signals and ensure the accuracy of gesture recognition, the embodiments of the present invention take this as an example, but the embodiments of the present invention are not limited thereto.

[0075] Optionally, in some embodiments, after determining the reflected signal in the ultrasonic signal, the position of the driver's hand or fingers during that time period can be located based on the reflected signal during that time period. For example, the position coordinates of the driver's hand or fingers during that time period can be determined based on the ultrasonic multi-point positioning principle.

[0076] Optionally, in some embodiments, after locating the position of the driver's hand or fingers over a period of time, the trajectory of the driver's current gesture can be determined based on the obtained position information. For example, the position coordinates of the driver's hand or fingers over that period of time can be fitted with a function to obtain a fitted function curve, which is the trajectory of the driver's current gesture.

[0077] Optionally, in some embodiments, the type of the current gesture can be determined from a preset range of gesture types based on the determined trajectory of the current gesture.

[0078] For example, the function curve of the current gesture trajectory can be compared with function curves in the database, and the function curve with the highest matching degree with the current gesture trajectory can be selected. The gesture type corresponding to the function curve with the highest matching degree is the type of the current gesture.

[0079] Optionally, in some embodiments, the preset gesture type may include at least one of the following gesture types:

[0080] Slide both fingers inwards or outwards;

[0081] Slide two fingers clockwise or counterclockwise;

[0082] Swipe up, down, left, or right with a single finger;

[0083] Double tap with a single finger;

[0084] Waving with one hand;

[0085] Slide a single finger along the inner circumference of the steering wheel;

[0086] Double tap with two fingers.

[0087] It should be understood that the two fingers mentioned above refer to the same finger of both the left and right hands. For example, sliding two fingers inward means sliding the left and right thumbs inward, and sliding two fingers clockwise means sliding the left and right thumbs clockwise. The single finger mentioned above can be any finger of the left or right hand, and the single hand can be any finger of the left or right hand.

[0088] It should also be understood that when a driver is driving a car, their hands are usually placed on the steering wheel, with the fingers other than the thumb usually gripping the steering wheel. The thumb can generally move freely. Therefore, the most commonly used finger when performing gestures should be the thumb. Thus, this embodiment of the invention only uses the thumb as an example to describe the above-mentioned gestures in detail, but this embodiment of the invention is not limited to this.

[0089] To facilitate the management of the above gesture types, the gesture of sliding two fingers inward or outward can be recorded as the first gesture, sliding two fingers clockwise or counterclockwise can be recorded as the second gesture, sliding one finger up, down, left, or right can be recorded as the third gesture, double-tapping one finger can be recorded as the fourth gesture, waving one hand can be recorded as the fifth gesture, sliding one finger along the inner circumference of the steering wheel can be recorded as the sixth gesture, and double-tapping two fingers can be recorded as the seventh gesture.

[0090] It should be understood that the above description of the preset gesture types is not a limitation on the preset gesture types. In the subsequent development process, the developers can also add gestures to control other functions to the preset gesture types. The embodiments of the present invention only use the above seven gestures as examples to describe the preset gestures in detail, but the embodiments of the present invention are not limited to this.

[0091] Figure 2 The diagram shows the application of the first gesture according to an embodiment of the present invention. The first gesture is to slide two fingers inward or outward, which can control the size of the display of the vehicle system or the size of the displayed content.

[0092] like Figure 2As shown, the gesture of sliding two fingers inward or outward is used to control the HUD. It can control the size of the HUD display or the font size of the information displayed on the HUD. For example, this gesture can be used to control the size of the HUD displayed on the windshield of the car, or when the driver needs to pay attention to fuel consumption information to determine whether the vehicle needs to refuel, the two fingers in the gesture can be slid outward to enlarge the font size of information such as fuel consumption and vehicle speed displayed on the HUD for the driver's convenience.

[0093] like Figure 2 As shown, when the gesture of sliding two fingers inward or outward is used to control the in-vehicle navigation system, the size of the map displayed on the in-vehicle central control screen can be adjusted. In some complex road conditions or when the vehicle needs to change lanes or turn, the map can be enlarged by sliding two fingers outward in the gesture, making it convenient for the driver to view smaller markers on the map at any time. When the road conditions are good or when the car is driving on a highway, the map can be zoomed out by sliding two fingers inward in the gesture.

[0094] like Figure 2 As shown, when the gesture of sliding two fingers inward or outward is used to control the in-vehicle menu system, the size of the menu font on the in-vehicle central control display screen can be adjusted through this gesture. For example, when the driver is an elderly person or someone with poor eyesight, the font in the in-vehicle system menu can be enlarged by sliding two fingers outward, making it easier for the driver to view.

[0095] Figure 3 The diagram shows the application of the second gesture in an embodiment of the present invention. The second gesture is a two-finger clockwise or counterclockwise swipe, which can control the change of the viewing angle of the content displayed in the vehicle system.

[0096] like Figure 3 As shown, when using the two-finger clockwise or counterclockwise swipe gesture to control the road conditions displayed on the HUD, the tilt angle of the display can be adjusted by swiping two fingers clockwise or counterclockwise in conjunction with the road conditions displayed on the HUD.

[0097] like Figure 3 As shown, when the two-finger clockwise or counterclockwise swipe gesture is used to control the in-vehicle navigation system, the driver can switch the map of the in-vehicle navigation system to a top view, front view, or other perspectives when they need to view specific buildings or road conditions on the map.

[0098] Figure 4 The diagram shows the application of the third gesture in this invention. The third gesture is a single-finger swipe up, down, left, and right, which can control the position of the vehicle system display, select menu options, or operate options in the multimedia system.

[0099] like Figure 4 As shown, when the single-finger swipe gesture is used to control the HUD, it can control the position of the HUD display on the windshield of the car. For example, the position of the HUD display can be adjusted according to the driver's height and the position of the driver's eyeballs in the driver's seat.

[0100] like Figure 4 As shown, when the single-finger swipe gesture is used to control the in-vehicle navigation system, it can control the position of the map displayed on the in-vehicle central control screen. For example, one can swipe up with one finger to view some location information above the current location, or swipe left with one finger to view some location information to the left of the current location.

[0101] like Figure 4 As shown, when the single-finger swipe gesture is used to control the in-vehicle menu system, it can realize the function of selecting specific options in the menu. For example, swiping down with a single finger can select the next option of the current option, and swiping right with a single finger can select the option to the right of the current option, etc.

[0102] like Figure 4 As shown, when the single-finger swipe gesture is used to control the multimedia system, it can control the operation of specific options in the multimedia system. For example, when the multimedia system is used to play music, swiping up with a single finger can control the playback of the previous song, swiping down with a single finger can control the playback of the next song, swiping left with a single finger can decrease the volume, and swiping right with a single finger can increase the volume, etc.

[0103] Figure 5 The diagram shows the application of the fourth gesture in this invention. This fourth gesture is a single-finger double-tap, which can enable different options in the vehicle system or pause playback.

[0104] like Figure 5 As shown, the single-finger double-tap gesture is used to control the HUD, which can enable access to different display options. For example, a single-finger double-tap can access the speed option, and a subsequent single-finger double-tap can access the fuel consumption option, etc.

[0105] like Figure 5 As shown, when the single-finger double-tap gesture is used to control the in-vehicle navigation system, it can be used to find the location of nearby businesses or target buildings. For example, a single-finger double-tap can be used to view the location of a business or other building. After that, a single-finger double-tap can be used to switch to different businesses or buildings.

[0106] like Figure 5 As shown, when using a single finger to double-tap to control the display of the in-vehicle menu, you can access specific options within the menu.

[0107] like Figure 5 As shown, when using a single-finger double-tap to control the multimedia system, a pause function can be achieved. For example, when playing a song, a single-finger double-tap can pause playback.

[0108] Figure 6 The diagram shown is an application illustration of the fifth gesture in an embodiment of the present invention. This fifth gesture is a single-handed wave and can control the switching of different applications or modes in the vehicle system.

[0109] like Figure 6 As shown, when the single-hand waving gesture is used to control the HUD, different display content can be switched using this gesture. For example, when the HUD is currently displaying real-time fuel consumption, it can be switched to average fuel consumption by waving one hand.

[0110] like Figure 6 As shown, when the single-hand waving gesture is used to control the in-vehicle central control screen, different applications on the screen can be switched by waving one hand. For example, when the current in-vehicle central control screen displays a map of in-vehicle navigation, one can switch to other applications such as radio broadcast by waving one hand.

[0111] like Figure 6 As shown, when the single-hand waving gesture is used to control the in-vehicle menu system, different setting modes can be switched using this gesture. For example, when the vehicle is currently in general mode, a single-hand waving gesture can be used to switch to safety mode.

[0112] like Figure 6 As shown, when the single-hand waving gesture is used to control the multimedia system, different applications can be switched using this gesture. For example, when the current multimedia system is playing music, the music playback can be switched to video playback, or to images, etc., by waving one hand.

[0113] Figure 7 The diagram shows the application of the sixth gesture in this embodiment of the invention. The sixth gesture is a single-finger touch swipe along the inside of the steering wheel, which can realize the function of returning to the previous menu or exiting the application.

[0114] like Figure 7 As shown, when the single-finger swipe gesture along the inside of the steering wheel is used to control the HUD, this gesture allows you to return to the previous menu from the currently displayed content.

[0115] like Figure 7As shown, when the single-finger swipe gesture along the inside of the steering wheel is used to control the in-vehicle navigation system, the gesture can be used to exit the in-vehicle navigation system.

[0116] like Figure 7 As shown, when the single-finger swipe gesture along the inside of the steering wheel is used to control the in-vehicle menu system, the gesture can be used to exit the in-vehicle menu system.

[0117] like Figure 7 As shown, when the single-finger swipe gesture along the inside of the steering wheel is used to control the multimedia system, the gesture can be used to exit the multimedia system.

[0118] Figure 8 The diagram shows the application of the seventh gesture in this embodiment of the invention. The seventh gesture is a double-tap with two fingers, which can realize the functions of closing the application and turning the gesture recognition mode on or off.

[0119] like Figure 8 As shown, when the double-tap gesture is used to control the HUD, the HUD can be turned off directly, or the gesture recognition mode can be turned on or off sequentially.

[0120] like Figure 8 As shown, when the double-tap gesture is used to control the in-vehicle navigation display, the navigation mode can be turned off directly by using this gesture, or the gesture recognition mode can be turned on or off sequentially by using this gesture.

[0121] like Figure 8 As shown, when the double-tap gesture is used to control the display of the in-vehicle menu, the gesture recognition mode can be turned on or off.

[0122] like Figure 8 As shown, when the double-tap gesture is used to control the multimedia system, the gesture recognition mode can be turned on or off.

[0123] Optionally, in some embodiments, when the gesture recognition mode needs to switch between multiple display modes, for example, when switching from HUD to in-vehicle navigation display mode, the user can use a touch-swipe gesture along the inside of the steering wheel to gradually return to the previous menu until returning to the top-level menu. After returning to the top-level menu, the user can perform the touch-swipe gesture along the inside of the steering wheel again, and the in-vehicle central control display will display a menu of multiple mode selections, for example, a selection menu of four modes: HUD, in-vehicle navigation display mode, in-vehicle menu display mode, and multimedia display mode. At this time, the user can select different modes by using a single-finger swipe gesture to move up, down, left, or right.

[0124] Optionally, in some embodiments, a plurality of sound holes are provided on the circumference of the steering wheel, and each of the plurality of sound holes corresponds to an ultrasonic sensor.

[0125] Figure 9 The diagram shows a sound outlet hole arranged on the circumference of a steering wheel according to an embodiment of the present invention.

[0126] like Figure 9 As shown, there can be multiple sound holes, which are evenly distributed around the circumference of the steering wheel.

[0127] Optionally, in some embodiments, when the sensor is an ultrasonic sensor, the number of sound outlets is at least two.

[0128] Optionally, in some embodiments, each of the plurality of sound holes corresponds one-to-one with an ultrasonic sensor.

[0129] Figure 10 The diagram shows an ultrasonic sensor installed in the steering wheel according to an embodiment of the present invention.

[0130] like Figure 10 As shown, a waterproof and breathable membrane 10 is provided between the sound outlet 05 and the ultrasonic sensor 15. This waterproof and breathable membrane is used to prevent moisture, such as sweat from the driver's hands or moisture in the air, from entering the ultrasonic sensor 15 through the sound outlet 05 and affecting the normal operation of the ultrasonic sensor 15.

[0131] Optionally, in some embodiments, the ultrasonic sensor 15 is mounted on the flexible printed circuit board 20 (FPC) using surface mount technology (SMT).

[0132] Figure 11 The diagram shown is a schematic representation of the layout of a flexible printed circuit board (FPC) according to an embodiment of the present invention.

[0133] like Figure 11 As shown, the FPC connects to the microprocessor by routing wires through the circumferential bracket of the steering wheel to the steering wheel shaft.

[0134] like Figure 10 As shown, a reinforcing plate 25 is installed after the FPC 20. The reinforcing plate 25 is used to reinforce the ultrasonic sensor and fixes the ultrasonic sensor 15 and the FPC 20 with screws 30.

[0135] like Figure 10 As shown, the direction of the sound outlet 05 can be determined by the area of ​​the driver's hand or fingers and the field of view (FOV) of the ultrasonic sensor, i.e., the coverage range of the ultrasonic signal.

[0136] like Figure 12The diagram shown is a schematic flowchart of a control method 200 for an in-vehicle system according to an embodiment of the present invention. In this method 200, the sensor installed in the steering wheel is an ultrasonic sensor. Figure 12 As shown, the method 200 includes:

[0137] 200, Initialize the ultrasonic sensor.

[0138] Optionally, in some embodiments, the initialization includes: powering on the ultrasonic sensor, setting the parameters of the ultrasonic sensor, such as setting the sampling time length of the ultrasonic sensor, setting an interrupt for the ultrasonic sensor, etc.

[0139] 210. Determine if the steering wheel is in a rotating state.

[0140] Optionally, in some embodiments, a gyroscope may be installed in the steering wheel. The gyroscope can determine whether the steering wheel is rotating based on the data it collects. When the steering wheel is rotating, it is not necessary to collect the ultrasonic signals sent by the ultrasonic sensor, that is, not to recognize the gestures performed by the driver, so as to ensure the accuracy of gesture recognition.

[0141] 220, Pre-judgment of ultrasonic signals.

[0142] Optionally, in some embodiments, after the receiving end of the ultrasonic sensor receives an ultrasonic signal within a certain period of time, it can perform a pre-judgment on the ultrasonic signal within that period of time to determine whether the time length between the ultrasonic signal being emitted and received meets a preset condition. The preset condition is that the time length between the ultrasonic signal being emitted and received is greater than or equal to a preset time threshold. When the preset condition is met, the received ultrasonic signal can be saved and further processed. When the preset condition is not met, the ultrasonic sensor does not save the current ultrasonic signal and re-receives the ultrasonic signal for the next period of time and performs a pre-judgment.

[0143] In this embodiment of the invention, by pre-judging the ultrasonic signal, signals in the ultrasonic signal that are not useful for gesture recognition can be removed in advance, thus ensuring the accuracy of gesture recognition.

[0144] 230, eliminate inherent noise signals.

[0145] Optionally, in some embodiments, the inherent noise signal may be a noise signal from a car engine or other mechanical structures.

[0146] Optionally, in some embodiments, these inherent frequency noise signals can be transformed in the time and frequency domains, and then a bandpass filter can be used to remove the inherent frequency noise signals.

[0147] 240, eliminate interference signals.

[0148] Optionally, in some embodiments, the interference signal is not a signal emitted by the ultrasonic sensor, so the interference signal is useless for driver gesture recognition. The interference signal can be converted between the time domain and the frequency domain, and a bandpass filter can be used to filter out the interference signal, retaining only the ultrasonic signal emitted by the ultrasonic sensor.

[0149] 250, determine the reflected signal.

[0150] Optionally, in some embodiments, after filtering out inherent noise and interference signals, the remaining signals in the ultrasonic signal are all signals emitted by the ultrasonic sensor. The ultrasonic signal includes direct signals and reflected signals. The direct signals are emitted from the transmitting end of the ultrasonic sensor and enter the receiving end of the ultrasonic sensor directly without being reflected by any object, so they are useless for gesture recognition. The reflected signals are emitted from the transmitting end of the ultrasonic sensor, reflected by an object, and then enter the receiving end. The object reflecting the ultrasonic waves may be the driver's hand or fingers, or other objects. Among the above reflected signals, those reflected by the driver's hand or fingers are the signals that are ultimately useful for gesture recognition.

[0151] Optionally, in some embodiments, the propagation distance of the ultrasonic signal can be calculated using the phase difference and delay parameters of the ultrasonic signal, and the reflected signal can be determined based on the different propagation distances. After determining the reflected signal, the propagation distance of the reflected signal can be calculated using the phase difference and delay parameters of the reflected signal, thereby determining the signal reflected by the driver's hand or fingers, that is, the reflected signal within the active area of ​​the driver's hand or fingers. This signal is the useful signal for the final gesture recognition.

[0152] 260. Based on the position information of each reflected signal at a certain moment, determine the outline information of the current gesture at a certain moment, and determine whether the outline information is valid.

[0153] By analyzing the reflected signal within the active area of ​​the driver's hand or fingers, the position information of the reflected signal at a certain moment is determined. Based on this position information, the contour information of the current gesture at that moment is determined, and the contour information is compared with pre-stored contour information. If the similarity is greater than or equal to a certain preset threshold, the contour information can be considered valid, that is, the contour information is the contour information of a human hand or fingers. Thus, the reflected signal can be further processed to determine the gesture represented by the reflected signal. If the contour information is invalid, the reflected signal can be left unprocessed, saving gesture recognition time.

[0154] By determining the contour information of the driver's hand or fingers, it can be determined whether the acquired reflection signal is a reflection signal reflected by the driver's hand or fingers, thereby further ensuring the accuracy of gesture recognition.

[0155] 270, determine the trajectory of the driver's current gesture.

[0156] Optionally, in some embodiments, after determining the reflected signal in the ultrasonic signal, the position of the driver's hand or fingers during that time period can be located based on the reflected signal during that time period. For example, the position coordinates of the driver's hand or fingers during that time period can be determined based on the ultrasonic multi-point positioning principle.

[0157] Optionally, in some embodiments, after locating the position of the driver's hand or fingers over a period of time, the trajectory of the driver's current gesture can be determined based on the obtained position information. For example, the position coordinates of the driver's hand or fingers over that period of time can be fitted with a function to obtain a fitted function curve, which is the trajectory of the driver's current gesture.

[0158] 280, Determine the type of the current gesture.

[0159] Optionally, in some embodiments, the type of the current gesture can be determined from a preset range of gesture types based on the determined trajectory of the current gesture.

[0160] For example, the function curve of the current gesture trajectory can be compared with function curves in the database, and the function curve with the highest matching degree can be selected. The gesture type corresponding to the function curve with the highest matching degree is the type of the current gesture.

[0161] Optionally, in some embodiments, the preset gesture type may include at least one of the following gesture types:

[0162] Slide both fingers inwards or outwards;

[0163] Slide two fingers clockwise or counterclockwise;

[0164] Swipe up, down, left, or right with a single finger;

[0165] Double tap with a single finger;

[0166] Waving with one hand;

[0167] Slide a single finger along the inner circumference of the steering wheel;

[0168] Double tap with two fingers.

[0169] 290. Based on the type of the current gesture, control the vehicle system to perform the operation corresponding to the type of the current gesture.

[0170] The in-vehicle system in this embodiment of the invention may be an in-vehicle head-up display (HUD), an in-vehicle navigation system, an in-vehicle menu system, an in-vehicle multimedia system, or other in-vehicle systems. This embodiment of the invention does not limit the specific in-vehicle system.

[0171] The above text combined Figures 1 to 12 The method embodiments of the present invention are described in detail below, and will be combined with the following description. Figure 13 and Figure 14 The embodiments of the vehicle system control device of the present invention are described in detail below. It should be understood that the embodiments of the vehicle system control device correspond to the method embodiments, and similar descriptions can be referred to the method embodiments.

[0172] Figure 13 This is a schematic block diagram of a control device 300 for an in-vehicle system according to an embodiment of the present invention. Figure 13 As shown, the device includes:

[0173] The first determining module 310 is used to determine the trajectory of the driver's current gesture by means of a sensor installed in the steering wheel while the driver is holding the steering wheel.

[0174] The second determining module 320 is used to determine the type of the current gesture from a preset range of gesture types based on the trajectory of the current gesture.

[0175] The control module 330 is used to control the vehicle system to perform the operation corresponding to the type of the current gesture.

[0176] Specifically, when the driver holds the steering wheel, sensors installed in the steering wheel can identify the types of gestures the driver makes and execute the corresponding operations on the vehicle system based on the different types of gestures. Throughout the process, the driver's hands can remain on the steering wheel, and different functions can be performed by using only some of the driver's fingers or hands to perform some gestures. This greatly facilitates the driver's needs for some functions while driving, ensures driving safety, and makes the interaction between the car and the driver closer.

[0177] It should be understood that the in-vehicle system in the embodiments of the present invention may be an in-vehicle head-up display (HUD), an in-vehicle navigation system, an in-vehicle menu system, an in-vehicle multimedia system, or other in-vehicle systems. The embodiments of the present invention do not limit this.

[0178] Optionally, in some embodiments, the sensor may be mounted in the circumference of the steering wheel.

[0179] Optionally, in some embodiments, there may be multiple sensors, each of which is connected in series to form a sensor array.

[0180] It should be understood that the specific number of sensors can be determined by the size of the steering wheel or by the required accuracy of gesture recognition. For example, when it is necessary to accurately recognize the gestures performed by the driver, the number of sensors can be increased appropriately.

[0181] Optionally, in some embodiments, the sensor may be an ultrasonic sensor.

[0182] Optionally, in some embodiments, there may be multiple ultrasonic sensors. The specific number of these multiple ultrasonic sensors may be determined by the size of the steering wheel or by the accuracy of the hand gesture recognition. These multiple ultrasonic sensors are connected in series to form an ultrasonic sensor array.

[0183] It should be understood that since gesture recognition is achieved through ultrasonic multi-point positioning, at least two ultrasonic sensors are required.

[0184] It should be understood that when the sensor is an ultrasonic sensor, the ultrasonic sensor itself can act as both a transmitter and a receiver.

[0185] Optionally, in some embodiments, when the sensor is an ultrasonic sensor, the ultrasonic sensor is specifically used to acquire ultrasonic signals collected by the plurality of ultrasonic sensors over a period of time; select reflected signals from the acquired ultrasonic signals; and locate the position of the driver's hand or fingers over a period of time based on the selected reflected signals to determine the trajectory of the current gesture.

[0186] Optionally, in some embodiments, the device 300 further includes a third determining module 340, which is used to determine whether the steering wheel is in a rotating state.

[0187] Optionally, in some embodiments, the preset gesture types include at least one of the following gesture types:

[0188] Slide both fingers inwards or outwards;

[0189] Slide two fingers clockwise or counterclockwise;

[0190] Swipe up, down, left, or right with one finger;

[0191] Double tap with a single finger;

[0192] Waving with one hand;

[0193] Slide a single finger along the inner circumference of the steering wheel;

[0194] Double tap with two fingers.

[0195] Optionally, in some embodiments, a plurality of sound holes are provided on the circumference of the steering wheel, and the plurality of sound holes correspond one-to-one with the plurality of ultrasonic sensors. The direction of the sound holes is determined by the area of ​​the driver's hand or fingers and the coverage of the ultrasonic signal.

[0196] Optionally, in some embodiments, a waterproof and breathable membrane is provided between the ultrasonic sensor and the sound outlet, the waterproof and breathable membrane being used to prevent water vapor from entering the ultrasonic sensor through the sound outlet.

[0197] It should be understood that the control device of the vehicle system according to the embodiments of the present invention may correspond to the control device of the vehicle system in the embodiments of the present invention, and the above-mentioned and other operations and / or functions of each module in the control device 300 of the vehicle system are respectively implemented. Figure 1 and Figure 12 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0198] Figure 14 This is a schematic structural diagram of a control device 400 for an in-vehicle system according to an embodiment of the present invention. Figure 14 As shown, the device 400 includes a memory 410 and a processor 420, which communicate with each other through an internal connection path to transmit control and / or data signals.

[0199] The memory 410 is used to store program code;

[0200] The processor 420 is used to call the program code to implement the methods in the above embodiments of the present invention.

[0201] In this embodiment of the invention, the processor 420 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof.

[0202] This invention provides a computer-readable storage medium for storing computer program code, the computer program including code for executing the above-described... Figures 1 to 12 The instructions for the fault location method in this embodiment of the invention are as follows. The readable storage medium may be a read-only memory (ROM) or a random access memory (RAM), and this embodiment of the invention does not limit it.

[0203] In embodiments of the present invention, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Video Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0204] It should be understood that the control device of the vehicle system according to the embodiments of the present invention may correspond to the control device of the vehicle system in the embodiments of the present invention, and the above-mentioned and other operations and / or functions of each module in the control device 400 of the vehicle system are respectively implemented. Figure 1 and Figure 12 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0205] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0206] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present invention.

[0207] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0208] In the several embodiments provided in this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0209] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0211] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A control method of an in-vehicle system, characterized by, The method comprises: determining a trajectory of a current gesture of a driver through sensors installed in a steering wheel; determining a category of the current gesture from preset gesture categories according to the trajectory of the current gesture; controlling a vehicle-mounted system to perform an operation corresponding to the category of the current gesture; wherein the steering wheel is installed with a plurality of sensors, and the plurality of sensors are a plurality of ultrasonic sensors; the determining of the trajectory of the current gesture of the driver through the sensors installed in the steering wheel comprises: acquiring ultrasonic signals collected by the plurality of ultrasonic sensors within a period of time; determining that the ultrasonic signals within an active area of a hand or a finger of the driver are reflection signals according to phase differences and time delay parameters of the ultrasonic signals; determining contour information of the current gesture at a first time according to position information of each reflection signal at the first time, and judging whether the contour information is valid; when the contour information is valid, positioning positions of the hand or the finger of the driver within the period of time according to the determined reflection signals to determine the trajectory of the current gesture.

2. The method of claim 1, wherein, Before the determining of the trajectory of the current gesture of the driver through the sensors installed in the steering wheel, the method further comprises: determining whether the steering wheel is in a rotating state; the determining of the trajectory of the current gesture of the driver through the sensors installed in the steering wheel comprises: determining the trajectory of the current gesture of the driver through the sensors installed in the steering wheel when the steering wheel is in a non-rotating state.

3. The method of claim 2, wherein, the steering wheel is installed with a gyroscope, the determining of whether the steering wheel is in the rotating state comprises: determining whether the steering wheel is in the rotating state according to the gyroscope.

4. The method according to any one of claims 1 to 3, characterized in that, the preset gesture categories at least include one of the following gesture categories: double-finger inward or outward sliding; double-finger clockwise or counterclockwise sliding; single-finger up-down or left-right sliding; single-finger double-clicking; single-hand waving; single-finger touch sliding along an inner side of a circumference of the steering wheel; double-finger double-clicking.

5. The method according to any one of claims 1 to 3, characterized in that, a plurality of sound holes are arranged on the circumference of the steering wheel, the plurality of sound holes correspond to the plurality of ultrasonic sensors one by one, and directions of the sound holes are determined by an active area of the hand or the finger of the driver and a coverage range of the ultrasonic signals.

6. The method according to any one of claims 1 to 3, characterized in that, the controlling of the vehicle-mounted system to perform the operation corresponding to the category of the current gesture comprises at least one of the following: controlling zooming of a map size in response to a first zooming operation corresponding to the category of the current gesture; or controlling zooming of a font size of display information in response to a second zooming operation corresponding to the category of the current gesture. the steering wheel has a steering angle of zero or is in the non-rotating state.

7. The method of claim 6, wherein, the device comprises:

8. A control device of a vehicle-mounted system, characterized by comprising: a first determining module configured to: determine a trajectory of a current gesture of a driver through sensors installed in a steering wheel; acquire ultrasonic signals collected by a plurality of ultrasonic sensors within a period of time; determine that the ultrasonic signals within an active area of a hand or a finger of the driver are reflection signals according to phase differences and time delay parameters of the ultrasonic signals; ​ The plurality of sensors are a plurality of ultrasonic sensors, According to position information of each of the reflected signals at the first time, contour information of the current hand gesture at the first time is determined, and it is determined whether the contour information is valid; When the contour information is valid, positions of the driver's hand or fingers in a period of time are located according to the determined reflected signals, so as to determine a trajectory of the current hand gesture; A second determining module is configured to determine a type of the current hand gesture from preset gesture types according to the trajectory of the current hand gesture. A control module is configured to control a vehicle-mounted system to perform an operation corresponding to the type of the current hand gesture.

9. The apparatus of claim 8, wherein, The device comprises a third determining module configured to determine whether the steering wheel is in a rotating state, The third determining module is specifically configured to determine the trajectory of the current hand gesture of the driver through the sensors installed in the steering wheel when the steering wheel is in a non-rotating state.

10. The apparatus of claim 9, wherein, The steering wheel is installed with a gyroscope, The third determining module is specifically configured to determine whether the steering wheel is in a rotating state according to the gyroscope.

11. The apparatus of any one of claims 8-10, wherein, The preset gesture types at least include one of the following gesture types: Double-finger inward or outward sliding; Double-finger clockwise or counterclockwise sliding; Single-finger up-down or left-right sliding; Single-finger double-clicking; Single-hand waving; Single-finger touch sliding along the inner side of the steering wheel circumference; Double-finger double-clicking.

12. The apparatus of any one of claims 8-10, wherein, The circumference of the steering wheel is provided with a plurality of sound holes, the plurality of sound holes correspond to the plurality of ultrasonic sensors one by one, and the direction of the sound hole is determined by the driver's hand or finger movement area and the coverage range of the ultrasonic signal.

13. The apparatus of any one of claims 8-10, wherein, The control module is configured to at least one of the following: In response to a first scaling operation corresponding to the type of the current hand gesture, the size of the map is scaled; or In response to a second scaling operation corresponding to the type of the current hand gesture, the font size of the display information is scaled.

14. The apparatus of claim 13, wherein, The steering wheel angle is zero or in a non-rotating state.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, when the instructions are run on the computer, the computer executes the method of any one of claims 1 to 7.

16. A vehicle characterized by comprising: The vehicle comprises the device of any one of claims 8 to 14. The vehicle comprises the device of any one of claims 8 to 14.

Images