Human-machine interaction device based on wireless communication

Abstract

A human-computer interaction device based on wireless communication, comprising: a wireless communication unit; a controller which communicates with a mobile terminal via the wireless communication unit and communicates with an in-vehicle infotainment system through an in-vehicle data bus; wherein the controller maintains a piano key operation interface at the mobile terminal and obtains operation information of a user on the piano key operation interface through wireless communication with the mobile terminal; and sends a control instruction related to an infotainment function corresponding to a user combined key operation represented by the operation information to the in-vehicle infotainment system. The human-computer interaction device simplifies user operation and avoids safety hazards.

Description

Technical Field

[0001] This invention relates to the development of human-computer interaction interface technology, and particularly to a wireless communication-based human-computer interaction device for in-vehicle infotainment systems. Background Technology

[0002] Currently, depending on the hardware configuration of the in-vehicle infotainment system, users can generally control the system via buttons or touchscreen. Furthermore, feedback information or function demonstrations for the corresponding operations can be obtained through the display screen included in the in-vehicle infotainment system.

[0003] However, with the continuous introduction of new technologies, the application functions and services offered by in-vehicle infotainment systems are becoming increasingly rich. Most mainstream car models integrate numerous infotainment functions such as radio, music playback, and climate control into their in-vehicle infotainment systems. Consequently, the user interface of in-vehicle infotainment systems contains an ever-growing number of menus.

[0004] The resulting problem is that, whether using buttons or touchscreens, users need to spend a significant amount of time navigating the user interface to find the desired application function from numerous menus. On one hand, this is inconvenient for users; on the other hand, while driving, this operation can distract them, potentially leading to traffic accidents. Summary of the Invention

[0005] The problem solved by this invention is to provide a human-computer interaction device based on wireless communication to simplify user operation and improve driving safety.

[0006] To address the above problems, the present invention provides a human-computer interaction device based on wireless communication, comprising:

[0007] Wireless communication unit;

[0008] The controller communicates wirelessly with the mobile terminal via a wireless communication unit and with the in-vehicle infotainment system via an in-vehicle data bus. The controller maintains a keyboard-style interface on the mobile terminal and acquires user operation information on the interface via wireless communication. It also sends control commands related to the infotainment functions corresponding to the user's key combinations, as represented by the operation information, to the in-vehicle infotainment system.

[0009] Compared with existing technologies, the above solution has the following advantages: it provides users with a way to operate in-vehicle infotainment functions through mobile terminals, so that passengers in the vehicle can also directly control the in-vehicle infotainment system, avoiding the safety hazards caused by the driver having to operate the in-vehicle infotainment system while driving; in addition, passengers in the vehicle can quickly operate various infotainment functions by using combination buttons, without having to go through multiple menu operations as in existing technologies to obtain the application functions, which also simplifies the operation for passengers in the vehicle. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of one embodiment of the human-computer interaction device of the present invention;

[0011] Figure 2 This is a schematic diagram illustrating the implementation of a combination key operation prompting the user via a white key display in one embodiment of the human-computer interaction device according to the present invention;

[0012] Figure 3 This is a schematic diagram illustrating the implementation of a combination key operation prompting the user via a white key display in another embodiment of the human-computer interaction device according to the present invention. Detailed Implementation

[0013] In the following description, numerous specific details are set forth to enable those skilled in the art to fully understand the invention. However, it will be apparent to those skilled in the art that implementation of the invention may not include some of these specific details. Furthermore, it should be understood that the invention is not limited to the specific embodiments described. Rather, the invention can be conceived to be practiced with any combination of the following features and elements, regardless of whether they relate to different embodiments. Therefore, the following aspects, features, embodiments, and advantages are illustrative only and should not be construed as elements or limitations of the claims unless expressly set forth in the claims.

[0014] The inventors of this invention believe that, considering the devices used for controlling in-vehicle infotainment systems, current in-vehicle environments already support various wireless access methods, such as Bluetooth and WiFi. Furthermore, current mobile terminals (smartphones, tablets, etc.) are standard equipment supporting wireless communication and are almost ubiquitous. Therefore, mobile terminals can be used as control panels for passengers other than the driver to operate the in-vehicle infotainment system. Based on this, the human-machine interface device applicable to this invention must have the ability to communicate wirelessly with the mobile terminal and also with the in-vehicle infotainment system, thereby enabling the transmission of control commands from the mobile terminal to the in-vehicle infotainment system.

[0015] Furthermore, considering the process of implementing in-vehicle infotainment control, the user interfaces of existing in-vehicle infotainment systems mostly categorize application functions into broad categories. This leads to the user's desired final service (such as listening to a radio program on a specific frequency) often being located in a second-level or even third-level submenu. Therefore, users have to navigate through level after level of submenus to find and select the desired application function. For users unfamiliar with menu hierarchies, the time required to find and select the final service can be even longer.

[0016] Analysis of the various application services provided by in-vehicle infotainment systems reveals that most application services can be covered by simply adding two variables to the system's core functions. For example, when listening to the radio, the user's desired final application service is limited to a few options such as "radio + volume adjustment," "radio + channel switching," and "radio + favorite channels." Each of these final application services can be determined by combining "radio" with another function option. Based on this, the inventors of this invention believe that by combining main function buttons (typically broad categories such as radio, music playback, and air conditioning adjustment) with auxiliary function buttons (optional options within each category), users can directly determine their desired final application service without having to navigate through multiple menu items as in existing technologies.

[0017] Reference Figure 1 As shown, according to one embodiment of the human-computer interaction device of the present invention, it includes:

[0018] Wireless communication unit 10;

[0019] The controller 20 communicates wirelessly with the mobile terminal 40 via the wireless communication unit 10 and with the in-vehicle infotainment system 30 via the vehicle data bus. The controller 20 maintains a keyboard-style interface on the mobile terminal 40 and acquires user operation information on the keyboard-style interface via wireless communication with the mobile terminal 40. It also sends control commands related to the infotainment functions corresponding to the user's key combinations represented by the operation information to the in-vehicle infotainment system 30.

[0020] In the above embodiments, the control commands refer to control commands corresponding to the aforementioned final application function services. For example, a control command to turn on a certain radio station frequency, or a control command to operate the air conditioner in economy mode. Therefore, when passengers in the vehicle perform combination button operations via the keyboard-style interface on the mobile terminal, the controller sends the corresponding control commands to the in-vehicle infotainment system, directly controlling the in-vehicle infotainment system to activate the corresponding application function services. The method of controlling the in-vehicle infotainment system through these control commands is essentially no different from the method of controlling the in-vehicle infotainment system using existing human-computer interaction devices such as buttons / touchscreens. For the controller, it only needs to ensure that the sent control commands meet the requirements of the in-vehicle infotainment system. This will not be elaborated further here. Furthermore, since the touch recognition and display capabilities of the mobile terminal itself can be utilized, from the perspective of control implementation, the human-computer interaction device of this invention does not require buttons / touchscreens or displays, and its hardware configuration requirements are greatly reduced compared to existing technologies. Therefore, the cost and complexity of hardware design are also significantly reduced.

[0021] According to various embodiments of the present invention, the controller may have different configurations. In one embodiment of the present invention, considering adaptability to various mobile terminals, the process of parsing user operations and converting them into control commands is configured to be implemented in the controller. In this case, the mobile terminal only performs the following functions: providing a keyboard-style operation interface display; and performing touch recognition on the user's combination key operations on the keyboard-style operation interface, and sending the touch recognition result (the user's position information on the keyboard-style operation interface) to the controller.

[0022] Accordingly, the controller may include: a data communication unit connected to an in-vehicle data bus, such as a CAN bus or a LIN bus; a user interface unit maintaining a key-style operating interface suitable for display on a mobile terminal, for example, providing image data suitable for displaying a key-style operating interface on a mobile terminal; and a recognition unit storing a list of control commands related to infotainment functions corresponding to various key combinations. After obtaining the above-mentioned touch recognition result through wireless communication with the mobile terminal, the recognition unit obtains the information of the combination keys based on the key-style operating interface provided by the user interface unit and the position information contained in the touch recognition result. Then, it determines the control command corresponding to the combination key operation based on the control command list, and sends the determined control command to the in-vehicle infotainment system via the in-vehicle data bus through the data communication unit.

[0023] Alternatively, in other embodiments of the present invention, the process of parsing user operations and converting them into control commands can be implemented in the mobile terminal. For example, the control command list described above can be stored in the mobile terminal, and the processor of the mobile terminal can convert the user's combination key operations into control commands based on the control command list. Accordingly, the controller can be simplified to: a data communication unit connected to the vehicle data bus; a user interface unit maintaining a keyboard-style operation interface suitable for display on the mobile terminal; and a processing unit that sends the control commands obtained from the mobile terminal to the vehicle infotainment system via the vehicle data bus through the data communication unit. That is, the controller here is more inclined to serve as a message processing interface between the mobile terminal and the vehicle infotainment system. Therefore, the design of the controller is simplified.

[0024] To optimize the user's operating experience, according to other embodiments of the present invention, when any white key is operated, the user interface unit in the controller displays a function identifier on other white keys that can be combined with the function to form a control command, based on the function corresponding to the white key, so as to prompt the user and facilitate the user's combination key operation.

[0025] For example, refer to Figure 2 As shown, assuming a piano key interface consisting of 4 white keys and 3 black keys, the 4 white keys initially display the radio function (FM), music playback function (MP3), air conditioning control function (AIR), and user settings function (FUNC), respectively. When the user presses the white key displaying the FM function, the user interface unit in the controller updates the maintained image data of the piano key interface after the user has pressed the white key displaying the FM function, adjusting the display on the other white keys and displaying the radio-related function icons on those other white keys. For example... Figure 3 As shown in the upper middle diagram, the other white keys currently display function icons for radio channels: 101.7, 103.7, and 105.7. When the user holds down the white key displaying the FM function icon and then presses another white key displaying the 101.7 function icon, a combination key operation is completed. This activates the FM 101.7 radio channel. As explained above, the control command to activate FM 101.7 is sent to the in-vehicle infotainment system via the vehicle data bus (e.g., CAN or LIN). Upon receiving this control command, the in-vehicle infotainment system activates the radio function, sets the current station channel to FM 101.7, and plays radio programs through the vehicle's speakers. Similarly, users can also perform quick radio listening operations by combining the white key displaying the FM function icon with other white keys displaying radio channel function icons (103.7 or 105.7).

[0026] Similarly, for example Figure 3 As shown in the lower image, when the white key displaying the MP3 function icon is pressed, other white keys can display functions such as pause (||) and loop playback (||). ) and fast forward ( The white keys displaying the MP3 function icon and the loop playback function icon are used to indicate music playback functionality. Users can set the music to loop by pressing and holding down the white key displaying the loop playback function, for example, while simultaneously pressing and holding down the white key displaying the loop playback function.

[0027] As can be seen from the examples above, displaying suggestive function indicators on other white keys after a user presses a white key makes it easier for the user to perform combination key operations, further improving the user's operating experience.

[0028] Similarly, according to other embodiments of the present invention, when any white key is operated, the user interface unit in the controller displays not only the function identifier of the operated white key itself, but also the function identifier of the adjacent black key, so that it can be combined with the function of the white key itself to form a control command.

[0029] For example, refer to Figure 3 As shown, when a user presses the white key displaying the MP3 function icon, the user interface unit in the controller updates the image data of the maintained keyboard-style operation interface after the user has pressed the white key representing the MP3 function, and displays the volume adjustment function on the white key. The function indicator (MP3) is used to display the function represented by the black key to the left or right of the white key. Therefore, by holding down the white key displaying the MP3 function indicator while continuing to press the black key to the left or right, the user can adjust the volume of the currently playing music.

[0030] Of course, to further optimize the user experience, the above-mentioned display prompts for function identification on other white keys and display prompts for function identification on the white key being operated and the adjacent black keys can be implemented simultaneously.

[0031] Furthermore, considering user safety while driving, in addition to providing the aforementioned shortcut operations for the combination keys, according to other embodiments of the present invention, a feedback sound can be given to the user when performing combination key operations to help the user confirm their key operations through sound. Specifically, the controller can store audio data corresponding to each key, and when the user operates the keys on the piano key interface, the controller sends the audio data of the corresponding key to the mobile terminal, through which the mobile terminal plays the corresponding audio.

[0032] Still with Figure 2Taking the piano key interface shown as an example, the simplest implementation is to assign the audio to each white key in sequence, corresponding to "Do", "Ruai", "Mi", and "Fa", and to each black key in sequence, corresponding to "Sou", "La", and "Xi". Alternatively, corresponding voice prompts can be assigned to the white and black keys. In this way, when the user presses a key, its corresponding function will be announced via voice on the mobile terminal.

[0033] Of course, instead of setting individual feedback sounds for each white and black key, a corresponding feedback sound can be set for each control command associated with a combination of keys. In this way, when the user completes a combination of key presses, the controller retrieves the audio data corresponding to the current control command and sends it to the mobile terminal for playback. Furthermore, setting feedback sounds for each white and black key and setting feedback sounds for each control command can also be used in combination.

[0034] Furthermore, considering users' personalized needs, the design of this invention can also provide an editing function for key combinations. For example, during user operation... Figure 3 When the “FUNC” button is pressed on the interface shown, function setting options are provided for each white key. Users can determine the desired function identifier in the white key area where function settings are required by clicking or swiping gestures. After the controller obtains the user's setting operation on the mobile terminal through wireless communication, it updates the image data of the piano key-style operation interface it maintains through its user interface unit.

[0035] While the present invention has been disclosed above with reference to preferred embodiments, it is not limited thereto. Any modifications and alterations made by those skilled in the art without departing from the spirit and scope of the invention should be included within the scope of protection of the invention. Therefore, the scope of protection of the invention should be determined by the scope defined in the claims.

Claims

1. A human-computer interaction device based on wireless communication, characterized in that, include: Wireless communication unit; The controller communicates wirelessly with the mobile terminal via the wireless communication unit and with the in-vehicle infotainment system via the vehicle data bus. The controller maintains a keyboard-style interface on the mobile terminal and acquires user operation information on the interface via wireless communication. It also sends control commands related to infotainment functions corresponding to user key combinations based on the operation information to the in-vehicle infotainment system. These key combinations include combinations of white keys on the keyboard interface or combinations of white and black keys. When any white key is operated, the controller displays a first function identifier on the operated white key that can be combined with that function to form a first control command, based on the function corresponding to that white key. The first function identifier is the function identifier of the black key adjacent to the operated white key on the keyboard interface. Furthermore, it displays a second function identifier on other white keys on the keyboard interface that can be combined with that function to form a second control command.

2. The human-computer interaction device based on wireless communication as described in claim 1, characterized in that, The controller includes: a data communication unit connected to the vehicle data bus; a user interface unit maintaining a keyboard-style operation interface suitable for display on the mobile terminal; and an identification unit storing a list of control commands related to infotainment functions corresponding to various key combinations. The identification unit obtains information about user key combinations from the operation information, determines the control command corresponding to the key combination operation based on the control command list, and sends the determined control command to the vehicle infotainment system via the vehicle data bus through the data communication unit. Alternatively, the controller may include: a data communication unit connected to the vehicle data bus; a user interface unit maintaining a keyboard-style operating interface suitable for display on the mobile terminal; and a processing unit that, through the data communication unit, transmits the control commands obtained from the mobile terminal to the vehicle infotainment system via the vehicle data bus.

3. The human-computer interaction device based on wireless communication as described in claim 1, characterized in that, The controller stores audio data corresponding to each key. When a user operates a key on the keyboard-style interface, the controller sends the audio data of the corresponding key to the mobile terminal, and the mobile terminal plays the corresponding audio. Alternatively, the controller stores audio data of each control command. When a user performs a combination key operation, the controller sends the audio data of the corresponding control command to the mobile terminal, and the mobile terminal plays the corresponding audio.

4. The human-computer interaction device based on wireless communication as described in claim 1, characterized in that, The controller obtains the user's settings on the mobile terminal via wireless communication and adjusts the content of the maintained keyboard-style interface accordingly based on the settings.

5. The human-computer interaction device based on wireless communication as described in claim 1, characterized in that, The wireless communication unit supports Bluetooth communication and / or WiFi communication.

6. The human-computer interaction device based on wireless communication as described in claim 1, characterized in that, The control commands are the control commands for the final application function services corresponding to the infotainment function.

Images