Vehicle-mounted voice interaction method and system

Abstract

The present application provides a vehicle-mounted voice interaction method and system, which relates to the field of voice interaction technology, specifically: the vehicle-mounted infotainment controller uploads the de-noised voice data and vehicle information to the mobile terminal; the mobile terminal uploads the de-noised voice data, vehicle information and user information to the cloud server; the cloud server recognizes the voice data to obtain the voice data text, and when the voice data text includes the text information of the voice command, the voice data text is sent to the mobile terminal; when the voice data text includes the text information of the voice question, based on the vehicle information, user information and the text information of the voice question, the reply voice is matched from the preset answer voice database, and the reply voice is sent to the mobile terminal; the vehicle-mounted infotainment controller performs the corresponding operation according to the data sent by the mobile terminal. The present application can perform operation interaction in the visible and audible voice command mode in the car, which improves the user experience.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a vehicle-mounted voice interaction method and system. Background Art

[0002] The rapid development of information technology and Internet technology has driven many industries to transform towards digitalization and intelligence. The car navigation system in the intelligent era has also ushered in changes. The functions in the car are constantly increasing, and the human-computer interaction operation of the in-car navigation is becoming more and more complicated, making it increasingly costly for users to quickly and efficiently operate the functions.

[0003] With the development and application of voice technology, more and more voice interaction products have appeared in people's daily lives, and more and more research has been conducted in the field of voice interaction. The emergence of in-vehicle voice systems has greatly changed people's driving habits. Drivers can achieve the same efficiency as touching the in-vehicle screen only through full voice interaction, while avoiding distraction of the driver's attention and effectively ensuring the driver's safety during the journey.

[0004] Mobile phone interconnection technology is a technology that connects a smartphone to a vehicle navigation system through wired or wireless means, thereby achieving communication between the smartphone and the vehicle navigation system, greatly improving the intelligence of the vehicle navigation system and playing a significant role in expanding the functions of the vehicle system.

[0005] At present, the in-vehicle navigation systems are all based on some basic voice command interactions, and it is impossible to realize all the functions of the navigation system that can satisfy the driver's needs and have no safety hazards in a visible and audible way. Therefore, the voice interaction in the current in-vehicle navigation system cannot meet the requirements of users. Summary of the invention

[0006] In view of this, the present application provides an in-vehicle voice interaction method and system to solve the technical problem that the existing technology cannot implement all the functions in the navigation system that can be seen by the driver and have no safety hazards in a visible and audible way.

[0007] In a first aspect, an embodiment of the present application provides a vehicle-mounted voice interaction method, which is applied to a vehicle-mounted voice interaction system, wherein the vehicle-mounted voice interaction system includes: a vehicle-mounted infotainment controller, a mobile terminal, and a cloud server, and the method includes:

[0008] The in-vehicle infotainment controller acquires voice data and vehicle information, performs noise elimination on the voice data, and uploads the noise eliminated voice data and vehicle information to the mobile terminal;

[0009] The mobile terminal obtains user information and uploads the noise-removed voice data, vehicle information and user information to the cloud server;

[0010] The cloud server recognizes the voice data to obtain a voice data text. When the voice data text includes text information of a voice command, the voice data text is sent to the mobile terminal. When the voice data text includes text information of a voice question, a reply voice is matched from a preset answer voice database based on vehicle information, user information and text information of the voice question, and the reply voice is sent to the mobile terminal.

[0011] The mobile terminal sends the voice data text or the reply voice to the in-vehicle infotainment controller;

[0012] The in-vehicle infotainment controller performs corresponding operations according to the received data.

[0013] Furthermore, the vehicle information includes: the brand and model of the vehicle.

[0014] Furthermore, the user information includes: user gender, user age, user driving experience and user hobbies.

[0015] Furthermore, the preset answer voice database includes matching items and matching results; the matching items include: vehicle type, user gender, user age, driving age, user driving age and text information of the voice question, and the matching result is the answer voice;

[0016] When the voice data text includes text information of the voice question, based on the vehicle information, user information and the text information of the voice question, matching the answer voice from the preset answer voice database includes:

[0017] Get the model from the vehicle information;

[0018] Get user gender, user age and driving age from user information;

[0019] The vehicle model, user gender, user age, driving age and text information of the voice question are used as matching items, and the reply voice is matched from the preset answer voice database.

[0020] Furthermore, the in-vehicle infotainment controller performs corresponding operations according to the received data; including:

[0021] The in-vehicle infotainment controller receives data from the mobile terminal;

[0022] When the received data is a voice data text, determining whether the voice command in the voice data text is a local command, if so, executing the action corresponding to the voice command; otherwise, sending the voice command to the vehicle controller;

[0023] When the received data is a reply voice, the reply voice is played using a voice player.

[0024] Furthermore, the method further includes: the mobile terminal is connected to the vehicle infotainment controller wirelessly or by wire.

[0025] Furthermore, when the mobile terminal is interconnected with the vehicle infotainment controller via a USB cable, a USB data transmission channel is established between the two; when the mobile terminal is wirelessly interconnected with the vehicle infotainment controller via WIFI, a WIFI data transmission channel is established between the two.

[0026] Furthermore, the in-vehicle infotainment controller stores WIFI information in a QR code, and the mobile terminal scans the QR code on the in-vehicle infotainment controller to interconnect with the in-vehicle infotainment controller in a wireless WIFI manner.

[0027] In a second aspect, an embodiment of the present application provides an in-vehicle voice interaction system, including: an in-vehicle infotainment controller, a mobile terminal, and a cloud server;

[0028] The in-vehicle infotainment controller is used to obtain voice data and vehicle information, perform noise elimination on the voice data, and upload the noise-eliminated voice data and vehicle information to the mobile terminal; and is also used to perform corresponding operations according to the data received from the mobile terminal;

[0029] The mobile terminal is used to obtain user information, upload the noise-removed voice data, vehicle information and user information to the cloud server; and send the voice data text or reply voice to the in-vehicle infotainment controller;

[0030] The cloud server is used to recognize voice data and obtain voice data text. When the voice data text includes text information of voice commands, the voice data text is sent to the mobile terminal. When the voice data text includes text information of voice questions, a reply voice is matched from a preset answer voice database based on vehicle information, user information and text information of the voice question, and the reply voice is sent to the mobile terminal.

[0031] This application utilizes the in-vehicle infotainment controller, mobile terminal and cloud server to perform operational interactions in a visible-and-spoken voice command manner, thereby improving the user experience, enhancing the intelligence level of the in-vehicle system, ensuring safety during the entire operation process, and bringing the driver a perfect, comfortable, convenient and safe driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A schematic diagram of information interaction provided in an embodiment of the present application;

[0034] Figure 2 A flow chart of the in-vehicle voice interaction method provided in an embodiment of the present application;

[0035] Figure 3 A flowchart of in-vehicle speech recognition and processing provided for an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0037] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0038] First, the design concept of the embodiments of the present application is briefly introduced.

[0039] At present, the in-vehicle navigation systems are all based on some basic voice command interactions, and it is impossible to realize all the functions of the navigation system that can satisfy the driver's needs and have no safety hazards in a visible and audible way. Therefore, the voice interaction in the current in-vehicle navigation system cannot meet the requirements of users.

[0040] In order to solve the above technical problems, Figure 1As shown, the present application provides a vehicle voice interaction method, which collects the visible and audible voice instructions of the speaker through the microphone in the car and transmits them to the vehicle infotainment controller; since the voice instruction data will be interfered by the noise in the car, the vehicle infotainment controller performs noise elimination processing on the acquired voice instruction data; then the noise-eliminating data is transmitted to the mobile phone interconnected with the vehicle infotainment controller, and the mobile phone uses the network to upload the visible and audible voice instructions to the voice platform of the cloud server for recognition and processing; finally, the voice platform feeds back the recognition success result to the mobile phone, and the mobile phone further transmits the recognition success result to the vehicle infotainment controller, thereby completing the execution of the visible and audible voice instructions. If the visible and audible instructions are related to the car control, such as voice instructions such as "turn on the air conditioner" and "open the sunroof", the vehicle infotainment controller will further interact with the car after receiving the recognition result, and perform operations such as turning on the car air conditioner, opening the sunroof, and opening the windows through the car control instructions. If the visible and audible instructions are related to the vehicle navigation system, such as "open settings", "increase / decrease screen brightness", "change theme wallpaper", and "increase / decrease media volume", the vehicle navigation system performs the corresponding actions.

[0041] This application utilizes the in-vehicle infotainment controller, mobile phone and cloud server to perform operation interaction in the form of visible and spoken voice commands, thereby improving the user experience, enhancing the intelligence level of the in-vehicle system, ensuring safety during the entire operation process, and bringing the driver a perfect, comfortable, convenient and safe driving experience.

[0042] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0043] like Figure 2 As shown, an in-vehicle voice interaction method provided by an embodiment of the present application includes the following steps:

[0044] Step 101: the in-vehicle infotainment controller acquires voice data and vehicle information, performs noise elimination on the voice data, and uploads the noise eliminated voice data and vehicle information to a mobile terminal;

[0045] The in-vehicle infotainment controller includes: ARM processor, MCU processor, CAN module, USB, Bluetooth and WIFI modules, display screen, storage module and microphone. The ARM processor runs one or more programs, the MCU processor and CAN module realize communication with the vehicle, the USB module can connect to the USB flash drive device on the one hand, and can connect to the mobile phone APP on the other hand to realize wired connection, the Bluetooth module connects to the mobile phone, and the WIFI module connects to the mobile phone APP to realize seamless connection.

[0046] The mobile phone interconnection APP on the mobile phone is a navigation assistant developed specifically for driving. It integrates navigation, music, and phone functions; it supports the interconnection of mobile phones and in-vehicle infotainment controllers, and can support full voice interaction. For example, if a user says the visible and spoken voice command "call 10086", after voice recognition and processing, the action of dialing 10086 will eventually be executed on the mobile phone. Combining the mobile phone interconnection APP application with the in-vehicle infotainment controller enriches the visible and spoken voice commands in the in-vehicle voice interaction.

[0047] The mobile phone interconnection APP application on the mobile phone connects to the Linux system vehicle infotainment controller wirelessly or by wire. The mobile phone interconnection APP refers to a software that can connect the car navigation device and the mobile phone, which can display and control the content on the user's mobile phone through the car navigation device, and assist the driver to operate navigation, telephone, music and other applications safely and conveniently. Specifically, the mobile phone interconnection APP is interconnected with the Linux vehicle infotainment controller, i.e. the navigation device, in a wired manner via a USB data cable, and realizes data communication between the mobile phone and the vehicle-mounted device through the USB channel. Specifically, the mobile phone interconnection APP is interconnected with the Linux vehicle infotainment controller, i.e. the navigation device, through WIFI wirelessly, and realizes data communication between the mobile phone and the vehicle-mounted device through the WIFI channel.

[0048] Specifically, the visible and audible voice commands are based on the visible functions displayed to the user on the navigation device, and these voice commands will not affect driving safety. For example, "open settings", "increase / decrease screen brightness", "change theme wallpaper", "increase / decrease media volume", "open / close sunroof", "open / close air conditioning", etc. Users can achieve fast, convenient and efficient interaction with the functions of the in-vehicle navigation device through multiple visible and audible voice commands.

[0049] The microphone collects the sound source signal in the car. The in-vehicle infotainment controller obtains the visible and spoken voice command data. Since the in-vehicle environment will be affected by noise data, it is necessary to perform noise reduction processing on the visible and spoken voice commands to make the voice recognition more accurate.

[0050] The mobile phone and the in-vehicle infotainment controller are interconnected via Bluetooth. The purpose of connecting Bluetooth is to communicate the mobile phone media-related data with the vehicle computer via Bluetooth. For example, when the user says "I want to listen to Jay Chou's songs", the music media player on the mobile phone and the vehicle computer media information transmission use the Bluetooth channel.

[0051] Before the mobile phone and the in-vehicle infotainment controller are interconnected, you need to enable relevant permissions on the mobile phone. For example, when the mobile phone interconnection APP application is used for the first time, you need to enable the permission to locate the mobile phone and the permission to access contacts.

[0052] Step 102: The mobile terminal obtains user information and uploads the noise-removed voice data, vehicle information and user information to the cloud server;

[0053] The vehicle information includes: the brand and model of the vehicle. The user information includes: user gender, user age, user driving experience and user hobbies.

[0054] Step 103: The cloud server recognizes the voice data, and when the voice data is a control operation instruction, sends the control operation instruction text to the mobile terminal; when the voice data is a voice question, obtains a voice answer text of the voice question based on the vehicle information and the user information, and sends the voice answer text to the mobile terminal;

[0055] In this embodiment, the cloud server recognizes the voice data through a semantic platform, which includes an autonomous learning module, a supervised learning module, a multi-round dialogue module, and a personalized semantic module;

[0056] The self-learning module is used to learn the user's true intention by using clustering and feedback analysis methods. For example, if a user says "I want to go shopping", the user's intention cannot be understood at first, but then a large number of users choose to go to the supermarket. Through advanced learning, the user is asked whether he wants to go to the supermarket. When a large number of users choose to go to the supermarket, it is determined that the user's intention to buy something is to go to the supermarket, and a self-learning process of the semantic platform can be completed.

[0057] The supervised learning module is used to perform clustering and summary analysis based on the collected data in an unsupervised state, manually annotate the clustering results, and retrain the model with the annotated results so that the machine can recognize new intent capabilities.

[0058] The multi-round dialogue processing module is used for example for user voice interaction "What is the air circulation for?" and the second round voice interaction "How to turn it on?", which refers to how to turn on the car's air circulation. Through multi-round dialogue, the visible and spoken instructions that the user needs to execute are completed. The multi-round dialogue enables the Linux system in-vehicle infotainment controller or the vehicle to perform the target action in the form of multi-round dialogue according to the visible and spoken voice instructions entered by the user.

[0059] The personalized semantic module is used to obtain the voice answer text of the voice question based on the vehicle information and user information when the voice data is a voice question; it is specifically used for:

[0060] Get the model from the vehicle information;

[0061] Get user gender, user age and driving age from user information;

[0062] Based on the vehicle model, user gender, user age, driving age and voice questions in the voice text, the voice reply text is matched from the preset answer voice database.

[0063] The preset answer voice database includes: vehicle type, user gender, user age, driving age, voice questions and voice answers;

[0064] In this way, different responses can be customized according to the users, and different responses can be given to user groups with different characteristics. For example, different responses can be given to male and female drivers, and different responses can be given to new and old drivers.

[0065] Speech recognition includes multiple speech recognition methods. Among them, wake-up-free recognition is to recognize voice commands without fixed wake-up words and then execute the corresponding actions of the commands; wake-up recognition first requires a fixed wake-up word to activate the speech recognition process, and then perform voice command recognition and corresponding actions.

[0066] like Figure 3 As shown, first, the visible and speakable voice instructions are monitored; when the visible and speakable voice instructions are received, preprocessing is performed; then it is determined whether there is a wake-up word in the wake-up recognition. If there is no wake-up word, semantic analysis is performed, and then the semantic recognition result information is broadcast, and finally the instruction is executed; if there is a wake-up word during the wake-up recognition, voice broadcast will be performed, and then semantic analysis will be performed. The visible and speakable instructions are understood through multiple rounds of dialogue, and finally the semantic recognition result information is played and the instruction is executed.

[0067] Step 104: the mobile terminal sends the control operation instruction text or the voice reply text to the in-vehicle infotainment controller;

[0068] Step 105: The in-vehicle infotainment controller performs corresponding operations according to the received data type.

[0069] Specifically, the in-vehicle infotainment controller receives data from a mobile terminal; when the received data type is a control operation instruction, it determines whether the control operation instruction is a local instruction, and if so, executes the action corresponding to the control operation instruction; otherwise, the control operation instruction is sent to the vehicle controller; when the received data is a voice reply text, a voice player is used to play the voice reply text.

[0070] The embodiment of the present application provides an in-vehicle voice interaction system, including: an in-vehicle infotainment controller, a mobile terminal and a cloud server;

[0071] The in-vehicle infotainment controller is used to obtain voice data and vehicle information, perform noise elimination on the voice data, and upload the noise-eliminated voice data and vehicle information to the mobile terminal; and is also used to perform corresponding operations according to the data received from the mobile terminal;

[0072] The mobile terminal is used to obtain user information, upload the noise-removed voice data, vehicle information and user information to the cloud server; and send the voice data text or reply voice to the in-vehicle infotainment controller;

[0073] The cloud server is used to recognize voice data and obtain voice data text. When the voice data text includes text information of voice commands, the voice data text is sent to the mobile terminal. When the voice data text includes text information of voice questions, a reply voice is matched from a preset answer voice database based on vehicle information, user information and text information of the voice question, and the reply voice is sent to the mobile terminal.

[0074] It should be noted that the principle of solving the technical problem by the in-vehicle voice interaction system provided in the embodiment of the present application is similar to the in-vehicle voice interaction method provided in the embodiment of the present application. Therefore, the implementation of the in-vehicle voice interaction system provided in the embodiment of the present application can refer to the implementation of the in-vehicle voice interaction method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0075] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0076] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle-mounted voice interaction method, characterized in that: Applied to an in-vehicle voice interaction system, the in-vehicle voice interaction system includes: an in-vehicle infotainment controller, a mobile terminal and a cloud server, and the method includes: The in-vehicle infotainment controller acquires voice data and vehicle information, performs noise elimination on the voice data, and uploads the noise eliminated voice data and vehicle information to the mobile terminal; The mobile terminal obtains user information and uploads the noise-removed voice data, vehicle information and user information to the cloud server; The cloud server recognizes the voice data to obtain a voice data text. When the voice data text includes text information of a voice command, the voice data text is sent to the mobile terminal. When the voice data text includes text information of a voice question, a reply voice is matched from a preset answer voice database based on vehicle information, user information and text information of the voice question, and the reply voice is sent to the mobile terminal. The mobile terminal sends the voice data text or the reply voice to the in-vehicle infotainment controller; The in-vehicle infotainment controller performs corresponding operations according to the received data; The vehicle information includes: the brand and model of the vehicle; The user information includes: user gender, user age, user driving experience and user hobbies; The preset answer voice database includes matching items and matching results; the matching items include: vehicle type, user gender, user age, driving age, user driving age and text information of the voice question, and the matching result is the answer voice; When the voice data text includes text information of the voice question, based on the vehicle information, user information and the text information of the voice question, matching the answer voice from the preset answer voice database includes: Get the model from the vehicle information; Get user gender, user age and driving age from user information; The vehicle model, user gender, user age, driving age and text information of the voice question are used as matching items, and the reply voice is matched from the preset answer voice database.

2. The vehicle-mounted voice interaction method according to claim 1, characterized in that: The in-vehicle infotainment controller performs corresponding operations according to the received data; including: The in-vehicle infotainment controller receives data from the mobile terminal; When the received data is a voice data text, determining whether the voice command in the voice data text is a local command, if so, executing the action corresponding to the voice command; otherwise, sending the voice command to the vehicle controller; When the received data is a reply voice, the reply voice is played using a voice player.

3. The vehicle-mounted voice interaction method according to claim 1, characterized in that: The method further includes: the mobile terminal is connected to the vehicle infotainment controller wirelessly or by wire.

4. The vehicle-mounted voice interaction method according to claim 3, characterized in that: When the mobile terminal is interconnected with the vehicle infotainment controller via a USB cable, a USB data transmission channel is established between the two; when the mobile terminal is wirelessly interconnected with the vehicle infotainment controller via WIFI, a WIFI data transmission channel is established between the two.

5. The vehicle-mounted voice interaction method according to claim 4, characterized in that: The vehicle infotainment controller stores WIFI information in a QR code, and the mobile terminal is interconnected with the vehicle infotainment controller in a wireless WIFI manner by scanning the QR code on the vehicle infotainment controller.

6. A vehicle-mounted voice interaction system, characterized in that: The system adopts the in-vehicle voice interaction method according to any one of claims 1 to 5, comprising: an in-vehicle infotainment controller, a mobile terminal and a cloud server; The in-vehicle infotainment controller is used to obtain voice data and vehicle information, perform noise elimination on the voice data, and upload the noise-eliminated voice data and vehicle information to the mobile terminal; and is also used to perform corresponding operations according to the data received from the mobile terminal; The mobile terminal is used to obtain user information, upload the noise-removed voice data, vehicle information and user information to the cloud server; and send the voice data text or reply voice to the in-vehicle infotainment controller; The cloud server is used to recognize the voice data and obtain the voice data text. When the voice data text includes the text information of the voice command, the voice data text is sent to the mobile terminal. When the voice data text includes the text information of the voice question, based on the vehicle information, the user information and the text information of the voice question, the reply voice is matched from the preset answer voice database, and the reply voice is sent to the mobile terminal. The vehicle information includes: the brand and model of the vehicle; The user information includes: user gender, user age, user driving experience and user hobbies; The preset answer voice database includes matching items and matching results; the matching items include: vehicle type, user gender, user age, driving age, user driving age and text information of the voice question, and the matching result is the answer voice; When the voice data text includes text information of a voice question, the cloud server is also used to: obtain the vehicle model from the vehicle information; obtain the user's gender, user age and driving age from the user information; and match the reply voice from the preset answer voice database using the vehicle model, user gender, user age, driving age and the text information of the voice question as matching items.

Images