Vehicle-mounted voice interaction method and device, vehicle, medium and product

Through the cloud intention recognition model, the user's voice intention is identified and recommended functions are generated, and the access controls are displayed, which solves the problem of insufficient voice interaction capabilities in the car, improves user operation convenience and demand satisfaction, and improves the driving experience.

CN118942459APending Publication Date: 2024-11-12CHONGQING CHANGAN AUTOMOBILE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411184628.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing vehicle voice interaction solutions have weak interaction capabilities, and cannot effectively identify user intentions and provide personalized functional recommendations, resulting in inconvenient operation and inconsistency in user needs.

Method used

Through cloud computing power training, intent recognition model, user voice and intent are received, response content and recommendation functions are generated, access controls are displayed, and users can directly select and execute recommendation functions to improve interaction convenience and demand satisfaction.

Benefits of technology

It significantly improves the convenience and interaction ability of users to operate on-board applications, provides higher demand satisfaction and accuracy, and improves the driving experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942459A_ABST
    Figure CN118942459A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of vehicle control, and discloses a vehicle-mounted voice interaction method and device, a vehicle, a medium and a product, and the method comprises the steps that the vehicle receives user voice, and sends the user voice to a cloud; receiving response content and a recommendation function sent by the cloud; playing the response content through voice, and displaying the response content on a vehicle-mounted display screen; obtaining a corresponding vehicle-mounted application access interface according to the recommendation function, and creating an access control corresponding to the vehicle-mounted application access interface; and displaying an access control corresponding to the vehicle-mounted application access interface on the vehicle-mounted display screen. According to the invention, the vehicle-mounted voice interaction capability is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to an in-vehicle voice interaction method, device, vehicle, medium and product. Background Art

[0002] The existing in-vehicle voice interaction solutions are relatively simple, and almost all of them display the text spoken by the user on the status bar, and then answer the user's questions, or operate the corresponding functions according to the user's voice commands. There are some improved technologies that can not only operate the corresponding functions according to the user's voice commands, but also detect other functions that the user often operates in conjunction with the current function before, and then the robot replies with voice suggestions for users to try other voice commands. The interactive capabilities of this voice method need to be further improved. Summary of the invention

[0003] In view of this, the present invention provides an in-vehicle voice interaction method, device, vehicle, medium and product to solve the problem of weak in-vehicle voice interaction capability.

[0004] In a first aspect, the present invention provides an in-vehicle voice interaction method, which is applied to a vehicle, and the method includes: receiving user voice and sending the user voice to the cloud; receiving response content and recommended functions sent by the cloud, the response content is reply language information generated according to the user intention after the cloud recognizes the intention of the user voice based on a pre-trained intention recognition model; the recommended function is a vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user intention; playing the response content by voice and displaying the response content on a vehicle display screen; obtaining a corresponding in-vehicle application access interface according to the recommended function, and creating an access control corresponding to the in-vehicle application access interface; and displaying the access control corresponding to the in-vehicle application access interface on the vehicle display screen.

[0005] According to the above technical means, the intention recognition model is trained by using the cloud computing power. When the user gives a voice command, the vehicle sends the user's voice to the cloud. The cloud recognizes the user's voice through the intention recognition model, determines the user's intention, and responds according to the user's intention. It also analyzes the subsequent vehicle function operations that most people like and are accustomed to based on the user's intention, and generates recommended functions. The response content and recommended functions are fed back to the vehicle by the cloud. The vehicle plays the response content, calls the corresponding vehicle application access interface according to the recommended function, and displays the access control corresponding to the interface on the vehicle screen, providing users with a variety of choices. Therefore, the user can directly select the recommended subsequent operation according to the access control displayed on the screen, start the corresponding application to perform the corresponding function, without the user having to issue an additional command to start the corresponding application, which significantly improves the convenience of users operating vehicle applications, and provides users with vehicle application choices with higher demand rates, improves the ability of vehicle voice and user interaction, and improves the user's driving experience.

[0006] In some optional implementations, the method further includes: if the response content includes logic for executing a target function, calling a target in-vehicle application according to the response content to execute the target function.

[0007] According to the above technical means, if the response content of the cloud reply includes the logic of starting a certain vehicle target function, the vehicle side will recognize the logic and call the target vehicle application to execute the target function according to the response content, ensuring the accurate and reliable execution of the user's voice commands.

[0008] In some optional implementations, the method further includes: detecting a triggering state of the access control; when the access control is triggered, calling a corresponding in-vehicle application through the in-vehicle application access interface to execute the recommended function.

[0009] In some optional embodiments, before pre-training the intent recognition model, the method also includes: collecting corresponding vehicle functions of vehicle-mounted applications installed on the current vehicle; and sending the vehicle functions to the cloud so that the cloud trains the intent recognition model through the vehicle functions.

[0010] According to the above-mentioned technical means, before using the voice function, the user's vehicle periodically uploads the available vehicle functions supported by the installed in-vehicle applications of the current vehicle, so that the trained intent recognition model can understand which functions the current vehicle supports. When recommending functions later, it ensures that the recommended functions are functions that the current vehicle can directly start and run, thereby realizing customized training of the intent recognition model and improving the accuracy of recommended functions.

[0011] In a second aspect, the present invention provides a vehicle-mounted voice interaction method, which is applied to the cloud, and the method includes: receiving a user voice sent by a vehicle end, wherein the user voice is a voice command collected by the vehicle end; inputting the user voice into a pre-trained intention recognition model, and recognizing the intention of the user voice through the intention recognition model to obtain response content and recommended functions, wherein the response content is reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention; sending the response content and the recommended function to the vehicle end, so that the vehicle end voice plays the response content and displays the response content on the vehicle-mounted display screen, and then obtaining the corresponding vehicle-mounted application access interface according to the recommended function, and creating an access control corresponding to the vehicle-mounted application access interface, and then displaying the access control corresponding to the vehicle-mounted application access interface on the vehicle-mounted display screen.

[0012] In some optional embodiments, the step of training the intention recognition model includes: receiving vehicle functions sent by the target vehicle; obtaining user usage habit big data, the user usage habit big data including various types of voices emitted by different users, intent labels corresponding to each type of voice, and user preference functions related to each type of intent; training an intention recognition model suitable for the target vehicle based on the vehicle functions and the user usage habit big data.

[0013] In some optional implementations, the intention of the user's voice is recognized through an intention recognition model, and the step of obtaining a recommended function includes: recognizing the intention of the user's voice to obtain intention information; analyzing candidate recommended functions from user usage habit big data based on the intention information; and screening target recommended functions available for the target vehicle from the candidate recommended functions.

[0014] According to the above technical means, the intention recognition model provided by the present invention is trained based on the big data of vehicle functions and user usage habits uploaded by users. On the one hand, when recommending functions, it is ensured that the recommended functions are functions that can be directly started and run by the current vehicle. On the other hand, the intention recognition model is trained in combination with big data statistics. The big data will also be updated in real time. The intention recognition model does not need to collect the habit data of the current user for a long time, ensuring that the intention recognition model has the ability to recommend functions more accurately at the beginning of use. This solution realizes customized training of the intention recognition model and improves the accuracy of the recommended functions.

[0015] In some optional implementations, the intent recognition model is trained using Tencent Hunyuan Large Model as the model structure.

[0016] Based on the above technical means, the Tencent Hunyuan large model architecture adopted is based on the Transformer neural network, has a trillion-parameter scale, powerful Chinese creation capabilities, logical reasoning capabilities in complex contexts, and reliable task execution capabilities, so that the trained model can significantly improve the accuracy of intent recognition and recommendation functions.

[0017] In a third aspect, the present invention provides an in-vehicle voice interaction device, which is applied to a vehicle, and the device includes: a first user voice receiving module, which is used to receive user voice and send the user voice to the cloud; a cloud information receiving module, which is used to receive response content and recommended functions sent by the cloud, wherein the response content is reply language information generated according to the user intention after the cloud recognizes the intention of the user voice based on a pre-trained intention recognition model; the recommended function is a vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user intention; an answer broadcast module, which is used to voice play the response content and display the response content on a vehicle display screen; an application interface calling module, which obtains the corresponding in-vehicle application access interface according to the recommended function and creates an access control corresponding to the in-vehicle application access interface; a recommended application display module, which displays the access control corresponding to the in-vehicle application access interface on the vehicle display screen.

[0018] In a fourth aspect, the present invention provides an in-vehicle voice interaction device, which is applied to the cloud, and the device includes: a second user voice receiving module, which is used to receive the user voice sent by the vehicle end, and the user voice is the voice command collected by the vehicle end; an intention recognition module, which is used to input the user voice into a pre-trained intention recognition model, and recognize the intention of the user voice through the intention recognition model to obtain response content and recommended functions, wherein the response content is reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention; a feedback module, which is used to send the response content and the recommended function to the vehicle end, so that the vehicle end voice plays the response content and displays the response content on the vehicle display screen, and then obtains the corresponding in-vehicle application access interface according to the recommended function, and creates an access control corresponding to the in-vehicle application access interface, and then displays the access control corresponding to the in-vehicle application access interface on the vehicle display screen.

[0019] In a fifth aspect, the present invention provides a vehicle, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0021] In a seventh aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0022] The technical solution provided by the present invention has the following advantages:

[0023] (1) According to the above technical means, the intention recognition model is trained by using the cloud computing power. When the user gives a voice command, the vehicle sends the user's voice to the cloud. The cloud recognizes the user's voice through the intention recognition model, determines the user's intention, and responds according to the user's intention. It also analyzes the subsequent vehicle function operations that most people like and are accustomed to based on the user's intention, and generates recommended functions. The response content and recommended functions are fed back to the vehicle by the cloud. The vehicle plays the response content and calls the corresponding vehicle application access interface according to the recommended function, and displays the access control corresponding to the interface on the vehicle screen, providing users with multiple choices. Therefore, the user can directly select the recommended subsequent operation according to the access control displayed on the screen, start the corresponding application to perform the corresponding function, without the user having to issue an additional command to start the corresponding application. This significantly improves the convenience of users operating vehicle applications, and provides users with vehicle application choices with higher demand rates, improves the ability of vehicle voice and user interaction, and improves the user's driving experience.

[0024] (2) According to the above technical means, if the response content of the cloud reply includes the logic of starting a certain vehicle target function, the vehicle side recognizes the logic and calls the target vehicle application to execute the target function according to the response content, thereby ensuring that the user's voice commands are executed accurately and reliably.

[0025] (3) According to the above technical means, before using the voice function, the user's vehicle periodically uploads the vehicle functions supported by the installed in-vehicle applications of the current vehicle, so that the trained intent recognition model can understand which functions the current vehicle supports. When recommending functions later, it ensures that the recommended functions are functions that can be directly started and run by the current vehicle, thereby realizing customized training of the intent recognition model and improving the accuracy of the recommended functions.

[0026] (4) According to the above technical means, the intention recognition model provided by the present invention is trained based on the big data of vehicle functions and user usage habits uploaded by users. On the one hand, when recommending functions, it is ensured that the recommended functions are functions that can be directly started and run by the current vehicle. On the other hand, the intention recognition model is trained in combination with big data statistics. The big data will also be updated in real time. The intention recognition model does not need to collect the habit data of the current user for a long time, ensuring that the intention recognition model has the ability to recommend functions more accurately at the beginning of use. This solution realizes customized training of the intention recognition model and improves the accuracy of the recommended functions.

[0027] (5) Based on the above technical means, the Tencent Hunyuan large model architecture is implemented based on the Transformer neural network, has a trillion-parameter scale, and has powerful Chinese creation capabilities, logical reasoning capabilities in complex contexts, and reliable task execution capabilities. The trained model significantly improves the accuracy of intent recognition and recommendation functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 is a flow chart of an in-vehicle voice interaction method according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a dialog flow card display effect according to an embodiment of the present invention;

[0031] Figure 3 is another flowchart of an in-vehicle voice interaction method according to an embodiment of the present invention;

[0032] Figure 4 is a structural schematic diagram of an in-vehicle voice interaction system according to an embodiment of the present invention;

[0033] Figure 5 is another flowchart of an in-vehicle voice interaction method according to an embodiment of the present invention;

[0034] Figure 6 is another flowchart of an in-vehicle voice interaction method according to an embodiment of the present invention;

[0035] Figure 7 is a schematic structural diagram of a vehicle-mounted voice interaction device according to an embodiment of the present invention;

[0036] Figure 8 is another structural schematic diagram of an in-vehicle voice interaction device according to an embodiment of the present invention;

[0037] Fig. 9 Schematic diagram of the hardware structure of a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0039] According to an embodiment of the present invention, an embodiment of an in-vehicle voice interaction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0040] In this embodiment, a vehicle-mounted voice interaction method is provided, which can be used in a vehicle. Figure 1 is a flow chart of a vehicle voice interaction method according to an embodiment of the present invention, the flow chart comprising the following steps:

[0041] Step S101, receiving user voice and sending the user voice to the cloud;

[0042] Step S102, receiving the response content and recommended functions sent by the cloud, wherein the response content is the reply language information generated by the cloud according to the user's intention after recognizing the intention of the user's voice based on the pre-trained intention recognition model; the recommended function is the vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user's intention;

[0043] Step S103, playing the response content by voice and displaying the response content on the vehicle display screen;

[0044] Step S104, obtaining a corresponding in-vehicle application access interface according to the recommended function, and creating an access control corresponding to the in-vehicle application access interface;

[0045] Step S105: displaying the access control corresponding to the vehicle-mounted application access interface on the vehicle-mounted display screen.

[0046] Specifically, the embodiment of the present invention uses the powerful computing power of the cloud to train the intention recognition model in advance. When the user is riding or driving a vehicle, when the user wants to query certain information or call a vehicle function, he speaks a voice command in the car, and the sound receiving device in the car collects the user's voice. Then the vehicle sends the user's voice to the cloud. The cloud recognizes the user's voice through the intention recognition model and determines the user's intention, that is, what the user really wants to do. After that, the intention recognition model also generates a response content based on the user's intention, and also conducts capability arbitration based on the user's intention, analyzes the subsequent vehicle function operations that most people like and are accustomed to, and generates recommended functions.

[0047] For example, the user inputs a voice: It is snowing heavily, what should I do? The vehicle uploads the content to the cloud. The cloud analyzes the voice through the intent recognition mode, recognizes the user's intention to obtain the driving response strategy for heavy snow, and then responds. The generated response content is: "It is not easy to drive in snowy weather, pay attention to reduce the speed". The generated recommended functions are: 1) Turn on the rearview mirror heating, 2) Turn on the front glass defrost, 3) Turn on the rear glass defrost. The above recommended functions are recommended to users to start the functions on the current vehicle. In some optional implementations, the cloud can also filter the recommended functions generated by the intent recognition model, sort multiple recommended functions according to the probability of occurrence, and then select the top preset ranking functions with higher probability as the recommended functions. For example, 90% of users choose to turn on the rearview mirror heating, 70% of users choose to turn on the front glass defrost, and 50% of users choose to turn on the rear glass defrost, so that the top-ranked rearview mirror heating and front glass defrost are selected as the actual recommended functions and sent to the vehicle to avoid the problem of too many recommended functions and difficulty in displaying them on the vehicle.

[0048] Afterwards, the cloud will feed back the response content and recommended functions to the vehicle side, and the vehicle side will play the response content to the user. The playback method can be to play the voice of the response content through the speakers, and display the text of the response content through the in-vehicle screen. At the same time, the vehicle side calls the corresponding in-vehicle application access interface according to the recommended function feedback from the cloud side. The in-vehicle application access interface refers to the API interface for the in-vehicle application to implement the corresponding function control, such as calling the rearview mirror heating interface, front glass defrost interface, and rear glass defrost interface of the vehicle control application. The vehicle side displays the access controls corresponding to the interface on the in-vehicle screen, providing users with a variety of choices. For example: a dialogue flow card is created on the in-vehicle screen for the voice interaction function. The dialogue flow card is used to display the text corresponding to the user's voice, the text corresponding to the response content, and the access controls corresponding to the recommended function in the designated area of ​​the in-vehicle screen. The display effect is as follows: Figure 2 As shown, users can directly select recommended subsequent operations based on the access controls displayed on the in-vehicle screen, such as starting the rearview mirror heating, front glass defrosting, and rear glass defrosting functions by clicking the screen.

[0049] Through this solution, combined with the big data model for function recommendation, compared with related technologies, there is no need to collect the current user's historical operation data for a long time in the early stage of function recommendation, which improves the accuracy of function recommendation. In addition, the access control of the recommended function is displayed through the dialogue flow card, without the user having to issue an additional instruction to start the corresponding application, which significantly improves the convenience of users operating in-vehicle applications, and provides users with in-vehicle application selections with higher demand rates, improves the ability of in-vehicle voice and user interaction, and improves the user's driving experience.

[0050] In some optional implementations, the present invention provides an in-vehicle voice interaction method, further comprising:

[0051] Step a1: If the response content includes logic for executing a target function, then calling a target vehicle-mounted application to execute the target function according to the response content.

[0052] Specifically, if the response content of the cloud reply includes the logic of starting a certain vehicle target function, in the embodiment of the present invention, the vehicle side recognizes the logic and calls the target vehicle application to execute the target function according to the response content, ensuring accurate and reliable execution of the user's voice command, rather than just voice or text reply, to ensure the integrity of the voice interaction capability. For example: the user's voice is to play music A. Through the analysis of the cloud, the vehicle side replies "OK, now play music A for you", and on the other hand calls the music application to execute the target function logic of playing music A.

[0053] In some optional implementations, the present invention provides an in-vehicle voice interaction method, further comprising:

[0054] Step b1, detecting the triggering state of the access control;

[0055] Step b2: when the access control is triggered, the corresponding in-vehicle application is called through the in-vehicle application access interface to execute the recommended function.

[0056] Specifically, the access controls displayed on the vehicle screen of the present invention can be triggered by the user touching the screen. When the user clicks on any access control, the corresponding vehicle application access interface calls the corresponding vehicle application to execute the recommended function. The detailed principle explanation can refer to the relevant description of the aforementioned steps S101 to S105, which will not be repeated here.

[0057] In some optional implementations, before pre-training the intention recognition model, the in-vehicle voice interaction method provided by the present invention further includes:

[0058] Step c1, collecting corresponding vehicle functions of vehicle applications installed on the current vehicle;

[0059] Step c2, sending the vehicle function to the cloud, so that the cloud trains the intent recognition model through the vehicle function.

[0060] Specifically, the embodiments of the present invention upload the vehicle functions supported by the installed in-vehicle applications of the current vehicle before the user uses the voice function, or periodically upload the vehicle functions supported by the installed in-vehicle applications of the current vehicle, so that the trained intent recognition model can understand which functions the current vehicle supports. When subsequently recommending functions, it ensures that the recommended functions are functions that can be directly started and run by the current vehicle. Different user vehicles are trained with different intent recognition models, which realizes customized training of the intent recognition model and improves the accuracy of recommended functions.

[0061] The embodiment of the present invention also provides a vehicle-mounted voice interaction method, which can be used in the cloud. Figure 3 is a flow chart of a vehicle voice interaction method according to an embodiment of the present invention, the flow chart comprising the following steps:

[0062] Step S301, receiving a user voice sent by the vehicle end, wherein the user voice is a voice command collected by the vehicle end;

[0063] Step S302, inputting the user voice into a pre-trained intention recognition model, and recognizing the intention of the user voice through the intention recognition model to obtain a response content and a recommended function, wherein the response content is reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention;

[0064] Step S303, sending the response content and the recommended function to the vehicle side, so that the vehicle side plays the response content by voice and displays the response content on the vehicle display screen, and then obtains the corresponding vehicle application access interface according to the recommended function, and creates an access control corresponding to the vehicle application access interface, and then displays the access control corresponding to the vehicle application access interface on the vehicle display screen.

[0065] Specifically, the explanation of the principles of steps S301 to S303 may refer to the relevant description of steps S101 to S105 of the aforementioned embodiment, which will not be repeated here.

[0066] In some optional implementations, the step of training the intent recognition model includes:

[0067] Step d1, receiving the vehicle function sent by the target vehicle;

[0068] Step d2, obtaining user usage habit big data, wherein the user usage habit big data includes various types of speech emitted by different users, the intent labels corresponding to each type of speech, and user preference functions related to each type of intent;

[0069] Step d3, training an intention recognition model applicable to the target vehicle based on the vehicle functions and the user usage habit big data.

[0070] Specifically, the present invention provides a customized training method for an intent recognition model. For the training of the intent recognition model, the training data that needs to be obtained include the vehicle functions sent by the target vehicle and the user usage habits big data. Among them, the user usage habits big data includes various types of voices issued by different users, the intent labels corresponding to each type of voice, and the user's favorite functions related to each type of intention. For example: the voice of turning on the air conditioner, the intention label is to turn on the air conditioner. The big data statistics show that 90% of people also operate to close the window, 80% of people also operate to turn off the child lock, 50% of people also operate to turn on the massage, and 10% of people play music. These statistical data can represent the user's favorite functions after turning on the air conditioner. The vehicle function sent by the target vehicle refers to the available vehicle functions supported by the installed vehicle application of the target vehicle, so that the trained intent recognition model can understand which functions the current vehicle supports. The training process of the natural language processing model is a prior art, and will not be described in detail. The model training is performed based on the above data to obtain an intent recognition model suitable for the target vehicle. Later, when recommending functions, the intent recognition model executes the process of capability arbitration, which specifically includes:

[0071] Step e1, identifying the intention of the user's voice and obtaining intention information;

[0072] Step e2, analyzing candidate recommended functions from the user usage habit big data based on the intention information;

[0073] Step e3: Filter the target recommended functions available for the target vehicle from the candidate recommended functions.

[0074] Through the above process, the intention recognition model first recognizes what the user currently wants to do (i.e., user intention) based on the user's voice, and then determines what most users will do in the future under the current intention conditions (i.e., candidate recommended functions) according to the big data statistics, and then filters the functions that most users like to operate in the future according to the functions supported by the current vehicle, so as to obtain the recommended functions recommended to the current vehicle (i.e., target recommended functions). For example, big data statistics show that after turning on the air conditioner, 90% of people closed the window, 80% of people also turned off the child lock, 50% of people also turned on the massage, and 10% of people played music, but the current vehicle only supports closing windows, turning off child locks, and playing music, and does not support massage, so the recommended functions finally recommended to the vehicle are: closing windows, turning off child locks, and playing music. Based on the technical solution provided by the embodiment of the present invention, on the one hand, when recommending functions, it is ensured that the recommended functions are functions that can be directly started and run by the target vehicle. On the other hand, the intention recognition model is trained in combination with big data statistics. The big data will also be updated in real time. The intention recognition model does not need to collect habit data of the target vehicle driver for a long time, ensuring that the intention recognition model has the ability to recommend functions more accurately at the beginning of use. This solution realizes customized training of the intention recognition model and improves the accuracy of recommended functions.

[0075] In some optional implementations, the intent recognition model is trained using Tencent Hunyuan Large Model as the model structure.

[0076] Specifically, Tencent's Hunyuan large model architecture is based on the Transformer neural network, has a trillion-parameter scale, and has powerful Chinese creation capabilities, logical reasoning capabilities in complex contexts, and reliable task execution capabilities, enabling the trained model to significantly improve the accuracy of intent recognition and recommendation functions.

[0077] In a complete embodiment, the vehicle-mounted voice interaction method provided by the present invention comprises: Figure 4The in-vehicle voice interaction system is realized, wherein the vehicle side and the cloud side are connected in communication, and the Tencent Hunyuan large model is deployed in the cloud side. The vehicle side can enable the voice service through the iFlytek SDK, monitor the user voice of the microphone, and send the user voice to the cloud; the cloud side recognizes the intention of the user voice through the Tencent Hunyuan large model, and generates the response content according to the user intention. The Tencent Hunyuan large model also generates the recommended function through capability arbitration according to the user intention, and the capability arbitration refers to the description of the above steps e1 to e3. The recommended functions include but are not limited to multimedia, vehicle control, navigation, wallpaper and other aspects. The vehicle receives the response content, plays the response content through voice, and displays the response content on the in-vehicle display screen through the dialogue flow card text; the vehicle side receives the recommended function, obtains the corresponding in-vehicle application access interface according to the recommended function, and creates the access control corresponding to the in-vehicle application access interface, and then displays the access control corresponding to the in-vehicle application access interface on the in-vehicle display screen. Figure 5 and Figure 6 , which are all flow charts of the complete process of the above-mentioned in-vehicle voice interaction method.

[0078] Examples of multimedia applications:

[0079] 1. User voice input: What is Faye Wong’s famous song?

[0080] 2. The car-side dialogue flow card displays text in real time: What is Faye Wong’s famous song?

[0081] 3. The vehicle uploads the text to the Tencent Hunyuan model in the cloud.

[0082] 4. Tencent Hunyuan Model returns the response content: "Faye Wong's famous song is "A Woman Easily Hurt"", and gives the recommended functions: play "A Woman Easily Hurt" immediately; add "A Woman Easily Hurt" to the favorite playlist.

[0083] 5. The car-side dialogue flow card displays the text in real time: Faye Wong’s famous song is "A Woman Easily Hurt", and displays the recommendation function: Play "A Woman Easily Hurt" immediately; add "A Woman Easily Hurt" to the favorite playlist.

[0084] 6. The in-car multimedia application provides: the function of immediately playing "A Woman Easily Hurt" and the function of adding "A Woman Easily Hurt" to the favorite playlist.

[0085] Navigation application example:

[0086] 1. User voice input: Go to Guanyin Bridge.

[0087] 2. The dialog flow card on the vehicle side displays the text in real time: Go to Guanyin Bridge.

[0088] 3. The vehicle uploads the text to the Tencent Hunyuan Model.

[0089] 4. Tencent Hunyuan Model returns the response content: Navigation to Guanyin Bridge is about to begin, and recommends the following functions: delicious food near Guanyin Bridge; cinema near Guanyin Bridge.

[0090] 5. The car-side dialogue flow card displays the text "About to navigate to Guanyin Bridge" in real time, and displays the recommended functions: navigate to the food store near Guanyin Bridge; navigate to the cinema near Guanyin Bridge.

[0091] 6. The navigation application provides: the ability to navigate to Guanyin Bridge, the ability to navigate to food stores near Guanyin Bridge, and the ability to navigate to movie theaters near Guanyin Bridge.

[0092] Vehicle control application examples:

[0093] 1. User voice input: It’s snowing heavily, what should I do?

[0094] 2. The car-side dialogue flow card displays text in real time: It’s snowing heavily, what should I do?

[0095] 3. The vehicle uploads the text to the Tencent Hunyuan Model.

[0096] 4. Tencent Hunyuan Model returns the response: "Please reduce your speed when driving in snowy weather. These functions should be enabled" and recommends the following functions: Turn on rearview mirror heating; turn on front defrost; turn on rear defrost.

[0097] 5. The car-side dialogue flow card displays text in real time: "Please reduce your speed when driving in snowy weather, and these functions should be enabled" and displays recommended functions: turn on rearview mirror heating; turn on front defrost; turn on rear defrost.

[0098] 6. The vehicle control application provides: turn on the rearview mirror heating function, turn on the front defrost function, and turn on the rear defrost function.

[0099] Wallpaper application examples:

[0100] 1. User voice input: Generate a picture of a kitten.

[0101] 2. The car-side dialogue flow card displays text in real time: Generate a picture of a kitten.

[0102] 3. The vehicle uploads the text to the Tencent Hunyuan Model.

[0103] 4. Tencent Hunyuan Model returns a response: "A picture of a kitten has been generated for you". The response contains a link to the picture and a recommended function: Do you want to set the picture as wallpaper?

[0104] 5. The dialog flow displays the text "A picture of a kitten has been generated for you" in real time and displays the picture through the link of the picture. It also displays the recommendation function: Do you want to set the picture as wallpaper?

[0105] 6. The wallpaper application provides: the ability to set the image as wallpaper.

[0106] The technical solution provided by the present invention provides a method for calculating user intention based on Tencent Hunyuan large model, and after identifying the user's real intention, the large model is used to calculate the user's possible intention and recommend it to the user. Another aspect of the present invention provides a method for fragmenting the in-vehicle functions and displaying them on the dialogue flow card, so that the user can directly operate the displayed in-vehicle application functions on the screen without opening the corresponding application to operate.

[0107] In this embodiment, a vehicle-mounted voice interaction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0108] This embodiment provides a vehicle-mounted voice interaction device, which is applied to a vehicle, such as Figure 7 As shown, including:

[0109] The first user voice receiving module 701 is used to receive the user voice and send the user voice to the cloud;

[0110] The cloud information receiving module 702 is used to receive the response content and recommended functions sent by the cloud, wherein the response content is the reply language information generated by the cloud according to the user's intention after the cloud recognizes the intention of the user's voice based on the pre-trained intention recognition model; the recommended function is the vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user's intention;

[0111] The answer broadcast module 703 is used to play the answer content in voice and display the answer content on the vehicle display screen;

[0112] The application interface calling module 704 obtains the corresponding vehicle application access interface according to the recommended function, and creates an access control corresponding to the vehicle application access interface;

[0113] The recommended application display module 705 is applied to display the access control corresponding to the vehicle-mounted application access interface on the vehicle-mounted display screen.

[0114] An embodiment of the present invention provides a vehicle-mounted voice interaction device, further comprising:

[0115] A logic execution module is used to call a target vehicle-mounted application to execute the target function according to the response content if the response content includes logic for executing the target function.

[0116] A control detection module, used to detect the triggering state of the access control;

[0117] The recommended function execution module is used to call the corresponding in-vehicle application through the in-vehicle application access interface to execute the recommended function when the access control is triggered.

[0118] An information collection module, used to collect corresponding vehicle functions of vehicle applications installed on the current vehicle;

[0119] The vehicle information is sent to the cloud module, and is used to send the vehicle function to the cloud, so that the cloud can train the intention recognition model through the vehicle function.

[0120] This embodiment provides a vehicle-mounted voice interaction device, which is applied to the cloud. Figure 8 As shown, including:

[0121] The second user voice receiving module 801 is used to receive the user voice sent by the vehicle end, and the user voice is the voice command collected by the vehicle end;

[0122] The intention recognition module 802 is used to input the user voice into a pre-trained intention recognition model, and recognize the intention of the user voice through the intention recognition model to obtain a response content and a recommended function, wherein the response content is a reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention;

[0123] Feedback module 803 is used to send the response content and the recommended function to the vehicle side, so that the vehicle side plays the response content by voice and displays the response content on the vehicle display screen, and then obtains the corresponding vehicle application access interface according to the recommended function, and creates an access control corresponding to the vehicle application access interface, and then displays the access control corresponding to the vehicle application access interface on the vehicle display screen.

[0124] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0125] The in-vehicle voice interaction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0126] An embodiment of the present invention further provides a vehicle that executes the vehicle-mounted voice interaction device of the aforementioned embodiment.

[0127] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a vehicle provided by an optional embodiment of the present invention, such as Fig. 9 As shown, the vehicle includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the vehicle, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple vehicles can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0128] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0129] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0130] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the vehicle, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the vehicle via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0132] The vehicle further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0133] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0134] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0135] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vehicle-mounted voice interaction method, characterized in that: Applied to a vehicle, the method comprises: Receive user voice and send the user voice to the cloud; Receiving a response content and a recommended function sent by the cloud, wherein the response content is a reply language information generated by the cloud according to the user's intention after the cloud recognizes the intention of the user's voice based on a pre-trained intention recognition model; the recommended function is a vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user's intention; Play the response content by voice and display the response content on the vehicle display screen; Acquire a corresponding in-vehicle application access interface according to the recommended function, and create an access control corresponding to the in-vehicle application access interface; The access control corresponding to the in-vehicle application access interface is displayed on the in-vehicle display screen.

2. The method according to claim 1, characterized in that The method further comprises: If the response content includes logic for executing the target function, the target in-vehicle application is called according to the response content to execute the target function.

3. The method according to claim 1, characterized in that The method further comprises: Detecting a trigger state of the access control; When the access control is triggered, the corresponding in-vehicle application is called through the in-vehicle application access interface to execute the recommended function.

4. The method according to claim 1, characterized in that: Before pre-training the intent recognition model, the method further includes: Collect the corresponding vehicle functions of the in-vehicle applications installed on the current vehicle; The vehicle function is sent to the cloud so that the cloud trains the intent recognition model through the vehicle function.

5. A vehicle-mounted voice interaction method, characterized in that: Applied to the cloud, the method includes: Receiving user voice sent by the vehicle end, wherein the user voice is a voice command collected by the vehicle end; Inputting the user voice into a pre-trained intention recognition model, and recognizing the intention of the user voice through the intention recognition model to obtain a response content and a recommended function, wherein the response content is reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention; The response content and the recommended function are sent to the vehicle side, so that the vehicle side plays the response content by voice and displays the response content on the vehicle display screen, and then obtains the corresponding vehicle application access interface according to the recommended function, creates an access control corresponding to the vehicle application access interface, and then displays the access control corresponding to the vehicle application access interface on the vehicle display screen.

6. The method according to claim 5, characterized in that The steps of training the intent recognition model include: Receiving vehicle functions sent by the target vehicle; Obtaining user usage habit big data, wherein the user usage habit big data includes various types of voices emitted by different users, intent labels corresponding to each type of voice, and user preference functions related to each type of intent; An intention recognition model suitable for the target vehicle is trained based on the vehicle functions and the user usage habit big data.

7. The method according to claim 6, characterized in that The step of identifying the intention of the user's voice by using the intention recognition model to obtain a recommended function includes: Recognizing the intention of the user's voice to obtain intention information; Analyzing candidate recommended functions from the user usage habit big data based on the intention information; A target recommended function available to the target vehicle is filtered from the candidate recommended functions.

8. The method according to claim 6, characterized in that The intent recognition model is trained using Tencent Hunyuan Large Model as the model structure.

9. A vehicle-mounted voice interaction device, characterized in that: Applied to a vehicle, the device comprises: A first user voice receiving module, used to receive user voice and send the user voice to the cloud; A cloud information receiving module, used to receive the response content and recommended functions sent by the cloud, wherein the response content is the reply language information generated by the cloud according to the user's intention after the cloud recognizes the intention of the user's voice based on the pre-trained intention recognition model; the recommended function is the vehicle function recommended by the pre-trained intention recognition model for the user to operate according to the user's intention; A response broadcast module, used to play the response content by voice and display the response content on the vehicle display screen; An application interface calling module, wherein the application obtains the corresponding vehicle application access interface according to the recommended function, and creates an access control corresponding to the vehicle application access interface; The recommended application display module is used to display the access control corresponding to the vehicle application access interface on the vehicle display screen.

10. A vehicle-mounted voice interaction device, characterized in that: Applied to the cloud, the device comprises: A second user voice receiving module, used to receive the user voice sent by the vehicle end, wherein the user voice is a voice command collected by the vehicle end; an intention recognition module, used to input the user voice into a pre-trained intention recognition model, and recognize the intention of the user voice through the intention recognition model to obtain a response content and a recommended function, wherein the response content is reply language information generated according to the user intention, and the recommended function is a vehicle function recommended for the user to operate according to the user intention; A feedback module is used to send the response content and the recommended function to the vehicle side, so that the vehicle side plays the response content by voice and displays the response content on the vehicle display screen, and then obtains the corresponding vehicle application access interface according to the recommended function, and creates an access control corresponding to the vehicle application access interface, and then displays the access control corresponding to the vehicle application access interface on the vehicle display screen.

11. A vehicle, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 4 or 5 to 8 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 4 or 5 to 8.

13. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 4 or 5 to 8.