Multi-intention voice interaction method and device, vehicle-mounted equipment and readable storage medium

Abstract

The invention discloses a multi-intention voice interaction method and device, vehicle-mounted equipment and a readable storage medium, and relates to the technical field of vehicle-mounted voice interaction, and the method comprises the steps: obtaining the voice information of a user, and converting the voice information into first text information; identifying the first text information to obtain execution intention information; when the execution intention information comprises a plurality of sub-intentions, distributing each sub-intention in parallel to a corresponding vehicle control module for execution according to a mapping rule; feedback information executed by each vehicle control module on the sub-intention is obtained; obtaining a TTS template corresponding to each piece of feedback information; and generating to-be-played voice data based on each piece of feedback information and the corresponding TTS template, and playing the to-be-played voice data. According to the method, the multiple sub-intentions are distributed to different vehicle control modules in parallel, and the to-be-played voice data is generated for playing based on the feedback information of each module and the corresponding TTS template, so that richer and more accurate voice feedback can be provided for the user while the multi-intention execution efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of in-vehicle voice interaction technology, and in particular to a multi-intention voice interaction method, device, in-vehicle equipment and readable storage medium. Background Art

[0002] With the rapid development of intelligent technology, voice interaction technology has become an indispensable part of modern automobiles, smart homes and other fields, such as text-to-speech technology (Text To Speech, referred to as "TTS"). Voice interaction technology, with its natural and convenient characteristics, enables users to control devices through simple voice commands, greatly improving the user experience.

[0003] In actual applications, users often need to perform multiple operations and provide feedback at the same time, such as closing the car window and turning on the air conditioner at the same time. However, traditional voice interaction systems can usually only process single commands and respond one by one, which cannot accurately reflect the system's understanding and execution of user commands. This may cause users to request the same task multiple times without being able to confirm whether the request has been fully received and executed, thereby reducing efficiency. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the purpose of the embodiments of the present application is to provide a multi-intent voice interaction method, a multi-intent voice interaction device, a vehicle-mounted device, a readable storage medium and a computer program product. Through the technical solution of the embodiments of the present application, the interactive effect of parallel recognition feedback of multi-intent voices can be achieved, and the execution efficiency of instructions can be improved while providing users with richer and more accurate voice feedback.

[0005] In the first aspect, the present application provides a multi-intention voice interaction method, which adopts the following technical solution:

[0006] A multi-intention voice interaction method, comprising:

[0007] Acquire voice information of the user, and convert the voice information into first text information;

[0008] Identifying the first text information to obtain execution intention information;

[0009] When the execution intention information includes multiple sub-intentions, each of the sub-intentions is distributed in parallel to a corresponding vehicle control module according to a mapping rule, and the vehicle control module is controlled to execute the corresponding sub-intention;

[0010] Obtaining feedback information of each vehicle control module on the execution of the sub-intention;

[0011] Obtaining a TTS template corresponding to each feedback information;

[0012] The voice data to be played is generated based on each of the feedback information and the corresponding TTS template, and the voice data to be played is played.

[0013] By adopting the above technical solution, multiple sub-intentions are distributed in parallel to different vehicle control modules, and then converted into voice data to be played according to the return results after the execution of each module. While improving the execution efficiency of multiple intentions, users can also be informed of the execution status of sub-intentions, reducing the possibility of users requesting the same task multiple times without being able to confirm whether the request has been completely received and executed, thereby further improving the execution efficiency of sub-intentions.

[0014] Optionally, the obtaining a TTS template corresponding to each piece of feedback information includes:

[0015] Determining whether the feedback information includes interaction information for the sub-intention;

[0016] If the feedback information includes interaction information for the sub-intention, the TTS template corresponding to the interaction information includes execution information for the sub-intention and further selection information for the sub-intention;

[0017] If the feedback information does not include the interaction information for the sub-intention, the TTS template corresponding to the interaction information includes the execution information for the sub-intention.

[0018] Optionally, if the feedback information includes interaction information of the sub-intent, the method further includes:

[0019] Acquiring voice information of the user selecting the further selection information;

[0020] Converting the selected voice information into second text information;

[0021] determining selection information based on the second text information;

[0022] The selection information is sent to the corresponding vehicle control module.

[0023] Optionally, after obtaining the feedback information of each vehicle control module on the execution of the sub-intention, the method further includes:

[0024] Obtaining the distribution time of each sub-intention to the corresponding vehicle control module;

[0025] Obtaining the time at which the feedback information is obtained;

[0026] Calculating a time difference between the acquisition time and the distribution time;

[0027] The sub-intention corresponding to the time difference being greater than the time threshold is regarded as a timed-out intention, and the sub-intention corresponding to the time difference being not greater than the time threshold and being successfully executed is regarded as a normal intention.

[0028] Optionally, the execution information includes execution timeout, normal execution and execution failure, and the obtaining of the TTS template corresponding to each feedback information further includes:

[0029] When the sub-intention is a timeout intention, determining that the execution information of the sub-intention is execution timeout, and the TTS template of the execution timeout is the execution timeout of the sub-intention;

[0030] When the sub-intention is a normal intention, determining that the execution information of the sub-intention is normal execution, and the normal execution TTS template is the normal execution of the sub-intention;

[0031] When the sub-intention is a failed intention, the execution information of the sub-intention is determined to be execution failure, and the TTS template of the failed execution is the sub-intention execution failure.

[0032] Optionally, when the sub-intent is a failure intent, it also includes:

[0033] Query whether the vehicle control module corresponding to the sub-intention is in a connected state. If not, generate a prompt message to remind the user to set the vehicle control module to a connected state.

[0034] Optionally, the generating the voice data to be played based on each feedback information and the corresponding TTS template includes:

[0035] Integrate the TTS templates of all sub-intents, and generate the voice data to be played based on the integrated TTS templates;

[0036] The voice data to be played includes execution information and / or further selection information.

[0037] In a second aspect, the present application provides a multi-intention voice interaction device, the device comprising:

[0038] An acquisition conversion module, used for acquiring voice information of a user and converting the voice information into first text information;

[0039] An identification module, used to identify the first text information and obtain execution intention information;

[0040] A distribution module, when the execution intention information includes multiple sub-intentions, is used to distribute each of the sub-intentions in parallel to the corresponding vehicle control module according to the mapping rule, and control the vehicle control module to execute the corresponding sub-intention;

[0041] A first acquisition module, used for acquiring feedback information of each vehicle control module on the execution of the sub-intention;

[0042] The second acquisition module is used to acquire the TTS template corresponding to each feedback information.

[0043] The playing module is used to generate the voice data to be played based on each feedback information and the corresponding TTS template, and play the voice data to be played.

[0044] By adopting the above technical solution, the user's voice information can be converted into the first text information through the acquisition and conversion module and the recognition module, and the execution intention therein can be identified. The distribution module then distributes the multiple sub-intentions contained in the execution intention to different vehicle control modules in parallel. The playback module then generates the voice data to be played based on the feedback information obtained by the first acquisition module and the second acquisition module and the corresponding TTS template, which improves the efficiency of multi-intention execution while providing users with richer and more accurate voice feedback.

[0045] In a third aspect, the present application provides a vehicle-mounted device, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions so that the vehicle-mounted device performs the multi-intent voice interaction method as described in any one of the first aspects.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-intention voice interaction method as described in the first aspect above.

[0047] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the multi-intention voice interaction method as described in the first aspect.

[0048] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-intention voice interaction method as described in the first aspect above.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a structural block diagram of an application scenario of human-computer interaction of an intelligent vehicle provided by the present application;

[0051] Figure 2It is a flowchart of a multi-intention voice interaction method provided in an embodiment of the present application;

[0052] Figure 3 It is a schematic diagram of a monitoring process of intention execution in a multi-intention voice interaction method provided in an embodiment of the present application;

[0053] Figure 4 is a structural diagram of a multi-intention language interaction device provided in an embodiment of the present application;

[0054] Figure 5 It is a schematic diagram of the structure of the vehicle-mounted equipment provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is sought, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0056] In the following, in combination with the accompanying drawings, the multi-intention voice interaction method, multi-intention voice interaction device, vehicle-mounted equipment, electronic device, readable storage medium and computer program product provided by the present application are described in detail through specific implementation methods and application scenarios.

[0057] Among them, the multi-intention voice interaction method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0058] The terminal may be a vehicle computer, a device including a microphone, a touch panel or other physical user interface, or a server.

[0059] The present application provides a multi-intent voice interaction method, the execution subject of which can be a vehicle computer or a functional module or entity in the vehicle computer that can implement the multi-intent voice interaction method. In a vehicle environment, due to the complex network situation, such as when the vehicle-mounted device is driving, the network status changes dynamically due to location switching, and the complexity of multi-intent voice interaction is much higher than that of multi-intent voice interaction in a home environment.

[0060] like Figure 1As shown, the present application embodiment provides a structural block diagram of an application scenario of a multi-intention voice interaction method. Specifically, the method is applicable to an application scenario of human-computer interaction in an intelligent vehicle. The application scenario includes an electronic device 110 and an intelligent vehicle 120.

[0061] The smart vehicle 120 establishes a communication connection with the electronic device 110 through a transmission network (such as a wireless communication network). The smart vehicle 120 uploads the collected user's voice information to the electronic device 110 through the transmission network, and the electronic device 110 can recognize the user's voice information when obtaining it.

[0062] Optionally, the electronic device 110 can be a server, and the server can be a server cluster composed of multiple physical servers, or a distributed system, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms and other technical computing services.

[0063] Optionally, the application scenario may also include a management device, which is used to manage the devices in the application scenario (such as managing the smart vehicle 120 and the connection status between each module in the smart vehicle 120 and the electronic device 110, etc.), and the management device is connected to the electronic device 110 via a communication network.

[0064] Optionally, the communication network is a wired network or a wireless network.

[0065] Optionally, the wireless network or wired network uses standard communication technology and / or protocols. The network is usually the Internet, but can also be any other network, including but not limited to any combination of local area network, metropolitan area network, wide area network, mobile, wired or wireless network, private network or virtual private network.

[0066] In some embodiments, data exchanged over the network is represented using techniques and / or formats including hypertext markup language, extensible markup language, etc. In addition, all or some links may be encrypted using conventional encryption techniques such as secure socket layer, transport layer security, virtual private network, Internet protocol security, etc. In other embodiments, customized and / or dedicated data communication techniques may also be used or supplement the above data communication techniques.

[0067] like Figure 2 As shown, a multi-intention voice interaction method provided by an embodiment of the present application is described as follows (steps S210 to S260):

[0068] Step S210: Acquire the user's voice information, and convert the voice information into first text information.

[0069] When a user wants to control a functional module, the user sends a voice message. When the terminal acquires the voice message, based on any current natural language processing (NLP) technology, the terminal converts the user's voice message into a first text message, thereby obtaining the first text message.

[0070] Understandably, in an actual interaction environment, due to vehicle hardware limitations, or because of network instability, the user's speech is colloquial or dialectal, etc., resulting in the text obtained after speech recognition not being clear and accurate enough. Some conventional text corrections need to be performed through preprocessing, such as correcting some expressions with synonyms or near-synonyms, and removing some meaningless function words or punctuation marks, etc.

[0071] The process of converting the user's voice message into a first text message mainly includes three parts: preprocessing of the voice message, feature extraction, and model matching. Preprocessing includes processes such as pre-filtering, sampling, quantization, windowing, endpoint detection, and pre-emphasis. The most important part of speech recognition conversion is to extract feature parameters. The extracted feature parameters must be able to effectively represent the speech features, have good discriminability; there is good independence between the feature parameters at each stage; the feature parameters should be easy to calculate to ensure the real-time implementation of speech recognition. In the recognition stage, the feature parameters are input into the model for matching. The model matches the feature parameters with the feature parameters in the template library, and takes the template with the highest matching score as the recognition conversion result, that is, the text corresponding to the template with the highest matching score is used as the recognition conversion result, thereby obtaining the first text message.

[0072] Exemplarily, in the embodiments of the present application, since punctuation marks and function words have no actual meaning in text expression, therefore, in order to reduce the matching complexity, the punctuation marks and function words in the user's voice control instruction can be removed through the preprocessing process. The specific method is: obtain a punctuation mark library, and compare the user's voice control instruction with each punctuation mark in the punctuation mark library in turn to filter out the punctuation marks in the user's voice control instruction; obtain a function word library, and compare the user's voice control instruction with each function word in the function word library in turn to filter out the function words in the user's voice control instruction.

[0073] Among them, function words refer to some meaningless words that are not related to the instruction, such as "please", "help me" at the beginning of a sentence, and words such as "ne", "le" at the end of a sentence.

[0074] By establishing a punctuation library to compare the user's voice control instructions, the punctuation marks contained therein can be removed. For example, the user's voice message "Turn on the air conditioner and dial Zhang San" becomes "Turn on the air conditioner and dial Zhang San" after processing; and by establishing a function word library, the user's voice message can be filtered for function words. For example, the user's voice message "Please help me turn on the air conditioner and dial Zhang San" becomes "Turn on the air conditioner and dial Zhang San" after filtering.

[0075] Step S220: Identify the first text information to obtain execution intention information.

[0076] In an embodiment of the present application, the terminal may identify and parse the first text information based on NLP technology to obtain execution intention information.

[0077] The execution intent in the first text message can be a single intent or a multi-intent including multiple sub-intents, where a single intent refers to having only one execution intent, for example, "turn on the air conditioner"; a multi-intent refers to having two or more sub-intents, for example, "turn on the air conditioner and call Zhang San" has two sub-intents, one sub-intent is to turn on the air conditioner, and the other sub-intent is to call Zhang San. In the embodiment of the present application, the description is mainly for the case where the execution intent is multiple intents.

[0078] Step S230: When the execution intention information includes multiple sub-intentions, each sub-intention is distributed in parallel to the corresponding vehicle control module according to the mapping rule, and the vehicle control module is controlled to execute the corresponding sub-intention.

[0079] When it is recognized that the execution intention of the first text information includes multiple sub-intentions, the terminal obtains the vehicle control module corresponding to each sub-intention, and sends the execution operation of each sub-intention to the corresponding vehicle control module at the same time.

[0080] The execution intention includes one or more of streaming media execution intention and vehicle execution intention.

[0081] Streaming media execution intentions, including but not limited to music broadcasting, navigation sound broadcasting, system prompt sound control, and other intentions related to streaming media execution.

[0082] Vehicle execution intentions, including but not limited to seat adjustment, air conditioning adjustment, or headlights, cameras, and other intentions related to vehicle execution.

[0083] Different vehicle control modules also correspond to different functions. For example, the vehicle control module includes an audio module, an air-conditioning control module, a navigation module, a lighting control module, and the like.

[0084] The terminal distributes different types of execution intentions to the corresponding vehicle control modules in parallel through mapping rules, processes multiple self-intentions simultaneously, reduces the total execution time of user voice control commands, and improves user experience and system response speed.

[0085] Among them, the mapping rule is a set of rules used to map a specific execution intention (or sub-intention) to a specific vehicle control module in the system. It specifies to which vehicle control module different categories of execution intentions should be sent for processing.

[0086] It is worth noting that the correspondence between each execution intention and the vehicle control module is stored in the terminal device, and the correspondence between the execution intention and the vehicle control module is the above-mentioned mapping rule. For example, the mapping rule includes: the music playing intention corresponds to the audio module; the navigation broadcasting intention corresponds to the navigation module; the seat adjustment intention corresponds to the seat control module; the air conditioning adjustment intention corresponds to the air conditioning control module.

[0087] Step S240: Obtain feedback information from each vehicle control module on the execution of the sub-intention.

[0088] After the terminal assigns each sub-intention to the corresponding vehicle control module, the vehicle control module performs the corresponding operation. At this time, the terminal will enter a waiting state, monitor the execution status of each vehicle control module and obtain the returned feedback information.

[0089] Feedback information includes interaction information and execution information.

[0090] The interaction information is used to describe the interaction process between the terminal and the vehicle control module, including further operations and confirmations of the current sub-intention. The execution information is used to describe the specific status of the vehicle control module executing the sub-intention, including:

[0091] If the execution is normal, the sub-intention is executed as expected without encountering any problems, and the vehicle control module will return the operation result and success information; if the execution fails, the sub-intention cannot be completed, and the vehicle control module will return the reason for the failure and error description to facilitate diagnosis and repair of the problem; if the execution times out, the sub-intention is still not completed after exceeding the scheduled time, the vehicle control module will report the timeout and provide the reason for the timeout, such as network problems or device failure.

[0092] The terminal can monitor the execution status of each vehicle control module in real time through polling or asynchronous callback to ensure timely feedback information.

[0093] Polling is an active monitoring method. The terminal will periodically send requests to each vehicle control module to query their execution status. However, the update delay depends on the polling interval, which may cause the status feedback to be not real-time enough. Asynchronous callback is a passive monitoring method. The vehicle control module actively notifies the terminal when the status changes or the operation is completed. Although the real-time performance is good and the vehicle control module can immediately notify the terminal when the status changes, it requires the design of callback interfaces and processing mechanisms. The system implementation and maintenance are relatively complex. Therefore, in practical applications, the above two methods are usually used in combination.

[0094] like Figure 3 As shown, after obtaining the feedback information of each vehicle control module on the execution of the sub-intention, the following contents are also included:

[0095] Specifically, obtain the distribution time of each sub-intention in parallel to the corresponding vehicle control module; obtain the acquisition time of the feedback information; calculate the time difference between the acquisition time and the distribution time; treat the sub-intention corresponding to the time difference greater than the time threshold as a timeout intention, and treat the sub-intention corresponding to the time difference not greater than the time threshold and successfully executed as a normal intention.

[0096] Sub-intentions outside of timeout intents and normal intents can be considered as failed intents. That is, if an error occurs during the execution of a sub-intention, the operation cannot be completed, or the feedback information indicates that the execution failed, these sub-intentions are marked as failed intents.

[0097] Exemplarily, the first sub-intention is "adjust the air-conditioning temperature to 22°C" and the second sub-intention is "play music". After the two sub-intentions are distributed to the corresponding air-conditioning control module and audio module, the terminal records the distribution time of the first sub-intention and the second sub-intention as 10:00:00; after the corresponding modules are executed, the feedback time of the first sub-intention is recorded as 10:00:25, and the feedback time of the second sub-intention is 10:00:05; at this time, the terminal will calculate the time difference between the distribution time and the feedback time, and obtain the time difference of the first sub-intention as 25 seconds, and the time difference of the second sub-intention as 5 seconds; assuming that the set time threshold is 10 seconds, the time difference of the first sub-intention is greater than the time threshold, so it is marked as a timeout intention, and the time difference of the second sub-intention is not greater than the time threshold, so it is marked as a normal intention.

[0098] Step S250: Obtain the TTS template corresponding to each feedback information.

[0099] Specifically, step S250 also includes: determining whether the feedback information contains interaction information for the sub-intention; if the feedback information contains interaction information for the sub-intention, the TTS template corresponding to the interaction information includes execution information for the sub-intention and further selection information for the sub-intention; if the feedback information does not contain interaction information for the sub-intention, the TTS template corresponding to the interaction information includes execution information for the sub-intention.

[0100] Among them, the execution information is determined as follows: when the sub-intention is a timeout intention, the execution information of the sub-intention is determined as execution timeout, and the TTS template of execution timeout is the sub-intention execution timeout; when the sub-intention is a normal intention, the execution information of the sub-intention is determined as normal execution, and the TTS template of normal execution is the sub-intention execution normal; when the sub-intention is a failed intention, the execution information of the sub-intention is determined as execution failure, and the TTS template of failed execution is the sub-intention execution failure.

[0101] When the sub-intent is a failure intent, it also includes the following:

[0102] Specifically, it is queried whether the vehicle control module corresponding to the sub-intention is in a connected state. If not, a prompt message is generated to remind the user to set the vehicle control module to a connected state.

[0103] In this embodiment, when the feedback information contains interactive information on sub-intentions, the user needs to make a selection based on the interactive information. The vehicle control module will identify the execution intention contained therein based on the selection voice information fed back by the user and execute the next step of operation.

[0104] Specifically, when the feedback information contains interactive information on the sub-intention, it further includes: obtaining the user's selection voice information for further selection information; converting the selection voice information into a second text information; determining the selection information based on the second text information; and sending the selection information to the corresponding vehicle control module.

[0105] Based on the above steps, multiple rounds of interaction of intentions can be achieved.

[0106] Exemplarily, the corresponding relationship between the feedback information of the execution intention and the TTS template is as follows:

[0107] ① Successful execution: The TTS template corresponding to successful execution is generally "OK, executed" or a synonym;

[0108] ② Execution failure: Generally speaking, it will succeed. Failure may be due to the vehicle model not being supported or the software status not being supported. For example, when Bluetooth is not connected, the execution intention of "playing Bluetooth music" will fail. At this time, the corresponding TTS template is the specific business content, such as "Bluetooth is not connected yet" or a synonym.

[0109] ③Execution timeout: The TTS template corresponding to the execution timeout is also specific business content. For example, if the Bluetooth connection times out, the corresponding TTS template is "Bluetooth connection timeout, please try again later" or a synonym.

[0110] ④ Interaction information: When the intent has multiple rounds of interaction attributes, the feedback information includes the interaction information. For example, when the intent is "Navigate to Baoan Center", the TTS template corresponding to the interaction information is "Multiple destinations found, which one do you want to choose?" or synonymous words.

[0111] Step S260: Generate voice data to be played based on each piece of feedback information and the corresponding TTS template, and play the voice data to be played.

[0112] Specifically, the TTS templates of all sub-intentions are integrated, and the voice data to be played is generated based on the integrated TTS templates; the voice data to be played includes execution information and / or further selection information.

[0113] At this time, the terminal obtains the TTS template corresponding to the feedback information of the vehicle module, and dynamically generates the language data to be played according to the TTS template and the actual results.

[0114] In addition to being able to directly choose to use the system's built-in TTS templates, users can also use mobile phones, tablets, PCs, or personal computers such as Macs to enter different TTS templates through pre-set scene configuration interfaces, and then send them to the terminal via wired / wireless network connections. Commonly used TTS templates are introduced as follows:

[0115] ① Success 1: “OK, executed”

[0116] ② Success 2: “OK, all executed”

[0117] ③Failure 1: “The execution failed” or a synonym

[0118] ④ Failure intent TTS: Determined based on specific business feedback. For example, when Bluetooth is not connected, the TTS content for the Bluetooth music service is "Bluetooth is not connected yet."

[0119] ⑤ Timeout TTS: defined according to specific business feedback. For example, if the Bluetooth connection times out, the TTS content for the Bluetooth music player is "Bluetooth connection timed out, please try again later."

[0120] The final voice data to be broadcast needs to be synthesized according to the business results of each sub-intent and matched according to the following templates:

[0121] 1. All executions are successful, corresponding to {Success 2};

[0122] 2. If part of the execution succeeds, but one fails and returns business feedback, then {Success 1}+{Failure Intent TTS} will be matched. Such prompt content can quickly remind the user which sub-intent has a problem. For example, "Turn on the air conditioner and call Zhang San", if Bluetooth is not connected at this time, you can prompt "OK, it has been executed, please connect Bluetooth and try again" to remind the user that one command has been executed, but the other command does not meet the precondition;

[0123] 3. If none of them are executed successfully, it corresponds to {Failure 1}, prompting the user that all execution intentions have failed;

[0124] 4. If a command execution times out, it will correspond to {Timeout TTS}, prompting the user that the command execution time is too long and may fail.

[0125] Among them, if only one execution intention times out (for example, air conditioning), and other execution intentions are executed successfully, the corresponding template {Success 2}+{Timeout TTS}, the generation result of the voice data to be played is "All have been executed, but the air conditioning execution has timed out" or synonymous words; if more than one sub-intention is executed successfully, and multiple sub-intentions time out, the matching template {Success 2}+{Timeout TTS}, the generation result of the voice data to be played is "All have been executed, but some commands have timed out" or synonymous words; if all sub-intentions have timed out, the corresponding template {Timeout TTS}, the generation result of the voice data to be played is "The command execution has timed out" or synonymous words.

[0126] It should be understood that the commonly used TTS template of this application should also include:

[0127] ⑥Multi-round intent TTS: determined based on specific business feedback.

[0128] For example, if you ask "Navigate to Baoan Center", you may be prompted with "Multiple destinations found, which one do you want to select?"

[0129] When the feedback information contains interaction information for a sub-intention, the terminal will determine that the sub-intention requires multiple rounds of interaction. For example, the feedback information is: all instructions are executed successfully, and one of the sub-intentions requires multiple rounds of interaction. The corresponding template is {Success 1}+{Multiple Rounds of Intent TTS}, prompting the user to focus more on the next round of interaction actions.

[0130] Among them, {Success 1} corresponds to execution information, {Multi-round Intent TTS} corresponds to further selection information, and then it is broadcast. The terminal will obtain the user's selection voice information for further selection information again, and then convert it into a second text information. Based on the second text information, the selection information is determined, and the selection information is sent to the corresponding vehicle control module to continue execution.

[0131] Exemplarily, the first text message is "Turn on the air conditioner and navigate to the Cultural Palace". Based on the feedback information and the TTS template, the language data to be broadcast is generated and broadcast "OK, executed. Multiple destinations are found. Which one do you want to choose?" The selection voice information fed back by the user is "Select the first one", which is then converted into the second text message. The selection information therein is determined, and the terminal will send it to the corresponding vehicle control module. If the execution is normal, the template {Success 1} is matched based on the feedback information to generate the first voice data "OK, executed".

[0132] The multi-intention voice interaction method of the present application distributes multiple sub-intentions in parallel to different vehicle control modules, and then converts them into voice data to be played based on the return results after the execution of each module. While improving the execution efficiency of multiple intentions, it can also provide users with richer and more accurate voice feedback.

[0133] The multi-intent voice interaction method provided in the present application introduces a monitoring mechanism. When the time difference between the acquisition time and the distribution time of the feedback information is greater than the sub-intention corresponding to the time threshold, the corresponding sub-intention will be marked as a timed-out intention. Then, based on the feedback information and the corresponding TTS template, the corresponding language data to be broadcast is generated to inform the user, thereby avoiding the user's anxiety caused by long waiting times and improving the user experience.

[0134] The above are all operations that the voice assistant can perform in the in-vehicle voice interaction scenario.

[0135] like Figure 4 As shown, a multi-intention voice interaction device 400 provided in an embodiment of the present application is shown, and the multi-intention voice interaction device 400 includes:

[0136] The acquisition conversion module 410 is used to acquire the user's voice information and convert the voice information into the first text information;

[0137] The recognition module 420 is used to recognize the first text information and obtain the execution intention information;

[0138] A distribution module 430, when the execution intention information includes multiple sub-intentions, is used to distribute each sub-intention in parallel to the corresponding vehicle control module according to the mapping rule, and control the vehicle control module to execute the corresponding sub-intention;

[0139] A first acquisition module 440 is used to obtain feedback information of each vehicle control module on the execution of the sub-intention;

[0140] The second acquisition module 450 is used to acquire the TTS template corresponding to each feedback information;

[0141] The playing module 460 is used to generate the voice data to be played based on each feedback information and the corresponding TTS template, and play the voice data to be played.

[0142] As an optional implementation of this embodiment, the second acquisition module 450 includes a judgment submodule and an acquisition conversion submodule, wherein:

[0143] The judgment submodule is used to judge whether the feedback information contains interactive information on the sub-intention. If the feedback information contains interactive information on the sub-intention, the TTS template corresponding to the feedback information includes execution information on the sub-intention and further selection information on the sub-intention; if the feedback information does not contain interactive information on the sub-intention, the TTS template corresponding to the feedback information includes execution information on the sub-intention.

[0144] Obtain a conversion submodule, which is used to obtain the user's selection voice information for further selection information if the feedback information contains interactive information on the sub-intention; convert the selection voice information into a second text information; determine the selection information based on the second text information; and send the selection information to the corresponding vehicle control module.

[0145] As an optional implementation of this embodiment, the multi-intention voice interaction device 400 further includes:

[0146] The monitoring module is used to obtain the distribution time of each sub-intention in parallel to the corresponding vehicle control module; obtain the distribution time of each sub-intention to the corresponding vehicle control module; obtain the acquisition time of feedback information; calculate the time difference between the acquisition time and the distribution time; regard the sub-intention corresponding to the time difference greater than the time threshold as a timed-out intention, and regard the sub-intention corresponding to the time difference not greater than the time threshold and successfully executed as a normal intention.

[0147] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment, which will not be repeated here.

[0148] like Figure 5 As shown, the embodiment of the present application further provides a vehicle-mounted device 500, including a memory 510, a processor 520 and a communication bus 530; the memory 510 and the processor 520 are connected via the communication bus 530. The memory 510 stores a multi-intention voice interaction method provided in the above embodiment that can be loaded and executed by the processor 520, and can achieve the same technical effect, so it will not be repeated here to avoid repetition.

[0149] The memory 510 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 510 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing a digital tablet adaptive control method provided in the above embodiment, etc.; the data storage area may store data involved in a digital tablet adaptive control method provided in the above embodiment, etc.

[0150] The processor 520 may include one or more processing cores. The processor 520 calls the data stored in the memory 510 by running or executing the instructions, programs, code sets or instruction sets stored in the memory 510, and executes various functions and processes data of the present application. The processor 520 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor 520 function can also be other, and the embodiment of the present application is not specifically limited.

[0151] The communication bus 530 may include a path to transmit information between the above components. The communication bus 530 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 530 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one double arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0152] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned multi-intention voice interaction method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware embodiments. Moreover, the present application may adopt one or more computer-usable storage media containing computer-usable program codes, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disk.

[0154] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0157] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned voice interaction method when executed by a processor.

[0158] The processor is the processor in the electronic device in the above example, and the readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0159] It should be understood that the chip provided in the present application may also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip chip, etc.

[0160] The present application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned multi-intention voice interaction method example, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0161] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0162] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0163] In the description of this specification, reference to a description belonging to "an example", "some examples", "illustrative examples", "examples", "specific examples", or "exemplarily" means that the specific features, structures, materials, or characteristics described in conjunction with the example or example are included in at least one example or example of the present application. In this specification, exemplary expressions of the above terms do not necessarily refer to the same example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more examples in a suitable manner.

[0164] The above is a detailed introduction to a multi-intention voice interaction method, device, vehicle-mounted equipment and readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A multi-intention voice interaction method, characterized in that: include: Acquire voice information of the user, and convert the voice information into first text information; Identifying the first text information to obtain execution intention information; When the execution intention information includes multiple sub-intentions, each of the sub-intentions is distributed in parallel to a corresponding vehicle control module according to a mapping rule, and the vehicle control module is controlled to execute the corresponding sub-intention; Obtaining feedback information of each vehicle control module on the execution of the sub-intention; Obtaining a TTS template corresponding to each feedback information; The voice data to be played is generated based on each of the feedback information and the corresponding TTS template, and the voice data to be played is played.

2. A multi-intention voice interaction method according to claim 1, characterized in that: The obtaining of the TTS template corresponding to each feedback information includes: Determining whether the feedback information includes interaction information for the sub-intention; If the feedback information includes interactive information on the sub-intention, the TTS template corresponding to the feedback information includes execution information on the sub-intention and further selection information on the sub-intention; If the feedback information does not include the interactive information of the sub-intention, the TTS template corresponding to the feedback information includes the execution information of the sub-intention.

3. A multi-intention voice interaction method according to claim 2, characterized in that: If the feedback information includes interactive information of the sub-intention, the method includes: Acquiring voice information of the user selecting the further selection information; Converting the selected voice information into second text information; determining selection information based on the second text information; The selection information is sent to the corresponding vehicle control module.

4. The multi-intention voice interaction method according to claim 3, characterized in that: After obtaining the feedback information of each vehicle control module on the execution of the sub-intention, the method further includes: Obtaining the distribution time of each sub-intention to the corresponding vehicle control module; Obtaining the time at which the feedback information is obtained; Calculating a time difference between the acquisition time and the distribution time; The sub-intention corresponding to the time difference being greater than the time threshold is regarded as a timed-out intention, and the sub-intention corresponding to the time difference being not greater than the time threshold and being successfully executed is regarded as a normal intention.

5. The multi-intention voice interaction method according to claim 4, characterized in that: The obtaining of the TTS template corresponding to each feedback information further includes: When the sub-intention is a timeout intention, determining that the execution information of the sub-intention is execution timeout, and the TTS template of the execution timeout is the execution timeout of the sub-intention; When the sub-intention is a normal intention, determining that the execution information of the sub-intention is normal execution, and the normal execution TTS template is the normal execution of the sub-intention; When the sub-intention is a failed intention, the execution information of the sub-intention is determined to be execution failure, and the TTS template of the failed execution is the sub-intention execution failure.

6. The multi-intention voice interaction method according to claim 5, characterized in that: When the sub-intent is a failed intent, it also includes: Query whether the vehicle control module corresponding to the sub-intention is in a connected state. If not, generate a prompt message to remind the user to set the vehicle control module to a connected state.

7. The multi-intention voice interaction method according to claim 6, characterized in that: The generating the voice data to be played based on each feedback information and the corresponding TTS template includes: Integrate the TTS templates of all sub-intents, and generate the voice data to be played based on the integrated TTS templates; The voice data to be played includes execution information and / or further selection information.

8. A multi-intention voice interaction device, characterized in that: include: An acquisition conversion module, used for acquiring voice information of a user and converting the voice information into first text information; An identification module, used to identify the first text information and obtain execution intention information; A distribution module, when the execution intention information includes multiple sub-intentions, distributes each of the sub-intentions in parallel to a corresponding vehicle control module according to a mapping rule, and controls the vehicle control module to execute the corresponding sub-intention; A first acquisition module, used for acquiring feedback information of each vehicle control module on the execution of the sub-intention; A second acquisition module is used to acquire the TTS template corresponding to each feedback information; The playing module is used to generate the voice data to be played based on each feedback information and the corresponding TTS template, and play the voice data to be played.

9. A vehicle-mounted device, characterized in that: It includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the multi-intention voice interaction method described in any one of claims 1-7 by executing the computer instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-intention voice interaction method as described in any one of claims 1 to 7 is implemented.

Images