Voice interaction method, server and computer readable storage medium

Abstract

The invention discloses a voice interaction method, computer equipment and a computer readable storage medium. The voice interaction method comprises the following steps: receiving a current round voice request forwarded by a vehicle; when the current round is not the first round, the execution condition of the voice request of the previous round is detected, and a detection result is obtained; performing sentence segmentation processing or correction processing on the previous round of voice request according to the detection result to obtain a first processing result; and issuing the first processing result and a second processing result of the current round of voice request to the vehicle to complete voice interaction. According to the voice interaction method provided by the embodiment of the invention, the execution condition of the previous round of voice request is detected, and the previous round of voice request is segmented or corrected according to the detection result to obtain a more complete and accurate execution result, so that the execution condition of the user instruction is detected and processed, and the user experience is improved. The accuracy of executing the user instruction is improved, and the experience of the user in voice interaction is improved.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and more specifically, to a voice interaction method, a server, and a computer-readable storage medium. Background Art

[0002] Voice interaction is a basic application of most smart devices and has been widely integrated into daily life, covering multiple fields such as smart furniture and in-vehicle interaction. However, after the user issues a command, if the system fails to correctly understand the command or the execution is deviated, it will affect the user experience. In the related technology, there is a lack of methods to detect whether the user's command is executed correctly. Summary of the invention

[0003] Embodiments of the present application provide a voice interaction method, a server, and a computer-readable storage medium.

[0004] The present application provides a voice interaction method, including:

[0005] Receive the current round voice request forwarded by the vehicle;

[0006] When the current round is not the first round, detecting the execution status of the voice request in the previous round to obtain a detection result;

[0007] According to the detection result, feedback detection processing is performed on the previous round of voice request to obtain a first processing result, wherein the feedback detection processing includes sentence segmentation processing and correction processing;

[0008] The first processing result and the second processing result of the current round of voice request are sent to the vehicle to complete the voice interaction.

[0009] In this way, in the voice interaction method, server and computer-readable storage medium of the implementation mode of the present application, by detecting the execution status of the previous round of voice requests, and segmenting or correcting the previous round of voice requests according to the detection results, a more complete and accurate execution result can be obtained, thereby realizing the detection and processing of the execution status of user instructions, improving the accuracy of the execution of user instructions, and improving the user experience in voice interaction.

[0010] In some implementations, the performing of the execution status detection on the previous round of voice request to obtain the detection result includes:

[0011] Acquiring status information of a target object, wherein the target object includes vehicle parts and / or vehicle-mounted system applications;

[0012] Determining a previous round processing result according to the previous round voice request;

[0013] When the state information is consistent with the previous round processing result, it is determined that the previous round voice request is executed successfully.

[0014] In this way, by comparing the status information of the target object and the processing results of the previous round, it is possible to determine whether the previous round of voice request is executed successfully. If they are consistent, it is determined that the previous round of voice request is executed successfully, thereby realizing the detection of the execution status of the previous round of voice request.

[0015] In some implementations, performing feedback detection processing on the previous round of voice request according to the detection result to obtain a first processing result includes:

[0016] In the case where the previous round of voice request is successfully executed, executing the sentence segmentation process to obtain a feedback processing result;

[0017] The first processing result is obtained according to the feedback processing result.

[0018] In this way, when it is determined that the previous round of voice request is executed successfully, sentence segmentation processing is performed, and a first processing result is obtained according to the feedback processing result to achieve further analysis of the previous round of voice request.

[0019] In some implementations, the performing the sentence segmentation process to obtain a feedback process result includes:

[0020] Performing sentence segmentation recognition on the previous round of voice request to determine sub-voice requests included in the previous round of voice request;

[0021] The obtaining the first processing result according to the feedback processing result includes:

[0022] The first processing result is determined according to the previous round processing result and the sub-voice request.

[0023] In this way, by performing sentence segmentation recognition on the previous round of voice requests, the sub-voice requests included in the previous round of voice requests can be determined.

[0024] In some implementations, determining the first processing result according to the previous round processing result and the sub-voice request includes:

[0025] Determining, according to the control instruction corresponding to the sub-voice request and the previous round processing result, an execution status of the control instruction, wherein the execution status includes executed and not executed;

[0026] According to the execution status, the unexecuted control instruction is determined as the first processing result.

[0027] In this way, the execution status of the control instruction can be determined based on the control instruction of the sub-voice request and the processing result of the previous round, and the unexecuted control instruction can be sent to the vehicle as the first processing result, so that the vehicle can execute the corresponding functional requirements and realize the supplementary execution of the previous round of voice request.

[0028] In some implementations, the detecting of the execution status of the previous round of voice requests includes:

[0029] Determine, based on the pre-trained classification model, the category of the current round of voice request, wherein the category includes negative user feedback;

[0030] In a case where it is determined that the category of the current round of voice request is the user negative feedback, the detection result is determined to be execution failure.

[0031] In this way, by inputting the previous round of voice requests into the pre-trained classification model, the category of the previous round of voice requests can be obtained. In the case of negative user feedback, it is determined that the previous round of voice requests failed to execute, so that the execution of the previous round of voice requests can be detected according to user feedback. In addition, it is possible to respond to and process negative feedback conveyed by users in a timely manner and provide corresponding solutions, which can not only improve the comfort and satisfaction of user voice interaction, but also effectively reduce the risk of user loss due to poor experience.

[0032] In some implementations, performing feedback detection processing on the previous round of voice request according to the detection result includes:

[0033] In the case where the execution of the previous round of voice request fails, the correction process is performed.

[0034] In this way, when it is determined that the execution of the previous round of voice request fails, a correction process is performed to correct the previous round of voice request so that the corrected voice request can be recognized to meet the functional requirements expected by the user.

[0035] In some embodiments, performing the correction process includes:

[0036] Based on the pre-trained user behavior sequence model, modifying the previous round of voice requests;

[0037] The first processing result is determined according to the corrected voice request of the previous round.

[0038] In this way, by inputting the previous round of voice requests into the pre-trained user behavior sequence model, the previous round of voice requests can be corrected. According to the corrected previous round of voice requests, the first processing result can be determined.

[0039] In some embodiments, the user behavior sequence model is trained by the following steps:

[0040] Obtaining the current round user behavior according to the current round training voice request and the previous round training voice request;

[0041] According to the current round of user behavior and historical user behavior, a training sequence set is obtained;

[0042] A reference model is trained according to the training sequence set to obtain the user behavior sequence model.

[0043] In this way, by training the reference model based on the training sequence set obtained according to the current round of user behavior and historical user behavior, a user behavior sequence model can be obtained, so that when a voice request is input into the user behavior sequence model, the user's real needs can be predicted and analyzed, thereby correcting the voice request.

[0044] An embodiment of the present application provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, a method of any of the above embodiments is implemented.

[0045] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by one or more processors, the method of any of the above embodiments is implemented.

[0046] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0048] Figure 1 It is one of the flowcharts of the voice interaction method of certain embodiments of the present application;

[0049] Figure 2 is a schematic diagram of a voice interaction processing process of certain embodiments of the present application;

[0050] Figure 3 This is the second flow chart of the voice interaction method of certain implementation modes of the present application;

[0051] Figure 4 This is the third flow chart of the voice interaction method of certain implementation modes of the present application;

[0052] Figure 5This is a fourth flowchart of a voice interaction method according to certain embodiments of the present application;

[0053] Figure 6 This is a fifth flow chart of a voice interaction method according to certain embodiments of the present application;

[0054] Figure 7 This is the sixth flow chart of the voice interaction method of certain implementation modes of the present application;

[0055] Figure 8 This is the seventh flow chart of the voice interaction method of certain implementation modes of the present application;

[0056] Fig. 9 This is the eighth flow chart of the voice interaction method of certain implementation modes of the present application;

[0057] Fig.10 It is a flowchart of a method for training a user behavior sequence model in certain embodiments of the present application;

[0058] Fig.11 This is the ninth flow chart of the voice interaction method of certain embodiments of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0060] Voice interaction is a basic application of most smart devices and has been widely integrated into daily life, covering multiple fields such as smart furniture and in-vehicle interaction. However, after the user issues a command, if the system fails to correctly understand the command or the execution is deviated, it will affect the user experience. In the related technology, there is a lack of methods to detect whether the user's command is executed correctly.

[0061] Based on the above unresolved issues, please refer to Figure 1 and Figure 2 , the present application embodiment provides a voice interaction method, including:

[0062] 011: Receive the current round voice request forwarded by the vehicle;

[0063] 012: When the current round is not the first round, the execution status of the voice request in the previous round is detected to obtain the detection result;

[0064] 013: performing feedback detection processing on the previous round of voice request according to the detection result to obtain a first processing result, wherein the feedback detection processing includes sentence segmentation processing and correction processing;

[0065] 014: Send the first processing result and the second processing result of the current round of voice request to the vehicle to complete the voice interaction.

[0066] The embodiment of the present application provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and the computer program can be executed by the processor. The processor can be used to receive a current round voice request forwarded by a vehicle; and to detect the execution status of the previous round voice request when the current round is not the first round, and obtain a detection result; and to perform feedback detection processing on the previous round voice request according to the detection result, and obtain a first processing result, wherein the feedback detection processing includes sentence segmentation processing and correction processing; and can also be used to send the first processing result and the second processing result of the current round voice request to the vehicle to complete the voice interaction.

[0067] The embodiment of the present application provides a voice interaction device, which includes a receiving module, a detection module, a processing module and a sending module. Among them, the receiving module can be used to receive the current round of voice requests forwarded by the vehicle; the detection module can be used to detect the execution of the previous round of voice requests when the current round is not the first round, and obtain the detection result; the processing module can be used to perform feedback detection processing on the previous round of voice requests according to the detection result, and obtain the first processing result, and the feedback detection processing includes sentence segmentation processing and correction processing; the sending module can be used to send the first processing result and the second processing result of the current round of voice requests to the vehicle to complete the voice interaction.

[0068] Specifically, the current round of voice request is the voice request information sent by the user through the voice interaction module of the vehicle. The vehicle forwards the current round of voice request information to the server, and the server processes and analyzes it.

[0069] After receiving the voice request, the corresponding recognition text can be generated through Automatic Speech Recognition (ASR) processing. Then the recognition text of the voice request is forwarded to the server. In the following text, no actual distinction is made between the recognition text and the voice request. It should be understood that what is expressed by the user is actually a voice request, and what is actually processed by the server is the recognition text of the voice request.

[0070] After processing each voice request, the server will store the processed voice request and the corresponding processing result in the set storage area. The server can retrieve the stored voice request and the corresponding processing result from the set storage area as needed.

[0071] Whether the current round of voice request is the first round of voice request can be realized by obtaining the previous round of voice request. If the previous round of voice request is successfully obtained, it is determined that the current round of voice request is not the first round, and the server performs feedback detection processing on the previous round of voice request. If the previous round of voice request is not obtained, it is determined that the current round of voice request is not the first round of voice request, and the server only processes the current round of voice request.

[0072] When the current round is not the first round, the execution status of the voice request in the previous round is detected to determine whether the voice request in the previous round is successfully executed. It should be understood that the successful execution here does not completely refer to the vehicle completing the requirements of the user's voice request expressed in natural language, but it can be that the function implemented by the vehicle is consistent with the processing result obtained by the server recognizing the voice request.

[0073] Based on the detection result of the execution status detection, the server performs sentence segmentation or correction processing on the previous round of voice request to perform feedback detection processing. Through sentence segmentation or correction processing, the processing result of the previous round of voice request can be further corrected so that the processing result of the recognition semantics of the previous round of voice request is closer to the content that the user actually wants to express.

[0074] Segmentation processing is performed on the previous round of voice request to determine whether the previous round of voice request includes multiple sentences. The processing results corresponding to the sentences that were not correctly identified are sent to the vehicle so that the vehicle can realize the functional requirements corresponding to the sentences that have not been executed. In one embodiment, the previous round of voice request is "turn on the air conditioner and open the car window". After the sentence segmentation processing, it can be determined that the previous round of voice request includes two sentences, "turn on the air conditioner" and "open the car window".

[0075] The previous round of voice requests can be corrected to complete the semantics of the previous round of voice requests. In the example of "next song", it can be understood that the user is more likely to want to express "play the next song". The server can complete and correct the previous round of voice requests based on the user's historical behavior, and correct "next song" to "play the next song".

[0076] By performing feedback detection processing on the previous round of voice requests, the incomplete voice requests can be further processed, or the voice requests that cannot be correctly recognized can be corrected to obtain a complete voice request that can be recognized, thereby further improving the recognition accuracy and execution correctness of the voice requests issued by the user and improving user satisfaction.

[0077] A first processing result obtained by processing a previous round of voice requests and a second processing result obtained by processing a current round of voice requests are sent to the vehicle, so that the vehicle can execute the functional requirements included in the previous round of voice requests and the current round of voice requests, or give feedback to the user, so as to complete the unfinished functions of the previous round of voice interaction while completing the function execution of the current round of voice interaction, thereby realizing feedback detection and processing of voice interaction.

[0078] See also Figure 2 In the implementation mode of the present application, a feedback detection module is added to the traditional voice interaction processing flow of "the user sends a voice request, the ASR recognition module generates a recognition text, the semantic understanding module performs semantic understanding on the recognition text, and the result delivery module delivers the processing result obtained by the semantic understanding" to detect the execution status of the previous round of voice requests according to the current round of voice requests, and further process it, thereby realizing the monitoring of the previous round of voice requests.

[0079] In this way, in the voice interaction method, server and computer-readable storage medium of the implementation mode of the present application, by detecting the execution status of the previous round of voice requests, and segmenting or correcting the previous round of voice requests according to the detection results, a more complete and accurate execution result can be obtained, thereby realizing the detection and processing of the execution status of user instructions, improving the accuracy of the execution of user instructions, and improving the user experience in voice interaction.

[0080] See also Figure 3 In some implementations, step 012 (detecting the execution status of the previous round of voice requests to obtain a detection result) includes:

[0081] 0121: Obtain status information of a target object, the target object including vehicle parts and / or vehicle system applications;

[0082] 0122: Determine the processing result of the previous round according to the voice request of the previous round;

[0083] 0123: When the status information is consistent with the processing result of the previous round, it is determined that the execution of the voice request in the previous round is successful.

[0084] In some embodiments, the processor can be used to obtain status information of a target object, which includes vehicle parts and / or vehicle-mounted system applications; and to determine a previous round of processing results based on a previous round of voice requests; and can also be used to determine that a previous round of voice requests was successfully executed when the status information and the previous round of processing results are consistent.

[0085] In some embodiments, the detection module includes an acquisition submodule, a first determination submodule, and a second determination submodule. The acquisition submodule can be used to acquire the state information of the target object, the target object includes vehicle parts and / or vehicle-mounted system applications; the first determination submodule can be used to determine the previous round processing result according to the previous round voice request; the second determination submodule can be used to determine that the previous round voice request is executed successfully when the state information is consistent with the previous round processing result.

[0086] Specifically, the server obtains the state information of the target object on the vehicle, obtains the previous round processing result corresponding to the previous round voice request, and compares the processing information of the target object with the previous round processing result, and determines that the previous round voice request is successfully executed if they are consistent. The previous round processing result and the previous round voice request can be stored together in a set storage area, and can be called when needed.

[0087] It should be noted that the state information of the target object here should be understood as the state information of the object that has a direct or indirect relationship with the vehicle. For example, the object with a direct relationship may be a vehicle component. The object with an indirect relationship may be an in-vehicle system application related to the vehicle cabin, such as multimedia functions, navigation functions, etc.

[0088] In one embodiment, the previous round processing result corresponding to the previous round voice request is "turn on the air conditioner". The server obtains the status information of the air conditioner on the vehicle and determines that the air conditioner is currently turned on. If the previous round processing result is consistent with the status information of the current target object, it is determined that the previous round voice request is executed successfully.

[0089] In addition, when the status information is inconsistent with the previous round processing result, it is determined that the previous round voice request execution failed, and the previous round processing result can be sent to the vehicle again to instruct the vehicle to try to execute the corresponding function again. If it fails after multiple sending, the user is prompted that there may be a functional failure.

[0090] In this way, by comparing the status information of the target object and the processing results of the previous round, it is possible to determine whether the previous round of voice request is executed successfully. If they are consistent, it is determined that the previous round of voice request is executed successfully, thereby realizing the detection of the execution status of the previous round of voice request.

[0091] See also Figure 4 In some implementations, step 013 (performing feedback detection processing on the previous round of voice requests according to the detection result to obtain a first processing result) includes:

[0092] 0131: When it is determined that the previous round of voice request execution is successful, perform sentence segmentation processing to obtain a feedback processing result;

[0093] 0132: According to the feedback processing result, a first processing result is obtained.

[0094] In some implementations, the processor may be used to perform sentence segmentation processing to obtain a feedback processing result when a previous round of voice request execution is successful; and may also be used to obtain a first processing result based on the feedback processing result.

[0095] In some embodiments, the processing module includes a first execution submodule and a third determination submodule. The first execution submodule can be used to perform sentence segmentation processing to obtain a feedback processing result when the previous round of voice request execution is successful; the third determination submodule can be used to obtain the first processing result according to the feedback processing result.

[0096] Specifically, when a voice request includes multiple sentences, the server may only be able to recognize the content of some of the sentences, causing the vehicle to only execute part of the functions and fail to fully execute the functional requirements required by the voice request.

[0097] Since the functional requirements of the vehicle at this time actually correspond to the processing results of the voice request recognized by the server, the vehicle successfully executes the partial voice request recognized by the server. In the example of "turn on the air conditioner and play music", the server may only recognize "turn on the air conditioner" and send the corresponding processing results to the vehicle, and the vehicle executes the function of "turn on the air conditioner". Therefore, the voice request of "turn on the air conditioner" is successfully executed.

[0098] However, since the voice request may actually include multiple sentences (functional requirements), for the user who issued the voice request, the vehicle does not fully execute the corresponding functional requirements, affecting the user experience.

[0099] Therefore, the previous round of voice request that is determined to be successfully executed is segmented to determine whether the previous round of voice request includes multiple sentences, and a feedback processing result is obtained. The feedback processing result may be that the previous round of voice request includes sentences, or the number of sentences included in the previous round of voice request.

[0100] The first processing result corresponding to the previous round of voice request is determined according to the feedback processing result to determine the specific instructions that need to be sent to the vehicle.

[0101] In this way, when it is determined that the previous round of voice request is executed successfully, sentence segmentation processing is performed, and a first processing result is obtained according to the feedback processing result to achieve further analysis of the previous round of voice request.

[0102] See also Figure 5In some implementations, step 0131 (performing sentence segmentation processing and obtaining feedback processing results) includes:

[0103] 01311: performing sentence segmentation recognition on the previous round of voice request to determine the sub-voice request included in the previous round of voice request;

[0104] Step 0132 (obtaining a first processing result according to the feedback processing result) includes:

[0105] 01321: Determine a first processing result according to the previous round processing result and the sub-voice request.

[0106] In some embodiments, the processor may be used to perform sentence segmentation recognition on a previous round of voice requests to determine sub-voice requests included in the previous round of voice requests; and may also be used to determine a first processing result based on the previous round of processing results and the sub-voice requests.

[0107] In some embodiments, the first execution submodule includes a first determination unit and a second determination unit. The first determination unit may be used to perform sentence segmentation recognition on a previous round of voice requests to determine sub-voice requests included in the previous round of voice requests; and the second determination unit may be used to determine the first processing result according to the previous round of processing results and the sub-voice requests.

[0108] Specifically, it can be understood that a voice request can be a whole sentence or a complex sentence composed of multiple clauses. When it is determined that the voice request in the previous round is a whole sentence, it is determined that the voice request in the previous round only includes one sub-voice request. When it is determined that the voice request in the previous round is a complex sentence, the voice request in the previous round is divided into multiple clauses, and each clause is processed as a sub-voice request. In the example of "turn on the air conditioner and play music", the voice command can be divided into two sub-voice requests of "turn on the air conditioner" and "play music".

[0109] By comparing the processing results of the previous round and the sub-voice requests, the sub-voice requests that were not processed in the previous round can be determined, and based on the unprocessed sub-voice requests, the first processing result can be determined to send the corresponding control instructions to the vehicle so that the vehicle can execute and realize the corresponding functional requirements.

[0110] In this way, by performing sentence segmentation recognition on the previous round of voice requests, the sub-voice requests included in the previous round of voice requests can be determined.

[0111] See also Figure 6 In some implementations, step 01321 (determining a first processing result based on a previous round of processing results and a sub-voice request) includes:

[0112] 013211: Determine the execution status of the control instruction according to the control instruction corresponding to the sub-voice request and the result of the previous round of processing, where the execution status includes executed and not executed;

[0113] 013212: According to the execution status, the unexecuted control instruction is determined as the first processing result.

[0114] In some embodiments, the processor can be used to determine the execution status of the control instruction based on the control instruction corresponding to the sub-voice request and the processing results of the previous round, and the execution status includes executed and not executed; it can also be used to determine the unexecuted control instruction as the first processing result based on the execution status.

[0115] In some embodiments, the second determination unit includes a first determination subunit and a second determination subunit. The first determination subunit may be used to determine the execution status of the control instruction according to the control instruction corresponding to the sub-voice request and the previous round processing result, wherein the execution status includes executed and unexecuted; and the second determination subunit may be used to determine the unexecuted control instruction as the first processing result according to the execution status.

[0116] Specifically, the server determines the control instruction corresponding to each sub-voice request. For example, the control instruction corresponding to "turn on the air conditioner" is to turn on the car air conditioner, and the control instruction corresponding to "play music" is to turn on the car player and start the music application in the car system.

[0117] In the previous round, the server has identified some sub-voice requests and issued their corresponding control instructions, so the execution status of these sub-voice requests can be considered as executed. However, the sub-voice requests that were not identified in the previous round of voice requests were not processed, and their control instructions were not issued, so the execution status of these sub-voice requests is not executed.

[0118] The server can determine the execution status of the sub-voice request by comparing the control instruction corresponding to the sub-voice request and the processing result of the previous round. If the processing result of the previous round includes the control instruction of the sub-voice request, the sub-voice request has been processed in the previous round, and the execution status of the control instruction of the sub-voice request is executed. If the processing result of the previous round does not include the control instruction of the sub-voice request, the sub-voice request has not been processed in the previous round, and the execution status of the control instruction of the sub-voice request is not executed.

[0119] Furthermore, the control instructions with an execution status of unprocessed are issued as the first processing result, so as to supplement the execution of the control instructions that have not been processed in the previous round, thereby improving the integrity of the processing of the voice request.

[0120] In some implementations, the number of control instructions sent each time is one, that is, the server sends one of the control instructions whose execution status is unfinished to the vehicle as the first processing result.

[0121] In some implementations, when it is determined that the number of sub-voice requests is 2, the server sends the unexecuted control instructions as the first processing result.

[0122] In some implementations, when it is determined that the number of sub-voice requests is 1 or greater than 2, the server ignores the current round of voice requests and does not further process them.

[0123] In one embodiment, after the sentence segmentation process, the first round of voice request is divided into two sub-voice requests, the execution status of the control instruction of the first sub-voice request is executed, and the execution status of the control instruction of the second sub-voice request is not executed. The server sends the control instruction of the second sub-voice request as the first processing result to the vehicle, so that the vehicle executes the corresponding functional requirements.

[0124] In this way, the execution status of the control instruction can be determined based on the control instruction of the sub-voice request and the processing result of the previous round, and the unexecuted control instruction can be sent to the vehicle as the first processing result, so that the vehicle can execute the corresponding functional requirements and realize the supplementary execution of the previous round of voice request.

[0125] See also Figure 7 In some implementations, step 012 (detecting the execution status of the previous round of voice requests) includes:

[0126] 0121: Based on the pre-trained classification model, determine the category of the current round of voice request, which includes negative user feedback;

[0127] 0122: When it is determined that the category of the current round of voice request is negative user feedback, the detection result is determined to be execution failure.

[0128] In some embodiments, the processor may be used to determine a category of a current round of voice request based on a pre-trained classification model, where the category includes negative user feedback; and may also be used to determine that a detection result is an execution failure when the category of the current round of voice request is determined to be negative user feedback.

[0129] Specifically, the pre-trained classification model is input into the previous round of voice requests, and the classification model can output the category of the previous round of voice requests. The classification model can adopt a general CNN neural network, or further add acoustic features on the basis of the CNN neural network to enhance the semantic recognition effect of the neural network model. The classification model can be obtained by training the reference classification model for multiple rounds using a preset voice data set.

[0130] The categories of voice requests in the current round include normal function categories, user negative feedback, and other categories. Among them, the processing results corresponding to the voice requests in the normal function category are functional instructions that the vehicle can execute, such as "turn on the air conditioner", "navigate to a certain place", etc. User negative feedback is the negative feedback raised by the user in the previous round of voice interaction, such as "navigation error", "playback error", "wrong", etc. Other categories of voice requests are voice requests whose corresponding processing results are functional instructions that the vehicle cannot execute, or sentences with unclear semantics.

[0131] In actual interactions, when a user issues negative feedback, it is usually because the function execution in the previous round of voice interaction was wrong or incomplete. Therefore, if the category of the voice request in the previous round is determined to be negative feedback from the user, it can be determined that the execution of the voice request in the previous round failed.

[0132] In addition, when it is determined that the category of the current round of voice request is a normal function category or other category, the previous round of voice request is not processed, only the current round of voice request is processed, and the obtained second processing result is sent to the vehicle.

[0133] In this way, by inputting the previous round of voice requests into the pre-trained classification model, the category of the previous round of voice requests can be obtained. In the case of negative user feedback, it is determined that the previous round of voice requests failed to execute, so that the execution of the previous round of voice requests can be detected according to user feedback. In addition, it is possible to respond to and process negative feedback conveyed by users in a timely manner and provide corresponding solutions, which can not only improve the comfort and satisfaction of user voice interaction, but also effectively reduce the risk of user loss due to poor experience.

[0134] In some embodiments, the detection module further includes a third determination subunit and a fourth determination subunit. The third determination subunit may be used to determine the category of the current round of voice request based on the pre-trained classification model, the category including user negative feedback; and the fourth determination subunit may be used to determine that the detection result is an execution failure when the category of the current round of voice request is determined to be user negative feedback.

[0135] See also Figure 8In some implementations, step 013 (performing feedback detection processing on the previous round of voice requests according to the detection results) includes:

[0136] 0131: When the previous round of voice request execution fails, perform correction processing.

[0137] In some embodiments, the processor may be configured to perform a correction process if a previous round of voice request execution fails.

[0138] In some embodiments, the processing module includes a second execution submodule, and the second execution submodule can be used to perform correction processing when the execution of the previous round of voice request fails.

[0139] Specifically, when it is determined that the category of the previous round of voice request is negative user feedback, it is determined that the execution of the previous round of voice request has failed. It should be noted that the execution failure here means that the relevant functions executed according to the previous round of voice request do not meet the user's expectations, rather than the functional requirements executed by the vehicle and the processing results issued by the server. Therefore, even if the same recognition as the previous round is repeated for the previous round of voice request, the recognition result will be the same as the first time, and the user's expectations will also not be met. At this point, it is necessary to consider whether the recognition error is caused by the incomplete voice request in the previous round.

[0140] Therefore, when it is determined that the execution of the previous round of voice request fails, a correction process is performed to correct or complete the text of the previous round of voice request, so that the semantics of the previous round of voice request is more complete.

[0141] In some implementations, the voice request in the previous round can be modified according to the voice request in the previous round. In one embodiment, the voice request in the previous round is "turn down the temperature", and the voice request in the previous round is "turn on the air conditioner", then the voice request in the previous round is modified to "turn down the air conditioner temperature".

[0142] In this way, when it is determined that the execution of the previous round of voice request fails, a correction process is performed to correct the previous round of voice request so that the corrected voice request can be recognized to meet the functional requirements expected by the user.

[0143] See also Fig. 9 In some embodiments, step 0131 (performing correction processing) includes:

[0144] 01313: Based on the pre-trained user behavior sequence model, correct the previous round of voice requests;

[0145] 01314: Determine a first processing result according to the corrected voice request of the previous round.

[0146] In some embodiments, the processor may be used to modify a previous round of voice requests based on a pre-trained user behavior sequence model; and may also be used to determine a first processing result based on the modified previous round of voice requests.

[0147] In some embodiments, the second execution submodule includes a correction unit and a third determination unit. The correction unit can be used to correct the previous round of voice requests based on the pre-trained user behavior sequence model; the third determination unit can be used to determine the first processing result according to the corrected previous round of voice requests.

[0148] Specifically, the user behavior sequence model is established based on the user behavior sequence and the user historical behavior sequence, using the classic encoder and decoder structure. The user historical behavior sequence includes multiple behaviors that appear frequently in the user's historical behavior. The user behavior sequence includes multiple voice requests issued by the user in multiple adjacent rounds of voice interaction, that is, a certain voice request and its context.

[0149] By inputting the previous round of voice request into the user behavior sequence model, the user's real intention can be predicted to correct the previous round of voice request. In one embodiment, the previous round of voice request is "next", and after the previous round of voice request is input into the user behavior sequence model for correction, the corrected voice request is "play the next song".

[0150] According to the control instruction corresponding to the modified voice request, a first processing result can be determined, and the first processing result can be sent to the vehicle so that the vehicle can execute the functional requirements corresponding to the modified voice request.

[0151] In this way, by inputting the previous round of voice requests into the pre-trained user behavior sequence model, the previous round of voice requests can be corrected. According to the corrected previous round of voice requests, the first processing result can be determined.

[0152] See also Fig.10 In some embodiments, the user behavior sequence model is trained by the following steps:

[0153] 021: Obtain the current round user behavior according to the current round training voice request and the previous round training voice request;

[0154] 022: Get a training sequence set based on the current round of user behavior and historical user behavior;

[0155] 023: Train the reference model according to the training sequence set to obtain the user behavior sequence model.

[0156] In some embodiments, the processor can be used to obtain the current round user behavior based on the current round training voice request and the previous round training voice request; and to obtain a training sequence set based on the current round user behavior and historical user behavior; and can also be used to train a reference model based on the training sequence set to obtain a user behavior sequence model.

[0157] The embodiment of the present application provides a training device, which includes a first training module, a second training module and a third training module. The first training module can be used to obtain the current round user behavior according to the current round training voice request and the previous round training voice request; the second training module can be used to obtain the training sequence set according to the current round user behavior and the historical user behavior; the third training module can be used to train the reference model according to the training sequence set to obtain the user behavior sequence model.

[0158] Specifically, the current round training voice request and the previous round training voice request can be voice requests in two adjacent rounds. Generally speaking, there is a logical connection between them. They are taken as a group of user behaviors to obtain the current round user behavior. The historical user behavior includes the behaviors that appear frequently in the user history, and multiple behaviors that appear frequently are taken as a group. In addition, the format of the current round user behavior and the historical user behavior is the same.

[0159] The current round of user behavior and historical user behavior are combined into a training sequence set, and the reference model is trained according to the training sequence set to obtain a user behavior sequence model.

[0160] In one embodiment, three consecutive rounds of voice requests are processed into a setting format, and the setting format is "command 1#sep#command 2#sep#command 3". The user's historical high-frequency behaviors are processed into a setting format such as "turn on music#sep#today's weather#sep#tell a story", where "turn on music", "today's weather" and "tell a story" are all historical high-frequency behaviors of the user.

[0161] In this way, by training the reference model based on the training sequence set obtained according to the current round of user behavior and historical user behavior, a user behavior sequence model can be obtained, so that when a voice request is input into the user behavior sequence model, the user's real needs can be predicted and analyzed, thereby correcting the voice request.

[0162] See also Fig.11 In some embodiments, the voice interaction method further includes:

[0163] 05: If the number of consecutive execution failures of the voice request in the previous round is greater than the set number, the target prompt information is used as the first processing result. The target prompt information is used to prompt the user to perform manual operation or change the expression of the voice request.

[0164] Specifically, the set number of times is a preset value and can be set according to actual needs. If the number of consecutive execution failures of the previous round of voice requests is greater than the set number, it may be due to the influence of the noise environment or the network, resulting in multiple recognition errors of the voice request. At this time, TTS feedback is executed and the target prompt information is sent to the vehicle to remind the user to perform manual operation or change the expression of the voice request.

[0165] In one embodiment, after multiple execution failures, the vehicle reminds the user through voice to perform manual operation or other means to remind the user to switch the command execution mode.

[0166] In this way, after multiple execution failures, the target prompt information is sent to the vehicle to remind the user, and the user can be informed of the abnormal situation in time, so that the user can handle it in time.

[0167] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the method of any of the above embodiments is implemented.

[0168] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium.

[0169] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0170] In addition, the term "connection" should be understood in a broad sense, for example, it can include fixed connection, detachable connection, or integral connection; it can include direct connection, indirect connection through an intermediate medium, and internal communication between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0171] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0172] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0173] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in the field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receive the current round voice request forwarded by the vehicle; When the current round is not the first round, detecting the execution status of the voice request in the previous round to obtain a detection result; According to the detection result, feedback detection processing is performed on the previous round of voice request to obtain a first processing result, wherein the feedback detection processing includes sentence segmentation processing and correction processing; The first processing result and the second processing result of the current round of voice request are sent to the vehicle to complete the voice interaction.

2. The voice interaction method according to claim 1, characterized in that: The detecting of the execution status of the previous round of voice requests to obtain the detection result includes: Acquiring status information of a target object, wherein the target object includes vehicle parts and / or vehicle-mounted system applications; Determining a previous round processing result according to the previous round voice request; When the state information is consistent with the previous round processing result, it is determined that the previous round voice request is executed successfully.

3. The voice interaction method according to claim 2, characterized in that: The step of performing feedback detection processing on the previous round of voice request according to the detection result to obtain a first processing result includes: In the case where the previous round of voice request is successfully executed, executing the sentence segmentation process to obtain a feedback processing result; The first processing result is obtained according to the feedback processing result.

4. The voice interaction method according to claim 3, characterized in that: The executing the sentence segmentation process to obtain a feedback processing result includes: Performing sentence segmentation recognition on the previous round of voice request to determine sub-voice requests included in the previous round of voice request; Obtaining the first processing result according to the feedback processing result includes: The first processing result is determined according to the previous round processing result and the sub-voice request.

5. The voice interaction method according to claim 4, characterized in that: The determining the first processing result according to the previous round processing result and the sub-voice request includes: Determining, according to the control instruction corresponding to the sub-voice request and the previous round processing result, an execution status of the control instruction, wherein the execution status includes executed and not executed; According to the execution status, the unexecuted control instruction is determined as the first processing result.

6. The voice interaction method according to claim 1, characterized in that: The detecting of the execution status of the previous round of voice requests includes: Determine, based on the pre-trained classification model, the category of the current round of voice request, wherein the category includes negative user feedback; When it is determined that the category of the current round of voice request is the user negative feedback, the detection result is determined to be execution failure.

7. The voice interaction method according to claim 6, characterized in that: The performing feedback detection processing on the previous round of voice request according to the detection result includes: In the case where the execution of the previous round of voice request fails, the correction process is performed.

8. The voice interaction method according to claim 7, characterized in that: The performing of the correction process comprises: Based on the pre-trained user behavior sequence model, modifying the previous round of voice requests; The first processing result is determined according to the corrected voice request of the previous round.

9. The voice interaction method according to claim 7, characterized in that: The user behavior sequence model is trained by the following steps: Obtaining the current round user behavior according to the current round training voice request and the previous round training voice request; According to the current round of user behavior and historical user behavior, a training sequence set is obtained; A reference model is trained according to the training sequence set to obtain the user behavior sequence model.

10. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 19 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 19 is implemented.

Images