Voice interaction method, server and computer readable storage medium

Abstract

The application discloses a voice interaction method, a computer device and a computer readable storage medium. The voice interaction method comprises the following steps: receiving a current round voice request forwarded by a vehicle; when the current round is not the first round, detecting the execution of a previous round voice request to obtain a detection result; performing sentence segmentation processing or correction processing on the previous round voice request according to the detection result to obtain a first processing result; and delivering the first processing result and a second processing result of the current round voice request to the vehicle to complete voice interaction. The voice interaction method detects the execution of the previous round voice request, performs sentence segmentation or correction on the previous round voice request according to the detection result to obtain a more complete and accurate execution result, thereby realizing the detection and processing of the execution of the user instruction, improving the accuracy of the execution of the user instruction and improving the experience of the user in the voice interaction.

Description

Technical Field

[0001] This application relates to the field of voice interaction technology, and more specifically, to a voice interaction method, a server, and a computer-readable storage medium. Background Technology

[0002] Voice interaction is a fundamental application of most smart devices and has been widely integrated into daily life, covering multiple fields such as smart furniture and in-vehicle interaction. However, if the system fails to correctly understand or executes a command incorrectly, it will affect the user experience. Currently, there is a lack of methods in related technologies to detect whether user commands have been executed correctly. Summary of the Invention

[0003] This application provides a voice interaction method, a server, and a computer-readable storage medium.

[0004] This application provides a voice interaction method, including:

[0005] Receive the current round's voice request forwarded by the vehicle;

[0006] If the current round is not the first round, the execution status of the voice requests in the previous rounds is detected, and the detection results are obtained.

[0007] Based on the detection results, feedback detection processing is performed on the previous rounds of voice requests to obtain a first processing result. The feedback detection processing includes sentence segmentation processing and correction processing.

[0008] The first processing result and the second processing result for the current round of voice request are sent to the vehicle to complete the voice interaction.

[0009] Thus, in the voice interaction method, server, and computer-readable storage medium of the embodiments of this application, by detecting the execution status of previous voice requests and segmenting or correcting the previous voice requests based on the detection results, a more complete and accurate execution result is obtained, thereby realizing the detection and processing of the execution status of user commands, improving the accuracy of user command execution, and enhancing the user's experience in voice interaction.

[0010] In some implementations, the step of detecting the execution status of previous rounds of voice requests and obtaining detection results includes:

[0011] Obtain the status information of the target object, which includes vehicle components and / or in-vehicle system applications;

[0012] The processing result of the previous round is determined based on the previous round of voice requests;

[0013] If the status information and the result of the previous round of processing are consistent, it is determined that the previous round of voice request was successfully executed.

[0014] In this way, by comparing the state information of the target object with the processing results of the previous rounds, it can be determined whether the voice request of the previous round was executed successfully. If they are consistent, it can be determined that the voice request of the previous round was executed successfully, thus realizing the detection of the execution status of the voice request of the previous round.

[0015] In some implementations, the step of performing feedback detection processing on the previous rounds of voice requests based on the detection results to obtain a first processing result includes:

[0016] If the previous round of voice requests is successfully executed, the sentence segmentation process is performed to obtain the feedback processing result;

[0017] Based on the feedback processing result, the first processing result is obtained.

[0018] Thus, if the previous round of voice requests is confirmed to have been successfully executed, sentence segmentation is performed, and the first processing result is obtained based on the feedback processing result, so as to achieve further analysis of the previous round of voice requests.

[0019] In some implementations, performing the sentence segmentation process and obtaining the feedback processing result includes:

[0020] Sentence segmentation recognition is performed on the previous rounds of voice requests to determine the sub-voice requests included in the previous rounds of voice requests;

[0021] The step of obtaining the first processing result based on the feedback processing result includes:

[0022] The first processing result is determined based on the previous round processing results and the sub-voice request.

[0023] Thus, by performing sentence segmentation recognition on the earlier rounds of voice requests, the sub-voice requests included in the earlier rounds of voice requests can be determined.

[0024] In some implementations, determining the first processing result based on the previous round processing result and the sub-voice request includes:

[0025] Based on the control command corresponding to the sub-voice request and the processing results of the previous rounds, the execution status of the control command is determined, and the execution status includes executed and not executed;

[0026] Based on the execution status, the unexecuted control instructions are determined as the first processing result.

[0027] Thus, the execution status of the control command can be determined based on the control command of the sub-voice request and the processing results of the previous rounds, and the unexecuted control command is sent to the vehicle as the first processing result so that the vehicle can execute the corresponding functional requirements and realize the supplementary execution of the voice request of the previous rounds.

[0028] In some implementations, the execution status detection of previous rounds of voice requests includes:

[0029] Based on the pre-trained classification model, the category of the current round of voice request is determined, and the category includes negative user feedback;

[0030] If the category of the current round of voice request is determined to be negative feedback from the user, the detection result is determined to be execution failure.

[0031] Thus, by inputting previous voice requests into a pre-trained classification model, the category of each previous voice request can be determined. If the category is negative user feedback, the previous voice request is considered to have failed, allowing for the detection of the execution status of previous voice requests based on user feedback. Furthermore, the ability to promptly respond to and handle negative user feedback, and to provide corresponding solutions, not only improves the comfort and satisfaction of user voice interaction but also effectively reduces the risk of user churn due to poor experiences.

[0032] In some implementations, the step of performing feedback detection processing on the previous round of voice requests based on the detection results includes:

[0033] If the previous round of voice request execution fails, the correction process is performed.

[0034] Thus, if it is determined that the voice request in the previous round failed to be executed, a correction process is performed to correct the voice request in the previous round so that the corrected voice request can be recognized to meet the user's expected functional requirements.

[0035] In some implementations, performing the correction process includes:

[0036] Based on the pre-trained user behavior sequence model, the previous rounds of voice requests are corrected;

[0037] The first processing result is determined based on the revised prior round voice request.

[0038] Thus, by inputting the earlier voice requests into the pre-trained user behavior sequence model, the earlier voice requests can be corrected. Based on the corrected earlier voice requests, the first processing result can be determined.

[0039] In some implementations, the user behavior sequence model is trained through the following steps:

[0040] The user behavior in the current round is obtained based on the voice requests in the current round of training and the voice requests in the previous rounds of training.

[0041] Based on the current round of user behavior and historical user behavior, a set of training sequences is obtained;

[0042] The reference model is trained based on the training sequence set to obtain the user behavior sequence model.

[0043] In this way, by training a reference model based on the training sequence set obtained from the current round of user behavior and historical user behavior, a user behavior sequence model can be obtained. This allows the user's real needs to be predicted and analyzed when a voice request is input into the user behavior sequence model, thereby correcting the voice request.

[0044] This application provides a server including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the method of any of the above embodiments.

[0045] This application provides a computer-readable storage medium storing a computer program that, when executed by one or more processors, implements the method of any of the above embodiments.

[0046] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0047] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0048] Figure 1 This is one of the flowcharts illustrating a voice interaction method according to certain embodiments of this application;

[0049] Figure 2 This is a schematic diagram of the voice interaction processing procedure in some embodiments of this application;

[0050] Figure 3 This is a second flowchart illustrating a voice interaction method according to certain embodiments of this application;

[0051] Figure 4 This is the third flowchart illustrating a voice interaction method according to certain embodiments of this application;

[0052] Figure 5This is the fourth flowchart illustrating a voice interaction method according to certain embodiments of this application;

[0053] Figure 6 This is the fifth flowchart illustrating a voice interaction method according to certain embodiments of this application;

[0054] Figure 7 This is a flowchart of a voice interaction method according to certain embodiments of this application, number six.

[0055] Figure 8 This is the seventh flowchart illustrating a voice interaction method according to certain embodiments of this application;

[0056] Figure 9 This is the eighth flowchart of a voice interaction method according to certain embodiments of this application;

[0057] Figure 10 This is a flowchart illustrating the training method of a user behavior sequence model according to certain embodiments of this application;

[0058] Figure 11 This is the ninth flowchart of a voice interaction method according to certain embodiments of this application. Detailed Implementation

[0059] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0060] Voice interaction is a fundamental application of most smart devices and has been widely integrated into daily life, covering multiple fields such as smart furniture and in-vehicle interaction. However, if the system fails to correctly understand or executes a command incorrectly, it will affect the user experience. Currently, there is a lack of methods in related technologies to detect whether user commands have been executed correctly.

[0061] Based on the above-mentioned issues that need to be resolved, please refer to Figure 1 and Figure 2 This application provides a voice interaction method, including:

[0062] 011: Receive the current round's voice request forwarded by the vehicle;

[0063] 012: If the current round is not the first round, perform execution checks on the voice requests from previous rounds and obtain the check results;

[0064] 013: Based on the detection results, feedback detection processing is performed on the voice requests in the previous rounds to obtain the first processing result. The feedback detection processing includes sentence segmentation processing and correction processing.

[0065] 014: Issue the first processing result and the second processing result for the current round's voice request to the vehicle to complete the voice interaction.

[0066] This application provides a server including a memory and a processor. The memory stores a computer program that can be executed by the processor. The processor can be used to receive a voice request for the current round forwarded by a vehicle; and to perform execution status detection on voice requests from previous rounds when the current round is not the first round, obtaining a detection result; and to perform feedback detection processing on voice requests from previous rounds based on the detection result, obtaining a first processing result, the feedback detection processing including sentence segmentation processing and correction processing; and can also be used to send the first processing result and a second processing result for the voice request for the current round to the vehicle to complete the voice interaction.

[0067] This application provides a voice interaction device, which includes a receiving module, a detection module, a processing module, and a sending module. The receiving module receives voice requests for the current round forwarded by a vehicle; the detection module, when the current round is not the first round, detects the execution status of voice requests from previous rounds and obtains a detection result; the processing module, based on the detection result, performs feedback detection processing on the voice requests from previous rounds to obtain a first processing result, which includes sentence segmentation and correction processing; the sending module sends the first processing result and a second processing result for the voice request of the current round to the vehicle to complete the voice interaction.

[0068] Specifically, the current round voice request is a voice request message issued by the user through the vehicle's voice interaction module. The vehicle forwards the current round voice request message to the server, where it is processed and analyzed.

[0069] Upon receiving a voice request, Automatic Speech Recognition (ASR) can be used to generate corresponding recognized text. This recognized text is then forwarded to the server. From this text onward, the distinction between recognized text and voice request will not be explicitly made; it should be understood that what the user expresses is the voice request, and what the server actually processes is the recognized text of the voice request.

[0070] After processing each voice request, the server stores the processed voice request and the corresponding processing result in a designated storage area. The server can retrieve the stored voice requests and corresponding processing results from the designated storage area as needed.

[0071] Whether the current round of voice requests is the first round can be determined by retrieving previous rounds of voice requests. If a previous round of voice requests is successfully retrieved, the current round of voice requests is determined to be non-first round, and the server performs feedback detection processing on the previous rounds of voice requests. If no previous round of voice requests is retrieved, the current round of voice requests is determined to be non-first round, and the server only processes the current round of voice requests.

[0072] If the current round is not the first round, the execution status of voice requests from previous rounds is checked to determine whether the voice requests from previous rounds were executed successfully. It should be understood that successful execution here does not necessarily mean that the vehicle has completed the requirements of the user's voice request expressed in natural language, but rather that the function performed by the vehicle is consistent with the processing result obtained by the server from recognizing the voice request.

[0073] Based on the execution status detection results, the server performs sentence segmentation or correction processing on previous rounds of voice requests for feedback detection. Through sentence segmentation or correction processing, the processing results of previous rounds of voice requests can be further refined, so that the semantically determined processing results of previous rounds of voice requests are closer to the content that the user actually wants to express.

[0074] Sentence segmentation processing of previous voice requests can determine whether a previous voice request includes multiple sentences. The processing results corresponding to the sentences that were not correctly identified are then sent to the vehicle, enabling the vehicle to implement the functional requirements corresponding to those unexecuted sentences. In one embodiment, a previous voice request is "turn on the air conditioning and open the windows." After sentence segmentation processing, it can be determined that the previous voice request includes two sentences: "turn on the air conditioning" and "open the windows."

[0075] Correcting previous voice requests allows for semantic completion. In the example of "next song," it's understood that the user likely meant "play the next song." The server can correct previous voice requests based on the user's historical behavior, changing "next song" to "play the next song."

[0076] By performing feedback detection processing on previous rounds of voice requests, incomplete voice requests can be further processed, or voice requests that cannot be correctly recognized can be corrected to obtain complete voice requests that can be recognized. This further improves the recognition accuracy and execution correctness of voice requests issued by users, thereby increasing user satisfaction.

[0077] The system processes previous voice requests to obtain a first processing result and processes the current voice request to obtain a second processing result, which are then sent to the vehicle. This enables the vehicle to execute the functional requirements contained in the previous and current voice requests, or to provide feedback to the user. In order to complete the functional execution of the current voice interaction, the system can also complete any unfinished functions from the previous voice interactions, thereby achieving feedback detection and processing of voice interactions.

[0078] Please see Figure 2 In this application, the implementation method adds a feedback detection module to the traditional voice interaction processing flow of "the user issues a voice request, the ASR recognition module generates the recognition text, the semantic understanding module performs semantic understanding on the recognition text, and the result delivery module delivers the processing result obtained from the semantic understanding". This module detects the execution status of previous voice requests based on the current voice request and processes them further, thereby realizing the monitoring of previous voice requests.

[0079] Thus, in the voice interaction method, server, and computer-readable storage medium of the embodiments of this application, by detecting the execution status of previous voice requests and segmenting or correcting the previous voice requests based on the detection results, a more complete and accurate execution result is obtained, thereby realizing the detection and processing of the execution status of user commands, improving the accuracy of user command execution, and enhancing the user's experience in voice interaction.

[0080] Please see Figure 3 In some implementations, step 012 (to detect the execution status of previous rounds of voice requests and obtain the detection results) includes:

[0081] 0121: Obtain the status information of the target object, which includes vehicle parts and / or in-vehicle system applications;

[0082] 0122: Determine the processing result of the previous round based on the previous round's voice request;

[0083] 0123: If the status information is consistent with the processing result of the previous round, it is determined that the voice request of the previous round was successfully executed.

[0084] In some implementations, the processor may be used to acquire status information of a target object, including vehicle components and / or in-vehicle system applications; and to determine the processing result of a previous round based on a previous round of voice requests; and may also be used to determine that the previous round of voice requests was successfully executed if the status information and the previous round of processing results are consistent.

[0085] In some implementations, the detection module includes an acquisition submodule, a first determination submodule, and a second determination submodule. The acquisition submodule is used to acquire status information of a target object, which may include vehicle components and / or in-vehicle system applications. The first determination submodule is used to determine the processing result of a previous round of voice requests based on that round. The second determination submodule is used to determine that the previous round of voice requests was successfully executed if the status information and the previous round's processing result are consistent.

[0086] Specifically, the server obtains the status information of the target object on the vehicle and the processing result of the previous round corresponding to the previous round's voice request. It then compares the processing information of the target object with the processing result of the previous round, and if they match, determines that the previous round's voice request was successfully executed. The previous round's processing result and the previous round's voice request can be stored together in a designated storage area and retrieved when needed.

[0087] It should be noted that the state information of the target object here should be understood as the state information of objects that have a direct or indirect relationship with the vehicle. For example, objects with a direct relationship may be vehicle parts. Objects with an indirect relationship may be in-vehicle system applications related to the vehicle cabin, such as multimedia functions and navigation functions.

[0088] In one embodiment, the previous processing result corresponding to the previous voice request is "Turn on the air conditioner". The server obtains the status information of the air conditioner in the vehicle, determines that the air conditioner is currently on, and if the previous processing result is consistent with the status information of the current target object, it is determined that the previous voice request was executed successfully.

[0089] Furthermore, if the status information and the results of previous processing iterations are inconsistent, indicating that the previous voice request failed, the previous processing results can be resent to the vehicle to instruct it to attempt to execute the corresponding function again. If multiple resentments fail, the user will be alerted that there may be a functional malfunction.

[0090] In this way, by comparing the state information of the target object with the processing results of the previous rounds, it can be determined whether the voice request of the previous round was executed successfully. If they are consistent, it can be determined that the voice request of the previous round was executed successfully, thus realizing the detection of the execution status of the voice request of the previous round.

[0091] Please see Figure 4 In some implementations, step 013 (based on the detection results, performing feedback detection processing on the previous rounds of voice requests to obtain a first processing result) includes:

[0092] 0131: If the previous round of voice requests was successfully executed, perform sentence segmentation processing and obtain the feedback processing result;

[0093] 0132: Based on the feedback processing results, the first processing result is obtained.

[0094] In some implementations, the processor can be used to perform sentence segmentation processing and obtain feedback processing results if the previous round of voice requests is successfully executed; it can also be used to obtain a first processing result based on the feedback processing results.

[0095] In some implementations, the processing module includes a first execution submodule and a third determination submodule. The first execution submodule can be used to perform sentence segmentation processing and obtain a feedback processing result if the previous round of voice requests was successfully executed; the third determination submodule can be used to obtain a first processing result based on the feedback processing result.

[0096] Specifically, when a voice request includes multiple sentences, the server may only be able to recognize the content of some of the sentences, causing the vehicle to only execute the corresponding part of the function and fail to fully execute the functional requirements requested by the voice request.

[0097] Since the functional requirement performed by the vehicle at this moment corresponds to the processing result of the voice request recognized by the server, the vehicle successfully executes the corresponding portion of the voice request recognized by the server. In the example of "turn on the air conditioner and play music," the server may only recognize "turn on the air conditioner" and send the corresponding processing result to the vehicle, which then executes the "turn on the air conditioner" function. Therefore, the voice request for "turn on the air conditioner" is successfully executed.

[0098] However, since a voice request may actually include multiple clauses (functional requirements), the vehicle does not fully perform the corresponding functional requirements for the user who makes the voice request, thus affecting the user experience.

[0099] Therefore, the previously executed voice requests that were successfully processed are segmented to determine whether they include multiple sentences, and a feedback processing result is obtained. This feedback result can indicate that the previous voice request included sentences, or it can indicate the number of sentences included in the previous voice request.

[0100] Based on the feedback processing results, determine the first processing result corresponding to the previous round of voice requests, and then determine the specific instructions that need to be issued to the vehicle.

[0101] Thus, if the previous round of voice requests is confirmed to have been successfully executed, sentence segmentation is performed, and the first processing result is obtained based on the feedback processing result, so as to achieve further analysis of the previous round of voice requests.

[0102] Please see Figure 5In some implementations, step 0131 (performing sentence segmentation and obtaining feedback processing results) includes:

[0103] 01311: Perform sentence segmentation recognition on the previous rounds of voice requests to determine the sub-voice requests included in the previous rounds of voice requests;

[0104] Step 0132 (obtaining the first processing result based on the feedback processing result) includes:

[0105] 01321: Determine the first processing result based on the results of previous rounds and the sub-voice requests.

[0106] In some implementations, the processor can be used to perform sentence segmentation recognition on previous rounds of voice requests to determine the sub-voice requests included in the previous rounds of voice requests; it can also be used to determine a first processing result based on the previous rounds of processing results and the sub-voice requests.

[0107] In some implementations, the first execution submodule includes a first determining unit and a second determining unit. The first determining unit can be used to perform sentence segmentation recognition on the previous rounds of voice requests to determine the sub-voice requests included in the previous rounds of voice requests; the second determining unit can be used to determine a first processing result based on the previous rounds of processing results and the sub-voice requests.

[0108] Specifically, a voice request can be understood as either a single sentence or a complex sentence composed of multiple clauses. When a previous voice request is determined to be a single sentence, it is determined that the previous voice request includes only one sub-voice request. When a previous voice request is determined to be a complex sentence, it is segmented into multiple clauses, and each clause is processed as a sub-voice request. In the example "turn on the air conditioner and play music," the voice command can be segmented into two sub-voice requests: "turn on the air conditioner" and "play music."

[0109] By comparing the processing results of previous rounds with the sub-voice requests, the sub-voice requests that were not processed in previous rounds can be identified. Based on the unprocessed sub-voice requests, the first processing result can be determined, and the corresponding control command can be sent to the vehicle so that the vehicle can perform the corresponding functional requirements.

[0110] Thus, by performing sentence segmentation recognition on the earlier rounds of voice requests, the sub-voice requests included in the earlier rounds of voice requests can be determined.

[0111] Please see Figure 6 In some implementations, step 01321 (determining the first processing result based on the results of previous rounds and the sub-voice request) includes:

[0112] 013211: Based on the control command corresponding to the sub-voice request and the processing results of previous rounds, determine the execution status of the control command, which includes executed and not executed;

[0113] 013212: Based on the execution status, the unexecuted control instructions are determined as the first processing result.

[0114] In some implementations, the processor can be used to determine the execution status of the control instruction based on the control instruction corresponding to the subvocal request and the processing results of previous rounds, the execution status including executed and unexecuted; it can also be used to determine the unexecuted control instruction as the first processing result based on the execution status.

[0115] In some embodiments, the second determining unit includes a first determining subunit and a second determining subunit. The first determining subunit can be used to determine the execution status of a control instruction based on the control instruction corresponding to the subvocal request and the results of previous processing rounds; the execution status includes executed and unexecuted. The second determining subunit can be used to determine the unexecuted control instruction as the first processing result based on the execution status.

[0116] Specifically, the server determines the control command corresponding to each sub-voice request. For example, the control command corresponding to "turn on the air conditioner" is to turn on the car air conditioner, and the control command corresponding to "play music" is to turn on the car player and launch the music application in the car system.

[0117] In the earlier rounds, the server has identified some sub-voice requests and issued their corresponding control commands; therefore, the execution status of these sub-voice requests can be considered as executed. However, sub-voice requests that were not identified in the earlier rounds were not processed, and their control commands were not issued; therefore, the execution status of these sub-voice requests is unexecuted.

[0118] The server can determine the execution status of a sub-voice request by comparing the control instructions corresponding to the sub-voice request with the processing results of previous rounds. If the processing results of previous rounds include the control instructions for the sub-voice request, then the sub-voice request has been processed in the previous round, and the execution status of the control instructions for the sub-voice request is "executed." If the processing results of previous rounds do not include the control instructions for the sub-voice request, then the sub-voice request has not been processed in the previous round, and the execution status of the control instructions for the sub-voice request is "not executed."

[0119] Furthermore, control commands that are in an unprocessed state are issued as the first processing result to supplement the execution of control commands that were not processed in previous rounds, thereby improving the completeness of voice request processing.

[0120] In some implementations, only one control command is issued at a time. That is, the server will execute one of the control commands that is in an incomplete state and send it to the vehicle as the first processing result.

[0121] In some implementations, if the number of subvocal requests is determined to be 2, the server sends out the control instructions that were not executed as the first processing result.

[0122] In some implementations, when the number of sub-voice requests is determined to be one or more than two, the server ignores the voice requests in the current round and does not process them further.

[0123] In one embodiment, after sentence segmentation, the previous round of voice requests is divided into two sub-voice requests. The execution status of the control command in the first sub-voice request is "executed," while the execution status of the control command in the second sub-voice request is "not executed." The server sends the control command of the second sub-voice request as the first processing result to the vehicle, so that the vehicle performs the corresponding functional requirements.

[0124] Thus, the execution status of the control command can be determined based on the control command of the sub-voice request and the processing results of the previous rounds, and the unexecuted control command is sent to the vehicle as the first processing result so that the vehicle can execute the corresponding functional requirements and realize the supplementary execution of the voice request of the previous rounds.

[0125] Please see Figure 7 In some implementations, step 012 (performing execution status checks on previous rounds of voice requests) includes:

[0126] 0121: Based on the pre-trained classification model, determine the category of the voice request in the current round, including negative user feedback;

[0127] 0122: If the current round of voice requests is determined to be negative user feedback, the detection result is determined to be execution failure.

[0128] In some implementations, the processor can be used to determine the category of the current round of voice requests based on a pre-trained classification model, where the category includes negative user feedback; it can also be used to determine the detection result as execution failure if the category of the current round of voice requests is determined to be negative user feedback.

[0129] Specifically, a pre-trained classification model is input into the voice requests from the previous rounds, and the classification model outputs the category of the voice request from the previous rounds. The classification model can employ a general CNN neural network, or it can be further enhanced with acoustic features to improve the semantic recognition performance of the neural network model. The classification model can be obtained by training a reference classification model multiple times using a pre-defined voice dataset.

[0130] The current round of voice requests is categorized into normal function categories, negative user feedback categories, and other categories. Normal function categories correspond to commands that the vehicle can execute, such as "turn on the air conditioning" or "navigate to a certain location." Negative user feedback refers to user feedback on previous rounds of voice interaction, such as "navigation error," "playback error," or "incorrect." Other categories of voice requests correspond to commands that the vehicle cannot execute, or are ambiguous statements.

[0131] In actual interactions, when users provide negative feedback, it usually indicates that the function was executed incorrectly or incompletely in the previous round of voice interaction. Therefore, if the category of the previous round of voice request is determined to be negative user feedback, it can be concluded that the previous round of voice request failed.

[0132] Furthermore, when it is determined that the current round of voice requests falls under the normal function category or other categories, previous rounds of voice requests are not processed; only the current round of voice requests are processed, and the resulting second processing result is sent to the vehicle.

[0133] Thus, by inputting previous voice requests into a pre-trained classification model, the category of each previous voice request can be determined. If the category is negative user feedback, the previous voice request is considered to have failed, allowing for the detection of the execution status of previous voice requests based on user feedback. Furthermore, the ability to promptly respond to and handle negative user feedback, and to provide corresponding solutions, not only improves the comfort and satisfaction of user voice interaction but also effectively reduces the risk of user churn due to poor experiences.

[0134] In some implementations, the detection module further includes a third determining subunit and a fourth determining subunit. The third determining subunit can be used to determine the category of the current round of voice requests based on a pre-trained classification model, where the category includes negative user feedback. The fourth determining subunit can be used to determine the detection result as execution failure if the category of the current round of voice requests is determined to be negative user feedback.

[0135] Please see Figure 8In some implementations, step 013 (performing feedback detection processing on previous rounds of voice requests based on the detection results) includes:

[0136] 0131: If the previous round of voice request execution fails, perform correction processing.

[0137] In some implementations, the processor can be used to perform corrective processing if the previous round of voice request execution fails.

[0138] In some implementations, the processing module includes a second execution submodule, which can be used to perform corrective processing if the previous round of voice request execution fails.

[0139] Specifically, if the category of a previous voice request is determined to be negative user feedback, then the execution of that previous voice request is considered a failure. It's important to note that "execution failure" here refers to a discrepancy between the function executed based on the previous voice request and the user's expectations, rather than a discrepancy between the vehicle's functional requirements and the server's processing results. Therefore, even if the previous voice request is re-performed using the same recognition method, the recognition result will be the same as the first time, failing to meet user expectations. In this case, it's necessary to consider whether the recognition error was caused by an incomplete previous voice request.

[0140] Therefore, if it is determined that the previous round of voice request execution failed, a correction process is performed to correct or supplement the text of the previous round of voice request, so that the semantics of the previous round of voice request are more complete.

[0141] In some implementations, the voice request of an earlier round can be modified based on the voice request of the round before that earlier round. In one embodiment, if the voice request of an earlier round is "turn it down" and the voice request of the round before that earlier round is "turn on the air conditioner", then the voice request of the earlier round will be modified to "turn down the air conditioner temperature".

[0142] Thus, if it is determined that the voice request in the previous round failed to be executed, a correction process is performed to correct the voice request in the previous round so that the corrected voice request can be recognized to meet the user's expected functional requirements.

[0143] Please see Figure 9 In some implementations, step 0131 (performing correction processing) includes:

[0144] 01313: Based on the pre-trained user behavior sequence model, correct the voice requests in previous rounds;

[0145] 01314: Determine the first processing result based on the corrected previous round voice request.

[0146] In some implementations, the processor can be used to correct previous rounds of voice requests based on a pre-trained user behavior sequence model; it can also be used to determine a first processing result based on the corrected previous rounds of voice requests.

[0147] In some implementations, the second execution submodule includes a correction unit and a third determination unit. The correction unit can be used to correct previous rounds of voice requests based on a pre-trained user behavior sequence model; the third determination unit can be used to determine a first processing result based on the corrected previous rounds of voice requests.

[0148] Specifically, the user behavior sequence model is built based on user behavior sequences and user historical behavior sequences, employing a classic encoder and decoder structure. The user historical behavior sequence includes multiple behaviors that occur frequently in the user's historical behavior. The user behavior sequence includes multiple voice requests issued by the user during adjacent rounds of voice interaction, that is, a specific voice request and its context.

[0149] The user behavior sequence model inputted from previous voice requests can predict the user's true intent and correct the previous voice requests. In one embodiment, if the previous voice request is "next", the corrected voice request obtained after correcting the user behavior sequence model inputted from the previous voice request is "play the next song".

[0150] Based on the control command corresponding to the corrected voice request, a first processing result can be determined and sent to the vehicle so that the vehicle can execute the functional requirements corresponding to the corrected voice request.

[0151] Thus, by inputting the earlier voice requests into the pre-trained user behavior sequence model, the earlier voice requests can be corrected. Based on the corrected earlier voice requests, the first processing result can be determined.

[0152] Please see Figure 10 In some implementations, the user behavior sequence model is trained through the following steps:

[0153] 021: Obtain the user behavior for the current round based on the current round's training voice requests and the previous round's training voice requests;

[0154] 022: Based on the user behavior in the current round and the historical user behavior, obtain the training sequence set;

[0155] 023: Train the reference model based on the training sequence set to obtain the user behavior sequence model.

[0156] In some implementations, the processor can be used to obtain the user behavior of the current round based on the training voice request of the current round and the training voice request of the previous round; and to obtain a training sequence set based on the user behavior of the current round and the historical user behavior; and can also be used to train a reference model based on the training sequence set to obtain a user behavior sequence model.

[0157] This application provides a training device, which includes a first training module, a second training module, and a third training module. The first training module is used to obtain the user behavior in the current round based on the training voice requests in the current round and the training voice requests in previous rounds. The second training module is used to obtain a training sequence set based on the user behavior in the current round and historical user behavior. The third training module is used to train a reference model based on the training sequence set to obtain a user behavior sequence model.

[0158] Specifically, the voice requests in the current training round and those in previous training rounds can be voice requests from two adjacent rounds, and they are usually logically related. Treating these as a group of user behaviors yields the user behavior for the current round. Historical user behavior includes frequently occurring behaviors from the user's history, and multiple frequently occurring behaviors are grouped together. Furthermore, the current round of user behavior and historical user behavior have the same format.

[0159] The current round of user behavior and historical user behavior are combined into a training sequence set. The reference model is then trained based on the training sequence set to obtain the user behavior sequence model.

[0160] In one embodiment, three consecutive voice requests are processed into a predefined format, namely "command 1#sep#command 2#sep#command 3". Frequently occurring user behaviors are processed into a predefined format such as "open music#sep#today's weather#sep#tell a story", where "open music", "today's weather", and "tell a story" are all frequently occurring user behaviors.

[0161] In this way, by training a reference model based on the training sequence set obtained from the current round of user behavior and historical user behavior, a user behavior sequence model can be obtained. This allows the user's real needs to be predicted and analyzed when a voice request is input into the user behavior sequence model, thereby correcting the voice request.

[0162] Please see Figure 11 In some implementations, the voice interaction method further includes:

[0163] 05: If the number of consecutive failures of the voice request in the previous round exceeds the set number, the target prompt message will be used as the first processing result. The target prompt message is used to prompt the user to perform manual operation or change the way the voice request is expressed.

[0164] Specifically, the set number of attempts is a pre-set value that can be adjusted according to actual needs. If the number of consecutive failed attempts in the initial round of voice requests exceeds the set number, it may be due to noise or network issues causing multiple misrecognitions of the voice requests. In this case, TTS fallback feedback is executed, sending the target prompt information to the vehicle to remind the user to perform manual operation or change the way the voice request is expressed.

[0165] In one embodiment, after multiple execution failures, the vehicle prompts the user via voice to perform manual operation or other means to remind the user to switch the command execution method.

[0166] In this way, after multiple failed attempts, the target prompt message is sent to the vehicle to remind the user, which can promptly inform the user of the abnormal situation and facilitate the user to deal with it in a timely manner.

[0167] This application provides a computer-readable storage medium storing a computer program that, when executed by one or more processors, implements the method of any of the above embodiments.

[0168] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, external hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0169] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with an embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, without contradiction, those skilled in the art can combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples.

[0170] Furthermore, the term "connection" should be interpreted broadly. For example, it can include fixed connections, detachable connections, or integral connections; it can include direct connections or indirect connections through an intermediate medium; and it can also include internal communication between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0171] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0172] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0173] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A voice interaction method, characterized in that, include: Receive the current round's voice request forwarded by the vehicle; If the current round is not the first round, the execution status of the voice requests in the previous rounds is detected to obtain the detection result. The detection result is used to determine whether the voice requests in the previous rounds were executed successfully. Based on the detection results, feedback detection processing is performed on the previous round voice requests to obtain a first processing result. The feedback detection processing includes sentence segmentation processing and correction processing. The first processing result is that the control instructions for the sub-voice requests included in the previous round voice requests have not been executed. The first processing result and the second processing result for the current round of voice request are sent to the vehicle to complete the voice interaction.

2. The voice interaction method according to claim 1, characterized in that, The execution status detection of previous rounds of voice requests, and the resulting detection results, include: Obtain the status information of the target object, which includes vehicle components and / or in-vehicle system applications; The processing result of the previous round is determined based on the previous round of voice requests; If the status information and the result of the previous round of processing are consistent, it is determined that the previous round of voice request was successfully executed.

3. The voice interaction method according to claim 2, characterized in that, The step of performing feedback detection processing on the previous rounds of voice requests based on the detection results to obtain a first processing result includes: If the previous round of voice requests is successfully executed, the sentence segmentation process is performed to obtain the feedback processing result; Based on the feedback processing result, the first processing result is obtained.

4. The voice interaction method according to claim 3, characterized in that, The execution of the sentence segmentation process yields feedback processing results, including: Sentence segmentation recognition is performed on the previous rounds of voice requests to determine the sub-voice requests included in the previous rounds of voice requests; The step of obtaining the first processing result based on the feedback processing result includes: The first processing result is determined based on the previous round processing results and the sub-voice request.

5. The voice interaction method according to claim 4, characterized in that, Determining the first processing result based on the previous round processing results and the sub-voice request includes: Based on the control command corresponding to the sub-voice request and the processing results of the previous rounds, the execution status of the control command is determined, and the execution status includes executed and not executed; Based on the execution status, the unexecuted control instructions are determined as the first processing result.

6. The voice interaction method according to claim 1, characterized in that, The execution status detection of previous rounds of voice requests includes: Based on the pre-trained classification model, the category of the current round of voice request is determined, and the category includes negative user feedback; If the category of the current round of voice request is determined to be negative feedback from the user, the detection result is determined to be execution failure.

7. The voice interaction method according to claim 6, characterized in that, The step of performing feedback detection processing on the previous rounds of voice requests based on the detection results includes: If the previous round of voice request execution fails, the correction process is performed.

8. The voice interaction method according to claim 7, characterized in that, The execution of the correction process includes: Based on the pre-trained user behavior sequence model, the previous rounds of voice requests are corrected; The first processing result is determined based on the revised prior round voice request.

9. The voice interaction method according to claim 8, characterized in that, The user behavior sequence model is trained through the following steps: The user behavior in the current round is obtained based on the voice requests in the current round of training and the voice requests in the previous rounds of training. Based on the current round of user behavior and historical user behavior, a set of training sequences is obtained; The reference model is trained based on the training sequence set to obtain the user behavior sequence model.

10. A server, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, which, when executed by the processor, implements the method according to any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method of any one of claims 1-9.

Images