Voice interaction method and apparatus thereof, vehicle, and storage medium

Abstract

The application discloses a voice interaction method and device, a vehicle and a storage medium. The voice interaction method comprises the following steps: in the case that the network connection of the vehicle is in an abnormal state, performing voice recognition on each frame of user voice data collected in real time to obtain a real-time voice request; matching the real-time voice request in a voice request library to obtain a preset voice request, wherein the preset voice request comprises a first content and a second content; in the case that the recognized real-time voice request is the same as the first content, if it is recognized that the user voice data in a preset frame meets a preset condition corresponding to the second content, it is determined that a voice instruction corresponding to the preset voice request is a first target voice instruction; and completing voice interaction according to the first target voice instruction. In the case that the network is abnormal, each frame of user voice data is collected in real time, voice recognition is performed to obtain a real-time voice request, and a preset voice request is matched according to a voice request library, so that fast response and high accuracy of voice recognition are realized.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a voice interaction method, apparatus, vehicle, and storage medium thereof. Background Technology

[0002] Current in-vehicle voice recognition solutions require uploading complete speech segments to a cloud server when network conditions are good, and then sending the cloud recognition results back to the vehicle. This entire process introduces a delay in voice recognition. Especially when network quality is poor, voice recognition is slow and accurate, resulting in a poor user experience. Summary of the Invention

[0003] This invention provides a voice interaction method, apparatus, vehicle, and storage medium thereof.

[0004] This invention provides a voice interaction method. The voice interaction method includes: when the network connection of a vehicle is abnormal, performing real-time voice recognition on each frame of user voice data collected within the vehicle to obtain a real-time voice request; matching the real-time voice request with a voice request library to obtain a preset voice request, the preset voice request including first content and second content; if the identified real-time voice request is identical to the first content, and if the identified user voice data within the preset frame satisfies a preset condition corresponding to the second content, then determining the voice command corresponding to the preset voice request as a first target voice command; and completing the voice interaction based on the first target voice command.

[0005] Thus, in the event of network anomalies, the voice interaction method of the present invention can collect user voice data in real time, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to the voice request library, thereby achieving rapid response and high accuracy of voice recognition.

[0006] The step of determining the voice instruction corresponding to the preset voice request as the first target voice instruction when the identified real-time voice request is the same as the first content includes: when the identified real-time voice request is the same as the first content, if the identified user voice data in the preset frame includes the voice data corresponding to the second content and the confidence level is greater than the confidence level threshold, then the voice instruction corresponding to the preset voice request is determined as the first target voice instruction.

[0007] In this way, if the real-time voice request that has been recognized is the same as the first content, the user can directly determine the voice command corresponding to the preset voice request as the first target voice command based on the user voice data in the preset frame. This enables a fast response of voice recognition even if the user has not completely input the entire voice.

[0008] The step of performing real-time speech recognition on each frame of user speech data collected in the vehicle to obtain a real-time speech request includes: extracting speech features of each frame of user speech data in the vehicle in real time; obtaining an embedding representation and acoustic state probability distribution in real time based on the speech features; and obtaining the real-time speech request based on the embedding representation and the acoustic state probability distribution.

[0009] In this way, the speech features of each frame of user speech data are extracted in real time inside the vehicle, and the embedding representation and acoustic state probability distribution are obtained in real time based on the speech features. Real-time speech requests are obtained based on the embedding representation and acoustic state probability distribution, thus realizing real-time response of speech recognition.

[0010] The voice interaction method includes: when a template library is set in the vehicle, determining the embedded representation template corresponding to the real-time voice request; matching the embedded representation template in the template library to obtain a target embedded representation template; determining the voice command corresponding to the target embedded representation template as a second target voice command; and completing the voice interaction according to the second target voice command.

[0011] Thus, by introducing an embedded representation template, matching the target embedded representation template in the template library based on the embedded representation template, determining the voice command corresponding to the target embedded representation template as the second target voice command, and completing the voice interaction based on the second target voice command, the present invention improves the recognition accuracy of voice commands.

[0012] When a template library is provided in the vehicle, determining the embedded representation template corresponding to the real-time voice request includes: determining the embedded representation template corresponding to the real-time voice request when the identified real-time voice request is the same as the first content.

[0013] Thus, when the recognized real-time voice request is the same as the first content, that is, when the current voice command is the same as the first content of a certain command in the pre-set preset voice request, the embedded representation template corresponding to the current voice request is determined in real time and the embedded representation template in the template library is matched. It is not necessary to determine the embedded representation template for the recognized voice request frame by frame, which further improves the efficiency of voice recognition.

[0014] The step of obtaining a target embedding representation template by matching the embedding representation template in the template library includes: matching the embedding representation template with a preset embedding representation module in the template library using a matching algorithm; and determining the preset embedding representation template whose matching result meets the matching threshold as the target embedding representation template.

[0015] Thus, by matching the embedded representation template with the preset embedded representation modules in the template library according to the matching algorithm, the preset embedded representation template that meets the matching threshold can be used as the target embedded representation template, which can improve the efficiency of template matching.

[0016] The interaction method includes: if the target embedded representation template cannot be matched in the template library according to the embedded representation template, then after completing the voice interaction according to the first target voice command, the embedded representation template corresponding to the first target voice command is added to the template library as the preset embedded representation template.

[0017] Thus, the present invention can automatically add new preset embedded representation templates to the template library to update the template library in real time, thereby achieving a fast response in speech recognition.

[0018] The step of adding the embedding representation template corresponding to the first target speech instruction as the preset embedding representation template to the template library includes: obtaining the duration of each recognition state corresponding to the embedding representation; and averaging the embedding representations corresponding to the first target speech instruction in each recognition state according to the duration of the recognition state to obtain the preset embedding representation template with a preset length.

[0019] Thus, the present invention can form multiple preset embedded representation templates of equal length, which can speed up template matching and improve speech recognition efficiency.

[0020] The step of adding the embedding representation template corresponding to the first target voice instruction as the preset embedding representation template to the template library includes: counting the usage frequency of the first target voice instruction; and when the usage frequency of the first target voice instruction is greater than the preset frequency, adding the embedding representation template corresponding to the first target voice instruction as the preset embedding representation template to the template library.

[0021] Thus, for high-frequency voice commands, when recognizing the voice command, template matching can be performed based on the preset embedded representation templates in the template library, thereby improving the efficiency of voice recognition on the basis of improving the accuracy of voice command recognition.

[0022] The present invention also provides a voice interaction device. The voice interaction device includes a data acquisition module, a matching module, an instruction determination module, and a voice interaction module. The data acquisition module is used to perform real-time voice recognition on each frame of user voice data acquired in real time within the vehicle to obtain a real-time voice request when the vehicle's network connection is abnormal. The matching module is used to match the real-time voice request with a voice request library to obtain a preset voice request, the preset voice request including a first content and a second content. The instruction determination module is used to determine the voice instruction corresponding to the preset voice request as a first target voice instruction if, when the identified real-time voice request is the same as the first content, the user voice data within the preset frame satisfies a preset condition corresponding to the second content. The voice interaction module is used to complete the voice interaction according to the first target voice instruction.

[0023] In the event of network anomalies, the voice interaction device of the present invention can collect each frame of user voice data in real time, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to the voice request library, thereby achieving rapid response and high accuracy of voice recognition.

[0024] The present invention also provides a vehicle. The vehicle includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the voice interaction method described in any of the above embodiments.

[0025] In the event of network anomalies, the vehicle of this invention can collect user voice data in real time for each frame, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to a voice request library, thereby achieving rapid response and high accuracy in voice recognition.

[0026] The present invention also provides a non-volatile computer-readable storage medium for a computer program. When the computer program is executed by one or more processors, it implements the voice interaction method described in any of the above embodiments.

[0027] In the event of network anomalies, the storage medium of this invention can collect user voice data in real time for each frame, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to a voice request library, thereby achieving rapid response and high accuracy in voice recognition.

[0028] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0030] Figure 1 This is a flowchart illustrating the voice interaction method of the present invention;

[0031] Figure 2 This is a schematic diagram of the voice interaction device of the present invention;

[0032] Figure 3 This is a flowchart illustrating the voice interaction method of the present invention;

[0033] Figure 4 This is a flowchart illustrating the voice interaction method of the present invention;

[0034] Figure 5 This is a flowchart illustrating the voice interaction method of the present invention;

[0035] Figure 6 This is a schematic diagram of the voice interaction device of the present invention;

[0036] Figure 7 This is a schematic diagram of a scenario for the voice interaction method of the present invention;

[0037] Figure 8 This is a flowchart illustrating the voice interaction method of the present invention;

[0038] Figure 9 This is a flowchart illustrating the voice interaction method of the present invention;

[0039] Figure 10 This is a flowchart illustrating the voice interaction method of the present invention;

[0040] Figure 11 This is a flowchart illustrating the voice interaction method of the present invention;

[0041] Figure 12 This is a flowchart illustrating the voice interaction method of the present invention;

[0042] Figure 13 This is a structural schematic diagram of the vehicle of the present invention;

[0043] Figure 14 This is a schematic diagram of the structure of the computer-readable storage medium of the present invention. Detailed Implementation

[0044] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present invention, and should not be construed as limiting the embodiments of the present invention.

[0045] Please see Figure 1This invention provides a voice interaction method. The voice interaction method includes:

[0046] 01: When the vehicle's network connection is abnormal, real-time voice requests are obtained by performing voice recognition on each frame of user voice data collected in the vehicle.

[0047] 03: Based on the real-time voice request, a preset voice request is obtained by matching it in the voice request library. The preset voice request includes the first content and the second content.

[0048] 05: If the identified real-time voice request is the same as the first content, and if the user voice data in the preset frame is found to meet the preset conditions corresponding to the second content, then the voice command corresponding to the preset voice request is determined to be the first target voice command.

[0049] 07: Complete the voice interaction according to the first target voice command.

[0050] Please see Figure 2 The present invention also provides a voice interaction device 10. The voice interaction device 10 includes: a data acquisition module 11, a request matching module 13, a first instruction determination module 15, and a voice interaction module 17.

[0051] Step 01 can be implemented by the acquisition module 11, step 03 by the request matching module 13, step 05 by the first instruction determination module 15, and step 07 by the voice interaction module 17. That is, the acquisition module 11 is used to perform real-time voice recognition on each frame of user voice data acquired in real time within the vehicle to obtain a real-time voice request when the vehicle's network connection is abnormal; the request matching module 13 is used to match a preset voice request in the voice request library based on the real-time voice request, the preset voice request including a first content and a second content; the first instruction determination module 15 is used to determine the voice instruction corresponding to the preset voice request as the first target voice instruction if the identified real-time voice request is the same as the first content and the identified user voice data within the preset frame meets the preset conditions corresponding to the second content; the voice interaction module 17 is used to complete the voice interaction based on the first target voice instruction.

[0052] The situation where the vehicle's network connection is in an abnormal state refers to the situation where the vehicle's network is offline or the network condition is poor.

[0053] In the event of a vehicle network malfunction, the vehicle performs real-time speech recognition on each frame of user voice data collected to obtain real-time voice requests. For example, the starting point of a voice segment can be obtained from the real-time user voice data stream using the Voice Activity Detection (VAD) module, and then fed into the offline Automatic Speech Recognition (ASR) module for offline speech recognition. The offline ASR module can act as an offline speech recognition server, thereby enabling rapid speech recognition by the vehicle in offline conditions.

[0054] The preset voice request includes a first part and a second part. That is, the preset voice request of the present invention can be divided into two parts: the first part is the prefix of the preset voice request, and the second part is the suffix of the preset voice request. Specifically, the second part can be the last word of the preset voice request. For example, if the preset voice request set in the voice request library is "Air conditioner set to 21 degrees", the first part can be "Air conditioner set to 21", and the second part can be "degree".

[0055] When the currently recognized real-time voice request has the same first content as the preset voice request and both are "set the air conditioner to 21 degrees", it is not necessary to wait for the word "degree" to be completely spoken and for the result to be deciphered. As long as the preset condition corresponding to the second content "degree" is met, the voice command corresponding to the preset voice request, "set the air conditioner to 21 degrees", can be directly determined as the first target voice command. The voice command can be executed directly according to the first target voice command without waiting for the user's voice to be completely parsed or for semantic analysis based on the Natural Language Understanding (NLU) module. This greatly reduces the execution delay of the command, and the command can be executed as soon as the voice is spoken, thus achieving a fast response of voice recognition.

[0056] Thus, in the event of network anomalies, the voice interaction method of the present invention can collect user voice data in real time, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to the voice request library, thereby achieving rapid response and high accuracy of voice recognition.

[0057] Please see Figure 3 Step 05 includes:

[0058] 051: If the identified real-time voice request is the same as the first content, and if the identified user voice data in the preset frame includes the voice data corresponding to the second content, and the confidence level is greater than the confidence level threshold, then the voice command corresponding to the preset voice request is determined to be the first target voice command.

[0059] Please see Figure 2 Step 051 can be implemented by the first instruction determination module 15. That is, the first instruction determination module 15 is used to determine the voice instruction corresponding to the preset voice request as the first target voice instruction if the user voice data in the preset frame includes the voice data corresponding to the second content and the confidence level is greater than the confidence level threshold when the recognized real-time voice request is the same as the first content.

[0060] Continuing with the example mentioned above, the preset condition can be that after the user says "set the air conditioner to 21 degrees", the first instruction determination module 15 identifies the user's voice data within a preset frame, including the voice data corresponding to the second content "degree". The preset frame is the number of frames for voice recognition within a preset time. For example, if the preset frame is 5 frames, at least 3 frames containing the second content "degree" are identified within 5 frames, and the confidence of the 3 frames containing the second content "degree" is greater than a set threshold. The set threshold is, for example, 90%. Then, the voice instruction corresponding to the preset voice request, "set the air conditioner to 21 degrees", can be directly determined as the first target voice instruction.

[0061] In this way, if the recognized real-time voice request is the same as the first content, the user can directly determine the voice command corresponding to the preset voice request as the first target voice command, even if the user has not completely input the entire voice message.

[0062] Please see Figure 4 Step 01 includes:

[0063] 011: Extract speech features from each frame of user speech data in real time inside the vehicle;

[0064] 013: Real-time acquisition of embedded representations and acoustic state probability distributions based on speech features;

[0065] 015: Real-time voice requests are obtained based on embedded representations and acoustic state probability distributions.

[0066] Please see Figure 2 Steps 011, 013, and 015 can be implemented by the acquisition module 11. That is, the acquisition module 11 is used to extract the speech features of each frame of user speech data in real time inside the vehicle; to obtain the embedding representation and acoustic state probability distribution in real time based on the speech features; and to obtain the real-time speech request based on the embedding representation and acoustic state probability distribution.

[0067] First, the starting point of the speech segment is obtained from the real-time user speech data by the VAD (Voice Activity Detection) module, and then input into the offline ASR (Automatic Speech Recognition) module in real time. The offline ASR module then begins to extract speech features frame by frame, including FBank features. That is, at this point, each frame of the user speech data is extracted in real time; as soon as the user speaks a sentence, the speech features of each frame of speech data are obtained in real time.

[0068] Then, based on the real-time acquired speech features, the embedding and acoustic state probability distribution can be obtained in real time. That is, the speech features are fed into the acoustic model in real time, thereby obtaining the acoustic model embedding and acoustic state probability distribution in real time.

[0069] Finally, the decoder obtains the current optimal speech path in real time based on the acoustic state probability distribution and language model. Based on this optimal speech path, a real-time speech request can be obtained. Specifically, if the current optimal speech path is a prefix of a pre-defined locally executable speech command, and only the last character remains to be decoded (e.g., the pre-defined locally executable speech command contains "set the air conditioner to 21 degrees"), and the currently identified optimal speech path frame by frame is "set the air conditioner to 21 degrees"), then it is not necessary to wait for the complete word "degree" to be spoken and for the decoding result to be obtained. It only needs to identify at least, for example, three more speech frames containing "degree" with a confidence level greater than a set threshold. The speech command can then be directly sent to the command execution module for execution, without waiting for complete speech decoding and semantic analysis by the NLU module. This greatly reduces the execution delay of the command, allowing execution to begin immediately after the speech is spoken.

[0070] In this way, the speech features of each frame of user speech data are extracted in real time inside the vehicle, and the embedding representation and acoustic state probability distribution are obtained in real time based on the speech features. Real-time speech requests are obtained based on the embedding representation and acoustic state probability distribution, thus realizing real-time response of speech recognition.

[0071] Please see Figure 5 Voice interaction methods include:

[0072] 02: If a template library is set up in the vehicle, determine the embedded representation template corresponding to the real-time voice request;

[0073] 04: Obtain the target embedding representation template by matching it in the template library based on the embedding representation template;

[0074] 06: Determine that the speech instruction corresponding to the target embedded representation template is the second target speech instruction;

[0075] 08: Complete the voice interaction according to the second target voice command.

[0076] Please combine Figure 6 The voice interaction device 10 includes a template determination module 12, a template matching template 14, and a second instruction determination module 16.

[0077] Please combine Figure 2 and Figure 6 Step 02 can be implemented by the template determination module 12, step 04 can be implemented by the template matching module 14, step 06 can be implemented by the second instruction determination module 16, and step 08 can be implemented by the voice interaction module 17. That is, the template determination module 12 is used to determine the embedded representation template corresponding to the real-time voice request when a template library is set up in the vehicle; the template matching module 14 is used to match the embedded representation template in the template library to obtain the target embedded representation template; the second instruction determination module 16 is used to determine that the voice instruction corresponding to the target embedded representation template is the second target voice instruction; and the voice interaction module 17 is used to complete the voice interaction according to the second target voice instruction.

[0078] Please see Figure 7 When a template library is installed in the vehicle, the corresponding embedded representation template can be determined in real time according to the voice request, and the target embedded representation template can be obtained by matching the embedded representation template in the template library.

[0079] In other words, this invention can update the embedding representation template of the current voice request in real time as the acoustic model outputs the voice request, and then match the updated embedding representation template with templates in the template library to obtain the target embedding representation template. For example, when the user's voice request outputs the embedding representation corresponding to "air conditioner" after passing through the acoustic model, the output embedding representation can be added to the template in real time. At this time, if there is no matching template in the template library, the voice request recognition can continue. When the user's voice request outputs the embedding representation corresponding to "air conditioner set to 20" after passing through the acoustic model, the output embedding representation can be added to the template in real time, updating the current voice request embedding representation template. At this time, if there is still no matching template in the template library, the voice request recognition can continue. When the user's voice request outputs the embedding representation corresponding to "air conditioner set to 21" after passing through the acoustic model, the output embedding representation can be added to the template in real time, updating the current voice request embedding representation template. At this time, if there is a matching template in the template library corresponding to "air conditioner set to 21", then the template corresponding to "air conditioner set to 21" is determined as the target embedding representation template.

[0080] Thus, by introducing an embedded representation template, matching the target embedded representation template in the template library based on the embedded representation template, determining the voice command corresponding to the target embedded representation template as the second target voice command, and completing the voice interaction based on the second target voice command, the present invention improves the recognition accuracy of voice commands.

[0081] Please see Figure 8 Step 02 includes:

[0082] 021: If the identified real-time voice request is the same as the first content, determine the embedded representation template corresponding to the real-time voice request.

[0083] Please see Figure 6 Step 021 can be implemented by the template determination module 12. That is, the template determination module 12 is used to determine the embedded representation template corresponding to the real-time voice request when the identified real-time voice request is the same as the first content.

[0084] As mentioned above, a preset voice request is obtained by matching the real-time voice request with the voice request library. The preset voice request includes a first content and a second content. For example, if the preset voice request is "set the air conditioner to 21 degrees", then the first content can be "set the air conditioner to 21" and the second content can be "degree".

[0085] Thus, when the recognized real-time voice request is the same as the first content, that is, when the current voice command is the same as the first content of a certain command in the pre-set preset voice request, the embedded representation template corresponding to the current voice request is determined in real time and matched with the template in the template library. It is not necessary to determine the embedded representation template for the recognized voice request frame by frame, which further improves the efficiency of voice recognition.

[0086] Please see Figure 9 Step 04 includes:

[0087] 041: Match the embedded representation template with the preset embedded representation modules in the template library using a matching algorithm;

[0088] 042: The preset embedding representation template whose matching result meets the matching threshold is determined as the target embedding representation template.

[0089] Please combine Figure 6 Steps 041 and 042 can be implemented by template matching template 14. That is, template matching template 14 is used to match the embedded representation template with the preset embedded representation modules in the template library using a matching algorithm; and the preset embedded representation template whose matching result meets the matching threshold is determined as the target embedded representation template.

[0090] Matching algorithms for embedded representation templates include Dynamic Time Warping (DTW) and its improved versions, as well as similarity algorithms such as cosine distance. DTW and its improved versions are suitable for variable-length templates, while similarity algorithms such as cosine distance are suitable for fixed-length templates. A variable-length template refers to a template where the duration of the recognition state corresponding to each embedded representation template is inconsistent, with some templates having longer recognition states. The embedding feature for a variable-length template is the direct output of the penultimate layer of the acoustic model, forming a variable-length embedded representation template. A fixed-length template refers to a template where the duration of the recognition state corresponding to each embedded representation template is consistent.

[0091] Thus, by matching the embedded representation template with the preset embedded representation modules in the template library according to the matching algorithm, the preset embedded representation template that meets the matching threshold can be used as the target embedded representation template, which can improve the efficiency of template matching.

[0092] Please see Figure 10 The interaction methods include:

[0093] 043: If no target embedded representation template can be matched in the template library according to the embedded representation template, then after the voice interaction is completed according to the first target voice command, the embedded representation template corresponding to the first target voice command is added to the template library as a preset embedded representation template.

[0094] Please combine Figure 6 Step 043 can be implemented by template matching template 14. That is, template matching template 14 is used to add the embedded representation template corresponding to the first target voice command as a preset embedded representation template to the template library if the target embedded representation template cannot be matched in the template library according to the embedded representation template.

[0095] When no target embedded representation template can be matched in the template library based on the embedded representation template, after the voice interaction is completed according to the first target voice command, the embedded representation template corresponding to the first target voice command is directly added to the template library as a preset embedded representation template for storage.

[0096] Thus, the present invention can automatically add new preset embedded representation templates to the template library to update the template library in real time, thereby achieving a fast response in speech recognition.

[0097] Please see Figure 11 Step 043 includes:

[0098] 0431: Obtain the duration of each recognition state corresponding to the embedded representation;

[0099] 0432: Based on the duration of the recognition state, the embedded representations corresponding to the first target speech command in each recognition state are averaged to obtain a preset embedded representation template with a preset length.

[0100] Please see Figure 6 Steps 0431 and 0432 can be implemented by template matching template 14. That is, template matching template 14 is used to obtain the duration of each recognition state corresponding to the embedded representation; according to the duration of the recognition state, the embedded representation corresponding to the first target speech instruction in each recognition state is averaged to obtain a preset embedded representation template with a preset length.

[0101] like Figure 7 As shown, the duration of each recognition state corresponding to the embedded representation can be obtained. The embedded representation corresponding to the first target speech command in each recognition state is averaged to become a preset embedded representation template with a preset length, and this template is placed into the template library as a reference template. According to the dwell time of different states, the original acoustic features corresponding to the dwell time of the corresponding state are averaged to obtain equal-length templates for the same speech command but with different durations, thus solving the problem of variable-length templates caused by different speech speeds.

[0102] Thus, the present invention can form multiple preset embedded representation templates of equal length, which can speed up template matching and improve speech recognition efficiency.

[0103] Please see Figure 12 Step 043 includes:

[0104] 0433: Statistical analysis of the frequency of use of the first target voice command;

[0105] 0434: When the frequency of use of the first target voice command is greater than the preset frequency, the embedded representation template corresponding to the first target voice command is added to the template library as a preset embedded representation template.

[0106] Please see Figure 6 Steps 0433 and 0434 can be implemented by template matching template 14. That is, template matching template 14 is used to count the usage frequency of the first target speech instruction; when the usage frequency of the first target speech instruction is greater than the preset frequency, the embedding representation template corresponding to the first target speech instruction is added to the template library as the preset embedding representation template.

[0107] Understandably, if the first target voice command is successfully executed after speech recognition by the offline ASR module, the frequency of use of the first target voice command will be counted on the executing electronic device. When the frequency of use is greater than a preset frequency, it is regarded as a high-frequency voice command, and finally a set of high-frequency voice commands is formed. Each high-frequency voice command corresponds to an embedding representation template, and the embedding representation template corresponding to each high-frequency voice command is added to the template library as a preset embedding representation template.

[0108] Among them, high-frequency voice commands refer to voice commands that users use frequently. For example, if a user's voice command "set the air conditioner to 24 degrees" is used multiple times in a certain period of time, then "set the air conditioner to 24 degrees" can be considered a high-frequency voice command.

[0109] Thus, for high-frequency voice commands, when recognizing the voice command, template matching can be performed based on the preset embedded representation templates in the template library, thereby improving the efficiency of voice recognition on the basis of improving the accuracy of voice command recognition.

[0110] It should be noted that whether the voice command corresponding to the preset voice request is determined by the scheme of determining the preset voice command based on the user voice data in the preset frame meeting the preset conditions corresponding to the second content, or by determining the voice command corresponding to the preset voice request based on the preset embedded representation template in the template library, depends on the specific processing speed of the two schemes. The voice command determined by the scheme with faster processing speed will be given priority.

[0111] Please see Figure 13 The present invention also provides a vehicle 20. The vehicle 20 includes a processor 21 and a memory 22. The memory 22 stores a computer program 221. When the computer program 221 is executed by the processor 21, it implements the voice interaction method in any of the above embodiments.

[0112] The vehicle 20 of the present invention can collect each frame of user voice data in real time in the event of network anomalies to obtain real-time voice requests and obtain preset voice requests by matching according to the voice request library, thereby achieving fast response and high accuracy of voice recognition.

[0113] Please see Figure 14 The present invention also provides a non-volatile computer-readable storage medium 30 for computer programs. When the computer program 31 is executed by one or more processors 40, it implements the voice interaction method of any of the above embodiments.

[0114] For example, when computer program 31 is executed by processor 40, it implements the following steps of the voice interaction method:

[0115] 01: When the vehicle's network connection is abnormal, real-time voice requests are obtained by performing voice recognition on each frame of user voice data collected in the vehicle.

[0116] 03: Based on the real-time voice request and the voice request library, a preset voice request is obtained. The preset voice request includes a first content and a second content.

[0117] 05: If the identified real-time voice request is the same as the first content, and if the user voice data in the preset frame is found to meet the preset conditions corresponding to the second content, then the voice command corresponding to the preset voice request is determined to be the first target voice command.

[0118] 07: Complete the voice interaction according to the first target voice command.

[0119] It is understood that a computer program 31 includes computer program code. Computer program code can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0120] The computer-readable storage medium 30 of the present invention can collect each frame of user voice data in real time under network abnormality, obtain real-time voice requests through voice recognition, and obtain preset voice requests by matching according to the voice request library, thereby achieving fast response and high accuracy of voice recognition.

Claims

1. A voice interaction method, characterized in that, include: In the event that the vehicle’s network connection is in an abnormal state, the system performs real-time voice recognition on each frame of user voice data collected inside the vehicle to obtain real-time voice requests. A preset voice request is obtained by matching the real-time voice request in the voice request library. The preset voice request includes a first content and a second content, where the first content is the prefix of the preset voice request and the second content is the suffix of the preset voice request. If the real-time voice request identified is the same as the first content, and if the user voice data in the preset frame is identified to include the voice data corresponding to the second content, and the confidence level of the voice data corresponding to the second content in the user voice data is greater than the confidence level threshold, then the voice command corresponding to the preset voice request is determined to be the first target voice command. If a template library is provided in the vehicle, determine the embedded representation template corresponding to the real-time voice request; When a target embedding representation template is obtained by matching the embedding representation template in the template library, the speech instruction corresponding to the target embedding representation template is determined to be the second target speech instruction; Complete the voice interaction based on the first target voice command or the second target voice command.

2. The voice interaction method according to claim 1, characterized in that, The process of obtaining real-time voice requests by performing speech recognition on each frame of user voice data collected in real time within the vehicle includes: The speech features of each frame of user speech data are extracted in real time inside the vehicle. Based on the aforementioned speech features, the embedding representation and acoustic state probability distribution are obtained in real time. The real-time voice request is obtained based on the embedded representation and the acoustic state probability distribution.

3. The voice interaction method according to claim 2, characterized in that, The voice interaction method includes: If a template library is provided in the vehicle, determine the embedded representation template corresponding to the real-time voice request; The target embedding representation template is obtained by matching the embedding representation template in the template library; The speech instruction corresponding to the target embedded representation template is determined to be the second target speech instruction; Complete the voice interaction based on the second target voice command.

4. The voice interaction method according to claim 3, characterized in that, When a template library is provided in the vehicle, determining the embedded representation template corresponding to the real-time voice request includes: If the identified real-time voice request is the same as the first content, the embedded representation template corresponding to the real-time voice request is determined.

5. The voice interaction method according to claim 3, characterized in that, The step of matching the target embedding representation template in the template library according to the embedding representation template includes: The embedded representation template is matched with a preset embedded representation template in the template library using a matching algorithm; The preset embedding representation template whose matching result meets the matching threshold is determined as the target embedding representation template.

6. The voice interaction method according to claim 3, characterized in that, The interaction method includes: If the target embedded representation template cannot be matched in the template library according to the embedded representation template, then after the voice interaction is completed according to the first target voice command, the embedded representation template corresponding to the first target voice command is added to the template library as a preset embedded representation template.

7. The voice interaction method according to claim 6, characterized in that, The step of adding the embedding representation template corresponding to the first target speech instruction as the preset embedding representation template to the template library includes: Obtain the duration of each recognition state corresponding to the embedded representation; Based on the duration of the recognition state, the embedded representations corresponding to the first target speech instruction in each recognition state are averaged to obtain the preset embedded representation template with a preset length.

8. The voice interaction method according to claim 6, characterized in that, The step of adding the embedding representation template corresponding to the first target speech instruction as the preset embedding representation template to the template library includes: Statistical analysis of the frequency of use of the first target voice command; If the frequency of use of the first target voice instruction is greater than the preset frequency, the embedded representation template corresponding to the first target voice instruction is added to the template library as the preset embedded representation template.

9. A voice interaction device, characterized in that, include: The acquisition module is used to perform real-time voice recognition on each frame of user voice data acquired in the vehicle in real time to obtain real-time voice requests when the network connection of the vehicle is in an abnormal state. A matching module is used to obtain a preset voice request by matching the real-time voice request with a voice request library. The preset voice request includes a first content and a second content, wherein the first content is a prefix of the preset voice request and the second content is a suffix of the preset voice request. The instruction determination module is used to determine the voice instruction corresponding to the preset voice request as the first target voice instruction when the identified real-time voice request is the same as the first content. If the identified user voice data in the preset frame includes the voice data corresponding to the second content, and the confidence level of the voice data corresponding to the second content in the user voice data is greater than the confidence level threshold, then the instruction determination module is used to determine the voice instruction corresponding to the preset voice request as the first target voice instruction. A template determination module is used to determine the embedded representation template corresponding to the real-time voice request when a template library is set in the vehicle. The second instruction determination module is used to determine the speech instruction corresponding to the target embedding representation template as the second target speech instruction when a target embedding representation template is obtained by matching the target embedding representation template in the template library according to the embedding representation template. A voice interaction module is used to complete voice interaction based on the first target voice command or the second target voice command.

10. A vehicle, characterized in that, The vehicle includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the voice interaction method according to any one of claims 1-8.

11. A non-volatile computer-readable storage medium containing a computer program, characterized in that, When the computer program is executed by one or more processors, it implements the voice interaction method according to any one of claims 1-8.

Images