Voice interaction method, system, device, equipment and storage medium

Abstract

The embodiments of the present application provide a voice interaction method, system, apparatus, device and storage medium. In the voice interaction system, the terminal device can obtain the user's voice data and send the voice data to the cloud server. The cloud server can perform voice recognition on the voice data through the voice recognition engine corresponding to the first language type, and determine whether the voice data matches the voice recognition engine based on the voice recognition result. If there is no match, a voice recognition engine that matches the voice data is selected from the backup voice recognition engines, voice recognition is performed again, and a reply message is generated based on the voice recognition result. Through this embodiment, when the language type used by the user is different, the terminal device can more accurately perform voice recognition on the voice data input by the user, and then provide the user with a reply that is more closely matched with the voice information.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent robot technology, and in particular to a voice interaction method, system, device, equipment and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, intelligent dialogue is becoming more and more popular. In scenarios such as shopping malls, supermarkets, and restaurants, intelligent devices (such as robots) that can conduct intelligent dialogues are widely used. In the prior art, usually, under the premise of knowing the user's language type, an ASR (Automatic Speech Recognition) engine corresponding to the user's language type is manually set in advance to recognize the voice information input by the user, convert the voice information into text information, recognize the text information and respond based on the recognition result. However, in many usage scenarios where the user's language type is unknown, the user's language type cannot be known in advance, and thus the ASR engine cannot be set before interacting with the user. Furthermore, it is impossible to accurately perform voice recognition on the voice information input by the user, resulting in the inability to provide the user with a response that is more consistent with the voice information. Therefore, a solution is urgently needed. Summary of the Invention

[0003] The embodiments of the present application provide a voice interaction method, system, apparatus, device and storage medium for more accurately performing voice recognition on voice data input by a user, thereby providing the user with a response that more closely matches the voice information.

[0004] An embodiment of the present application provides a voice interaction method, including: obtaining voice data sent by a user to a device; performing voice recognition on the voice data through a first voice recognition engine corresponding to a preset first language type to obtain a first voice recognition result; judging, based on the first voice recognition result, whether the voice data matches the first voice recognition engine; if not, selecting a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; and generating reply information for the voice data based on a second voice recognition result of the target voice recognition engine on the voice data.

[0005] Further optionally, based on the first speech recognition result, determining whether the speech data matches the first speech recognition engine includes: obtaining text information in the first speech recognition result; calculating the recognition accuracy of the text information; if the recognition accuracy is less than a set accuracy threshold, determining that the speech data does not match the first speech recognition engine.

[0006] Further optionally, the method also includes: if the recognition accuracy is greater than or equal to the set accuracy threshold, using a question-answer matching model to perform question-answer matching on the text information to obtain reply information and the confidence of the reply information; if the confidence of the reply information is less than the preset confidence threshold, it is determined that the voice data does not match the first voice recognition engine.

[0007] Further optionally, selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine includes: selecting any speech recognition engine from the at least one backup speech recognition engine as a second speech recognition engine; performing speech recognition on the speech data through the second speech recognition engine to obtain a second speech recognition result; judging whether the speech data matches the second speech recognition engine based on the second speech recognition result; if the speech data matches the second speech recognition engine, using the second speech recognition engine as the target speech recognition engine.

[0008] Further optionally, the at least one backup speech recognition engine is sorted according to a set priority order; and any one speech recognition engine is selected from the at least one backup speech recognition engine as the second speech recognition engine, including: selecting the at least one backup speech recognition engine in turn according to the sorting of the at least one backup speech recognition engine to obtain the second speech recognition engine.

[0009] Further optionally, before selecting a target voice recognition engine that matches the voice data from at least one backup voice recognition engine, the method further includes: sorting the at least one backup voice recognition engine according to the current region of the device; or sorting the at least one backup voice recognition engine according to the historical usage frequency of the at least one voice recognition engine of the device.

[0010] An embodiment of the present application also provides a voice interaction system, comprising: a terminal device and a cloud server; wherein the terminal device is mainly used to: obtain voice data sent by a user to the terminal device; send the voice data to the cloud server; the cloud server is mainly used to: receive the voice data; perform voice recognition on the voice data through a first voice recognition engine corresponding to a preset first language type to obtain a first voice recognition result; based on the first voice recognition result, determine whether the voice data matches the first voice recognition engine; if not, select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; and generate reply information for the voice data based on a second voice recognition result of the target voice recognition engine on the voice data.

[0011] An embodiment of the present application also provides a voice interaction device, including: an acquisition module, used to: acquire voice data sent by a user to a device; a recognition module, used to: perform voice recognition on the voice data through a first voice recognition engine corresponding to a preset first language type to obtain a first voice recognition result; a judgment module, used to: judge whether the voice data matches the first voice recognition engine based on the first voice recognition result; a selection module, used to: if not, select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; a generation module, used to generate reply information of the voice data based on a second voice recognition result of the target voice recognition engine on the voice data.

[0012] An embodiment of the present application also provides a cloud server, comprising: a memory, a processor, and a communication component; wherein the memory is used to: store one or more computer instructions; the processor is used to execute the one or more computer instructions, so as to: execute the steps in the voice interaction method.

[0013] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the voice interaction method.

[0014] The embodiments of the present application provide a voice interaction method, system, apparatus, device and storage medium, in which the terminal device can obtain the voice data sent by the user to the terminal device and send the voice data to the cloud server. After receiving the voice data, the cloud server can perform voice recognition on the voice data through the first voice recognition engine corresponding to the first language type to obtain a first voice recognition result, and based on the first voice recognition result, determine whether the voice data matches the first voice recognition engine. If there is no match, a target voice recognition engine that matches the voice data is selected from the backup voice recognition engines, and a reply message for the voice data is generated based on the second voice recognition result of the target voice recognition engine for the voice data. Through this embodiment, when the language types used by the user are different, the terminal device can more accurately perform voice recognition on the voice data input by the user, and thus can provide the user with a reply that is more closely matched with the voice information. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A schematic diagram of the structure of a voice interaction system provided by an exemplary embodiment of the present application;

[0017] Figure 2 A schematic diagram of the structure of a voice interaction system in an actual scenario provided by an exemplary embodiment of the present application;

[0018] Figure 3 A schematic structural diagram of a voice interaction system in an actual scenario provided by another exemplary embodiment of the present application;

[0019] Figure 4 A flowchart of a voice interaction method provided by an exemplary embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of a voice interaction device provided by an exemplary embodiment of the present application;

[0021] Figure 6 A schematic diagram of a cloud server provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] In the prior art, when users use different languages, robots cannot accurately recognize the user's voice input, resulting in an inability to provide the user with a response that closely matches the voice information. To address this technical problem, some embodiments of the present application provide a solution. The following, combined with the accompanying drawings, details the technical solutions provided by various embodiments of the present application.

[0024] Figure 1 A schematic diagram of the structure of a voice interaction system provided by an exemplary embodiment of the present application is shown in FIG. Figure 1 As shown, the voice interaction system 100 includes: a cloud server 10 and a terminal device 20.

[0025] The cloud server 10 can be implemented as a cloud host, a cloud virtual center, a cloud elastic computing instance, etc., and this embodiment does not limit this. The cloud server 10 mainly includes a processor, a hard disk, a memory, a system bus, etc., which is similar to a general computer architecture and will not be described in detail.

[0026] The terminal device 20 can be implemented as a variety of terminal devices in different scenarios. For example, in scenarios such as hotels, guesthouses, and restaurants, it can be implemented as a service-providing robot; in scenarios such as intelligent driving assistance or autonomous driving, it can be implemented as a controlled vehicle; in scenarios such as banks, it can be implemented as a multi-functional financial terminal; in scenarios such as hospitals, it can be implemented as a registration and payment terminal; in scenarios such as movie theaters, it can be implemented as a ticket collection terminal, and so on.

[0027] In the voice interaction system 100, a wireless communication connection can be established between the cloud server 10 and the terminal device 20. The specific communication connection method can be determined according to different application scenarios. In some embodiments, the wireless communication connection can be implemented based on a dedicated virtual network (VPN) to ensure communication security.

[0028] In the voice interaction system 100 , the terminal device 20 is mainly used to obtain voice data sent by the user to the terminal device 20 and send the voice data to the cloud server 10 .

[0029] Accordingly, the cloud server 10 is primarily configured to receive the voice data and perform voice recognition on the voice data using a first voice recognition engine corresponding to a preset first language type, thereby obtaining a first voice recognition result. The term "first" is used to define the voice recognition engine and the voice recognition result, and is only used to distinguish between the two. The voice recognition result may include text information. For example, the cloud server 10 may perform voice recognition on the user's voice data and obtain a first voice recognition result containing the text information "Where is the nearest movie theater to me?"

[0030] After performing speech recognition, the cloud server 10 can determine whether the speech data matches the first speech recognition engine based on the first speech recognition result. If not, a target speech recognition engine that matches the speech data is selected from at least one backup speech recognition engine.

[0031] The target speech recognition engine refers to the speech recognition engine that matches the speech data. At least one backup speech recognition engine corresponds to at least one language type. For example, the at least one backup speech recognition engine includes: a speech recognition engine for Chinese, a speech recognition engine for Arabic, and a speech recognition engine for an Indic language. If the user is speaking Arabic, and the speech data is determined not to match the first speech recognition engine based on the first speech recognition result, a speech recognition engine for Arabic that matches the speech data can be selected from the multiple backup speech recognition engines.

[0032] Based on the above steps, the cloud server 10 can generate a reply message for the voice data based on the second voice recognition result of the target voice recognition engine on the voice data. The reply message can be implemented as text information or audio information used to provide a reply to the user. For example, if the user says to the terminal device 20, "What time is dinner served in the afternoon?", the cloud server 10 can generate a reply message of "6 pm". Further, the cloud server 10 can optionally send the generated reply message to the terminal device 20 in the form of text or audio, so that the terminal device 20 can output the reply message to the user through an audio component or a display component.

[0033] In this embodiment, the terminal device 20 can obtain the voice data sent by the user to the terminal device and send the voice data to the cloud server 10. After receiving the voice data, the cloud server 10 can perform voice recognition on the voice data through the first voice recognition engine corresponding to the first language type to obtain a first voice recognition result, and based on the first voice recognition result, determine whether the voice data matches the first voice recognition engine. If there is no match, a target voice recognition engine that matches the voice data is selected from the backup voice recognition engines, and a reply message for the voice data is generated based on the second voice recognition result of the target voice recognition engine for the voice data. Through this embodiment, when the language types used by the user are different, the terminal device 20 can more accurately perform voice recognition on the voice data input by the user, and then provide the user with a reply that is more closely matched with the voice information.

[0034] In some optional embodiments, the step of “determining whether the voice data matches the first voice recognition engine according to the first voice recognition result” described in the aforementioned embodiment can be implemented based on the following steps:

[0035] The cloud server 10 can obtain the text information in the first speech recognition result and calculate the recognition accuracy of the text information. The recognition accuracy can be calculated by a preset speech recognition model or by a preset algorithm. For example, a plurality of evaluation indicators such as the sentence error rate (SER), sentence correct rate (S.Corr) or character error rate (CER) of the text information can be calculated by a preset model or algorithm, and the recognition accuracy of the text information can be calculated based on the plurality of evaluation indicators and their respective weights.

[0036] If the calculated recognition accuracy is less than the set accuracy threshold, it is determined that the voice data does not match the first voice recognition engine, where the threshold can be set to 90%, 85% or 80%, etc., which is not limited in this embodiment.

[0037] If the calculated recognition accuracy is greater than or equal to the set accuracy threshold, the voice data can be preliminarily determined to match the first voice recognition engine. Based on this, the cloud server 10 can further determine whether the voice data matches the first voice recognition engine based on the confidence level of the response information generated during the question-answer matching phase. This will be explained in detail below.

[0038] If the recognition accuracy of the text information in the first speech recognition result is greater than or equal to a set accuracy threshold, the cloud server 10 may perform question-answer matching on the text information using a question-answer matching model based on NLP (Natural Language Processing) technology. After pre-training, the question-answer matching model can search the model's built-in dataset for multiple pre-selected information corresponding to the input text information, each with different confidence levels. Furthermore, the question-answer matching model can select the pre-selected information with the highest confidence level from the multiple pre-selected information as the response information. For example, if the cloud server 10 performs question-answer matching on the text information "Which street is the nearest bank to me on?" using the question-answer matching model, it may obtain the pre-selected information "On Street A" with an 80% confidence level and the pre-selected information "On Street B" with an 85% confidence level. The pre-selected information "On Street B" with an 85% confidence level can then be selected as the response information from these two pre-selected information.

[0039] Through the above question-answer matching method, the cloud server 10 can obtain the reply information and the confidence level of the reply information. If the confidence level of the reply information is less than the preset confidence level threshold, it is determined that the voice data does not match the first voice recognition engine.

[0040] If it is determined that the voice data does not match the first voice recognition engine, the cloud server 10 may select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine.

[0041] In some optional embodiments, when the cloud server 10 selects a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, it may select any speech recognition engine from the at least one backup speech recognition engine as the second speech recognition engine. For example, if the at least one backup speech recognition engine includes a speech recognition engine for Chinese and a speech recognition engine for French, the cloud server may select the speech recognition engine for French from the at least one backup speech recognition engine as the second speech recognition engine.

[0042] After the cloud server 10 selects the second speech recognition engine, it can use the engine to perform speech recognition on the speech data to obtain a second speech recognition result. The second speech recognition result refers to the speech recognition result obtained by performing speech recognition using the second speech recognition engine. The term "second" is used to define the speech recognition result and is only used to distinguish speech recognition results obtained from multiple speech recognition attempts.

[0043] After the speech recognition, the cloud server 10 can determine whether the speech data matches the second speech recognition engine based on the second speech recognition result. Detailed description will be given below.

[0044] The cloud server 10 can obtain the text information in the second speech recognition result and calculate the recognition accuracy of the text information. Among them, the calculation of the recognition accuracy can be performed by a preset speech recognition model or by a preset algorithm. For example, multiple evaluation indicators such as the sentence error rate (SER), sentence correct rate (S.Corr) and or character error rate (CER) of the text information can be calculated by a preset model or algorithm, and the recognition accuracy of the text information can be calculated based on the multiple evaluation indicators and their respective weights. If the recognition accuracy is greater than or equal to the set accuracy threshold, it is determined that the voice data matches the second recognition engine, and the cloud server 10 can use the second speech recognition engine as the target speech recognition engine. If the recognition accuracy is less than the set accuracy threshold, it is determined that the voice data does not match the second speech recognition engine, wherein the threshold can be set to 90%, 85% or 80%, etc., which is not limited in this embodiment.

[0045] It should be noted that the at least one backup speech recognition engine can be sorted according to a set priority order. The priority order can be preset by the user or according to the recognition results of big data. For example, the user can preset the priority order so that the speech recognition engine corresponding to Korean is in front and the speech recognition engine corresponding to German is in the back. Alternatively, the recognition results of big data indicate that the shopping mall where the terminal device 20 is located is mainly frequented by Arabs and French people. In this case, the cloud server 10 can arrange the order so that the speech recognition engine corresponding to Arabic is in front and the speech recognition engine corresponding to French is in the back according to the recognition results of the big data.

[0046] Based on the above-mentioned sorting process, when the cloud server 10 selects any one of the at least one backup speech recognition engines as the second speech recognition engine, it can select at least one backup speech recognition engine in sequence according to the sorting of the at least one backup speech recognition engine to obtain the second speech recognition engine. Continuing with the above-mentioned example, the engines are sorted so that the speech recognition engine corresponding to Korean is first and the speech recognition engine corresponding to German is last. If the cloud server 10 fails to recognize the first recognition engine, it can select in sequence according to this sorting order, that is, first select the speech recognition engine corresponding to Korean, then select the speech recognition engine corresponding to German, and obtain the second speech recognition engine.

[0047] In some optional embodiments, before selecting a target speech recognition engine that matches the speech data from the at least one backup speech recognition engine, the cloud server 10 may prioritize the at least one backup speech recognition engine based on the current location of the terminal device 20. For example, if the terminal device 20 is currently located in a community with a large Korean population and a small number of Germans, the cloud server 10 may preset the priority order so that the speech recognition engine corresponding to Korean is ranked first and the speech recognition engine corresponding to German is ranked last.

[0048] In addition to the above embodiments, the cloud server 10 may further rank the at least one backup voice recognition engine based on the historical usage frequency of the at least one voice recognition engine of the terminal device 20. For example, if the terminal device is implemented as a bank withdrawal terminal, the historical usage frequency of the bank withdrawal terminal is 30 times for Korean, 20 times for French, and 10 times for German. This means that the terminal is most frequently used by Korean speakers, followed by French speakers and German speakers. Therefore, the cloud server 10 may rank the at least one backup voice recognition engine based on the historical usage frequency into Korean, French, and German.

[0049] The following will be combined Figure 2 、 Figure 3 As well as actual application scenarios, the voice interaction system is further explained.

[0050] like Figure 2 and Figure 3 As shown, the terminal device collects the user's initial voice data through a microphone and sends the initial voice data to the voice endpoint detection module. The voice endpoint detection module can intercept valid voice data from the initial voice data. After successful interception, the terminal device can perform voice recognition on the voice data using the first voice recognition engine (i.e., the main engine) corresponding to the first language type in the main module to obtain text information corresponding to the voice data. Afterwards, the terminal device can perform question-answer matching on the text information using the question-answer matching model corresponding to the first language type to obtain reply information corresponding to the text information.

[0051] If the confidence level of the reply information is greater than or equal to the confidence level threshold, the reply information is converted into speech by a text-to-speech module corresponding to the first language type, and the speech is output.

[0052] If the confidence level of the reply information is less than the confidence level threshold, a target speech recognition engine that matches the speech data is selected from at least one backup speech recognition engine to perform speech recognition on the speech data again.

[0053] Taking the target speech recognition engine as the backup speech recognition engine corresponding to Korean as an example, the terminal device performs speech recognition on the speech data through the backup speech recognition engine corresponding to Korean to obtain the corresponding text information. Afterwards, the terminal device can perform question-answer matching on the text information through the backup question-answer matching model corresponding to Korean in the main module to obtain the corresponding reply information. If the confidence of the reply information is greater than or equal to the confidence threshold, the reply information is converted into speech through the text-to-speech module corresponding to Korean and the speech is output. If the confidence of the reply information is less than the confidence threshold, the backup speech recognition engine is reselected and the speech data is recognized again.

[0054] The present application also provides a voice interaction method. Figure 4 Provide detailed explanation.

[0055] Step 401: Acquire voice data sent by the user to the device.

[0056] Step 402: Perform speech recognition on the speech data using a first speech recognition engine corresponding to a preset first language type to obtain a first speech recognition result.

[0057] Step 403: Determine whether the voice data matches the first voice recognition engine based on the first voice recognition result.

[0058] Step 404: If not, select a target speech recognition engine that matches the speech data from at least one backup speech recognition engine; the at least one backup speech recognition engine corresponds to at least one language type.

[0059] Step 405: Generate response information for the voice data according to the second voice recognition result of the target voice recognition engine on the voice data.

[0060] Further optionally, based on the first speech recognition result, determining whether the speech data matches the first speech recognition engine includes: obtaining text information in the first speech recognition result; calculating the recognition accuracy of the text information; if the recognition accuracy is less than a set accuracy threshold, determining that the speech data does not match the first speech recognition engine.

[0061] Further optionally, the method also includes: if the recognition accuracy is greater than or equal to the set accuracy threshold, then using a question-answer matching model to perform question-answer matching on the text information to obtain reply information and the confidence of the reply information; if the confidence of the reply information is less than the preset confidence threshold, then determining that the voice data does not match the first voice recognition engine.

[0062] Further optionally, selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine includes: selecting any speech recognition engine from the at least one backup speech recognition engine as a second speech recognition engine; performing speech recognition on the speech data through the second speech recognition engine to obtain a second speech recognition result; judging whether the speech data matches the second speech recognition engine based on the second speech recognition result; if the speech data matches the second speech recognition engine, using the second speech recognition engine as the target speech recognition engine.

[0063] Further optionally, the at least one backup voice recognition engine is sorted in a set priority order; and any one voice recognition engine is selected from the at least one backup voice recognition engine as the second voice recognition engine, including: selecting the at least one backup voice recognition engine in turn according to the sorting of the at least one backup voice recognition engine to obtain the second voice recognition engine.

[0064] Further optionally, before selecting a target voice recognition engine that matches the voice data from at least one backup voice recognition engine, the method further includes: sorting the at least one backup voice recognition engine according to the current region of the device; or sorting the at least one backup voice recognition engine according to the historical usage frequency of the at least one voice recognition engine of the device.

[0065] In this embodiment, the first speech recognition engine corresponding to the first language type can be used to perform speech recognition on the speech data to obtain a first speech recognition result. Based on the first speech recognition result, it is determined whether the speech data matches the first speech recognition engine. If there is no match, a target speech recognition engine that matches the speech data is selected from the backup speech recognition engines, and a response message for the speech data is generated based on the second speech recognition result of the target speech recognition engine on the speech data. Through this embodiment, when the user uses different language types, the terminal device can more accurately perform speech recognition on the speech data input by the user, and further, can provide the user with a response that more closely matches the speech message.

[0066] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 401 to 405 can be device A; for another example, the execution entity of steps 401 to 403 can be device A, and the execution entity of steps 404 and 405 can be device B; and so on.

[0067] In addition, some of the processes described in the above embodiments and the accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 401 and 402, are merely used to distinguish between different operations and do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0068] It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types.

[0069] The present application embodiment provides a voice interaction device, such as Figure 5 As shown, the voice interaction device includes: an acquisition module 501, a recognition module 502, a judgment module 503, a selection module 504 and a generation module 505.

[0070] Among them, the acquisition module 501 is used to: acquire the voice data sent by the user to the device; the recognition module 502 is used to: perform voice recognition on the voice data through the first voice recognition engine corresponding to the preset first language type to obtain a first voice recognition result; the judgment module 503 is used to: judge whether the voice data matches the first voice recognition engine based on the first voice recognition result; the selection module 504 is used to: if not, select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; the generation module 505: generates reply information of the voice data based on the second voice recognition result of the target voice recognition engine on the voice data.

[0071] Further optionally, when the judgment module 503 judges whether the voice data matches the first voice recognition engine based on the first voice recognition result, it is specifically used to: obtain the text information in the first voice recognition result; calculate the recognition accuracy of the text information; if the recognition accuracy is less than the set accuracy threshold, determine that the voice data does not match the first voice recognition engine.

[0072] Further optionally, the judgment module 503 is also used to: if the recognition accuracy is greater than or equal to the set accuracy threshold, then use the question-answer matching model to perform question-answer matching on the text information to obtain reply information and the confidence of the reply information; if the confidence of the reply information is less than the preset confidence threshold, then determine that the voice data does not match the first voice recognition engine.

[0073] Further optionally, when the selection module 504 selects a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, it is specifically used to: select any speech recognition engine from the at least one backup speech recognition engine as a second speech recognition engine; perform speech recognition on the speech data through the second speech recognition engine to obtain a second speech recognition result; determine whether the speech data matches the second speech recognition engine based on the second speech recognition result; if the speech data matches the second speech recognition engine, use the second speech recognition engine as the target speech recognition engine.

[0074] Further optionally, the at least one backup speech recognition engine is sorted according to a set priority order. When selecting any speech recognition engine from the at least one backup speech recognition engine as the second speech recognition engine, the selection module 504 is specifically configured to: select the at least one backup speech recognition engine in sequence according to the sorting of the at least one backup speech recognition engine to obtain the second speech recognition engine.

[0075] Further optionally, before selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, the selection module 504 is also used to: sort the at least one backup speech recognition engine according to the current region of the device; or sort the at least one backup speech recognition engine according to the historical usage frequency of the at least one speech recognition engine of the device.

[0076] In this embodiment, the first speech recognition engine corresponding to the first language type can be used to perform speech recognition on the speech data to obtain a first speech recognition result. Based on the first speech recognition result, it is determined whether the speech data matches the first speech recognition engine. If there is no match, a target speech recognition engine that matches the speech data is selected from the backup speech recognition engines, and a response message for the speech data is generated based on the second speech recognition result of the target speech recognition engine on the speech data. Through this embodiment, when the user uses different language types, the terminal device can more accurately perform speech recognition on the speech data input by the user, and further, can provide the user with a response that more closely matches the speech message.

[0077] Figure 6 This is a schematic diagram of the structure of a cloud server provided by an exemplary embodiment of the present application. The server is applicable to the voice interaction system provided by the aforementioned embodiment, such as Figure 6 As shown, the server includes: a memory 601 , a processor 602 and a communication component 603 .

[0078] The memory 601 is used to store computer programs and can be configured to store various other data to support operations on the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, contact data, phone book data, messages, pictures, videos, etc.

[0079] The processor 602 is coupled to the memory 601 and is used to execute the computer program in the memory 601, so as to: obtain voice data sent by the user to the device; perform voice recognition on the voice data through a first voice recognition engine corresponding to a preset first language type to obtain a first voice recognition result; determine whether the voice data matches the first voice recognition engine based on the first voice recognition result; if not, select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; and generate reply information for the voice data based on a second voice recognition result of the target voice recognition engine on the voice data.

[0080] Further optionally, when the processor 602 determines whether the voice data matches the first voice recognition engine based on the first voice recognition result, it is specifically used to: obtain text information in the first voice recognition result; calculate the recognition accuracy of the text information; if the recognition accuracy is less than the set accuracy threshold, determine that the voice data does not match the first voice recognition engine.

[0081] Further optionally, the processor 602 is also used to: if the recognition accuracy is greater than or equal to the set accuracy threshold, then use the question-answer matching model to perform question-answer matching on the text information to obtain reply information and the confidence of the reply information; if the confidence of the reply information is less than the preset confidence threshold, then determine that the voice data does not match the first voice recognition engine.

[0082] Further optionally, when the processor 602 selects a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, it is specifically used to: select any speech recognition engine from the at least one backup speech recognition engine as a second speech recognition engine; perform speech recognition on the speech data through the second speech recognition engine to obtain a second speech recognition result; determine whether the speech data matches the second speech recognition engine based on the second speech recognition result; if the speech data matches the second speech recognition engine, use the second speech recognition engine as the target speech recognition engine.

[0083] Further optionally, the at least one backup speech recognition engine is sorted according to a set priority order. When the processor 602 selects any speech recognition engine from the at least one backup speech recognition engine as the second speech recognition engine, it is specifically configured to: select the at least one backup speech recognition engine in sequence according to the sorting of the at least one backup speech recognition engine to obtain the second speech recognition engine.

[0084] Further optionally, before selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, the processor 602 is also used to: sort the at least one backup speech recognition engine according to the current region of the device; or sort the at least one backup speech recognition engine according to the historical usage frequency of the at least one speech recognition engine of the device.

[0085] Further, if Figure 6 As shown, the cloud server also includes: a power supply component 604 and other components. Figure 6 Only some components are shown schematically, which does not mean that the cloud server only includes Figure 6 Components shown.

[0086] In this embodiment, the first speech recognition engine corresponding to the first language type can be used to perform speech recognition on the speech data to obtain a first speech recognition result. Based on the first speech recognition result, it is determined whether the speech data matches the first speech recognition engine. If there is no match, a target speech recognition engine that matches the speech data is selected from the backup speech recognition engines, and a response message for the speech data is generated based on the second speech recognition result of the target speech recognition engine on the speech data. Through this embodiment, when the user uses different language types, the terminal device can more accurately perform speech recognition on the speech data input by the user, and further, can provide the user with a response that more closely matches the speech message.

[0087] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the cloud server in the above method embodiment.

[0088] above Figure 6 The memory 601 in the embodiment can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0089] above Figure 6 The communication component 603 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0090] above Figure 6 The power supply component 604 in the embodiment provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0091] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0095] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0096] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0097] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0098] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0099] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A voice interaction method, characterized in that: include: Obtain the voice data sent by the user to the device; performing speech recognition on the speech data using a first speech recognition engine corresponding to a preset first language type to obtain a first speech recognition result; Calculating multiple evaluation indicators of the text information in the first speech recognition result through a preset speech recognition model, and calculating the recognition accuracy rate of the text information based on the multiple evaluation indicators and their respective weights; The multiple evaluation indicators include: sentence error rate, sentence accuracy rate and word error rate; If the recognition accuracy is less than a set accuracy threshold, determining that the voice data does not match the first voice recognition engine; If the voice data does not match the first voice recognition engine, selecting a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; Generate response information of the voice data according to a second voice recognition result of the voice data by the target voice recognition engine.

2. The method according to claim 1, characterized in that Also includes: If the recognition accuracy is greater than or equal to the set accuracy threshold, the question-answer matching model is used to perform question-answer matching on the text information to obtain answer information and the confidence level of the answer information; If the confidence level of the reply information is less than a preset confidence level threshold, it is determined that the voice data does not match the first voice recognition engine.

3. The method according to claim 1, characterized in that Selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine includes: Selecting any one speech recognition engine from the at least one backup speech recognition engine as the second speech recognition engine; Performing speech recognition on the speech data using a second speech recognition engine to obtain a second speech recognition result; Determining, based on the second speech recognition result, whether the speech data matches the second speech recognition engine; If the voice data matches the second voice recognition engine, the second voice recognition engine is used as the target voice recognition engine.

4. The method according to claim 3, characterized in that The at least one backup speech recognition engine is sorted according to a set priority order; Selecting any one of the at least one backup speech recognition engine as the second speech recognition engine comprises: According to the ranking of the at least one backup speech recognition engine, the at least one backup speech recognition engine is selected in sequence to obtain the second speech recognition engine.

5. The method according to claim 4, characterized in that Before selecting a target speech recognition engine that matches the speech data from at least one backup speech recognition engine, the method further includes: sorting the at least one backup speech recognition engine according to the current region of the device; or, The at least one backup speech recognition engine is sorted according to a historical usage frequency of the at least one speech recognition engine of the device.

6. A voice interaction system, characterized in that: include: Terminal devices and cloud servers; The terminal device is mainly used to: obtain voice data sent by the user to the terminal device; send the voice data to the cloud server; The cloud server is mainly used to: receive the voice data; perform voice recognition on the voice data through a first voice recognition engine corresponding to a preset first language type to obtain a first voice recognition result; calculate multiple evaluation indicators of the text information in the first voice recognition result through a preset voice recognition model, and calculate the recognition accuracy of the text information based on the multiple evaluation indicators and their respective weights; the multiple evaluation indicators include: sentence error rate, sentence correct rate and word error rate; if the recognition accuracy is less than the set accuracy threshold, it is determined that the voice data does not match the first voice recognition engine; if the voice data does not match the first voice recognition engine, a target voice recognition engine that matches the voice data is selected from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; and generate reply information for the voice data based on the second voice recognition result of the target voice recognition engine on the voice data.

7. A voice interaction device, characterized in that: include: The acquisition module is used to: acquire the voice data sent by the user to the device; a recognition module, configured to perform speech recognition on the speech data using a first speech recognition engine corresponding to a preset first language type to obtain a first speech recognition result; a judgment module, configured to calculate a plurality of evaluation indicators of the text information in the first speech recognition result by using a preset speech recognition model, and calculate a recognition accuracy rate of the text information based on the plurality of evaluation indicators and their respective weights; The multiple evaluation indicators include: sentence error rate, sentence correct rate and word error rate; if the recognition accuracy rate is less than a set accuracy rate threshold, it is determined that the voice data does not match the first voice recognition engine; A selection module is configured to: if the voice data does not match the first voice recognition engine, select a target voice recognition engine that matches the voice data from at least one backup voice recognition engine; the at least one backup voice recognition engine corresponds to at least one language type respectively; Generating module: generating reply information of the voice data according to the second voice recognition result of the target voice recognition engine on the voice data.

8. A cloud server, characterized in that: include: memory, processors, and communication components; Wherein, the memory is used to: store one or more computer instructions; The processor is configured to execute the one or more computer instructions to perform the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps of the method according to any one of claims 1 to 5.

Images