Method, device and computer readable storage medium for speech recognition

Abstract

The application discloses a speech recognition method, device, equipment and computer readable storage medium, and belongs to the technical field of artificial intelligence, and is particularly suitable for human-computer interaction in a car or an electric vehicle. The method comprises the following steps: obtaining or generating a dynamic target language model according to reply information of a first intention, the dynamic target language model comprising a front-end part and a core part; obtaining a speech signal, and generating a keyword by analyzing the speech signal; and calling the dynamic target language model to determine a second intention and service content, wherein the front-end part of the dynamic target language model analyzes the second intention according to the keyword, and the core part of the dynamic target language model analyzes the service content according to the keyword. The application can realize the ignoring of irrelevant speech, recognize non-continuous multi-intention speech, avoid the deviation of provided service content from user demand, and has good recognition effect.

Description

[0001] This application is a divisional application. The original application has the application number 201910470966.4 and the original application date is May 31, 2019. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence technology, and in particular to methods, apparatus, devices and computer-readable storage media for speech recognition. Background Technology

[0003] With the development of artificial intelligence (AI) technology, AI systems are widely used in daily life, and voice recognition systems are one such example. When in use, users issue voice commands to the voice recognition system. The system then needs to recognize and understand these commands, and based on those commands, ask the user a question. Next, it recognizes and understands the user's response, providing feedback and instructions to meet the user's needs. Therefore, how to perform voice recognition is crucial to meeting user needs.

[0004] Related technologies provide a speech recognition method. This method, after calling a language model to recognize and understand user commands, firstly asks a question to the user; secondly, it adjusts the language model based on the question, such as integrating a vocabulary set related to the question into the language model, enabling the adjusted language model to recognize words in the vocabulary set. When the user responds with a speech using words from the vocabulary set, the adjusted language model can recognize that response, thus meeting the user's needs.

[0005] The inventors have discovered that the related technology has at least the following problems:

[0006] Besides voice commands and responses, users may also emit irrelevant voice messages related to communication with third parties. For example, in typical multi-user and multi-scenario scenarios, when users interact with the in-vehicle module in a car or electric vehicle via voice, irrelevant voice messages may include interruptions from other users or other users' voice messages. The in-vehicle module's voice recognition system may also recognize and understand these irrelevant voice messages as voice commands or responses, causing the provided services to deviate from the user's needs and resulting in a poor user experience. Summary of the Invention

[0007] This application provides a method, apparatus, device, and computer-readable storage medium for speech recognition to address the problems of poor recognition performance and poor user experience in related technologies. The technical solution is as follows:

[0008] On one hand, a speech recognition method is provided, the method comprising: acquiring or generating a dynamic target language model based on response information to a first intent, wherein the dynamic target language model includes a front-end part and a core part, the core part being used to determine possible descriptions related to the response information, and the front-end part being used to determine descriptions of confirmatory information of the response information; therefore, after acquiring a speech signal, parsing the speech signal to generate keywords, the dynamic target language model can be invoked to determine a second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords.

[0009] Taking a voice recognition scenario involving a dialogue between a user and an in-vehicle module as an example, the first intent includes the intent obtained by parsing the user's voice signal after the voice dialogue between the user and the in-vehicle module begins. The response information for the first intent includes one or more response messages returned by the in-vehicle module to the user in response to the first intent. The in-vehicle module obtains a dynamic target language model, including the front-end and core parts, based on the response information for the first intent. After returning one or more response messages to the user, the in-vehicle module will acquire the voice signal again. It should be noted that the voice signal acquired again by the in-vehicle module may include the voice signal from the user's dialogue with the in-vehicle module, i.e., the voice signal related to the response information, as well as unrelated voice signals from conversations between the user and other users.

[0010] Next, the in-vehicle module analyzes the acquired voice signal to generate keywords, and calls the dynamic target language model to extract the words related to the response information from the generated keywords. The dynamic target language model consists of a front-end and a core part. The front-end is used to determine the user's confirmatory information regarding the response information, which may include confirmation, correction, and cancellation. By analyzing the keywords through the front-end, the user's second intent can be obtained. For example, if the response information for the first intent includes one item, and the confirmatory information obtained by the front-end from the keyword analysis includes the confirmation message "Yes, that's right," then it can be confirmed that the user's second intent is the intent indicated by the response information for the first intent.

[0011] The core component is used to determine possible descriptions related to the response information. Through the core component, the vocabulary used by the user to describe the response information can be parsed from keywords, thereby obtaining service content based on this vocabulary to provide the user with the service indicated by the service content. In this embodiment, the service indicated by the service content can be provided by a third-party cloud service, an in-vehicle module, an in-vehicle terminal, or even by the vehicle manufacturer. The in-vehicle terminal can be other terminals in the vehicle besides the in-vehicle module, such as an in-vehicle display screen, in-vehicle air conditioning, in-vehicle speakers, etc. Of course, two or more execution entities among the third-party cloud service, in-vehicle module, in-vehicle terminal, and vehicle manufacturer can jointly provide the service indicated by the service content. It should be noted that since both the front-end and core components of the dynamic target language model are obtained based on the response information, the second intent obtained through the front-end and the service content obtained through the core component are both related to the first intent, while voice signals unrelated to the first intent are ignored. Therefore, the speech recognition effect of this embodiment is better, avoiding the deviation of the provided service from user needs due to interference from irrelevant voice signals, thus improving the user experience.

[0012] Optionally, the dynamic target language model further includes a tail portion, which is used to confirm whether there is an additional intent. The method further includes: calling the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on the keyword.

[0013] Optionally, the tail portion includes a tail marker; the step of calling the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on the keyword, includes: the tail portion parses a reference tail marker based on the keyword, and the time point where the reference tail marker is located; based on the reference tail marker, combined with the first intent and the second intent, the dynamic target language model is updated to obtain an updated target language model; the updated target language model is called, and the additional intent is parsed based on the keyword and the time point where the reference tail marker is located. By parsing the additional intent, a more accurate service can be provided.

[0014] Optionally, before acquiring the speech signal, the method further includes: caching historical speech signals; and parsing the speech signal to generate keywords, which includes: parsing the speech signal, performing context detection using the historical speech signal, and then generating the keywords. Context detection using historical speech signals allows the recognized keywords to better fit the current context, thereby further improving the accuracy of speech recognition.

[0015] Optionally, after invoking the dynamic target language model to determine the second intent and service content, the method further includes: confirming the second intent to obtain the confirmed second intent.

[0016] Optionally, confirming the second intent and obtaining the confirmed second intent includes: sending confirmation information of the second intent to the user, obtaining the second intent fed back by the user, and using the second intent fed back by the user as the confirmed second intent. By confirming the second intent, the second intent becomes more accurate, thereby providing more accurate service content.

[0017] Optionally, obtaining the dynamic target language model based on the response information to the first intent includes: converting the response information to the first intent into a reference format to obtain the response information in the reference format, and obtaining or generating the dynamic target language model based on the response information in the reference format. Since different vendors may provide response information in different formats, converting the response information into a reference format ensures a unified response information format for easier reception. For different application domains, the response information is converted into different reference formats to ensure that the response information format is the same within the same application domain.

[0018] Optionally, obtaining or generating the dynamic target language model based on the response information in the reference format includes: converting the trained language model into a weighted finite state transition automaton model, and using the weighted finite state transition automaton model as the dynamic target language model, wherein the trained language model is obtained by training on the response information in the reference format and reference vocabulary. The reference vocabulary includes, but is not limited to, class names corresponding to words in the response information in the reference format, and referential expressions.

[0019] Optionally, obtaining or generating the dynamic target language model based on the response information in the reference format includes: converting the trained language model into a weighted finite state transition automaton model, using the weighted finite state transition automaton model as a first language model, wherein the trained language model is obtained by training response information in a reference format with a length not less than the reference length; obtaining a second language model based on response information in a reference format with a length less than the reference length, and obtaining a third language model based on reference vocabulary; merging the first language model, the second language model, and the third language model to obtain a total language model, and using the total language model as the dynamic target language model.

[0020] Optionally, the step of obtaining or generating the dynamic target language model based on the response information in the reference format includes: obtaining a word confusion network based on the response information in the reference format with a length not less than the reference length, wherein each word in the word confusion network has a transition probability; calculating the penalty weight of each word; converting the word confusion network into a weighted finite state transition automaton model based on the penalty weight of each word; using the weighted finite state transition automaton model as the first language model; obtaining a second language model based on the response information in the reference format with a length less than the reference length; obtaining a third language model based on the reference words; merging the first language model, the second language model, and the third language model to obtain a total language model; and using the total language model as the dynamic target language model.

[0021] Optionally, calculating the penalty weight for each word includes: for any word, using the negative logarithm of the word's transition probability as the penalty weight. The transition probability of a word indicates its frequency of occurrence within its category. A higher frequency of occurrence within a category corresponds to a higher transition probability and a smaller negative logarithm of the transition probability; that is, the penalty weight is inversely proportional to the frequency of occurrence, so that the target language model can better parse words with high frequency within their category.

[0022] Optionally, calculating the penalty weight for each word includes: for any word, using the logarithm of the number of response messages containing the reference format of that word as the penalty weight. Words with strong discriminative power, i.e., words with fewer response messages containing the reference format of that word, are given smaller penalty weights so that the target language model can better parse these highly discriminative words.

[0023] Optionally, calculating the penalty weight for each word includes: for any word, using the logarithm of the number of times the word appears in the response information of the reference format as the penalty weight. For words with strong discriminative power, i.e., words that appear less frequently, the penalty probability of the word is smaller, so that the dynamic target language model can better parse out words with strong discriminative power.

[0024] On one hand, a speech recognition device is provided, the device comprising: a first acquisition module, for acquiring or generating a dynamic target language model based on response information to a first intent, the dynamic target language model including a front-end part and a core part, the core part being used to determine possible descriptions related to the response information, and the front-end part being used to determine a description of confirmatory information of the response information; a second acquisition module, for acquiring a speech signal and parsing the speech signal to generate keywords; and a first determination module, for calling the dynamic target language model to determine a second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords.

[0025] Optionally, the dynamic target language model further includes a tail portion, which is used to confirm whether there is an additional intent. The device further includes a second determining module, which is used to call the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on the keyword.

[0026] Optionally, the end part includes an end-of-line marker; the second determining module is used to parse a reference end-of-line marker and the time point where the reference end-of-line marker is located based on the keyword; based on the reference end-of-line marker, combined with the first intent and the second intent, update the dynamic target language model to obtain an updated target language model; call the updated target language model to parse the additional intent based on the keyword and the time point where the reference end-of-line marker is located.

[0027] Optionally, the device further includes: a caching module for caching historical voice signals; and a second acquisition module for parsing the voice signals and generating the keywords after performing context detection using the historical voice signals.

[0028] Optionally, the device further includes: a confirmation module, configured to confirm the second intention and obtain the confirmed second intention.

[0029] Optionally, the confirmation module is used to send confirmation information of the second intent to the user, obtain the second intent fed back by the user, and use the second intent fed back by the user as the confirmed second intent.

[0030] Optionally, the first acquisition module is used to convert the response information of the first intent into a reference format to obtain the response information in the reference format, and to acquire or generate the dynamic target language model based on the response information in the reference format.

[0031] Optionally, the first acquisition module is used to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as the dynamic target language model, wherein the trained language model is obtained by training the response information of the reference format and the reference vocabulary.

[0032] Optionally, the first acquisition module is configured to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as a first language model, wherein the trained language model is obtained by training response information in a reference format with a length not less than a reference length; acquire a second language model based on response information in a reference format with a length less than a reference length, and acquire a third language model based on reference vocabulary; merge the first language model, the second language model, and the third language model to obtain a total language model, and use the total language model as the dynamic target language model.

[0033] Optionally, the first acquisition module includes: a first acquisition unit, configured to acquire a word confusion network based on response information in a reference format with a length not less than a reference length, wherein each word in the word confusion network has a transition probability; a calculation unit, configured to calculate the penalty weight of each word, convert the word confusion network into a weighted finite state transition automaton model according to the penalty weight of each word, and use the weighted finite state transition automaton model as a first language model; a second acquisition unit, configured to acquire a second language model based on response information in a reference format with a length less than a reference length, and acquire a third language model based on reference words; and a merging unit, configured to merge the first language model, the second language model, and the third language model to obtain a total language model, and use the total language model as the dynamic target language model.

[0034] Optionally, the calculation unit is configured to use the negative logarithm of the transition probability of any word as the penalty weight.

[0035] Optionally, the calculation unit is configured to, for any word, use the logarithm of the number of response messages containing the reference format of the word as the penalty weight.

[0036] Optionally, the calculation unit is configured to, for any word, use the logarithm of the number of times the word appears in the response information of the reference format as the penalty weight.

[0037] On one hand, a speech recognition device is provided, the device comprising: a memory and a processor; the memory stores at least one instruction, the at least one instruction being loaded and executed by the processor to implement the method in the first aspect of the embodiments of this application or any possible implementation of the first aspect.

[0038] Optionally, the processor may be one or more, and the memory may be one or more.

[0039] Optionally, the memory may be integrated with the processor, or the memory may be separated from the processor.

[0040] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. The embodiments of this application do not limit the type of memory or the way the memory and processor are set.

[0041] On the other hand, a computer-readable storage medium is provided that stores a program or instructions which are loaded by a processor and executed by any of the above-described speech recognition methods.

[0042] A computer program (product) is also provided, the computer program (product) comprising: computer program code, which, when executed by a computer, causes the computer to perform any of the above-described speech recognition methods.

[0043] A chip is also provided, including a processor for calling and executing instructions stored in a memory, causing a communication device equipped with the chip to perform any of the above-described speech recognition methods.

[0044] Another chip is also provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is used to execute code in the memory, wherein when the code is executed, the processor is used to perform any of the above-described speech recognition methods.

[0045] The beneficial effects of the technical solutions provided in this application include at least the following:

[0046] This application embodiment obtains or generates a dynamic target language model including a front-end and a core part based on the response information of the first intent. After parsing the speech signal to obtain keywords, the dynamic target language model is then called to parse the keywords to obtain the second intent and service content. Since the dynamic target language model is obtained based on the response information of the first intent, the second intent and service content obtained by parsing through the dynamic target language model are all related to the first intent. Therefore, this application embodiment realizes the ignoring of speech unrelated to the first intent, that is, it can recognize speech with non-continuous multi-intents, avoids the provided service content from deviating from user needs, has good recognition effect, and improves the user experience. Attached Figure Description

[0047] Figure 1 A schematic diagram of the implementation environment provided for the embodiments of this application;

[0048] Figure 2 A module structure diagram of the speech recognition method provided in the embodiments of this application;

[0049] Figure 3 A flowchart of a speech recognition method provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of the language model provided in the embodiments of this application;

[0051] Figure 5 A schematic diagram of the speech recognition process provided in an embodiment of this application;

[0052] Figure 6 This is a schematic diagram of the structure of the language model provided in the embodiments of this application;

[0053] Figure 7 This is a schematic diagram of the structure of the language model provided in the embodiments of this application;

[0054] Figure 8 This is a schematic diagram of the word confusion network provided in an embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the structure of the speech recognition device provided in the embodiments of this application. Detailed Implementation

[0056] The above description is merely an embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application. The terminology used in the implementation section of this application is only used to explain specific embodiments of this application and is not intended to limit this application.

[0057] With the development of artificial intelligence (AI) technology, AI systems are widely used in daily life, and voice recognition systems are one such example. When in use, users issue voice commands to the voice recognition system. The system then needs to recognize and understand these commands, and based on those commands, ask the user a question. Next, it recognizes and understands the user's response, providing feedback and instructions to meet the user's needs. Therefore, how to perform voice recognition is crucial to meeting user needs.

[0058] Related technologies provide a speech recognition method. This method, after calling a language model to recognize and understand user commands, firstly asks a question to the user; secondly, it adjusts the language model based on the question, such as integrating a vocabulary set related to the question into the language model, enabling the adjusted language model to recognize words in the vocabulary set. When the user responds with a speech using words from the vocabulary set, the adjusted language model can recognize that response, thus meeting the user's needs.

[0059] However, user-generated voice commands are often quite flexible. For example, in a multi-user in-vehicle navigation scenario, the user and the in-vehicle module might have the following voice dialogue:

[0060] User; (Regarding the vehicle module) Could you help me find a Sichuan restaurant nearby?

[0061] Onboard module: (To user) Are you going to Sichuan restaurant A?

[0062] User: (To other users in the car) It's noon right now, will there be any problems with parking? (To the vehicle module) Yes, it's Sichuan restaurant A.

[0063] Using the methods provided by the relevant technologies, the in-vehicle module's voice recognition system, after asking a question based on a voice command, can integrate the phrase "Sichuan Restaurant A" from the question into its language model, resulting in an adjusted language model. Then, if the user responds with "Yes, it's Sichuan Restaurant A," the adjusted language model can recognize that response. However, in the above dialogue, the user first uttered irrelevant speech related to conversations with other users in the vehicle. Therefore, the adjusted language model misinterpreted this irrelevant speech as a response, leading to a misunderstanding. It can be seen that the voice recognition method provided by the relevant technologies has poor recognition performance and a poor user experience.

[0064] This application provides a speech recognition method, which can be applied to, for example... Figure 1 The implementation environment shown. Figure 1The system includes audio equipment, memory, and a central processing unit (CPU). The audio equipment includes a microphone array and a speaker, while the memory stores programs or instructions for modules used for speech recognition. The audio equipment, memory, and CPU are connected via a data bus (D-Bus) so that the CPU can use the microphone array to collect the user's voice signal, execute the programs or instructions stored in the memory based on the collected voice signal, and use the speaker to output a voice signal to the user based on the results.

[0065] Additionally, see Figure 1 The CPU can also access cloud services through a gateway to obtain data returned by the cloud services. The CPU can also access the Controller Area Network Bus (CAN-Bus) through the gateway, thereby enabling it to read and control the status of other devices.

[0066] Optionally, Figure 1 In the schematic diagram of the implementation environment shown, the program or instructions for the speech recognition module stored in the memory include: Figure 2 The circular speech buffer module, AM module, SL module, Dynamic LM module, SLU module, DM module, and NCM process, etc., are provided by... Figure 1 The CPU executes programs or instructions from various modules in the memory to achieve speech recognition. Next, combining... Figure 2 The module functions that implement the speech recognition method provided in this embodiment are shown, and the speech recognition process is explained:

[0067] The front-end speech module is used to distinguish the user's voice signal from non-voice signals such as road noise and music. It is also used to perform noise reduction and enhancement on the user's voice signal to improve the accuracy of subsequent recognition and understanding.

[0068] The circular buffer module is used to buffer the speech signal processed by the front-end language model, so that the stored speech signal can be recognized and understood multiple times. The circular buffer has a reference time length. When the time length of the buffered speech signal is longer than the reference time length, the speech signal with the longest storage time will be overwritten by the new speech signal.

[0069] Acoustic model (AM) module: used to acquire the speech signal stored in the circular speech buffer module and convert the speech signal into a phoneme sequence.

[0070] The Selective Listening (SL) module calls the Dynamic Language Model (LM) module to convert the phoneme sequence output by the AM model into keywords and then sends the keywords to the Spoken Language Understanding (SLU) module.

[0071] SLU module: Used to extract intent and semantic slots from keywords to understand the user's primary intent, secondary intent, and additional intent indicated by the voice signal.

[0072] Dialogue Manager (DM) module: Used to request response information from cloud services based on the initial intent.

[0073] The Application Manager (APP Manager) module is used to convert the response information returned by the cloud service into a reference format.

[0074] The Dialogue Manager (DM) module is also used to initiate non-continuous multi-intent (NCM) processes in relevant domains based on response information in a reference format returned by the App Manager module, and to control the response generator (RG) module to generate response content and play it aloud. It is also used to send instructions to the App Manager module based on second and additional intents to control the application or terminal device to execute service content and additional intents.

[0075] The Application Manager (APP Manager) module is also used for word segmentation and proper noun annotation of response information. It also manages applications and terminal devices based on instructions sent by the DM module, controlling applications or terminal devices to execute service content and additional intents.

[0076] Based on the above Figure 1 The implementation environment shown is described in the following document. Figure 3 This application provides a method for speech recognition. For example... Figure 3 As shown, the method includes:

[0077] Step 201: Obtain or generate a dynamic target language model based on the response information to the first intent. The dynamic target language model includes a front-end part and a core part. The core part is used to determine possible descriptions related to the response information, and the front-end part is used to determine the description of confirmatory information of the response information.

[0078] The first intent refers to the intent obtained by parsing the user's voice command signal after the voice dialogue between the user and the system begins. Taking the above voice dialogue as an example, the user's voice command signal is the voice of "help me find a nearby Sichuan restaurant". Parsing the voice command signal includes: calling the acoustic model to convert the voice command signal into a phoneme sequence. A phoneme is the smallest unit of speech in a language; for example, in Chinese, a phoneme refers to an initial consonant or a final vowel. Then, calling the language model to convert the phoneme sequence into a text sequence, which is the voice command. This language model refers to a language model that has been trained on a training set, and an appropriate language model can be called according to the domain in which speech recognition is applied.

[0079] After obtaining the text sequence, it can be parsed to obtain the first intent. The first intent includes the intent itself and semantic slots. Semantic slots refer to words in the text sequence that have clear definitions or concepts. Continuing with the voice dialogue example above, if the text sequence is "Help me find a nearby Sichuan restaurant," the parsed intent is "navigation," and the semantic slots are "nearby" and "Sichuan restaurant," resulting in the first intent being "Navigate to a nearby Sichuan restaurant." Then, the response information for the first intent can be obtained based on the obtained first intent. The content of the response information for the first intent satisfies the requirements of the semantic slots. The response information can be obtained by sending the first intent to a cloud service to obtain the response information returned by the cloud service. Alternatively, multiple mapping relationships between intents and response information can be stored in memory, and the response information corresponding to the first intent can be found based on the mapping relationship to obtain the response information.

[0080] It should be noted that regardless of the method used to obtain the response information, the response information can be one or more items, each of which is a text string. Furthermore, if there are multiple responses, all of them can be considered as options for the user to choose from. Taking the voice dialogue example above, the response information can be one item, "Sichuan Restaurant A," or multiple items, such as "Sichuan Restaurant A," "Sichuan Restaurant B," and "Sichuan Restaurant C." This embodiment does not limit the number of response items.

[0081] Next, a dynamic target language model can be obtained or generated based on the acquired response information of the first intent. The dynamic target language model includes a front-end and a core. The front-end is used to determine the description of confirmatory information regarding the response, including but not limited to confirmation, correction, or cancellation information. For example, confirmation information may include "correct" or "that's right," correction information may include "incorrect" or "wrong," and cancellation information may include "never mind" or "I don't need it." The core is used to determine possible descriptions related to the response, such as whether the user directly restates the response or selectively restates it.

[0082] It should be noted that the process of acquiring or generating a dynamic target language model based on the response information will be explained in detail later, and will not be repeated here. Of course, regardless of the process used to acquire or generate the dynamic target language model, once the dynamic target language model is obtained, speech signals can be further received.

[0083] Step 202: Acquire the speech signal and parse the speech signal to generate keywords.

[0084] In this embodiment, after obtaining the response information to the first intent, the in-vehicle module, in addition to acquiring or generating a dynamic target language model based on the response information, also sends the response information to the user to obtain a voice signal. It should be noted that the voice signal may include both the voice signal from the user's dialogue with the in-vehicle module (i.e., the voice signal responding to the first intent) and unrelated voice signals from the user's dialogue with other users. Taking the aforementioned voice dialogue as an example, the voice signal from the user's dialogue with the in-vehicle module is "Yes, it's Sichuan restaurant A," while the unrelated voice signal from the user's dialogue with other users is "It's noon right now, will there be a parking problem?" Of course, the aforementioned unrelated voice signals can include not only voice signals from the user's active dialogue with other users, but also voice signals from other users' active dialogue with the user (i.e., voice signals from other users interjecting). This embodiment does not limit the scope of unrelated voice signals.

[0085] After the vehicle module acquires the voice signal, it can parse the voice signal to generate keywords. Optionally, in this embodiment, before acquiring the voice signal, the method further includes: caching historical voice signals. Then, parsing the voice signal to generate keywords includes: parsing the voice signal, performing context detection using historical voice signals, and generating keywords.

[0086] Historical voice signals refer to voice signals from past times. For example, in the aforementioned voice dialogue, the voice command signal "Help me find a nearby Sichuan restaurant" used to obtain the first intention can be used as a historical voice signal. In this embodiment, a circular buffer can be used to cache historical voice signals. The circular buffer has a reference time length. If the time length of the cached historical voice signal is longer than the reference time length, the historical voice signal with the longest cache time will be overwritten by the new voice signal. Therefore, if a historical voice signal is needed, it can be read from the circular buffer. Of course, this embodiment does not limit the method of caching historical voice signals; other methods can be selected as needed to implement historical voice caching.

[0087] Furthermore, regarding the parsing method of the speech signal, the in-vehicle module can still call appropriate acoustic and language models based on the application domain of speech recognition. These models are used to parse the speech signal and obtain initial keywords. Since the speech signal from the user's dialogue with the in-vehicle module is a response to the first intent, the initial keywords generated from parsing this dialogue are relevant to the first intent. However, the initial keywords generated from parsing irrelevant speech signals from conversations with other users are irrelevant to the first intent. Therefore, context detection using historical speech signals is necessary to ensure that the keywords generated based on the initial keywords are only relevant to the first intent, i.e., ignoring those initial keywords irrelevant to the first intent.

[0088] Context detection using historical speech signals can include detecting keywords related to historical speech signals in the initial keywords, and then using keywords related to the text sequence corresponding to the historical speech signals as the generated keywords. For example, parsing the speech signal "It's noon, will there be a parking problem? Yes, it's Sichuan restaurant A" yields initial keywords including "noon," "parking," "yes," and "Sichuan restaurant A." Among the initial keywords, the keywords related to the historical speech signal "Help me find a nearby Sichuan restaurant" include "yes" and "Sichuan restaurant A." Therefore, "noon" and "parking" can be ignored, and only "yes" and "Sichuan restaurant A" can be used as the generated keywords.

[0089] Of course, this embodiment does not limit the method of using historical speech signals for context detection. Regardless of the method used to detect and generate keywords, after the keywords are generated, the dynamic target language model can be triggered to parse the keywords, thereby determining the second intent and service content. See step 203 for details.

[0090] Step 203: Invoke the dynamic target language model to determine the second intent and service content. The front-end part of the dynamic target language model parses the second intent based on keywords, and the core part of the dynamic target language model parses the service content based on keywords.

[0091] As explained in step 201, the dynamic target language model includes a front-end and a core. Since the dynamic target language model obtains information from the response to the first intent, the second intent and service content determined by the dynamic target language model are both related to the first intent. The front-end is used to determine the description of the confirmatory information in the response. Therefore, by parsing keywords through the front-end, the confirmatory information in the keywords can be obtained, and thus the user's second intent can be obtained. Taking the above voice dialogue as an example, the response to the first intent is "Do you want to go to Sichuan restaurant A?" The parsed keywords are "Yes, that's it" and "Sichuan restaurant A". The front-end can then parse "Yes, that's it" from the keywords, thus obtaining the user's second intent as "go to Sichuan restaurant A". Furthermore, the core part also parses "Sichuan restaurant A" from the keywords, and combined with the current in-vehicle navigation scenario, the service content is obtained as "navigate to Sichuan restaurant A".

[0092] As can be seen, when the response to the first intent includes only one item, the user's second intent can be determined through the front-end. However, when the response to the first intent includes two or more items, it is necessary to determine the user's second intent through both the front-end and core components. For example, if the response to the first intent is "Which of the following do you want to choose? The first option is Sichuan Restaurant A, and the second option is Sichuan Restaurant B," and the parsed keywords are still "Yes, that's it" and "Sichuan Restaurant A," then the front-end can still parse the confirmatory information "Yes, that's it" from the keywords. However, "Yes, that's it" alone cannot determine whether the user's second intent is "Sichuan Restaurant A" or "Sichuan Restaurant B." Therefore, it is also necessary to parse "Sichuan Restaurant A" from the keywords through the core component, thereby ultimately determining that the user's second intent is "Go to Sichuan Restaurant A," and the service content is "Navigate to Sichuan Restaurant A."

[0093] It should be noted that if the confirmatory information obtained from keyword parsing in the front-end includes confirmation messages, such as "Yes, that's it" in the aforementioned voice dialogue, then the core part can be used to further parse the keywords to obtain the service content. If the confirmatory information obtained from keyword parsing in the front-end includes correction or cancellation messages, such as words like "no" or "incorrect," it indicates that the user does not accept the response and may not have responded to it. In this case, it is not necessary to obtain the service content through the core part. Instead, other response information is obtained, and a new dynamic target language model is acquired based on this other response information to complete the speech recognition.

[0094] Of course, in addition to the second intent and service content, calling the dynamic target language model can also obtain information such as the confidence level of the second intent and service content, and the silence signal segment in the speech signal. The confidence level is used to indicate the accuracy of the second intent and server content.

[0095] After obtaining the second intent and service content, the service indicated by the service content can be triggered. For example, if the service content in the above voice dialogue is "navigate to Sichuan Restaurant A", then executing the service content includes calling the navigation device to navigate the user from the current location (i.e., the location where the above voice dialogue occurred) to the location of "Sichuan Restaurant A".

[0096] In an optional implementation, before executing the service content, the method provided in this embodiment further includes: confirming a second intent, obtaining a confirmed second intent, and executing the confirmed second intent. In this implementation, considering that although the dynamic target language model is obtained or generated based on the response information to the first intent, the second intent and service content determined by the dynamic target language model may still be inconsistent with the first intent. Therefore, before executing the service content, the second intent is confirmed to ensure consistency between the second intent and the first intent. After obtaining the confirmed second intent, the confirmed second intent is then executed.

[0097] The second intention being consistent with the first intention includes, but is not limited to: the second intention corresponding to the response information of the first intention (e.g., the second intention "go to Sichuan restaurant A" corresponds to the response information "Sichuan restaurant A" of the first intention). Or, the second intention satisfying the conditional constraints included in the first intention (e.g., the second intention "go to Sichuan restaurant A" satisfies the distance condition constraint "nearby" included in the first intention).

[0098] Optionally, the method for confirming the second intent and obtaining the confirmed second intent includes: sending confirmation information of the second intent to the user, obtaining the second intent fed back by the user, and using the second intent fed back by the user as the confirmed second intent.

[0099] As explained above, the confidence level of the second intent and service content can be obtained through the dynamic target language model. Therefore, this embodiment can send different confirmation messages to the user based on different confidence levels to confirm the second intent. Taking the second intent as "go to Sichuan restaurant A" as an example, if the confidence level is higher than the threshold, it indicates that the second intent is relatively credible, and therefore an indirect confirmation method can be used. For example, the voice message "You have selected Sichuan restaurant A" is sent to the user as the confirmation message of the second intent, assuming the second intent is correct, to obtain the second intent returned by the user. If the confidence level is not higher than the threshold, it indicates that the credibility of the second intent is low, and therefore a direct confirmation method is used. For example, the voice message "Do you confirm selecting Sichuan restaurant A?" is sent to the user.

[0100] In both the indirect and direct confirmation methods described above, the confirmation information sent is voice confirmation. If the user's second intent cannot be obtained through voice confirmation, other forms of confirmation information, such as text confirmation, can be selected to confirm the second intent with the user. Optionally, the terminal displays a response message for the first intent to the user, allowing the user to select any response message. The intent indicated by the selected response message is then used as the confirmed second intent, and the confirmed second intent is executed, thereby completing the voice recognition.

[0101] Next, the above voice dialogue is expanded to obtain the following complex dialogue:

[0102] User; (Regarding the vehicle module) Could you help me find a Sichuan restaurant nearby?

[0103] Onboard module: (To user) Are you going to Sichuan restaurant A?

[0104] User: (To other users in the car) It's noon right now, will there be any parking problems? (To the vehicle module) Yes, it's Sichuan restaurant A, please find me a parking space.

[0105] It can be seen that in this complex dialogue, after the user expresses "Yes, it's Sichuan restaurant A", they also express the additional intent "Please find me a parking space", thus making the complex dialogue a multi-intent dialogue.

[0106] In one optional implementation, the dynamic target language model further includes a tail portion used to determine whether there is an additional intent. Therefore, the method provided in this embodiment further includes: invoking the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on keywords, thereby identifying each intent in the multi-intent dialogue described above.

[0107] In this implementation, in addition to obtaining the second intent through the front-end and the service content through the core, additional intents are also obtained by parsing keywords through the tail section. A schematic diagram of the front-end, core, and tail section can be found in [reference needed]. Figure 4 . Figure 4 In this context, "out-of-vocabulary" (OOV) refers to vocabulary outside the dictionary, which is used to derive words based on phoneme sequences. "eps" represents a jump edge, used to indicate optional parts.

[0108] Optionally, the concluding part includes concluding markers, which include, but are not limited to, words such as "again," "also," and "by the way." For example, in the multi-intent dialogue mentioned above, the concluding marker is "again." Since users' descriptions of concluding markers are often relatively fixed, a set including multiple concluding markers can be used as corpus to train the language model, and the trained language model can be used as the concluding part. Therefore, the dynamic target language model is invoked to determine the additional intent, wherein the concluding part of the dynamic target language model parses the additional intent based on keywords, including: the concluding part parses the reference concluding marker based on keywords, and the time point where the reference concluding marker is located; based on the reference concluding marker, combined with the first intent and the second intent, the dynamic target language model is updated to obtain the updated target language model; the updated target language model is invoked, and the additional intent is parsed based on the keywords and the time point where the reference concluding marker is located.

[0109] The reference suffix is ​​one of a set of multiple suffixes in the corpus. If there is no reference suffix, it means there is no additional intent, and the service indicated by the above service content can be provided directly; if there is a reference suffix, it means there is an additional intent, and in this case, the suffix also obtains the time point where the reference suffix is ​​located.

[0110] If a reference term exists, the language model is invoked based on the first and second intentions. This language model can be the language model of the domain in which the first and second intentions reside. For example, in the above multi-intention dialogue, the domain in which the first and second intentions reside is "navigation". In this case, the language model of the navigation domain can be obtained to replace the dynamic target language model, thereby obtaining the updated target language model.

[0111] After that, the updated target language model is invoked to parse the keywords after the time point where the reference ending marker word is located, so as to obtain the user's additional intent. For example, in the above-mentioned multi-intent voice dialogue, the reference ending marker word is "zai". The keywords included in the voice signal before the time point where "zai" is located, that is, "This time period is exactly noon, will there be a problem with parking? Yes, it is Sichuan Restaurant A", have been parsed by the front-end part and the core part in the dynamic target language model. Therefore, the updated target language model can be invoked to parse the keywords included in the voice signal after the time point where "zai" is located, that is, the keywords included in "help me find a parking space", so as to obtain the user's additional intent.

[0112] It should be noted that this embodiment further provides another method for updating the target language model as follows: after the language model is obtained according to the first intent and the second intent, the combined model of the language model and the ending part is taken as the updated target language model. Therefore, referring to Figure 5 , the updated target language model can cyclically and iteratively detect whether there are more additional intents after parsing one additional intent, which increases the number of intents that can be identified.

[0113] In addition, if the additional intent exists, after the additional intent is parsed and obtained by the updated target language model, the following method is adopted to execute the second intent. The method includes: if the additional intent exists, executing the service content and the additional intent. After the service content is obtained, the service content is not executed immediately, but the ending part is used to confirm whether there is an additional intent in the voice signal. If the additional intent exists, the additional intent is acquired, and finally the service content and the additional intent are executed. Only when it is confirmed through the ending information that there is no additional intent in the voice signal, the obtained service content is executed.

[0114] Furthermore, the execution of service content and additional intent includes: merging the execution of service content and additional intent, or executing service content and additional intent sequentially. For example, if the service content is "navigate to Sichuan restaurant A" and the additional intent is "play a song," then the additional intent can be executed during the execution of the service content, i.e., the service content and additional intent are merged. If the service content is "navigate to Sichuan restaurant A" and the additional intent is "find a parking space," then the service content and additional intent need to be executed sequentially. Additionally, different execution entities can execute different service content and additional intent. For example, it can be executed by a third-party cloud service, by an in-vehicle module, by an in-vehicle terminal, or by the vehicle manufacturer. The in-vehicle terminal can be any terminal on the vehicle other than the in-vehicle module, such as an in-vehicle display screen, in-vehicle air conditioner, in-vehicle speakers, etc. Of course, it can also be executed jointly by two or more execution entities among the third-party cloud service, in-vehicle module, in-vehicle terminal, and vehicle manufacturer; this embodiment does not limit this.

[0115] Next, the process of obtaining or generating a dynamic target language model based on the response information to the first intent, as mentioned in step 201 above, will be described in detail. Optionally, obtaining or generating a dynamic target language model based on the response information to the first intent includes: converting the response information to the first intent into a reference format to obtain the response information in the reference format, and obtaining or generating a dynamic target language model based on the response information in the reference format.

[0116] As explained above, a dynamic target language model includes at least a front-end and a core, and may also include a tail section. The front-end is used to determine the description of confirmatory information in the response. Similar to the tail section, since users' descriptions of confirmatory information in responses are relatively fixed, the front-end can be trained using a set of multiple confirmatory messages for confirmation, correction, or cancellation as corpus. This trained language model then serves as the front-end, enabling it to parse keywords to obtain confirmatory information such as confirmation, correction, or cancellation. The core section, however, requires obtaining the response information according to the reference format described above.

[0117] The response information may be provided by multiple vendors. Since different vendors may provide response information in different formats, it is necessary to convert the response information into a reference format to ensure uniformity and facilitate reception. For different application areas, the response information can be converted into different reference formats to ensure consistency within the same application area. For example, in the field of car navigation, the response information is often an address, so the address can be standardized to a format of country (or region), province (or state), city, district, street, and house number. As another example, in the field of points of interest (POIs), the response information is often related to POIs, so the response information can be standardized to a format of category name, address, contact number, and user reviews. Category names can include hotels, restaurants, shopping malls, museums, concert halls, cinemas, stadiums, hospitals, and pharmacies, etc.

[0118] Furthermore, before converting the response information into the reference format, the response information can be segmented and tagged to facilitate the conversion. Segmentation and tagging refers to breaking down a text string into words. If the resulting words include proper nouns, these proper nouns can also be tagged. Both segmentation and tagging can be implemented using artificial intelligence algorithms. In this embodiment, artificial intelligence algorithms include, but are not limited to, conditional random fields (CRF), long short-term memory (LSTM) networks, and hidden Markov models (HMMs).

[0119] In this embodiment, after obtaining the response information in the reference format, a dynamic target language model is further obtained or generated based on the response information in the reference format. Optionally, the methods for obtaining the target language model based on the response information in the reference format include the following three:

[0120] The first method involves converting the trained language model into a weighted finite state transition automaton model, and using the weighted finite state transition automaton model as the dynamic target language model. The trained language model is obtained by training on the response information of the reference format and the reference vocabulary.

[0121] The reference vocabulary includes, but is not limited to, the category names corresponding to words in the reference format response information, as well as referential expressions. Words in the reference format response information can be obtained through methods such as word segmentation and tagging, thereby further retrieving the corresponding category names. For example, the category name for "Sichuan Restaurant A" is "Restaurant". Referential expressions are used to refer to any item in the reference format response information. For example, when there are multiple reference format response information, referential expressions include "first item", "middle one", "second to last", "last item", etc.

[0122] The trained language model includes an initial language model trained on response information in the reference format and reference vocabulary as corpus. Optionally, the initial language model can be an N-gram model; a diagram of an N-gram model can be found here. Figure 6 The N-gram model assumes that the probability of a word appearing depends only on the N words preceding it, and is independent of other words. For example, when N is 3, the N-gram model is a 3rd-order model, where the probability of a word appearing depends on the two words preceding it, i.e., the probability of the i-th word X. i The probability of X occurring is P(X) i |X i-1 X i-2 Therefore, the N-gram model can statistically determine the probability that one word appears after another, that is, the probability that two words appear adjacently. By training the N-gram model on the corpus, a trained N-gram model is obtained, which has statistically determined the probability of adjacent occurrences of each word in the corpus.

[0123] Furthermore, the trained language model can be converted into a weighted finite state transducer (WFST) model. WFST converts the input phoneme sequence into words based on a dictionary, and then calculates the weights of adjacent occurrences of each word based on the probabilities of their occurrences as statistically analyzed by the trained language model. The core information is then output based on these weights. The core information can be viewed as a word sequence, and therefore, the probability of its occurrence is the product of the weights of all adjacent occurrences of the words within that word sequence.

[0124] Furthermore, conversion can expand the analytical scope of trained language models. Trained language models can parse keywords to obtain vocabulary and reference vocabulary in response information. The converted WFST, in addition to resolving vocabulary and reference vocabulary in response information, can also obtain two or three combinations of vocabulary, corresponding class names, or referential expressions in the response information. For example, WFST can parse combinations of referential expressions and corresponding class names such as "the restaurant in the middle," etc.

[0125] It can be seen that the WFST is the core part of the dynamic target language model. Subsequently, the WFST and the front-end (or WFST, the front-end, and the tail) can be used as the dynamic target language model.

[0126] The second method involves transforming the trained original model into a weighted finite state transition automaton model, using the weighted finite state transition automaton model as the first language model. The trained language model is obtained by training response information in a reference format with a length not less than the reference length. The second language model is obtained based on response information in a reference format with a length less than the reference length. The third language model is obtained based on reference vocabulary. The first, second, and third language models are then merged to obtain the overall language model, which is used as the dynamic target language model.

[0127] The reference vocabulary can be found in the explanation of the first acquisition method, and will not be repeated here. Compared to the first acquisition method, in the second acquisition method, response information shorter than the reference length and the reference vocabulary are not used as corpus; only response information of a length not shorter than the reference length is used as corpus. The trained language model is the initial language model trained using response information of a length not shorter than the reference length as corpus. The initial language model can still be an N-gram model. In an optional implementation, the reference length is 2, i.e., two words.

[0128] The reason for this is that the N-gram model uses a back-off algorithm. The back-off algorithm means that for word sequences that do not appear in the corpus, the probability of occurrence of a lower-order word sequence can be used as the probability of occurrence of that word sequence, ensuring that the N-gram model can output a result for any input phoneme sequence. For example, in the corpus of a 3rd-order model, there is no word sequence (X). i-2 X i-1 X i In the case of X, the model did not count the Xth number. i The probability of each word appearing, P(X) i |X i-1 X i-2 If the user used word sequence (X) i-2 X i-1 X i Then, based on the lower-order (2nd order) P(X) i |X i-1 To estimate P(X) i |X i-1 X i-2 ), thereby achieving (X) i-2 X i-1 X i Analysis of ).

[0129] The trained language model is designed to determine possible descriptions related to the response information. Users typically emit different speech signals to repeat the response information for different lengths, in order to confirm or select the response. For responses shorter than the reference length, users often repeat the entire response rather than just certain words. If an N-gram model, including a backoff algorithm, is trained using responses shorter than the reference length as corpus, the trained language model may statistically identify some word sequences with low probability of occurrence, thus affecting the parsing performance of the trained language model. The reference length can be set based on the scenario or experience, or it can be adjusted during the speech recognition process; this embodiment does not limit this.

[0130] For example, in a car navigation scenario, "Oriental Pearl Tower" could be a response message with a length of 1. If "Oriental Pearl Tower" is used as the corpus, the trained language model would provide word sequences such as "Dongming" and "Fangzhu," which obviously have a low probability of occurrence. Therefore, in this embodiment, a second language model is obtained based on response messages with a length lower than the reference length, without using a backoff algorithm. The second language model only parses response messages with a total length lower than the reference length within the keywords.

[0131] In addition, for reference vocabulary including the class names corresponding to the words and the referential expressions, the user's expression is relatively fixed, and the number of combinations of the class names corresponding to the words and the referential expressions is also relatively limited. Therefore, the class names corresponding to the words, the referential expressions, and the combinations of class names and referential expressions can be used as corpus to train a third language model without using the backoff algorithm.

[0132] For responses of length no less than the reference length, users often select a portion of the entire response for repetition. Therefore, these responses of length no less than the reference length can be used as corpus to train an N-gram model, resulting in a trained language model. This trained model is then converted to WFST (Warranty-Free Scripting), yielding a first language model using the backslip algorithm. This first language model can parse the entire response or combinations of words contained within the entire response. For example, in a car navigation scenario, assuming a reference length of 2, "No. 1, D Avenue, District C, City B, Province A" is a response longer than the reference length. Users might select word sequences such as "City B" and "No. 1, D Avenue" for repetition. Therefore, the first language model using the backslip algorithm can parse the keywords included in the user's repetitive speech signal.

[0133] After obtaining the first language model, the second language model, and the third language model, as follows: Figure 7As shown, the first language model, the second language model, and the third language model are merged to obtain the overall language model, which is the core part of the dynamic target language model. The overall language model and the front-end part (or the overall language model, the front-end part, and the tail part) constitute the dynamic target language model.

[0134] The third acquisition method is as follows: A word confusion network is obtained based on response information in a reference format with a length not less than the reference length. Each word in the word confusion network has a transition probability. A penalty weight is calculated for each word, and the word confusion network is transformed into a weighted finite state transition automaton model based on the penalty weight of each word. This weighted finite state transition automaton model is used as the first language model. A second language model is obtained based on response information in a reference format with a length less than the reference length. A third language model is obtained based on the reference words. The first, second, and third language models are merged to obtain the overall language model, which is used as the dynamic target language model.

[0135] For an explanation of the reference vocabulary, please refer to the first acquisition method. For obtaining the second language model based on response information shorter than the reference length, and for obtaining the third language model based on the reference vocabulary, please refer to the second acquisition method; these details will not be repeated here. Next, the process of obtaining the first language model will be explained:

[0136] The method for obtaining a word confusion network includes: performing word alignment on words of the same category in responses of at least the reference length, and incrementing the number of categories by one to obtain the number of states in the word confusion network. Then, states are connected by arcs, each arc containing a word and its corresponding transition probability. The transition probability indicates the frequency of the word's occurrence in its category, and the sum of the transition probabilities across all arcs between two adjacent states is 1.

[0137] Furthermore, considering that users often skip words when repeating long replies, it's necessary to add a skip edge between every two states in the word confusion network to parse the keywords included in the user's skipped word speech signal. For example, in... Figure 8 In the word confusion network shown, eps represents a jump edge, F i Used to distinguish different categories.

[0138] Next, the penalty weight for each word is calculated, and the word confusion network is converted into a WFST based on the penalty weights to obtain the first language model. It should be noted that when parsing keywords using the first language model, the model calculates the penalty weights for the various word sequences that the phoneme sequence of the speech signal may correspond to. The penalty weight of a word sequence is equal to the product of the penalty weights of the words included in that word sequence, and the word sequence with the smallest penalty weight value is output. Optionally, the methods for calculating the penalty weight of each word include, but are not limited to, the following three:

[0139] The first calculation method: For any word, use the negative logarithm of the transition probability of that word as the penalty weight.

[0140] As explained above, the transition probability of a word is used to indicate the frequency of the word's occurrence in its category. The higher the frequency of a word's occurrence in its category, the greater the transition probability, and the smaller the negative logarithm of the transition probability. In other words, the penalty weight is inversely proportional to the frequency of occurrence, so that the target language model can better parse words that occur frequently in their category.

[0141] The second calculation method: For any word, the logarithm of the number of responses containing the reference format of that word is used as the penalty weight.

[0142] When users are summarizing potential responses from multiple lengthy replies, they tend to choose words that are clearly distinct from the others—words with strong discernibility. For example, when summarizing "No. 1, D Avenue, District C, City B, Province A" versus "No. 2, F Road, District E, City B, Province A," a user typically won't choose "Province A" or "City B," which appear in both, but rather "District C" or "No. 1, D Avenue," which only appear in the first response.

[0143] In this embodiment, the distinguishability of words is defined according to the following formula:

[0144]

[0145] The inverse presence frequency (IPF) is used to indicate the distinctiveness of words; the higher the IPF value, the more distinctive the words are. Fi Used to represent category F i In the context of word T, N is the total number of items in the response information of the reference format, and n is the number of words containing word T. FiThe number of responses to each reference format is considered. It can be seen that the more responses containing the reference format of a particular word, the lower the IPF value, and the weaker the word's discriminative power.

[0146] Considering the jump edges, the total number of response messages in the reference format changes from N to (N+1), IPF(T) Fi The expression is updated to be represented by the following formula:

[0147]

[0148] Furthermore, assuming that skip edges are not distinctive, meaning they appear in every reference format's response information, then the IPF (skip) of a skip edge can be expressed as:

[0149]

[0150] In this embodiment, the above IPF(skip) can also be rewritten to avoid the IPF value of skip edges being always equal to 0. The rewritten IPF(skip) is expressed by the following formula:

[0151]

[0152] Furthermore, based on IPF(T) Fi The penalty weight of a word is defined according to the following formula: Penalty(T) Fi It can be defined that the penalty weight of a word is the logarithmic value of the number of response messages containing that word in the reference format:

[0153]

[0154] Accordingly, the penalty weight Penalty(skip) for skip edges can be defined as:

[0155]

[0156] It can be seen that in this calculation method, words with strong discriminative power, that is, words with fewer responses containing the reference format of the word, are given a smaller penalty weight, so that the target language model can better parse these words with strong discriminative power.

[0157] The third calculation method: For any word, the logarithm of the number of times the word appears in the response information of each reference format is used as the penalty weight.

[0158] The third calculation method can still define the distinctiveness of words according to the following formula:

[0159]

[0160] However, unlike the second calculation method, N represents the total number of words contained in the response information of each reference format, and n represents the number of words T. Fi The number of times it appears in the response information of each reference format. Then, based on the formulas in the second calculation method, the vocabulary T can be defined. Fi Penalty (T) Fi The penalty weight for a word can be defined as follows, which is the logarithmic value of the number of times the word appears in the response information of each reference format:

[0161] Penalty(T Fi ) = Log(n)

[0162] It can be seen that for words with strong discriminative power, that is, words that appear less frequently, the penalty probability of the word is smaller, so that the dynamic target language model can better parse out words with strong discriminative power.

[0163] Regardless of the computational method used to obtain the first language model, after obtaining the first language model, the first language model, the second language model, and the third language model can be merged to obtain the overall language model, which is the core part of the dynamic target language model. Subsequently, the overall language model and the front-end part (or the overall language model, the front-end part, and the tail part) can be used as the dynamic target language model.

[0164] In summary, this embodiment of the application acquires or generates a dynamic target language model including a front-end and a core part based on the response information of the first intent, parses the speech signal to obtain keywords, and then calls the dynamic target language model to parse the keywords to obtain the second intent and service content. Since the dynamic target language model is acquired based on the response information of the first intent, the second intent and service content obtained by parsing through the dynamic target language model are all related to the first intent. Therefore, this embodiment of the application achieves the ignoring of speech unrelated to the first intent, avoids the provided service content deviating from user needs, has good recognition effect, and improves the user experience.

[0165] In addition, embodiments of this application also use the tail portion in the dynamic target language model to determine whether there are multiple intentions in the voice signal, so as to provide the service indicated by each intention of the user, thereby further improving the user experience.

[0166] like Figure 9 As shown in the figure, this application embodiment also provides a speech recognition device, which includes:

[0167] The first acquisition module 901 is used to acquire or generate a dynamic target language model based on the response information to the first intent. The dynamic target language model includes a front-end part and a core part. The core part is used to determine possible descriptions related to the response information, and the front-end part is used to determine the description of the confirmatory information of the response information.

[0168] The second acquisition module 902 is used to acquire speech signals and parse speech signals to generate keywords.

[0169] The first determination module 903 is used to call the dynamic target language model to determine the second intent and service content. The front-end part of the dynamic target language model parses the second intent based on keywords, and the core part of the dynamic target language model parses the service content based on keywords.

[0170] Optionally, the dynamic target language model further includes a tail section used to confirm the presence of additional intents, and the device further includes:

[0171] The second determination module is used to call the dynamic target language model to determine the additional intent. The end part of the dynamic target language model parses the additional intent based on the keywords.

[0172] Optionally, the concluding part may include a concluding marker word;

[0173] The second determining module is used to parse the reference ending marker and the time point where the reference ending marker is located based on the keywords in the end part; based on the reference ending marker, combined with the first intent and the second intent, the dynamic target language model is updated to obtain the updated target language model; the updated target language model is called to parse the additional intent based on the keywords and the time point where the reference ending marker is located.

[0174] Optionally, the device further includes:

[0175] The caching module is used to cache historical voice signals;

[0176] The second acquisition module 902 is used to parse the speech signal and generate keywords after performing context detection using historical speech signals.

[0177] Optionally, the device further includes: a confirmation module for confirming the second intent and obtaining the confirmed second intent.

[0178] Optionally, the confirmation module is used to send confirmation information of the second intent to the user, obtain the second intent fed back by the user, and use the second intent fed back by the user as the confirmed second intent.

[0179] Optionally, the first acquisition module 901 is used to convert the response information of the first intent into a reference format, obtain the response information of the reference format, and acquire or generate a dynamic target language model based on the response information of the reference format.

[0180] Optionally, the first acquisition module is used to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as a dynamic target language model, wherein the trained language model is obtained by training response information in a reference format and reference vocabulary.

[0181] Optionally, the first acquisition module 901 is used to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as the first language model, wherein the trained language model is obtained by training response information of a reference format with a length not less than the reference length; to acquire a second language model based on response information of a reference format with a length less than the reference length, and to acquire a third language model based on reference vocabulary; to merge the first language model, the second language model and the third language model to obtain a total language model, and to use the total language model as the dynamic target language model.

[0182] Optionally, the first acquisition module 901 includes:

[0183] The first acquisition unit is used to acquire a word confusion network based on response information in a reference format with a length not less than the reference length. Each word in the word confusion network has a transition probability.

[0184] The computational unit is used to calculate the penalty weight for each word, and transforms the word confusion network into a weighted finite state transition automaton model based on the penalty weight of each word, and uses the weighted finite state transition automaton model as the first language model.

[0185] The second acquisition unit is used to acquire a second language model based on response information in a reference format with a length shorter than the reference length, and to acquire a third language model based on reference vocabulary.

[0186] The merging unit is used to merge the first language model, the second language model, and the third language model to obtain the overall language model, which is then used as the dynamic target language model.

[0187] Optionally, the calculation unit is used to use the negative logarithm of the transition probability of any word as a penalty weight.

[0188] Optionally, the calculation unit is used to use the logarithm of the number of response messages containing the reference format of the word as a penalty weight for any given word.

[0189] Optionally, the calculation unit is used to use the logarithm of the number of times the word appears in the response information of the reference format as a penalty weight for any word.

[0190] In summary, this embodiment of the application acquires or generates a dynamic target language model including a front-end and a core part based on the response information of the first intent, parses the speech signal to obtain keywords, and then calls the dynamic target language model to parse the keywords to obtain the second intent and service content. Since the dynamic target language model is acquired based on the response information of the first intent, the second intent and service content obtained by parsing through the dynamic target language model are all related to the first intent. Therefore, this embodiment of the application achieves the ignoring of speech unrelated to the first intent, avoids the provided service content deviating from user needs, has good recognition effect, and improves the user experience.

[0191] In addition, embodiments of this application also use the tail portion in the dynamic target language model to determine whether there are multiple intentions in the voice signal, so as to provide the service indicated by each intention of the user, thereby further improving the user experience.

[0192] It should be understood that the above Figure 9 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0193] This application also provides a speech recognition device, which includes a memory and a processor. The memory stores at least one instruction, which is loaded and executed by the processor to implement the speech recognition method provided in this application. The method includes: acquiring or generating a dynamic target language model based on response information to a first intent. The dynamic target language model includes a front-end part and a core part. The core part is used to determine possible descriptions related to the response information, and the front-end part is used to determine a description of confirmatory information about the response information. Acquiring a speech signal and parsing the speech signal to generate keywords. Calling the dynamic target language model to determine a second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords.

[0194] Optionally, the dynamic target language model also includes a tail part, which is used to confirm whether there is an additional intent. The method further includes: calling the dynamic target language model to determine the additional intent, and the tail part of the dynamic target language model parsing out the additional intent based on the keywords.

[0195] Optionally, the tail portion includes a tail marker; the dynamic target language model is invoked to determine the additional intent, and the tail portion of the dynamic target language model is parsed from the keywords to derive the additional intent, including: the tail portion is parsed from the keywords to derive a reference tail marker, and the time point where the reference tail marker is located; based on the reference tail marker, combined with the first intent and the second intent, the dynamic target language model is updated to obtain the updated target language model; the updated target language model is invoked to derive the additional intent based on the keywords and the time point where the reference tail marker is located.

[0196] Optionally, before acquiring the speech signal, the method further includes: caching historical speech signals; and parsing the speech signal to generate keywords, including: parsing the speech signal and generating keywords after performing context detection using historical speech signals.

[0197] Optionally, after calling the dynamic target language model to determine the second intent and service content, the method further includes: confirming the second intent and obtaining the confirmed second intent.

[0198] Optionally, confirming the second intent and obtaining the confirmed second intent includes: sending confirmation information of the second intent to the user, obtaining the second intent fed back by the user, and using the second intent fed back by the user as the confirmed second intent.

[0199] Optionally, obtaining or generating a dynamic target language model based on the response information to the first intent includes: converting the response information to the first intent into a reference format to obtain the response information in the reference format, and obtaining or generating a dynamic target language model based on the response information in the reference format.

[0200] Optionally, obtaining or generating a dynamic target language model based on the response information in the reference format includes: converting the trained language model into a weighted finite state transition automaton model, and using the weighted finite state transition automaton model as the dynamic target language model, wherein the trained language model is obtained by training the response information in the reference format and the reference vocabulary.

[0201] Optionally, obtaining or generating a dynamic target language model based on response information in a reference format includes: converting a trained language model into a weighted finite state transition automaton model, using the weighted finite state transition automaton model as a first language model, wherein the trained language model is obtained by training response information in a reference format with a length not less than the reference length; obtaining a second language model based on response information in a reference format with a length less than the reference length, and obtaining a third language model based on reference vocabulary; merging the first language model, the second language model, and the third language model to obtain a total language model, and using the total language model as the dynamic target language model.

[0202] Optionally, a dynamic target language model is obtained or generated based on the response information in the reference format, including: obtaining a word confusion network based on the response information in the reference format with a length not less than the reference length, wherein each word in the word confusion network has a transition probability; calculating the penalty weight of each word, converting the word confusion network into a weighted finite state transition automaton model based on the penalty weight of each word, and using the weighted finite state transition automaton model as the first language model; obtaining a second language model based on the response information in the reference format with a length less than the reference length, and obtaining a third language model based on the reference words; merging the first language model, the second language model, and the third language model to obtain a total language model, and using the total language model as the dynamic target language model.

[0203] Optionally, the penalty weight for each word is calculated, including: for any word, using the negative logarithm of the word's transition probability as the penalty weight.

[0204] Optionally, the penalty weight for each word is calculated, including: for any word, using the logarithm of the number of response items containing the reference format of the word as the penalty weight.

[0205] Optionally, the penalty weight for each word is calculated, including: for any word, using the logarithm of the number of times the word appears in the response information of the reference format as the penalty weight.

[0206] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement a speech recognition method provided in this application. The method includes: acquiring or generating a dynamic target language model based on response information to a first intent, the dynamic target language model including a front-end part and a core part, the core part being used to determine possible descriptions related to the response information, and the front-end part being used to determine a description of confirmatory information about the response information; acquiring a speech signal, parsing the speech signal to generate keywords; and calling the dynamic target language model to determine a second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords.

[0207] Optionally, the dynamic target language model also includes a tail part, which is used to confirm whether there is an additional intent. The method further includes: calling the dynamic target language model to determine the additional intent, and the tail part of the dynamic target language model parsing out the additional intent based on the keywords.

[0208] Optionally, the tail portion includes a tail marker; the dynamic target language model is invoked to determine the additional intent, and the tail portion of the dynamic target language model is parsed from the keywords to derive the additional intent, including: the tail portion is parsed from the keywords to derive a reference tail marker, and the time point where the reference tail marker is located; based on the reference tail marker, combined with the first intent and the second intent, the dynamic target language model is updated to obtain the updated target language model; the updated target language model is invoked to derive the additional intent based on the keywords and the time point where the reference tail marker is located.

[0209] Optionally, before acquiring the speech signal, the method further includes: caching historical speech signals; and parsing the speech signal to generate keywords, including: parsing the speech signal and generating keywords after performing context detection using historical speech signals.

[0210] Optionally, after calling the dynamic target language model to determine the second intent and service content, the method further includes: confirming the second intent and obtaining the confirmed second intent.

[0211] Optionally, confirming the second intent and obtaining the confirmed second intent includes: sending confirmation information of the second intent to the user, obtaining the second intent fed back by the user, and using the second intent fed back by the user as the confirmed second intent.

[0212] Optionally, obtaining or generating a dynamic target language model based on the response information to the first intent includes: converting the response information to the first intent into a reference format to obtain the response information in the reference format, and obtaining or generating a target language model based on the response information in the reference format.

[0213] Optionally, obtaining or generating a dynamic target language model based on the response information in the reference format includes: converting the trained language model into a weighted finite state transition automaton model, and using the weighted finite state transition automaton model as the dynamic target language model, wherein the trained language model is obtained by training the response information in the reference format and the reference vocabulary.

[0214] Optionally, obtaining or generating a dynamic target language model based on response information in a reference format includes: converting a trained language model into a weighted finite state transition automaton model, using the weighted finite state transition automaton model as a first language model, wherein the trained language model is obtained by training response information in a reference format with a length not less than the reference length; obtaining a second language model based on response information in a reference format with a length less than the reference length, and obtaining a third language model based on reference vocabulary; merging the first language model, the second language model, and the third language model to obtain a total language model, and using the total language model as the dynamic target language model.

[0215] Optionally, a dynamic target language model is obtained or generated based on the response information in the reference format, including: obtaining a word confusion network based on the response information in the reference format with a length not less than the reference length, wherein each word in the word confusion network has a transition probability; calculating the penalty weight of each word, converting the word confusion network into a weighted finite state transition automaton model based on the penalty weight of each word, and using the weighted finite state transition automaton model as the first language model; obtaining a second language model based on the response information in the reference format with a length less than the reference length, and obtaining a third language model based on the reference words; merging the first language model, the second language model, and the third language model to obtain a total language model, and using the total language model as the dynamic target language model.

[0216] Optionally, the penalty weight for each word is calculated, including: for any word, using the negative logarithm of the word's transition probability as the penalty weight.

[0217] Optionally, the penalty weight for each word is calculated, including: for any word, using the logarithm of the number of response items containing the reference format of the word as the penalty weight.

[0218] Optionally, the penalty weight for each word is calculated, including: for any word, using the logarithm of the number of times the word appears in the response information of the reference format as the penalty weight.

[0219] This application also provides a chip, including a processor, which is used to call and execute instructions stored in a memory, causing a communication device equipped with the chip to perform any of the above-described speech recognition methods.

[0220] This application embodiment also provides another chip, including: an input interface, an output interface, a processor, and a memory. The input interface, the output interface, the processor, and the memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute any of the above-described speech recognition methods.

[0221] Optionally, there may be one or more processors and one or more memories.

[0222] Alternatively, the memory can be integrated with the processor, or the memory can be set up separately from the processor.

[0223] In the specific implementation process, the memory can be integrated with the processor on the same chip, or it can be set on different chips. The embodiments of this application do not limit the type of memory or the way the memory and processor are set.

[0224] It should be understood that the aforementioned processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0225] Furthermore, in an alternative embodiment, the memory described above may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0226] The memory can be volatile or non-volatile, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0227] This application provides a computer program that, when executed by a computer, causes the processor or computer to perform the corresponding steps and / or processes in the above method embodiments.

[0228] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0229] The above description is merely an embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A speech recognition method, characterized in that, The method includes: A dynamic target language model is obtained or generated based on the response information to the first intent. The dynamic target language model includes a front-end part and a core part. The core part is used to determine possible descriptions related to the response information, and the front-end part is used to determine descriptions of confirmatory information about the response information. Acquire speech signals and generate keywords by parsing the speech signals based on historical speech signals; The dynamic target language model is invoked to determine the second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords; The step of acquiring or generating a dynamic target language model based on the response information to the first intent includes: The response information of the first intent is converted into a reference format to obtain the response information of the reference format. The dynamic target language model is obtained or generated based on the response information of the reference format. The step of obtaining or generating the dynamic target language model based on the response information in the reference format includes: The trained language model is transformed into a weighted finite state transition automaton model, and the weighted finite state transition automaton model is used as the first language model, wherein the trained language model is obtained by training response information in a reference format with a length not less than the reference length. The second language model is obtained from the response information in the reference format whose length is shorter than the reference length, and the third language model is obtained from the reference vocabulary. The first language model, the second language model, and the third language model are merged to obtain a total language model, which is then used as the dynamic target language model.

2. The method according to claim 1, characterized in that, The dynamic target language model further includes a tail section, which is used to confirm whether there is any additional intent. The method further includes: The dynamic target language model is invoked to determine the additional intent, and the tail portion of the dynamic target language model is parsed to extract the additional intent based on the keywords.

3. The method according to claim 2, characterized in that, The closing part includes closing marker words; The step of calling the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on the keywords, includes: The concatenation part parses the reference concatenation marker word and the time point where the reference concatenation marker word is located based on the keyword; Based on the reference suffix, and combined with the first intent and the second intent, the dynamic target language model is updated to obtain the updated target language model; The updated target language model is invoked to parse the additional intent based on the time points of the keywords and the reference suffixes.

4. The method according to any one of claims 1-3, characterized in that, Before acquiring the speech signal, the method further includes: Cache historical voice signals; The process of parsing the speech signal to generate keywords includes: The speech signal is parsed, and the keywords are generated after context detection using the historical speech signal.

5. The method according to claim 1, characterized in that, The step of obtaining or generating the dynamic target language model based on the response information in the reference format includes: The trained language model is transformed into a weighted finite state transition automaton model, and the weighted finite state transition automaton model is used as the dynamic target language model, wherein the trained language model is obtained by training the response information of the reference format and the reference vocabulary.

6. A speech recognition device, characterized in that, The device includes: The first acquisition module is used to acquire or generate a dynamic target language model based on the response information to the first intent. The dynamic target language model includes a front-end part and a core part. The core part is used to determine possible descriptions related to the response information, and the front-end part is used to determine descriptions of confirmatory information about the response information. The second acquisition module is used to acquire voice signals and generate keywords by parsing the voice signals based on historical voice signals. The first determining module is used to call the dynamic target language model to determine the second intent and service content, wherein the front-end part of the dynamic target language model parses the second intent based on the keywords, and the core part of the dynamic target language model parses the service content based on the keywords; The first acquisition module is used to convert the response information of the first intent into a reference format to obtain the response information in the reference format, and to acquire or generate the dynamic target language model based on the response information in the reference format. The first acquisition module is used to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as the first language model, wherein the trained language model is obtained by training response information of a reference format with a length not less than the reference length; to acquire a second language model based on response information of a reference format with a length less than the reference length, and to acquire a third language model based on reference vocabulary; to merge the first language model, the second language model and the third language model to obtain a total language model, and to use the total language model as the dynamic target language model.

7. The apparatus according to claim 6, characterized in that, The dynamic target language model further includes a tailing part, which is used to confirm whether there is additional intent; the device further includes: The second determining module is used to call the dynamic target language model to determine the additional intent, wherein the tail portion of the dynamic target language model parses the additional intent based on the keywords.

8. The apparatus according to claim 7, characterized in that, The closing part includes closing marker words; The second determining module is used to parse a reference suffix word and the time point where the reference suffix word is located based on the keyword in the end part; update the dynamic target language model based on the reference suffix word, combined with the first intent and the second intent, to obtain an updated target language model; and call the updated target language model to parse the additional intent based on the keyword and the time point where the reference suffix word is located.

9. The apparatus according to any one of claims 6-8, characterized in that, The device further includes: The caching module is used to cache historical voice signals; The second acquisition module is used to parse the speech signal and generate the keywords after performing context detection using the historical speech signal.

10. The apparatus according to claim 6, characterized in that, The first acquisition module is used to convert the trained language model into a weighted finite state transition automaton model, and use the weighted finite state transition automaton model as the dynamic target language model, wherein the trained language model is obtained by training the response information of the reference format and the reference vocabulary.

11. A speech recognition device, characterized in that, The device includes a memory and a processor; the memory stores at least one instruction, which is loaded and executed by the processor to implement the speech recognition method as described in any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the speech recognition method as described in any one of claims 1 to 5.

Images