Decoder-based speech wake-up method and related devices thereof

Abstract

The embodiment of the application belongs to the field of artificial intelligence, is applied to the field of smart community, and relates to a voice wake-up method based on a decoder and related equipment thereof, which comprises receiving an initial voice wake-up model, the output layer of the initial voice wake-up model comprising a silence pdf node, a useless pdf node and a wake-up word pdf node, the silence pdf node corresponding to a pdf corresponding to silence, the wake-up word pdf node corresponding to a pdf corresponding to a wake-up word, and the useless pdf node corresponding to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to silence; performing an alignment operation on a voice training data set to obtain a voice frame training data set, and training the initial voice wake-up model to obtain a target voice wake-up model; inputting a voice to be recognized into the target voice wake-up model to obtain a target pdf probability; inputting the target pdf probability into a decoder to obtain a decoding result; and if the decoding result is a wake-up word, performing a wake-up operation. The target voice wake-up model can be stored in a blockchain. The application improves the accuracy of voice wake-up.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a decoder-based voice wake-up method and a related device thereof. BACKGROUND

[0002] A voice wake-up system has been widely applied to many intelligent devices, such as smart speakers. It is the front end of voice interaction. Unlike other more complex voice interaction functions, the voice wake-up will always be executed in the background, and it will detect in real time whether the user has said a wake-up word. Only after detecting that the user has said the wake-up word, will the subsequent more complex voice interaction logic be triggered.

[0003] The existing voice wake-up system generally needs a large number of wake-up word voices for model training after determining the wake-up word. This brings great inconvenience because such specific word recording data is very difficult to obtain, and the voice data that meets the requirements accounts for a very small proportion in ordinary voice data sets. Especially if the wake-up word is a self-made word, it is even more difficult to obtain. Thus, it is necessary to specially record the wake-up word, which is high in cost on the one hand, and the quality of the specially recorded data set is not as good as that of the data collected in real scenarios on the other hand, for example, it is difficult to cover complex noise scenarios and difficult to ensure the diversity of speakers, etc. The model performance is greatly affected by the recording quality. Moreover, once the wake-up word needs to be changed, new wake-up words need to be re-recorded, and even a large amount of continuous speech recognition training data cannot be utilized.

[0004] In addition, the existing voice wake-up system generally directly connects a confidence calculation module after a neural network model to determine whether to wake up. This method also has defects, and it is difficult to distinguish the case where the wake-up word is reversed, for example: the wake-up word is "Hello Xiaohai", but the user says "Xiaohai, hello" to wake up. Based on the above two cases, the accuracy of the existing voice wake-up system is not high. SUMMARY

[0005] The purpose of the embodiments of the present application is to propose a decoder-based voice wake-up method, device, computer equipment and storage medium, which improves the accuracy of voice wake-up.

[0006] In order to solve the above technical problems, the embodiments of the present application provide a decoder-based voice wake-up method, which adopts the following technical solutions:

[0007] A decoder-based voice wake-up method, comprising the following steps:

[0008] receive a voice training data set and an initial voice wake-up model, wherein an output layer of the initial voice wake-up model includes a silence pdf node, a useless pdf node and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf corresponding to silence in voice, the wake-up word pdf node corresponds to a pdf corresponding to a wake-up word in voice, and the useless pdf node corresponds to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in voice;

[0009] perform an alignment operation on the voice training data set to obtain a voice frame training data set, train the initial voice wake-up model through the voice frame training data set and the voice training data set, and obtain a target voice wake-up model;

[0010] receive a voice to be recognized, input the voice to be recognized frame by frame into the target voice wake-up model, and obtain a target pdf probability corresponding to each frame of output;

[0011] input each target pdf probability into a pre-trained decoder to obtain a decoding result output;

[0012] determine whether the decoding result is a wake-up word, and if the decoding result is a wake-up word, perform a wake-up operation.

[0013] Further, before the step of inputting each target pdf probability into a pre-trained decoder to obtain a decoding result output, the method further comprises:

[0014] constructing a target decoding graph through a preset grammar;

[0015] The step of inputting each target pdf probability into a pre-trained decoder to obtain a decoding result output comprises:

[0016] inputting each target pdf probability into a pre-trained decoder, the decoder finding an optimal path of the target pdf probability sequence in the target decoding graph to obtain a decoding result output.

[0017] Further, the step of constructing a target decoding graph through a preset grammar comprises:

[0018] constructing a G.fst graph through a preset grammar, wherein the G.fst graph has and only has one valid path, and the valid path corresponds to one wake-up word;

[0019] performing a plurality of combination operations on the G.fst graph and a preset L.fst graph to obtain a wake-up HCLG.fst graph;

[0020] An edge with an empty output is added in parallel in the wake-up HCLG.fst graph to absorb silence and useless pronunciation, and the target decoding graph is obtained.

[0021] Further, the output layer of the continuous speech recognition model includes a plurality of pdf nodes, the pre-trained continuous speech recognition model performs alignment operation on the speech training data set, and the step of obtaining the speech frame training data set includes:

[0022] The pre-trained continuous speech recognition model performs frame-level labeling on the speech training data set to obtain a labeled data set.

[0023] According to a preset mapping relationship table, the training label of the labeled data set is replaced by a corresponding target label to obtain the speech frame training data set, wherein the mapping relationship table includes a corresponding relationship between the name of the pdf node of the continuous speech recognition model and the target label, and the target label is the name of the silence pdf node, the name of the useless pdf node and the name of the wake-up word pdf node.

[0024] Further, the step of determining whether the decoding result is a wake-up word includes:

[0025] If the decoding result is not empty, it is determined that the decoding result is a wake-up word.

[0026] Further, the step of training the initial speech wake-up model based on the speech frame training data set and the speech training data set to obtain a target speech wake-up model includes:

[0027] The labeled data of the speech frame training data set is used as the training label of the corresponding data of the speech training data set to obtain a target training data set, and the initial speech wake-up model is trained based on the target training data set to obtain the target speech wake-up model.

[0028] Further, the step of inputting each target pdf probability into a pre-trained decoder to obtain an output decoding result includes:

[0029] Each target pdf probability is input into a pre-trained static decoder based on a weighted finite state transducer to obtain an output decoding result.

[0030] To solve the above technical problems, the embodiment of the application also provides a speech wake-up device based on a decoder, which adopts the technical scheme as follows:

[0031] A speech wake-up device based on a decoder, comprising:

[0032] The receiving module is configured to receive a voice training data set and an initial voice wake-up model, wherein an output layer of the initial voice wake-up model comprises a silence pdf node, a useless pdf node and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf corresponding to silence in voice, the wake-up word pdf node corresponds to a pdf corresponding to a wake-up word in voice, and the useless pdf node corresponds to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in voice;

[0033] The training module is configured to perform an alignment operation on the voice training data set to obtain a voice frame training data set, and train the initial voice wake-up model based on the voice frame training data set and the voice training data set to obtain a target voice wake-up model.

[0034] The input module is configured to receive a voice to be recognized, input the voice to be recognized into the target voice wake-up model frame by frame, and obtain a target pdf probability corresponding to each output frame.

[0035] The output module is configured to input each target pdf probability into a pre-trained decoder to obtain a decoding result.

[0036] The judgment module is configured to determine whether the decoding result is a wake-up word, and if the decoding result is a wake-up word, perform a wake-up operation, and if the decoding result is not a wake-up word, determine not to wake up.

[0037] To solve the above technical problems, the embodiment of the present application also provides a computer device, which adopts the technical scheme as follows:

[0038] A computer device comprises a memory and a processor, the memory stores computer readable instructions, and the processor executes the computer readable instructions to realize the steps of the above-mentioned decoder-based voice wake-up method.

[0039] To solve the above technical problems, the embodiment of the present application also provides a computer readable storage medium, which adopts the technical scheme as follows:

[0040] A computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by a processor to realize the steps of the above-mentioned decoder-based voice wake-up method.

[0041] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0042] The present application uses an existing voice training data set, without the need to specially record the wake-up word voice, greatly reducing the data cost; the wake-up word can be arbitrarily replaced, improving the flexibility of the model; at the same time, the output layer of the initial voice wake-up model of the present application is provided with fine-grained silence pdf nodes, useless pdf nodes and multiple wake-up word pdf nodes, the silence pdf node corresponds to the pdf corresponding to the silence in the voice, the wake-up word pdf node corresponds to the pdf corresponding to the wake-up word in the voice, and the useless pdf node corresponds to the pdf in the voice except the pdf corresponding to the wake-up word and the pdf corresponding to the silence, while realizing finer-grained voice recognition, the complex noise, speaker and other information in the voice training data set can be fully utilized, and the accuracy of the voice wake-up system can be improved to a certain extent. In the wake-up logic part, the decoder is used for decoding, which can solve the problem of false wake-up caused by reversed word order, and further improve the accuracy of voice wake-up. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the scheme in the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0044] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0045] Figure 2 is a flowchart of one embodiment of the decoder-based voice wake-up method according to the present application;

[0046] Figure 3a is an architecture diagram of a continuous speech recognition model of one embodiment of the decoder-based voice wake-up method according to the present application;

[0047] Figure 3b is an architecture diagram of a target voice wake-up model of one embodiment of the decoder-based voice wake-up method according to the present application;

[0048] Figure 4 is a structure diagram of one embodiment of the decoder-based voice wake-up device according to the present application;

[0049] Figure 5 is a structure diagram of one embodiment of the computer device according to the present application.

[0050] Label: 200, computer device; 201, memory; 202, processor; 203, network interface; 300, decoder-based voice wake-up device; 301, receiving module; 302, training module; 303, input module; 304, output module; 305, judging module. DETAILED DESCRIPTION

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application; the present application will be described with reference to the drawings in which is shown by way of illustration the application in accordance with embodiments described herein using exemplary language. No limitation on the scope of the present application is intended by the use of such language. Numerous specific details are set forth in order to provide a thorough understanding of the application often without some of these specific details. Otherwise, well-known methods, procedures, components, and circuits have been described without detail in order not to unnecessarily obscure aspects of the application.

[0052] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to a single alternative embodiment.

[0053] For those skilled in the art, better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings.

[0054] As shown in Figure 1 The system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0055] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0056] The terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and the like.

[0057] The server 105 can be a server providing various services, for example, a background server providing support for a page displayed on the terminal devices 101, 102, 103.

[0058] It should be noted that the decoder-based voice wake-up method provided by the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the decoder-based voice wake-up apparatus is generally arranged in a server / terminal device.

[0059] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0060] With reference to Figure 2 , a flowchart of one embodiment of the decoder-based voice wake-up method according to the present application is shown. The decoder-based voice wake-up method includes the following steps:

[0061] S1: receiving a voice training data set and an initial voice wake-up model, wherein an output layer of the initial voice wake-up model includes a silence pdf node, a useless pdf node, and a plurality of wake-up word pdf nodes, the silence pdf node corresponding to a pdf of silence in a voice, the wake-up word pdf node corresponding to a pdf of a wake-up word in the voice, and the useless pdf node corresponding to a pdf other than the pdf of the wake-up word and the pdf of the silence in the voice.

[0062] In the embodiment, the output layer of the initial speech wake-up model of the application includes a silence pdf node, a useless pdf node and a plurality of wake-up word pdf nodes. Pdf is a concept closely related to pronunciation, and the full name of pdf in English is probability distribution function, that is, probability distribution function. In fact, pdf is not a function, but only a concept defined by kaldi, which is used to represent more fine-grained speech elements. The total number of silence pdf nodes, useless pdf nodes and a plurality of wake-up word pdf nodes in the application is the number of output nodes of the initial speech wake-up model.

[0063] It is necessary to explain pdf, and first explain phone and Tri-phone: there is a one-to-many correspondence between phone and pdf, and a many-to-many correspondence between pdf and phone. Among them, Phone: similar to initial and final consonants, for example: the pronunciation of the word Xiaohai is xiao3hai3, then x is a phone, iao3 is a phone, h is a phone, and ai3 is also a phone. In Chinese Mandarin, there are about 100-200 such phones. Tri-phone is a further subdivision of phone considering the previous phone and the next phone. For example: “Hello Xiaohai”, “insurance” are converted into phone strings respectively: n i3 h ao3 x iao3 h ai3, b ao3 x ian3. Both phone strings contain “ao3”. The two phones are the same, but the two tri-phones are different. Because tri-phone considers the previous phone and the next phone of the current phone. The ao3 in “Hello Xiaohai” can be represented as: h-ao3+x (indicating that the previous phone is h and the next phone is x). While ao3 in “insurance” is b-ao3+x. This is two different tri-phones. Such subdivision is to consider that the pronunciation of a phone may change due to different contexts. The number of tri-phones is the cube of the number of phones. The concepts of phone and tri-phone are common sense in the field of speech recognition. Pdf is a term defined by kaldi. Because kaldi is a standard tool kit for speech recognition, it is actually common sense in the field of asr, especially in the asr industry. Pdf further subdivides each tri-phone into a number of pronunciation states. All these states are clustered to finally obtain pdf (or called: pdf-class). Among them, the subdivision of a number of pronunciation states belongs to the prior art, which will not be described herein.

[0064] In this embodiment, the electronic device (e.g., the server / terminal device shown in FIG. 1) on which the decoder-based voice wake-up method runs can receive the voice training dataset and the initial voice wake-up model through a wired connection or a wireless connection. It should be noted that the wireless connection can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other now known or future developed wireless connection. Figure 1

[0065] The initial voice wake-up model (referred to as Mw model) of the present application is obtained based on an existing trained continuous speech recognition model (referred to as Mc model), which includes but is not limited to a DNN model. The architecture of the initial voice wake-up model and the continuous speech recognition model can be the same or different. The key point is to determine the classification categories (i.e., the silence pdf node, the useless pdf node, and the multiple wake-up word pdf nodes) and the training labels (i.e., the voice frame training dataset) of the initial voice wake-up model with the help of the continuous speech recognition model, and then start training the initial voice wake-up model. Specifically as follows:

[0066] ​The trained continuous speech recognition model uses PDFs as nodes in its output layer. A typical continuous speech recognition model has several thousand PDF nodes, meaning the output layer has several thousand output nodes. For the initial voice wake-up model, due to strict limitations on computation, several thousand output nodes are not allowed. Therefore, this application only retains PDF nodes related to the pronunciations appearing in the wake-up word. Furthermore, the PDF nodes corresponding to silence are retained separately, while the PDF nodes corresponding to the remaining pronunciations are grouped into a single useless PDF node (i.e., the garbage PDF), thus obtaining the PDF nodes of the voice wake-up model. Simultaneously, a mapping table is obtained between the PDF node names (pdf-id) of the continuous speech recognition model and the PDF node names of the voice wake-up model. In the mapping table, the PDF node names corresponding to the remaining pronunciations and the useless PDF node names have a many-to-one relationship. The mapping table can be directly obtained from the output nodes of the model. For example, in the continuous speech recognition model, 0, 1, and 2 correspond to silence, and in the voice wake-up model, 0, 1, and 2 still correspond to silence, so in the mapping table, it is written as: 0-0, 1-1, 2-2; in the continuous speech recognition model, 3 and 4 correspond to a1, but a1 does not exist in the voice wake-up model, so 3 and 4 both correspond to 100 (a useless PDF node name); in the continuous speech recognition model, n may correspond to 2000 and 2001, while in the voice wake-up model, n corresponds to 3 and 4, so 2000 corresponds to 3, and 2001 corresponds to 4. And so on. Generally speaking, the number of PDF nodes in a voice wake-up model is roughly between several dozen and one hundred, which can meet the needs of the voice wake-up model. An example of the continuous speech recognition model in this application is... Figure 3a As shown, voice wake-up models include, for example Figure 3b As shown. For example, an audio file in WAV format is input to a continuous speech recognition model or a voice wake-up model. The continuous speech recognition model (i.e., the Mc model) includes an input layer, a hidden layer, and an output layer. Its output layer includes over 3000 PDF nodes, specifically including silence PDF nodes and multiple speech PDF nodes (e.g., a1 speech PDF node, a2 speech PDF node, i1 speech PDF node, uo4 speech PDF node). The voice wake-up model also includes an input layer, a hidden layer, and an output layer. Its output layer includes over 100 PDF nodes, specifically including silence PDF nodes, garbage PDF nodes, and multiple wake-up word PDF nodes (e.g., n wake-up word PDF node, i3 wake-up word PDF node, h wake-up word PDF node, ao3 wake-up word PDF node).

[0067] Compared with the data of the specific wake-up word, the training data of the continuous speech recognition model is easier to obtain and is convenient to collect from the actual scene. The voice training data set of the present application is actually the training data of the continuous speech recognition model. The present application enables the voice wake-up model to fully utilize the training data of the continuous speech recognition model, and thus facilitates the model to exhibit more effects. First, the present application changes each node of the output layer of the voice wake-up model from an initial and final consonant to a more fine-grained pdf, which is consistent with the continuous speech recognition model. In this way, although the wake-up word may not appear in the voice training data set, the voice segment containing a pronunciation of the wake-up word can be used to train the initial voice wake-up model, so that the training data of the continuous speech recognition model can be fully utilized. Secondly, the voice wake-up judgment logic based on confidence in the subsequent content is changed to the judgment logic based on the decoder. Since the decoder naturally considers the order of words, such a wake-up system can also distinguish the order of characters and avoid false wake-up when the order is reversed.

[0068] S2: performing an alignment operation on the voice training data set to obtain a voice frame training data set, training the initial voice wake-up model based on the voice frame training data set and the voice training data set to obtain a target voice wake-up model.

[0069] In the present embodiment, it is necessary to perform an alignment operation on the voice training data set according to the trained model to implement frame-level labeling on the voice training data set, to obtain a voice frame training data set, and to train an initial voice wake-up model based on the voice frame training data set.

[0070] Further, the output layer of the continuous speech recognition model includes a plurality of pdf nodes, and the step of performing an alignment operation on the voice training data set to obtain a voice frame training data set includes:

[0071] performing frame-level labeling on the voice training data set based on the pre-trained continuous speech recognition model to obtain a labeled data set;

[0072] According to a preset mapping relationship table, the training label of the labeled data set is replaced by a corresponding target label to obtain the voice frame training data set, wherein the mapping relationship table includes a corresponding relationship between the name of the pdf node of the continuous speech recognition model and the target label, and the target label is the name of the silence pdf node, the name of the useless pdf node and the name of the wake-up word pdf node.

[0073] In the embodiment, the training labels for obtaining the initial voice wake-up model are divided into two steps: first, a continuous speech recognition model is used to perform alignment on a continuous speech data set to generate a labeled data set (alignment, referred to as Ac). The alignment is a standard operation in speech recognition. The general principle is as follows (only for non-end-to-end speech recognition): The training of the model requires an input-output pair. In speech recognition, the input is the speech features (such as mfcc), and the features are frame by frame, for example, 10 milliseconds are set as one frame, so that 100 frames of features are generated for one second of speech, and correspondingly, 100 labels need to be known, so as to know which pdf each frame corresponds to. However, the training data of the speech can only give a word-level label (such as “hello”), and it is impossible to directly give a frame-level label (such as “sil sil n n n I I II I h h ao ao ao….”. (Here, only phone is taken as an example, and actually it should be pdf). Therefore, an operation is needed to convert the word-level label into a frame-level label, which is alignment. The alignment process is similar to recognition, which needs a pre-trained model, and then a decoding graph containing only one sentence is constructed from the labeled text, which ensures that only the content of the labeled text can be recognized, and other contents cannot be recognized, and then decoding is performed, and according to some information in the decoding search process, the frame-level label, that is, the labeled data set, can be obtained. The labeled data of the speech must be converted into the labeled data set to be directly used for model training. Then, the name (pdf-id) of the pdf node corresponding to the continuous speech recognition model in the labeled data set is converted into the corresponding target label by using the mapping relationship table, wherein in the mapping relationship table, the target label refers to the name of the pdf node corresponding to the initial voice wake-up model (the name of the pdf node corresponding to the initial voice wake-up model includes the name of the silence pdf node, the name of the useless pdf node, or the name of the wake-up word pdf node), to obtain a voice frame training data set.

[0074] In addition, the step of training the initial voice wake-up model through the voice frame training data set and the voice training data set to obtain a target voice wake-up model includes:

[0075] The labeled data of the voice frame training data set is taken as the training label of the corresponding data of the voice training data set to obtain a target training data set, the initial voice wake-up model is trained based on the target training data set to obtain the target voice wake-up model.

[0076] In this embodiment, the voice frame training data set serves as the training label of the voice training data set. During the training process, the voice training data set (referred to as: D data set) and the voice frame training data set (referred to as Aw data set) are used to train the initial voice wake-up model. Training requires input and label. When training the initial voice wake-up model, the input is the voice training data set, and the training label is the voice frame training data set.

[0077] S3: Receive the voice to be recognized, input the voice to be recognized frame by frame into the target voice wake-up model, and obtain the target pdf probability corresponding to each output frame.

[0078] In this embodiment, the output of the voice wake-up model is the probability distribution of a certain voice "frame" on all pdfs. A characteristic of voice-related tasks is that both their input and output are not single data points but a sequence. For example, for a voice "Hello, Xiaohai", it is not "input the whole sentence and get an output, and then know which category the whole sentence belongs to", but "divide the voice into several frames, then input frame by frame, and each frame can obtain a probability distribution". If the voice has 100 frames, 100 probability distributions can be obtained, and finally the decoder needs to combine the probability distributions of these 100 frames to obtain the final result. The output layer of the target voice wake-up model of this application includes a silent pdf node, a useless pdf node, and multiple wake-up word pdf nodes. After inputting the current frame of the voice to be recognized into the target voice wake-up model, the probability corresponding to the current frame of the voice to be recognized for the silent pdf node, the probability corresponding to the useless pdf node, and the probability corresponding to multiple wake-up word pdf nodes obtained are a probability sequence, serving as the target pdf probability corresponding to the current frame.

[0079] S4: Input each of the target pdf probabilities into a pre-trained decoder to obtain the output decoding result.

[0080] In this embodiment, the decoder decodes according to the target pdf probabilities corresponding to each voice frame and outputs the decoding result.

[0081] Further, before the step of inputting each of the target pdf probabilities into a pre-trained decoder to obtain the output decoding result, it further includes:

[0082] Construct a target decoding graph through a preset grammar; [[ID=第十九]] [[ID=第二十]]

[0083] [[ID=第二十一]]The step of inputting each of the target pdf probabilities into a pre-trained decoder to obtain the output decoding result includes: [[ID=二十二]] [[ID=二十三]]

[0084] The target pdf probabilities are input into a pre-trained decoder, which finds an optimal path of the target pdf probability sequence in the target decoding graph to obtain an output decoding result.

[0085] In this embodiment, a decoder finds an optimal path of the target pdf in a target decoding graph (i.e., HCLG.fst) to obtain an output wake-up word (e.g., "Hello, little sea") or null; if the output wake-up word is output, a wake-up operation is performed; and if the output null is output, it is determined not to wake up.

[0086] The step of constructing the target decoding graph through the preset grammar comprises:

[0087] A G.fst graph is constructed through a preset grammar, wherein the G.fst graph has and only has one valid path, and the valid path corresponds to one wake-up word.

[0088] The G.fst graph and a preset L.fst graph are subjected to a plurality of combination operations to obtain a wake-up HCLG.fst graph.

[0089] An edge outputting null is added in the wake-up HCLG.fst graph in a parallel manner to absorb silence and useless pronunciation to obtain the target decoding graph.

[0090] In this embodiment, the wake-up HCLG.fst graph is composed of four fst graphs through a series of algorithms (combination, determination, and minimization, etc.). The four fst graphs are H.fst graph, C.fst graph, L.fst graph, and G.fst graph, which are respectively an fst graph corresponding to an HMM model, an fst graph corresponding to a context environment, an fst graph corresponding to a dictionary, and a G.fst graph constructed through a grammar. A plurality of G.fst graphs are constructed through a plurality of preset grammars (i.e., grammar rules) in this application, and the grammar and the G.fst graph are in one-to-one correspondence. There is one valid path in each G.fst graph, which corresponds to one wake-up word. The plurality of G.fst graphs, H.fst graph, C.fst graph, and L.fst graph are combined to obtain the wake-up HCLG.fst graph.

[0091] The composition process specifically includes: 1) constructing a G.fst containing only one valid path (i.e. the path only outputs one wake-up word) using grammar, and then using the G.fst and a continuous speech recognition model for subsequent composition. The G.fst in the continuous speech recognition model is generated by an N-gram language model. The G.fst corresponding to the speech wake-up model of the present application is generated by grammar, which contains only one valid path, and the path only outputs one wake-up word, which ensures that the model can only recognize one wake-up word and cannot recognize other content. 2) converting the input symbol (input code) on the wake-up HCLG.fst from the pdf-id (name of the pdf node) of the continuous speech recognition model to the pdf-id (name of the pdf node) of the speech wake-up model. Because the present application actually calls the trained continuous speech recognition model to construct the wake-up HCLG.fst, the input code of the wake-up HCLG.fst is the name of the pdf node of the continuous speech recognition model. In order to match the speech wake-up model, the name of the pdf node of the continuous speech recognition model needs to be converted to the name of the pdf node of the speech wake-up model according to the mapping relationship table to obtain the intermediate HCLG.fst. 3) parallel a null output edge on the intermediate HCLG.fst to absorb silence and garbage pronunciation, and obtain the target decoding graph. To achieve the effect of recognizing the wake-up word as the wake-up word and recognizing the silence or other words as null.

[0092] In addition, the step of inputting each target pdf probability into a pre-trained decoder to obtain an output decoding result includes:

[0093] Inputting each target pdf probability into a pre-trained static decoder based on a weighted finite state transducer to obtain an output decoding result.

[0094] In the embodiment, the decoder of the application is a static decoder based on wfst (Weighted Finite-state Transducer), and the decoding principle is the same as continuous speech recognition. It is equivalent to directly performing speech recognition, and if the recognition result is a wake-up word, the wake-up is successful, and if the recognition result is empty, the wake-up is not performed. The wfst is a directed graph, which can represent the relationship between sequences. For example, a language model (the relationship between word strings) can be represented by a wfst graph (the language model is generally represented by G.fst), and a pronunciation dictionary (the relationship between phone strings and words) can also be represented by a wfst graph (the pronunciation dictionary is generally represented by L.fst). Meanwhile, the wfst also defines a series of operations, of which the most important is the compose operation, which can "combine" the "relationship". For example, the compose of G.fst and L.fst can obtain LG.fst, which is also a relationship graph, representing the relationship between phone strings and word strings after considering the information of the language model and the pronunciation dictionary.

[0095] The decoder based on wfst is a necessary component of speech recognition, and the speech recognition function can be realized only if the decoder is provided. The HCLG.fst can be finally obtained through multiple compose operations (i.e., combination operations), wherein the specific combination operation process can adopt the existing combination process, which will not be described herein. The HCLG.fst represents the relationship between the pdf string and the word string (i.e., the sentence). All candidate results of speech recognition are contained in the final relationship graph. Then, only the optimal path needs to be searched in the relationship graph by using the decoding search algorithm in combination with the output of the target speech wake-up model, so that the speech recognition can be realized. The process of constructing the target decoding graph (i.e., HCLG.fst) is called "graph construction", and the process of searching the optimal path on the target decoding graph is called "decoding search".

[0096] S5: determining whether the decoding result is a wake-up word, and if the decoding result is a wake-up word, performing a wake-up operation.

[0097] In the embodiment, if the decoding result is not a wake-up word, it is determined that the wake-up is not performed.

[0098] Further, the step of determining whether the decoding result is a wake-up word comprises:

[0099] determining whether the decoding result is empty, and if the decoding result is not empty, determining that the decoding result is a wake-up word.

[0100] In the embodiment, there is an edge with empty output in the target decoding graph, which is used to absorb silence and useless pronunciation. If the output of the decoding result is not empty, the output must be a wake-up word.

[0101] It should be emphasized that, in order to further ensure the privacy and security of the target voice wake-up model, the target voice wake-up model can also be stored in a node of a blockchain.

[0102] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each of which contains the information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0103] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0104] The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0105] The present application can be applied in the field of smart communities, thereby promoting the construction of smart cities.

[0106] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by computer readable instructions instructing related hardware, which can be stored in a computer readable storage medium. When the computer readable instructions are executed, they can include the processes of the above-mentioned embodiments. The storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0107] It should be understood that although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise explicitly stated herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0108] Further referring to FIG. 3, as an implementation of the method described above Figure 2 , the present application provides an embodiment of a decoder-based voice wake-up device, which corresponds to the method embodiment shown in Figure 2 , and the device can be specifically applied to various electronic devices.

[0109] As shown in FIG. 3, the decoder-based voice wake-up device 300 described in the embodiment includes a receiving module 301, a training module 302, an input module 303, an output module 304, and a judgment module 305. The receiving module 301 is configured to receive a voice training data set and an initial voice wake-up model, wherein the output layer of the initial voice wake-up model includes a silence pdf node, a useless pdf node, and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf corresponding to silence in a voice, the wake-up word pdf node corresponds to a pdf corresponding to a wake-up word in a voice, and the useless pdf node corresponds to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in the voice; the training module 302 is configured to perform an alignment operation on the voice training data set to obtain a voice frame training data set, train the initial voice wake-up model through the voice frame training data set and the voice training data set, and obtain a target voice wake-up model; the input module 303 is configured to receive a voice to be recognized, input the voice to be recognized into the target voice wake-up model frame by frame, and obtain a target pdf probability corresponding to each output frame; the output module 304 is configured to input each target pdf probability into a pre-trained decoder to obtain a decoding result; and the judgment module 305 is configured to determine whether the decoding result is a wake-up word, and if the decoding result is a wake-up word, perform a wake-up operation, and if the decoding result is not a wake-up word, determine not to wake up.

[0110] In the embodiment, the present application uses an existing voice training dataset, without the need for special recording of wake-up word voice, greatly reducing the data cost; the wake-up word can be arbitrarily replaced, improving the flexibility of the model; meanwhile, the output layer of the initial voice wake-up model is provided with fine-grained silence pdf nodes, useless pdf nodes and multiple wake-up word pdf nodes, the silence pdf nodes correspond to the pdf corresponding to the silence in the voice, the wake-up word pdf nodes correspond to the pdf corresponding to the wake-up word in the voice, and the useless pdf nodes correspond to the pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in the voice, while realizing finer-grained voice recognition, the voice training dataset can be fully utilized to complex noise, speaker and other information, and the accuracy of the voice wake-up system can be improved to a certain extent. In the wake-up logic part, the decoder is used for decoding, which can solve the problem of mis-wake-up caused by reversed word order, and further improve the accuracy of voice wake-up.

[0111] The output layer of the continuous speech recognition model includes multiple pdf nodes, and the training module 302 includes a labeling submodule and a replacement submodule, wherein the labeling submodule is configured to label the voice training dataset at a frame level based on the pre-trained continuous speech recognition model to obtain a labeled dataset; and the replacement submodule is configured to replace training labels of the labeled dataset with corresponding target labels according to a preset mapping relationship table to obtain the voice frame training dataset, wherein the mapping relationship table includes a correspondence between names of pdf nodes of the continuous speech recognition model and the target labels, and the target labels are names of the silence pdf nodes, names of the useless pdf nodes, and names of the wake-up word pdf nodes.

[0112] In some optional implementations of the embodiment, the training module 302 is further configured to obtain a target training dataset by taking labeled data of the voice frame training dataset as training labels of corresponding data of the voice training dataset, train the initial voice wake-up model based on the target training dataset, and obtain the target voice wake-up model.

[0113] In some optional implementations of the embodiment, the apparatus 300 further includes a construction module configured to construct a target decoding graph through a preset grammar; and the output module 304 is further configured to input each target pdf probability into a pre-trained decoder, the decoder finds an optimal path of the target pdf probability sequence in the target decoding graph, and obtains an output decoding result.

[0114] The construction module comprises a construction submodule, a combination submodule and an addition submodule, wherein the construction submodule is configured to construct a G.fst graph through a preset grammar, wherein the G.fst graph has only one valid path, and the valid path corresponds to a wake-up word; the combination submodule is configured to perform a plurality of combination operations on the G.fst graph and a preset L.fst graph to obtain a wake-up HCLG.fst graph; and the addition submodule is configured to add an edge with an empty output in the wake-up HCLG.fst graph in a parallel manner to absorb silence and useless pronunciation, and obtain the target decoding graph.

[0115] In addition, in some optional implementations of the embodiment, the output module 304 is further configured to input each target pdf probability into a pre-trained static decoder based on a weighted finite state transducer to obtain an output decoding result.

[0116] In some optional implementations of the embodiment, the judgment module 305 is further configured to judge whether the decoding result is empty, and if the decoding result is not empty, determine that the decoding result is a wake-up word.

[0117] The present application uses an existing voice training data set, without the need to specially record a wake-up word voice, greatly reducing the data cost; the wake-up word can be arbitrarily replaced, improving the flexibility of the model; at the same time, the output layer of the initial voice wake-up model of the present application is provided with a fine-grained silence pdf node, a useless pdf node and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf corresponding to silence in the voice, the wake-up word pdf node corresponds to a pdf corresponding to a wake-up word in the voice, and the useless pdf node corresponds to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in the voice, which can fully utilize the relatively complex noise, speaker and other information in the voice training data set, and can also improve the accuracy of the voice wake-up system to a certain extent. In the wake-up logic part, the decoder is used for decoding, which can solve the problem of false wake-up caused by the reverse order of words, and further improve the accuracy of voice wake-up.

[0118] To solve the above technical problems, the embodiment of the present application further provides a computer device. For details, please refer to Figure 5 , Figure 5 The basic structure block diagram of the computer device of the embodiment is shown in the following figure.

[0119] The computer device 200 includes a memory 201, a processor 202, and a network interface 203 which are communicatively connected by a system bus. It should be noted that the computer device 200 is only shown with components 201-203, but it should be understood that not all of the shown components are required to be implemented, and more or less components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device herein is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0120] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, and the like.

[0121] The memory 201 includes at least one type of readable storage medium, including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 201 can be an internal storage unit of the computer device 200, such as a hard disk or a memory of the computer device 200. In other embodiments, the memory 201 can also be an external storage device of the computer device 200, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Of course, the memory 201 can also include both the internal storage unit and the external storage device of the computer device 200. In this embodiment, the memory 201 is generally used to store an operating system and various application software installed in the computer device 200, such as computer readable instructions of the voice wake-up method based on a decoder, and the like. In addition, the memory 201 can also be used to temporarily store various data that have been output or will be output.

[0122] The processor 202 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 202 is generally used to control the overall operation of the computer device 200. In the present embodiment, the processor 202 is configured to execute computer readable instructions stored in the memory 201 or process data, such as computer readable instructions of the decoder-based voice wake-up method.

[0123] The network interface 203 may include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 200 and other electronic devices.

[0124] In the present embodiment, while achieving finer-grained voice recognition, the complex noise, speaker, and other information in the voice training data set can be fully utilized, and the accuracy of the voice wake-up system can be improved to some extent. Using the decoder for decoding can solve the problem of miswaking caused by reversed word order, and further improve the accuracy of voice wake-up.

[0125] The present application also provides another embodiment, that is, a computer readable storage medium storing computer readable instructions, which can be executed by at least one processor to enable the at least one processor to perform the steps of the decoder-based voice wake-up method as described above.

[0126] In the present embodiment, while achieving finer-grained voice recognition, the complex noise, speaker, and other information in the voice training data set can be fully utilized, and the accuracy of the voice wake-up system can be improved to some extent. Using the decoder for decoding can solve the problem of miswaking caused by reversed word order, and further improve the accuracy of voice wake-up.

[0127] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software and a general hardware platform as required, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device) to perform the methods described in the various embodiments of the present application.

[0128] Obviously, the above-described embodiments are only some embodiments but not all the embodiments of the present application, the preferred embodiments of the present application are shown in the drawings, but do not limit the patent scope of the present application. The present application can be implemented in many different forms, and conversely, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent replacements to some technical features therein. Any equivalent structure made by using the content of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the patent protection scope of the present application.

Claims

1. A decoder-based voice wake-up method, characterized by, The method comprises the following steps: receiving a voice training data set and an initial voice wake-up model, wherein an output layer of the initial voice wake-up model comprises a silence pdf node, a useless pdf node and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf of silence in voice, the wake-up word pdf node corresponds to a pdf of a wake-up word in voice, the useless pdf node corresponds to a pdf other than the pdf of the wake-up word and the pdf of the silence in voice, and the pdf is a probability distribution function used to represent fine-grained voice elements; frame-level labeling is performed on the voice training data set based on a pre-trained continuous voice recognition model to obtain a labeled data set, a training label of the labeled data set is replaced with a corresponding target label according to a preset mapping relationship table to obtain a voice frame training data set, and an output layer of the continuous voice recognition model comprises a plurality of pdf nodes, wherein the mapping relationship table comprises a corresponding relationship between a name of a pdf node of the continuous voice recognition model and the target label, and the target label is a name of the silence pdf node, a name of the useless pdf node and a name of the wake-up word pdf node; the initial voice wake-up model is trained based on the voice frame training data set and the voice training data set to obtain a target voice wake-up model; receiving a voice to be recognized, inputting the voice to be recognized into the target voice wake-up model frame by frame to obtain a target pdf probability corresponding to each frame of output; inputting each target pdf probability into a pre-trained decoder to obtain a decoding result of output; judging whether the decoding result is a wake-up word, and if the decoding result is a wake-up word, performing a wake-up operation.

2. The decoder-based voice wake-up method of claim 1, wherein, Before the step of inputting each target pdf probability into a pre-trained decoder to obtain a decoding result of output, the method further comprises: constructing a target decoding graph through a preset grammar; the step of inputting each target pdf probability into a pre-trained decoder to obtain a decoding result of output comprises: inputting each target pdf probability into a pre-trained decoder, and the decoder finds an optimal path of a target pdf probability sequence in the target decoding graph to obtain a decoding result of output.

3. The decoder-based voice wake-up method of claim 2, wherein, The step of constructing a target decoding graph through a preset grammar comprises: constructing a G.fst graph through a preset grammar, wherein the G.fst graph has only one valid path, and the valid path corresponds to one wake-up word; performing a plurality of combination operations on the G.fst graph and a preset L.fst graph to obtain a wake-up HCLG.fst graph; adding an edge with an empty output in the wake-up HCLG.fst graph in a parallel manner to absorb silence and useless pronunciation to obtain the target decoding graph.

4. The decoder-based voice wake-up method of claim 1, wherein, The step of judging whether the decoding result is a wake-up word comprises: judging whether the decoding result is empty, and if the decoding result is not empty, determining that the decoding result is a wake-up word.

5. The decoder-based voice wake-up method of claim 1, wherein, The step of training the initial voice wake-up model based on the voice frame training data set and the voice training data set to obtain a target voice wake-up model comprises: The target training data set is obtained by taking the labeled data of the speech frame training data set as the training label of the corresponding data of the speech training data set, and the initial speech wake-up model is trained based on the target training data set to obtain the target speech wake-up model.

6. The decoder-based voice wake-up method of claim 1, wherein, The step of inputting each of the target pdf probabilities into a pre-trained decoder to obtain a decoded result output includes: The step of inputting each of the target pdf probabilities into a pre-trained decoder to obtain a decoded result output includes:

7. A decoder-based voice wake-up apparatus, characterized by, Comprise: The receiving module is configured to receive a speech training data set and an initial speech wake-up model, wherein an output layer of the initial speech wake-up model comprises a silence pdf node, a useless pdf node, and a plurality of wake-up word pdf nodes, the silence pdf node corresponds to a pdf corresponding to silence in speech, the wake-up word pdf node corresponds to a pdf corresponding to a wake-up word in speech, and the useless pdf node corresponds to a pdf other than the pdf corresponding to the wake-up word and the pdf corresponding to the silence in the speech, and the pdf is a probability distribution function used to represent fine-grained speech elements; The training module is configured to perform frame-level labeling on the speech training data set based on a pre-trained continuous speech recognition model to obtain a labeled data set, and replace training labels of the labeled data set with corresponding target labels according to a preset mapping relationship table to obtain a speech frame training data set, wherein an output layer of the continuous speech recognition model comprises a plurality of pdf nodes, and the mapping relationship table comprises a corresponding relationship between names of the pdf nodes of the continuous speech recognition model and the target labels, and the target labels are names of the silence pdf node, names of the useless pdf node, and names of the wake-up word pdf node. The training module is further configured to train the initial speech wake-up model based on the speech frame training data set and the speech training data set to obtain a target speech wake-up model. The input module is configured to receive a to-be-recognized speech, input the to-be-recognized speech frame by frame into the target speech wake-up model, and obtain target pdf probabilities corresponding to each frame output. The output module is configured to input each of the target pdf probabilities into a pre-trained decoder to obtain a decoded result output. The judgment module is configured to determine whether the decoded result is a wake-up word, and if the decoded result is a wake-up word, perform a wake-up operation, and if the decoded result is not a wake-up word, determine not to wake up.

8. A computer device, comprising: The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the decoder-based speech wake-up method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor to implement the steps of the decoder-based speech wake-up method according to any one of claims 1 to 6.

Images