Voice control method, device and equipment and readable storage medium

Abstract

The invention discloses a voice control method, device and equipment and a readable storage medium, and electronic equipment obtains a voice signal and converts the voice signal into a to-be-corrected text. Then, the to-be-corrected text is input into the error correction model, so that the error correction model performs error correction on the to-be-corrected text by combining the characters and the corresponding pinyin in the to-be-corrected text to obtain a target text, and finally, the electronic equipment generates a control instruction according to the target text to control the terminal equipment. By adopting the scheme, the characteristics of pinyin are introduced into the error correction process, so that the problem of error cascading caused by error correction of texts by adopting a traditional multi-model pipeline method is relieved, the voice interaction capability of terminal equipment is improved, the user interaction experience is improved to a great extent, and the purpose of improving the voice interaction quality is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of voice control technology, and in particular to a voice control method, device, equipment and readable storage medium. Background Art

[0002] With the development of artificial intelligence technology, various autonomous mobile devices are increasingly entering people's lives, such as cleaning robots, air purification robots, lawn mowing robots, welcoming robots, etc.

[0003] Voice control technology is a common method for controlling self-mobile devices. During the voice interaction process, the self-mobile device collects the voice signal emitted by the user, converts the voice signal into text, and then performs intent recognition based on the text to achieve voice control. In order to ensure the quality of voice interaction, a multi-model pipeline method is usually used to correct text errors, which mainly includes the following steps: error detection, candidate recall, and candidate sorting. The main purpose of error detection is to identify and locate the position of the wrong word in the text and pass the error backward. Candidate recall refers to the recall of candidate words for the wrong position. Candidate sorting refers to sorting the recall results and returning the only correct result.

[0004] However, in the above-mentioned text error correction method, since the errors are transmitted step by step, the errors in the previous stage will generate noise for the next stage, resulting in speech recognition errors and even failure of speech interaction. Summary of the invention

[0005] The embodiments of the present application provide a voice control method, apparatus, device and readable storage medium, which introduce the characteristics of pinyin into the error correction process, alleviate the problem of error cascade caused by the traditional multi-model pipeline method for text error correction, and achieve the purpose of improving the quality of voice interaction.

[0006] In a first aspect, an embodiment of the present application provides a voice control method, comprising:

[0007] Acquire speech signals;

[0008] Converting the speech signal into text to be corrected;

[0009] Inputting the text to be corrected into the error correction model, so that the error correction model combines each character and the corresponding pinyin in the text to be corrected, and corrects the text to be corrected to obtain the target text;

[0010] A control instruction is generated according to the target text to control the terminal device.

[0011] In a second aspect, an embodiment of the present application provides a voice control device, comprising:

[0012] An acquisition module, used for acquiring a voice signal;

[0013] A conversion module, used for converting the speech signal into a text to be corrected;

[0014] A processing module, used for inputting the text to be corrected into an error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin, and corrects the text to be corrected to obtain a target text;

[0015] The control module is used to generate a control instruction according to the target text to control the terminal device.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect or various possible implementation methods of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the method described in the first aspect or various possible implementation methods of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computing program, and when the computer program is executed by a processor, the method described in the first aspect or various possible implementation methods of the first aspect is implemented.

[0019] The voice control method, device, equipment and readable storage medium provided in the embodiments of the present application, the electronic device obtains a voice signal and converts the voice signal into a text to be corrected. Afterwards, the text to be corrected is input into the error correction model, so that the error correction model combines each character and the corresponding pinyin in the text to be corrected, corrects the text to be corrected to obtain the target text, and finally, the electronic device generates a control instruction according to the target text to control the terminal device. By adopting this scheme, by introducing the characteristics of pinyin into the error correction process, the problem of error cascade caused by the traditional multi-model pipeline method for text correction is alleviated, the voice interaction capability of the terminal device is improved, the user interaction experience is greatly improved, and the purpose of improving the quality of voice interaction is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 Schematic diagram of the implementation environment of the voice control method provided in the embodiment of the present application;

[0022] Figure 2 is a flow chart of the voice control method provided by an embodiment of the present application;

[0023] Figure 3 is a flow chart for determining a first probability and a second probability in a voice control method provided in an embodiment of the present application;

[0024] Figure 4 is a flow chart of determining a fused feature vector in a voice control method provided in an embodiment of the present application;

[0025] Figure 5 It is a flowchart of error correction based on fused feature vectors in the voice control method provided in an embodiment of the present application;

[0026] Figure 6 Schematic diagram of the training process of the error correction model in the voice control method provided in the embodiment of the present application;

[0027] Figure 7 is a schematic diagram of an error correction model in a voice control method provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a voice control device provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In the voice interaction scenario, voice recognition includes two stages: the first stage: the voice system converts the user's voice into correct text; the second stage: subsequent intention recognition and interaction based on the text. Automatic speech recognition technology (Automatic Speech Recognition, ASR) is a technology that converts voice into text. ASR error correction is the basic work of error detection and correction of the text generated in the first stage. It is an indispensable preparatory work for intelligent voice applications such as intelligent question and answer and dialogue systems. An excellent error correction system can greatly improve the user experience.

[0031] The mainstream frameworks of traditional ASR error correction include multi-model pipeline and text generation. The multi-model pipeline method includes three steps: error detection, candidate recall, and candidate sorting. The main purpose of error detection is to use the sequence labeling model to locate the location where the error occurs. Candidate recall refers to recalling candidate words at the error location. Candidate sorting refers to sorting all candidate results and returning the only correct result.

[0032] The advantage of the multi-model pipeline method is that it is convenient to analyze problems in a targeted manner. However, there are the following problems:

[0033] 1. Errors are passed on step by step, so that the errors in the previous step will generate noise for the next step. For example, when detecting errors, a non-typo is mistakenly identified as a typo, and the correction of that position during candidate sorting must also be incorrect.

[0034] 2. A confusion set dictionary needs to be maintained. The quality of the dictionary greatly affects the recall effect of candidate words.

[0035] 3. Error detection, candidate recall, and candidate sorting are performed serially, resulting in poor timeliness.

[0036] The text generation method can be regarded as a machine translation method. An end-to-end generative model is constructed. The wrong text is input into the encoder to obtain the text feature representation, and the decoder decodes it one by one to obtain the corrected sentence. This method has poor controllability and is prone to miscorrecting low-frequency expressions into high-frequency expressions, resulting in a high miscorrection rate. For example, "cleaning the vicinity of eggs" is corrected to "cleaning the vicinity of the base station". Although this correction is correct, "cleaning the egg shell" is subsequently mistakenly corrected to "cleaning the base station shell".

[0037] Based on this, the embodiments of the present application provide a voice control method, apparatus, device and readable storage medium, which introduce the characteristics of pinyin into the error correction process, alleviate the problem of error cascades caused by the traditional multi-model pipeline method for text error correction, and achieve the purpose of improving the quality of voice interaction.

[0038] The voice control method provided in the embodiment of the present application trains an error correction model based on the BERT model, and uses the error correction model to correct the text to be corrected. In one method, the error correction model is deployed on the user's terminal device. During voice interaction, the terminal device collects voice signals, converts the voice signals into text to be corrected, and then inputs the text to be corrected into the error correction model. The error correction model combines the characters and corresponding pinyin in the text to be corrected to correct the text to obtain the target text. Afterwards, the terminal device generates a control instruction based on the target text and executes the control instruction.

[0039] In another way, the error correction model is deployed on a remote server, and the remote server corrects the text to be corrected. Figure 1 .

[0040] Figure 1 Schematic diagram of the implementation environment of the voice control method provided in the embodiment of the present application. Figure 1, the implementation environment includes: a terminal device 11 and a server 12, a network connection is established between the terminal device 11 and the server 12, and an error correction model is deployed on the server 12. The user sends a voice signal, the terminal device 11 collects the voice signal and sends it to the server 12, the server 12 converts the voice signal into a text to be corrected, and inputs the text to be corrected into the error correction model, and the error correction model combines each character in the text to be corrected and the corresponding pinyin to correct the text to obtain the target text. Afterwards, the server 12 generates a control instruction according to the target text, and sends the control instruction to the terminal device 11 to control the terminal device 11.

[0041] In the embodiment of the present application, the terminal device 11 is, for example, a self-moving device, a vehicle, a smart speaker, a mobile phone, a tablet computer, etc. When the terminal device 11 is a self-moving device, it can be a sweeping robot, a lawn mowing robot, an air purification robot, an automatic spraying device, a nursing robot, an automated guided vehicle (AVG), etc., which can realize unattended operation of various tasks.

[0042] The error correction model is deployed on the server 12. The server 12 can be hardware or software. When the server 12 is hardware, the server 12 is a single server or a distributed server cluster composed of multiple servers. When the server 12 is software, it can be multiple software modules or a single software module, etc., which is not limited by the embodiments of the present application.

[0043] It should be understood that Figure 1 The number of terminal devices 11 and servers 12 in the figure is only for illustration. In actual implementation, any number of terminal devices 11 and servers 12 can be deployed according to actual needs.

[0044] Figure 2 1 is a flow chart of the voice control method provided by an embodiment of the present application. The execution subject of this embodiment is an electronic device, such as the above-mentioned terminal device or server, and this embodiment includes:

[0045] 201. Acquire a voice signal.

[0046] In the voice interaction scenario, the user sends a voice signal, and the electronic device obtains the language signal. For example, when the electronic device is a terminal device, the terminal device has a microphone, etc., and uses the microphone, etc. to collect voice signals. For another example, when the electronic device is a server, after the terminal device collects the voice signal, it sends the voice signal to the server.

[0047] 202. Convert the speech signal into text to be corrected.

[0048] Exemplarily, the electronic device converts the voice signal into a digital signal using digital signal processing technology, and then pre-processes the digital signal by eliminating environmental noise, enhancing the signal, etc. Afterwards, the electronic device inputs the pre-processed voice signal into a voice recognition model, thereby converting the voice signal into text, hereinafter referred to as text to be corrected.

[0049] 203. Input the text to be corrected into an error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin to correct the text to obtain a target text.

[0050] A pre-trained error correction model is deployed on the electronic device. The error correction model is a model that introduces pinyin features during the model training process, so it can correct words with the same or similar pronunciation.

[0051] After converting the speech signal into the text to be corrected, the electronic device inputs the text to be corrected into the correction model. The correction model analyzes the characters in the text to be corrected and the pinyin corresponding to the characters, thereby determining the wrong characters and the correct characters corresponding to the wrong characters, and replacing the wrong characters with the correct characters to obtain the target text. Among them, characters refer to the individual words in the text to be corrected. For example, if the text to be corrected is: clean the area near eggs, then the text to be corrected contains 6 characters, namely: clean, sort, chicken, egg, attach, near.

[0052] During the error correction process, when the error correction model determines the wrong character, the weight of the character is reduced, the weight of the pinyin corresponding to the character is increased, and then the vector of the character and the vector of the pinyin are weighted fused according to the weight to avoid error cascading. After that, the character used to replace the wrong character is determined based on the fused feature vector.

[0053] 204. Generate a control instruction according to the target text to control a terminal device.

[0054] Exemplarily, the self-mobile device generates a control instruction based on the corrected text, that is, the target text, and controls the terminal device according to the control instruction. For example, the terminal device is a sweeping robot, and the target text is: clean the vicinity of the base station, then the electronic device generates a control instruction to control the terminal device to move to the vicinity of the base station and clean the vicinity of the base station. For example, with the base station as the center, clean a circular area with a radius of 1 meter. For another example, with the base station as the center, clean a square area with a side length of 1.5 meters.

[0055] In the voice control method provided in the embodiment of the present application, an electronic device obtains a voice signal and converts the voice signal into a text to be corrected. Afterwards, the text to be corrected is input into the error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin, and corrects the text to be corrected to obtain the target text. Finally, the electronic device generates a control instruction according to the target text to control the terminal device. By adopting this solution, by introducing the characteristics of pinyin into the error correction process, the problem of error cascade caused by the traditional multi-model pipeline method for text correction is alleviated, the voice interaction capability of the terminal device is improved, the user interaction experience is greatly improved, and the purpose of improving the quality of voice interaction is achieved.

[0056] Optionally, in the above embodiment, the electronic device inputs the text to be corrected into the error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin, and in the process of correcting the text to be corrected to obtain the target text, first, the error correction model is used to determine the first probability of each character in the text to be corrected, and the second probability of the pinyin corresponding to each character, and the first probability is used to indicate the probability of the character error. Then, the electronic device determines the fused feature vector of the text to be corrected based on the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character. Finally, the electronic device corrects the text to be corrected based on the fused feature vector to obtain the target text.

[0057] Exemplarily, the electronic device uses the error correction model to determine the error probability of each character in the text to be corrected, that is, the first probability mentioned above, denoted as P detect For example, if the text to be corrected is: Clean the area near the eggs, then P detect Including 6 first probabilities, namely P 0 , P 1 , P 2 , P 3 , P 4 and P 5 .P i The larger the value, the greater the probability of character error at that position. Where 0≤i≤5, and is an integer.

[0058] The second probability of the pinyin corresponding to each character is: 1-P i Obviously, if the first probability of a character is larger, the probability that the character is an incorrect character is larger, and when the fused feature vector is calculated later, the pinyin vector occupies the dominant part of the fused feature vector; if the first probability of a character is smaller, the probability that the character is a correct character and does not need to be corrected is larger, and when the fused feature vector is calculated later, the word vector occupies the dominant part of the fused feature vector.

[0059] After the electronic device determines the first probability and the corresponding second probability of each character included in the text to be corrected, it performs weighted fusion on the word vectors and pinyin responses corresponding to each character, so as to obtain a fused feature vector. Then, the electronic device corrects the text to be corrected according to the fused feature vector to obtain a target text.

[0060] Adopting this solution, when the electronic device determines the fused feature vector, by combining the first probability and the corresponding second probability of each character included in the text to be corrected, it can accurately determine the fused feature vector, and further achieve the purpose of improving voice interaction instructions.

[0061] Figure 3 It is a flowchart for determining the first probability and the second probability in the voice control method provided by the embodiments of the present application. This embodiment includes:

[0062] 301. Use the error correction model to determine the text sequence and pinyin sequence corresponding to the text to be corrected.

[0063] The electronic device performs character segmentation on the text to be corrected to obtain a text sequence, denoted as {c 1 , c 2 , ……, c n}, and the corresponding pinyin sequence is {p 1 , p 2 , …… p n}. For example, if the text to be corrected is "Clean the area near the base station", the text sequence is {qing, li, ji, zhan, fu, jin}, and the corresponding pinyin sequence is {qing, li, ji, zhan, fu, jin}.

[0064] 302. Encode the text sequence to obtain a first one-hot sequence, and determine the word vector corresponding to the text to be corrected according to the first one-hot sequence.

[0065] The electronic device encodes the text sequence into a first one-hot sequence, and uses a word embedding matrix to map the first one-hot sequence into a vector representation of a fixed length, so as to obtain the word vector of the text to be corrected, denoted as: Ec = {e c1 , e c2 , …, e cn}.

[0066] 303. Determine the hidden layer features of the word vector corresponding to the text to be corrected, and determine the first probability of each character in the text to be corrected according to the hidden layer features.

[0067] After the electronic device determines the word vector, the word vector Ec = {e c1 , e c2 , …, e cn} is input to the bidirectional-gate recurrent unit (Bi-GRU) of the error correction model to obtain the hidden layer features. After the fully connected layer transforms the dimension, the sigmoid activation function is used to output the first probability of each character, that is, P detect .

[0068] 304. Determine a second probability of the pinyin corresponding to each character according to the first probability of each character in the text to be corrected.

[0069] After the electronic device determines the first probability of each character in the text to be corrected, it calculates the difference between 1 and the first probability for each character, thereby obtaining a second probability.

[0070] By adopting this solution, the electronic device determines the first probability of each character in the text to be corrected based on the hidden layer features of the word vector of the text to be corrected, and then determines the second probability of the pinyin corresponding to each character, which is fast and accurate.

[0071] Figure 4 This is a flow chart for determining a fused feature vector in the voice control method provided in an embodiment of the present application.

[0072] This embodiment includes:

[0073] 401. Determine the text sequence and pinyin sequence corresponding to the text to be corrected by using the error correction model.

[0074] For details, please refer to the above Figure 3 The description of step 301 is not repeated here.

[0075] 402. Encode the text sequence to obtain a first one-hot sequence, and determine a word vector corresponding to the text to be corrected according to the first one-hot sequence.

[0076] The electronic device encodes the text sequence into a first one-hot sequence, and uses the word embedding matrix to map the first one-hot sequence into a vector representation of a fixed length, thereby obtaining the word vector of the text to be corrected, which is recorded as: Ec = {e c1 ,e c2 ,…,e cn}.

[0077] 403. Encode the pinyin sequence to obtain a second one-hot sequence, and determine the pinyin vector corresponding to the text to be corrected according to the second one-hot sequence.

[0078] The electronic device encodes the pinyin sequence into a second one-hot sequence, and uses the word embedding matrix to map the second one-hot sequence into a vector representation of a fixed length, thereby obtaining the pinyin vector of the text to be corrected, which is recorded as: Ep = {e p1 ,e p2,…,e pn}.

[0079] It should be noted that there is no strict sequence between step 402 and step 403.

[0080] 404. According to the first probability of each character in the text to be corrected and the second probability of the pinyin, weighted sum of the word vector and the pinyin vector is performed to obtain a fused feature vector of the text to be corrected.

[0081] The electronic device performs a weighted summation of the word vector and the pinyin vector according to the first probability of each character in the text to be corrected and the second probability of the corresponding pinyin. The formula is as follows: m =(1-P i )×e ci +P i ×e pi . It can be seen from this that: if the first probability of a character is larger, the probability that the character is an incorrect character is larger, and when calculating the fused feature vector, the pinyin vector occupies the dominant part of the fused feature vector; if the first probability of a character is smaller, the probability that the character is a correct character and does not need to be corrected is larger, and when calculating the fused feature vector later, the word vector occupies the dominant part of the fused feature vector

[0082] By adopting this scheme, when the electronic device determines the fused feature vector, if the error probability of a character is higher, the pinyin vector will occupy the dominant part of the fused feature vector, thus avoiding error cascades to a great extent, achieving the purpose of accurately determining the fused feature vector, and further achieving the purpose of improving voice interaction instructions.

[0083] Figure 5 This is a flow chart of error correction based on fusion feature vectors in the voice control method provided in an embodiment of the present application. This embodiment includes:

[0084] 501. Determine multiple probability sets according to the fused feature vector.

[0085] Among them, the multiple probability sets correspond to characters at different positions in the text to be corrected, the probabilities in the probability sets correspond one-to-one to the characters in the vocabulary, and the vocabulary is a set of remaining characters after removing duplicate characters contained in each sample in the sample set.

[0086] When the electronic device performs model training, it uses each sample in the sample set to train the initial model, thereby obtaining an error correction model. The electronic device extracts the characters contained in each sample in the sample set and removes duplicates, thereby obtaining a set of non-repeating characters, namely, a vocabulary. The number of non-repeating characters in the vocabulary is called the vocabulary size. For example, if there are 3326 characters in the vocabulary, the vocabulary size is 3326; for another example, if there are 3300 characters in the vocabulary, the vocabulary size is 3300; for another example, if there are 3400 characters in the vocabulary, the vocabulary size is 3400.

[0087] The electronic device determines the fusion feature vector E of the text to be corrected m After that, the fusion feature vector E m Input to the translation (Transformer) module of the error correction model for encoding, and output the semantic representation of the text to be corrected, denoted as H m Then, after the fully connected layer and softmax calculation of the current position, the probability P of each character in the vocabulary correct Assuming that the text to be corrected is "clean up the area near the eggs", there are 6 characters in total, and the vocabulary size is 3326. Then each character corresponds to a probability set containing 3326 probabilities, and there are 6 probability sets in total, P correct The quantity is 6×3326.

[0088] 502. Determine a first position of an erroneous character according to a first probability of each character in the text to be corrected.

[0089] The electronic device determines the first probability P of each character in the text to be corrected detect Then, traverse P one by one detect =[P 0 ,P 1 ,…,P n-1 ], if p i > preset probability, then the position is represented as 1; if p i ≤ the preset probability, then the position is represented as 0, thus obtaining the search sequence of the text to be corrected. For example, the text to be corrected is "clean up the vicinity of eggs", and the search sequence is [0,0,1,1,0,0], which respectively correspond to the 0th position (clear), 1st position (reason) ... 5th position (near) of the text to be corrected, indicating that the characters at the 2nd and 3rd positions of the text to be corrected are wrong, that is, the first position is the 2nd and 3rd positions.

[0090] It should be noted that the above steps 501 and 502 are not in a strict order.

[0091] 503. Determine a target probability set corresponding to the first position from the multiple probability sets.

[0092] In the above step 501, the electronic device determines the probability set corresponding to each character in the text to be corrected, that is, the probability set corresponding to the characters at different positions in the text to be corrected, and the number of probabilities in each probability set is equal to the size of the vocabulary. After the electronic device determines the first position, it determines the probability set corresponding to the first position from the multiple probability sets, thereby obtaining the target probability set.

[0093] Continuing with the above example, the target probability set is the probability set corresponding to the characters at the second and third positions of the text to be corrected, that is, the probability set corresponding to "chicken" and the probability set corresponding to "egg".

[0094] 504. Determine a maximum probability from the target probability set, and replace the erroneous character at the first position in the text to be corrected with the character corresponding to the maximum probability, so as to obtain the target text.

[0095] After the electronic device determines the target probability set, it sorts the probabilities in the target probability set from large to small to obtain the maximum probability. After that, the electronic device determines the character corresponding to the maximum probability from the vocabulary, and replaces the erroneous character at the first position in the text to be corrected with the character corresponding to the maximum probability to obtain the target text.

[0096] For example, in the above example, in the target probability set corresponding to "chicken", the probability of "base" is the largest, so "chicken" in the text to be corrected is replaced with "base"; in the target probability set corresponding to "egg", the probability of "station" is the largest, so "egg" in the text to be corrected is replaced with "station".

[0097] By adopting this solution, the electronic device replaces the character at the first position with the character corresponding to the maximum probability in the target probability set, thereby achieving the purpose of accurate error correction and further achieving the purpose of improving the quality of voice interaction.

[0098] In the above error correction method, the electronic device replaces the erroneous character with the word corresponding to the maximum probability in the target probability set. However, the implementation of the present application is not limited to this. In other implementations, the electronic device determines candidate characters from the word list to obtain a candidate set, and only when there is no original erroneous character in the candidate set, the erroneous character is replaced with the word corresponding to the maximum probability.

[0099] Exemplarily, when the electronic device determines the maximum probability from the target probability set and replaces the erroneous character at the first position in the text to be corrected with the character corresponding to the maximum probability, first, multiple candidate characters are determined from the vocabulary to obtain a candidate set, and the candidate characters are characters corresponding to a preset number of probabilities that are ranked at the top after the probabilities in the target probability set are sorted from large to small. After that, the electronic device determines whether the candidate set contains the erroneous character. When the candidate set does not contain the erroneous character, the character with the highest probability is determined from the target probability set, and the erroneous character at the first position in the text to be corrected is replaced with the character with the highest probability, thereby obtaining the target text. When the candidate set contains the erroneous character, the character in the text to be corrected remains unchanged, that is, there is no need to correct the erroneous character at the first position.

[0100] For example, the text to be corrected is: Clean the eggshells. The electronic device uses the error correction model to determine that the erroneous characters are "chicken" and "egg". There are two target probability sets, namely the probability sets corresponding to "chicken" and "egg", and the number of probabilities in each probability set is equal to the vocabulary size, for example, 3326. The electronic device sorts the probabilities in the two target probability sets. The character with the highest probability in the target probability set corresponding to "chicken" is "base", and the character with the highest probability in the target probability set corresponding to "egg" is "station".

[0101] Before replacing the wrong character with the character corresponding to the maximum probability, the electronic device processes the target probability set corresponding to "chicken" as follows: sort them in order of probability from large to small, and determine the characters corresponding to the top-ranked and preset number of probabilities from the vocabulary to obtain a candidate set, such as the top five probabilities. After that, the electronic device determines whether the character "chicken" exists in the candidate set. If "chicken" does not exist in the candidate set, replace "chicken" with the character corresponding to the maximum probability "base". If the character "chicken" exists, no matter where the probability of "chicken" ranks in the candidate set, such as any one of the 2nd to 5th positions, the electronic device does not need to replace "chicken" with the character corresponding to the maximum probability "base".

[0102] Similarly, the electronic device performs the same processing on the target probability set corresponding to "egg". If "egg" does not exist in the candidate set, the character corresponding to the maximum probability is "station" to replace "egg". If the character "egg" exists, no matter where the probability of "egg" ranks in the candidate set, such as any one of the 2nd to 5th positions, the electronic device does not need to replace "egg" with the character corresponding to the maximum probability "station".

[0103] Optionally, in the embodiment of the present application, the error correction rate represents the proportion of correct sentences that are corrected, and a large error correction rate will have a negative effect on the error correction model and user experience. In order to reduce the error correction rate, in the decoding stage, the electronic device increases the preset probability of error detection to determine whether there are typos in the text, that is, the preset probability in the above step 502; in another way, when determining the candidate set, the confusion set is used to determine the candidate characters included in the candidate set, that is, most of the candidate characters in the candidate set are homophones or similar-sounding characters.

[0104] With this solution, if the candidate set contains the erroneous character at the first position in the text to be corrected, there is no need to correct the erroneous character, thus avoiding miscorrection to a great extent and achieving the purpose of improving the quality of voice control.

[0105] The above describes in detail the process of an electronic device using an error correction model to correct a text to be corrected. Before using the error correction model, the error correction model must be trained first. Next, how to train the error correction model is described in detail.

[0106] The speech control method provided in the embodiment of the present application uses an error correction model to correct the text to be corrected. The error correction model is a model that integrates the characteristics of word pronunciation, combining the characteristics of pinyin with the BERT language model. The training process includes two stages: pre-training and fine-tuning. The samples come from data generated in real interaction scenarios and are used for model training after being annotated by annotators.

[0107] Figure 6 This is a schematic diagram of the training process of the error correction model in the voice control method provided in the embodiment of the present application. Figure 6 In order to train the error correction model, ASR data is first labeled to obtain a training data set and a paired data set. The training data set is used for pre-training to train the initial error correction model. The paired data set is used to fine-tune the error correction model to obtain the final online error correction model. The following describes ASR data labeling, pre-training, and fine-tuning.

[0108] First, ASR data annotation.

[0109] For neural network models, the quality of labeled data greatly affects the effect of model training. In the embodiment of the present application, for ASR data labeling, a training data set and a paired data set are required. When labeling ASR data, batch data automatic cleaning is first performed, including deleting all-English samples, deleting duplicate samples, deleting samples containing only modal particles, etc.

[0110] Next, the cleaned data is annotated in batches, and texts with smooth semantics and no typos are considered positive samples. On the contrary, texts with typos and incoherent semantics are considered noise samples. Batch annotation can be performed simultaneously, avoiding repeated annotation work and improving annotation efficiency.

[0111] In an embodiment of the present application, mask samples can be obtained in the following ways: In one way, noise samples with typos and incoherent semantics in the original data that has been automatically cleaned are regarded as mask samples. In another way, the electronic device constructs mask samples based on positive samples. For example, a preset proportion of characters in the positive sample is randomly covered; for another example, a preset number of characters in the positive sample are randomly deleted. For another example, the same position in the positive sample is replaced with a different word to obtain a mask sample.

[0112] After the electronic device constructs a labeled data set including positive samples and mask samples, it uses the samples in the training data set to train the initial model to obtain an error correction model.

[0113] With this solution, after the electronic device obtains the positive sample, it uses the positive sample to construct the mask sample, thereby achieving rapid acquisition of the mask sample while improving the generalization ability of the error correction model.

[0114] Optionally, in the above embodiment, when the electronic device constructs a mask sample using a positive sample in a positive sample set, first, the electronic device determines at least one masking position from the positive sample, and different masking positions correspond to different characters in the positive sample. Afterwards, for each masking position, the electronic device determines a target masking strategy from a strategy set, and masks according to the target masking strategy to obtain a masked sample, and the masking strategies in the strategy set at least include: replacing with words in a confusing sound dictionary, and the confusing sound dictionary is used to indicate words with the same or similar pronunciation.

[0115] In the embodiment of the present application, the Masked Language Model (MLM) in the BERT model is related to the ASR error correction task. MLM can be understood as a cloze test, that is, randomly masking characters or words at certain positions in the input sequence, that is, the positive sample, and then the model predicts the original characters at these positions based on the context.

[0116] In an embodiment of the present application, a strategy set is pre-set, and the strategy set includes multiple masking strategies. After the electronic device determines at least one masking position, for each masking position, a target masking strategy is determined from the strategy set, and the masking position is masked according to the target masking strategy. For example, 15% of the words in the positive sample are randomly masked. When the positive sample contains 100 characters, there are 15 masking positions; when the positive sample contains 6 characters, there is 1 masking position.

[0117] In an embodiment of the present application, the masking strategy in the strategy set at least includes: replacing words with words in a confusing sound dictionary, wherein the confusing sound dictionary is used to indicate words with the same or similar pronunciation. The electronic device uses a pinyin tool or the like to perform statistics on a pre-trained data set in advance to obtain a confusing sound dictionary. For example, please refer to Table 1.

[0118] Table 1

[0119]

[0120]

[0121] When the electronic device selects the masking position, the pinyin of the character at the selected masking position has a homophone or similar pronunciation. In this way, the target masking strategy can be used. For example, the positive sample is "cleaning near the base station", and the pinyin of "base" is "ji". The electronic device replaces "base" with the homophone "chicken" to obtain a mask sample.

[0122] In the embodiment of the present application, in addition to the above-mentioned replacement with words in the obfuscated sound dictionary, the masking strategy also includes other strategies, such as replacement with random characters in the vocabulary, keeping the original word unchanged, etc. During the masking process, taking a positive sample containing 100 characters and 15 masking positions as an example, among the 15 masking positions, 80% of the masking positions are replaced with words in the obfuscated sound dictionary, 10% of the masking positions are replaced with random characters in the vocabulary, and the remaining 10% of the masking positions keep the original word unchanged.

[0123] In addition, some fields contain many proper nouns and common terms, such as base stations, dust boxes, and map building in the field of sweeping robots. For these easy-to-error words, we cover them in units of words, and the covering strategy is the same as that of characters. For example, replace "cleaning" with "please come".

[0124] In BERT's original static MLM strategy, 15% of the positive samples are randomly masked. The masking strategy includes [MASK] tag replacement, random character replacement in the vocabulary, keeping the original word unchanged, etc. Among them, [MASK] tag replacement means replacing some characters in the positive sample with MASK, which leads to migration problems. That is, the "[MASK]" tag is input during the training process, but MASK does not appear in the input sequence of the real scene, resulting in inconsistent model output results.

[0125] In order to ensure the consistency of input in the model training phase and the use phase, the embodiment of the present application uses the "obfuscated character (word) replacement "[MASK]" mark. After the replacement, when the electronic device uses the positive sample to construct the mask sample, for each positive sample, 15% of the words in the positive sample are masked. Among the 15% positions, 80% of the masked positions are replaced with words in the obfuscated sound dictionary, 10% of the masked positions are replaced with random characters in the vocabulary, and the remaining 10% of the masked positions remain unchanged.

[0126] With this solution, when the electronic device uses positive samples to construct mask samples, it uses words in the obfuscated sound dictionary to replace the characters at the masked positions, thereby quickly and high-quality constructing a training data set, thereby achieving the purpose of training a high-quality error correction model.

[0127] Optionally, in the above embodiment, after the original data is automatically cleaned in batches, the semantically coherent and typo-free text becomes a positive sample. Text with typos and incoherent semantics is regarded as a mask sample, which is also called a noise sample. For the noise sample, the error is analyzed and the correctly labeled text is obtained to obtain a label sample, and a noise sample and a label sample are regarded as a pair of samples. In a pair of samples, the label sample is, for example, the text after error correction, and the noise sample is, for example, the text to be corrected. Afterwards, the electronic device uses the paired samples in the paired data set to fine-tune the error correction model, thereby improving the accuracy of the error correction model.

[0128] With this solution, the electronic device uses paired data sets to fine-tune the pre-trained error correction model to improve the accuracy of the error correction model, thereby achieving the goal of improving the quality of voice interaction.

[0129] Second, pre-training.

[0130] In an embodiment of the present application, the electronic device trains an initial model using samples in a training data set to obtain the error correction model, and the training data set includes the positive samples and the mask samples. The initial model is, for example, a BERT model. BERT is based on a multi-layer Transformer Encoder structure and obtains general knowledge by pre-training on a large-scale unsupervised corpus.

[0131] The traditional BERT model has a vocabulary size of more than 20,000 and a hidden unit of 768. Considering the issue of online performance, the embodiment of the present application uses a BERT model with 4 layers, 384 hidden units, and a vocabulary size of 3326 for pre-training. The BERT model is a lightweight model that greatly reduces the amount of calculation and improves reasoning efficiency.

[0132] After the electronic device constructs the mask samples based on the positive samples in the original positive sample set, a training data set is obtained. After that, the electronic device converts the positive samples and mask samples in the training data set into an inputtable format, trains the initial model, and calculates the loss value of the masked position during the training process to update the model parameters.

[0133] Finally, the fine-tuning phase.

[0134] In the pre-training stage, the embodiment of the present application constructs an end-to-end error correction model through pre-training. In the fine-tuning stage, the electronic device uses the paired data to fine-tune the error correction model, and uses the noise samples in each pair of samples in the paired data set as the input of the error correction model.

[0135] Figure 7 Schematic diagram of the error correction model in the voice control method provided in the embodiment of the present application. Figure 7 The error correction model in the embodiment of the present application includes an error detection module and an error correction module. During the fine-tuning process, for each pair of paired samples in the paired data set, the electronic device determines a first loss value and a second loss value, wherein the first loss value is used to indicate: the error of the wrong character determined using the error correction model, and the second loss value is used to indicate: the error of the replacement character determined using the error correction model. Afterwards, the electronic device fine-tunes the error correction model according to the first loss value and the second loss value.

[0136] Exemplarily, the electronic device uses an error detection module to determine the position of an erroneous character in a noise sample, and determines a first loss value based on the error of the position of the erroneous character. Afterwards, the electronic device uses an error correction module to determine a replacement character to correct the erroneous character in the noise sample, and determines a second loss value based on the error of the replacement character. Below, taking the noise sample of "cleaning near eggs" and the label sample of "cleaning near base stations" as an example, the fine-tuning process is described in detail. The fine-tuning process described in the embodiment of the present application includes the following steps:

[0137] Step 1: inputting the noise sample into the error correction model to obtain a third probability of each character in the noise sample;

[0138] This step includes the following sub-steps:

[0139] Step 11: Perform text segmentation on the noise sample to obtain a text sequence, denoted as {c 1 , c 2 , ..., c 6}; Use the pinyin tool to obtain the pinyin of each word in the noise sample and obtain the pinyin sequence, recorded as {p 1 , p 2 ,……p 6}.

[0140] Step 12: Encode the text sequence into a third one-hot sequence, and use the word embedding matrix to map the first one-hot sequence into a vector representation of a fixed length, thereby obtaining the word vector of the text to be corrected, denoted as: Ec = {e c1 ,e c2 ,…,e c6 At the same time, the pinyin sequence is encoded into a fourth unique hot sequence, and the fourth unique hot sequence is mapped into a vector representation of a fixed length using a word embedding matrix, thereby obtaining the pinyin vector of the text to be corrected, which is recorded as: Ep = {e p1 ,e p2 ,…,e p6}.

[0141] Step 12: Transform the word vector Ec into {e c1 ,e c2 ,…,e c6} is input to the Bi-GRU layer of the error correction model to obtain the hidden layer features. After the fully connected layer transforms the dimension, the sigmoid activation function is used to output the error probability of each character, that is, the third probability, denoted as P detect .

[0142] Step 2: Determine a first loss according to a third probability of each character in the noise sample and a fourth probability of each character in the label sample.

[0143] The electronic device obtains the fourth probability L of each character in the label sample detect Then, according to the third probability P of each character in the noise sample detect and the fourth probability L of each character in the label sample detect Determine the first loss loss1.

[0144] Step 3: According to the third probability of each character in the noise sample, determine a plurality of first sets, wherein the plurality of first sets correspond to characters at different positions in the noise sample, and the probability P in the first set correct There is a one-to-one correspondence with the characters in the vocabulary, where the vocabulary is a set of characters remaining after removing duplicate characters contained in each sample in the sample set.

[0145] This step includes the following sub-steps:

[0146] Step 31: According to the third probability of each character in the noise sample and the fourth probability of the pinyin, the word vector and the pinyin vector are weighted and summed to obtain the fused feature vector E of the noise sample. m .

[0147] Step 32: Fusion feature vector E m Input to the translation layer for encoding, and output the semantic representation of the noisy text, denoted as H mThen, after the fully connected layer and softmax calculation of the current position, the third probability P of each character in the vocabulary correct .

[0148] Step 4: Determine a second loss according to the plurality of first sets and the plurality of second sets.

[0149] Exemplarily, after the electronic device determines multiple second sets according to the fourth probability of each character in the label sample, it determines a second loss loss2 according to the multiple first sets and the multiple second sets. The multiple second sets correspond to different characters in the label sample, and the probability L in the second set is correct There is a one-to-one correspondence with the characters in the vocabulary.

[0150] Step 5: Fine-tune the error correction model according to the first loss and the second loss.

[0151] In the embodiment of the present application, the first loss value is used to measure the error of the error position predicted by the error correction model, and the second loss value is used to measure the error of the replacement character predicted by the error correction model. detect and sample label L detect The first loss value is recorded as loss1, and the P obtained by the error correction module is correct and sample label L correct The second loss value is recorded as loss2. The total loss value is determined according to the first loss value loss1 and the second loss value loss2, and the gradient descent algorithm is used to fine-tune the parameters of the error correction model with a smaller learning rate until the total loss value is minimized.

[0152] This solution combines the two stages of inspection and error correction to train and optimize the model, alleviating the problem of error cascades caused by the traditional multi-model pipeline method for text error correction, thereby improving the quality of voice interaction.

[0153] The control method of the cleaning robot is described in detail below in conjunction with specific application scenarios.

[0154] Application scenario 1:

[0155] The terminal device is specifically a sweeping robot, which is connected to the server and has an error correction model deployed on the server. The user finds dirt near the base station and says to the sweeping robot: "Clean near the base station", but due to the user's pronunciation problem, it sounds more like "Clean near the eggs". After the sweeping robot collects the voice signal, it sends the voice signal to the server. The server converts the voice signal into text to be corrected: Clean near the eggs. After that, the text to be corrected is input into the error correction model. The error correction model is used on the server to determine that the wrong characters are "chicken" and "egg".

[0156] After the server determines the error characters, it further determines the target probability sets corresponding to "chicken" and "egg" respectively. Each target probability set contains 3,326 probabilities. The server sorts the probabilities in the target probability set corresponding to "chicken" and finds that the character corresponding to the maximum probability is "base" in the vocabulary. Moreover, there is no character "chicken" among the characters corresponding to the top 5 probabilities before sorting. Therefore, the server replaces "chicken" in the text to be corrected with "base".

[0157] Similarly, the server sorts the probabilities in the target probability set corresponding to "egg" and finds that the character corresponding to the maximum probability is "station" in the vocabulary. Moreover, there is no character "egg" among the characters corresponding to the top 5 probabilities before sorting. Therefore, the server replaces "egg" in the text to be corrected with "station".

[0158] The target text obtained by the server is: Clean near the base station. Then, the server sends a control instruction to the sweeping robot. After receiving the control instruction, if the current distance from the base station is very far, the sweeping robot navigates to near the base station and cleans the area near the base station. If the sweeping robot is currently near the base station, it directly cleans the area near the base station.

[0159] Application scenario 2:

[0160] The terminal device is specifically a sweeping robot. The sweeping robot is connected to the server, and an error correction model is deployed on the server. When the user finds that the area near the base station is dirty and says to the sweeping robot, "Clean the eggshells", after the sweeping robot collects the voice signal, it sends the voice signal to the server. The server converts the voice signal into a text to be corrected: Clean the eggshells. Then, the text to be corrected is input into the error correction model. The server uses the error correction model to determine that the error characters are "chicken" and "egg".

[0161] After the server determines the error characters, it further determines the target probability sets corresponding to "chicken" and "egg" respectively. Each target probability set contains 3,326 probabilities. The server sorts the probabilities in the target probability set corresponding to "chicken" and finds that the character corresponding to the maximum probability is "base" in the vocabulary. However, there is a character "chicken" among the characters corresponding to the top 5 probabilities before sorting. Therefore, the server will not replace "chicken" in the text to be corrected with "base".

[0162] Similarly, the server sorts the probabilities in the target probability set corresponding to "egg" and finds that the character corresponding to the maximum probability is "station" in the vocabulary. However, there is a character "egg" among the characters corresponding to the top 5 probabilities before sorting. Therefore, the server will not replace "egg" in the text to be corrected with "station".

[0163] The target text obtained by the server is: clean the egg shell. After that, the server sends a control command to the sweeping robot. After receiving the control command, the sweeping robot collects the surrounding environment image to identify the location of the egg shell, moves to the egg shell and cleans the egg shell.

[0164] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0165] Figure 8 A schematic diagram of a voice control device provided in an embodiment of the present application. The voice control device 800 comprises: an acquisition module 81 , a conversion module 82 , a processing module 83 and a control module 84 .

[0166] An acquisition module 81, used for acquiring a voice signal;

[0167] A conversion module 82, used for converting the speech signal into a text to be corrected;

[0168] A processing module 83 is used to input the text to be corrected into an error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin to correct the text to obtain a target text;

[0169] The control module 84 is used to generate a control instruction according to the target text to control the terminal device.

[0170] In a feasible implementation, the processing module 83 is used to use the error correction model to determine the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character, wherein the first probability is used to indicate the probability of the character error; based on the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character, determine the fused feature vector of the text to be corrected; based on the fused feature vector, correct the text to be corrected to obtain the target text.

[0171] In a feasible implementation, when the processing module 83 corrects the text to be corrected to obtain the target text according to the fused feature vector, it is used to determine multiple probability sets according to the fused feature vector, the multiple probability sets correspond to characters at different positions in the text to be corrected, the probabilities in the probability sets correspond one-to-one to the characters in the vocabulary, and the vocabulary is a set of remaining characters after deduplication of characters contained in each sample in the sample set; determine the first position of the erroneous character according to the first probability of each character in the text to be corrected; determine the target probability set corresponding to the first position from the multiple probability sets; determine the maximum probability from the target probability set, and replace the erroneous character at the first position in the text to be corrected with the character corresponding to the maximum probability, so as to obtain the target text.

[0172] In a feasible implementation, the processing module 83 determines the character with the highest probability from the target probability set, and replaces the erroneous character located at the first position in the text to be corrected with the character with the highest probability, thereby obtaining the target text. It is used to determine multiple candidate characters from the vocabulary to obtain a candidate set, wherein the candidate characters are characters corresponding to a preset number of probabilities that are ranked at the top after the probabilities in the target probability set are sorted from large to small; determine whether the candidate set contains the erroneous character; when the candidate set does not contain the erroneous character, determine the character with the highest probability from the target probability set, and replace the erroneous character located at the first position in the text to be corrected with the character with the highest probability, thereby obtaining the target text.

[0173] In a feasible implementation, when the processing module 83 determines the fused feature vector of the text to be corrected based on the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character, it is used to determine the text sequence and pinyin sequence corresponding to the text to be corrected using the correction model; encode the text sequence to obtain a first one-hot sequence, and determine the word vector corresponding to the text to be corrected based on the first one-hot sequence; encode the pinyin sequence to obtain a second one-hot sequence, and determine the pinyin vector corresponding to the text to be corrected based on the second one-hot sequence; and weightedly sum the word vector and the pinyin vector based on the first probability of each character in the text to be corrected and the second probability of the pinyin to obtain the fused feature vector of the text to be corrected.

[0174] In a feasible implementation, when the processing module 83 uses the error correction model to determine the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character, it is used to use the error correction model to determine the text sequence and pinyin sequence corresponding to the text to be corrected; encode the text sequence to obtain a first one-hot sequence, and determine the word vector corresponding to the text to be corrected according to the first one-hot sequence; determine the hidden layer features of the word vector corresponding to the text to be corrected; determine the first probability of each character in the text to be corrected according to the hidden layer features; and determine the second probability of the pinyin corresponding to the character according to the first probability of each character in the text to be corrected.

[0175] For a feasible implementation, please refer to Figure 8 , the above-mentioned voice control device 800 further includes:

[0176] The training module 85 is used to input the text to be corrected into the error correction model in the processing module 83, so that the error correction model combines each character and the corresponding pinyin in the text to be corrected, and before correcting the text to be corrected to obtain the target text, construct a mask sample using the positive samples in the positive sample set, and the positive sample is a text with smooth semantics and no erroneous characters; use the samples in the training data set to train the initial model to obtain the error correction model, and the training data set contains the positive samples and the mask samples.

[0177] In a feasible implementation, when the training module 85 constructs a mask sample using a positive sample in a positive sample set, it is used to determine at least one masking position from the positive sample, and different masking positions correspond to different characters in the positive sample; for each masking position, a target masking strategy is determined from a strategy set, and masking is performed according to the target masking strategy to obtain a masked sample, and the masking strategies in the strategy set at least include: replacing with words in a confusing sound dictionary, and the confusing sound dictionary is used to indicate words with the same or similar pronunciation.

[0178] In a feasible implementation, the training module 85 uses samples in the training data set to train the initial model to obtain the error correction model, and then obtains a paired data set, wherein the paired data set includes multiple pairs of paired samples, and the paired samples include noise samples and label samples corresponding to the noise samples, and the label samples are samples obtained after error correction of the noise samples; the error correction model is fine-tuned according to the paired data set.

[0179] In a feasible implementation, when the training module 85 fine-tunes the error correction model according to the paired data set, it is used to determine a first loss value and a second loss value for each pair of paired samples in the paired data set, wherein the first loss value is used to indicate: the error of the erroneous character determined using the error correction model, and the second loss value is used to indicate: the error of the replacement character determined using the error correction model; the error correction model is fine-tuned according to the first loss value and the second loss value.

[0180] The voice control device provided in the embodiment of the present application can execute the actions of the electronic device in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0181] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 900 includes:

[0182] Processor 91 and memory 92;

[0183] The memory 92 stores computer instructions;

[0184] The processor 91 executes the computer instructions stored in the memory 92, so that the processor 91 executes the voice control method implemented by the electronic device as described above.

[0185] The specific implementation process of the processor 91 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0186] Optionally, the electronic device 900 further includes a communication component 93 , wherein the processor 91 , the memory 92 and the communication component 93 may be connected via a bus 94 .

[0187] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the voice control method implemented by the electronic device as above.

[0188] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the voice control method implemented by the above electronic device is implemented.

[0189] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0190] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A voice control method, It is characterized in that include: Acquire speech signals; Converting the speech signal into text to be corrected; Inputting the text to be corrected into the error correction model, so that the error correction model combines each character and the corresponding pinyin in the text to be corrected, and corrects the text to be corrected to obtain the target text; A control instruction is generated according to the target text to control the terminal device.

2. The method according to claim 1, It is characterized in that The step of inputting the text to be corrected into the error correction model so that the error correction model combines each character in the text to be corrected and the corresponding pinyin to correct the text to obtain the target text includes: Determine a first probability of each character in the text to be corrected and a second probability of the pinyin corresponding to each character by using the error correction model, wherein the first probability is used to indicate the probability of the character being wrong; Determine a fused feature vector of the text to be corrected according to a first probability of each character in the text to be corrected and a second probability of a pinyin corresponding to each character; According to the fused feature vector, the text to be corrected is corrected to obtain a target text.

3. The method according to claim 2, It is characterized in that The step of correcting the text to be corrected according to the fused feature vector to obtain a target text includes: Determine a plurality of probability sets according to the fused feature vector, the plurality of probability sets corresponding to characters at different positions in the text to be corrected, the probabilities in the probability sets corresponding one-to-one to characters in a vocabulary, the vocabulary being a set of remaining characters after deduplication of characters contained in each sample in the sample set; Determining a first position of an erroneous character according to a first probability of each character in the text to be corrected; Determine a target probability set corresponding to the first position from the multiple probability sets; The maximum probability is determined from the target probability set, and the erroneous character located at the first position in the text to be corrected is replaced with the character corresponding to the maximum probability, thereby obtaining the target text.

4. The method according to claim 3, It is characterized in that The step of determining a character with the highest probability from the target probability set and replacing the erroneous character at the first position in the text to be corrected with the character with the highest probability, thereby obtaining the target text, comprises: Determine a plurality of candidate characters from the word list to obtain a candidate set, wherein the candidate characters are characters corresponding to a preset number of probabilities that are ranked first after the probabilities in the target probability set are sorted from large to small; Determining whether the candidate set includes the erroneous character; When the candidate set does not contain the erroneous character, the character with the highest probability is determined from the target probability set, and the erroneous character at the first position in the text to be corrected is replaced with the character with the highest probability, thereby obtaining the target text.

5. The method according to claim 2, It is characterized in that The step of determining the fused feature vector of the text to be corrected according to the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character comprises: Determine the text sequence and pinyin sequence corresponding to the text to be corrected by using the error correction model; Encode the text sequence to obtain a first one-hot sequence, and determine the word vector corresponding to the text to be corrected according to the first one-hot sequence; Encode the pinyin sequence to obtain a second one-hot sequence, and determine the pinyin vector corresponding to the text to be corrected according to the second one-hot sequence; According to the first probability of each character in the text to be corrected and the second probability of the pinyin, the word vector and the pinyin vector are weighted and summed to obtain a fused feature vector of the text to be corrected.

6. The method according to claim 2, It is characterized in that The method of using the error model to determine the first probability of each character in the text to be corrected and the second probability of the pinyin corresponding to each character includes: Determine the text sequence and pinyin sequence corresponding to the text to be corrected by using the error correction model; Encode the text sequence to obtain a first one-hot sequence, and determine the word vector corresponding to the text to be corrected according to the first one-hot sequence; Determine the hidden layer features of the word vector corresponding to the text to be corrected; Determine a first probability of each character in the text to be corrected according to the hidden layer features; According to the first probability of each character in the text to be corrected, a second probability of the pinyin corresponding to the character is determined.

7. The method according to any one of claims 1 to 6, It is characterized in that Before inputting the text to be corrected into the error correction model so that the error correction model combines each character in the text to be corrected and the corresponding pinyin to correct the text to be corrected to obtain the target text, the method further includes: Constructing mask samples using positive samples in the positive sample set, wherein the positive samples are texts with smooth semantics and no erroneous characters; The initial model is trained using samples in a training data set to obtain the error correction model, wherein the training data set includes the positive samples and the mask samples.

8. The method according to claim 7, It is characterized in that The step of constructing a mask sample using a positive sample in a positive sample set includes: Determine at least one masking position from the positive sample, where different masking positions correspond to different characters in the positive sample; For each masking position, a target masking strategy is determined from a strategy set, and masking is performed according to the target masking strategy to obtain a mask sample, wherein the masking strategies in the strategy set at least include: replacing with words in a confusing sound dictionary, wherein the confusing sound dictionary is used to indicate words with the same or similar pronunciation.

9. The method according to claim 7, It is characterized in that After the initial model is trained using samples in the training data set to obtain the error correction model, the method further includes: Acquire a paired data set, wherein the paired data set includes multiple pairs of paired samples, wherein the paired samples include a noise sample and a label sample corresponding to the noise sample, and the label sample is a sample obtained after error correction of the noise sample; The error correction model is fine-tuned based on the paired data set.

10. The method according to claim 9, It is characterized in that The step of fine-tuning the error correction model according to the paired data set comprises: For each pair of paired samples in the paired data set, determining a first loss value and a second loss value, wherein the first loss value is used to indicate an error of an erroneous character determined using the error correction model, and the second loss value is used to indicate an error of a replacement character determined using the error correction model; The error correction model is fine-tuned according to the first loss value and the second loss value.

11. A voice control device, It is characterized in that include: An acquisition module, used for acquiring a voice signal; A conversion module, used for converting the speech signal into a text to be corrected; A processing module, used for inputting the text to be corrected into an error correction model, so that the error correction model combines each character in the text to be corrected and the corresponding pinyin, and corrects the text to be corrected to obtain a target text; The control module is used to generate a control instruction according to the target text to control the terminal device.

12. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images