Training method, recognition method and device for Nigerian soft speech model

Abstract

The present invention relates to the field of speech recognition technology, and specifically to a training method, recognition method, and device for a Nigerian Piruan speech model. The training method comprises obtaining a Nigerian Piruan audio sample and its corresponding first text sample; inputting the Nigerian Piruan audio sample into a convolution unit of the Nigerian Piruan speech model to obtain audio features, and inputting the audio features into a fully connected layer to obtain a first predicted text; based on the first predicted text and the first text sample, adjusting the parameters of the convolution unit and the fully connected layer to determine the Nigerian Piruan speech model. Since a fully convolutional network can simultaneously observe historical information and future information of an audio sequence and does not require additional features to record, the parameters of the network model can also be designed to be relatively small, the computing resources required to be consumed can also be greatly reduced, and it can be run on a mobile terminal.

Description

Technical Field

[0001] The present invention relates to the technical field of speech recognition, and in particular to a training method, a recognition method and a device for a Nigerian Piruan speech model. Background Art

[0002] Speech recognition, the process of converting human voice signals into text, is a key technology in the field of artificial intelligence perception. With the development of deep learning technology, speech recognition accuracy and speed have achieved significant progress. Today, speech recognition technology has permeated many applications in our daily lives, such as smart speakers and shopping guide robots. However, existing speech recognition research has largely focused on the most widely used languages, such as English and Chinese, limiting its application to more developed regions and cities.

[0003] There are 6,809 languages spoken in the world, most of which are minority languages with very few speakers. Research on speech recognition for minority languages is a key bridge for fostering closer communication between people speaking different languages. Nigerian Piran, a Chadic language from the Afro-Asiatic language family, is one of the three most important languages in Africa. Training a speech recognition model typically requires thousands of hours of audio. When training a speech model using audio files, a bidirectional recurrent neural network model is typically used. This model uses a set of hidden state features to process audio sequences and store historical information for prediction in the next time period. Therefore, the dimensionality of the hidden state features cannot be too small, otherwise it will affect the network's prediction performance. This model also requires a large number of parameters, consumes a lot of computing resources, and has low training efficiency. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method for training, a method for recognizing, and an apparatus for a Nigerian linguistic speech model to solve the problem of low speech model training efficiency.

[0005] According to a first aspect, an embodiment of the present invention provides a method for training a Nigerian soft speech model, comprising:

[0006] Obtain a Nigerian Piruan audio sample and its corresponding first text sample;

[0007] Inputting the Nigerian Piruan audio sample into a convolutional unit of a Nigerian Piruan speech model to obtain audio features, and inputting the audio features into a fully connected layer to obtain a first predicted text;

[0008] Based on the first predicted text and the first text sample, parameters of the convolution unit and the fully connected layer are adjusted to determine a Nigerian Piruan speech model.

[0009] The present invention provides a method for training a Nigerian Piruan speech model. The Nigerian Piruan speech model includes convolutional units and fully connected layers connected to the convolutional units. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information in an audio sequence without requiring additional features to record, the network model parameters can be designed to be relatively small, significantly reducing the computing resources required and enabling operation on mobile devices.

[0010] In combination with the first aspect, in the first embodiment of the first aspect, the convolution unit includes at least one convolution sub-unit, and each of the convolution sub-units in the convolution unit is connected in a residual manner; wherein the convolution sub-unit includes a connected depth-separable convolution layer, a normalization layer, and an activation function layer.

[0011] The training method for the Nigerian Piruan speech model provided in the embodiment of the present invention adopts a depthwise separable convolutional layer in each convolution subunit for feature extraction, which greatly reduces the amount of calculation and improves the efficiency of training.

[0012] In combination with the first aspect, in the second implementation of the first aspect, inputting the Nigerian Piruan audio sample into a convolution unit of a Nigerian Piruan speech model to obtain audio features, and inputting the audio features into a fully connected layer to obtain a first predicted text includes:

[0013] Obtaining an English audio sample and its corresponding second text sample;

[0014] Forming a training data set based on the English audio sample, the Nigerian Piruan audio sample, the corresponding second text, and the first text sample;

[0015] sequentially inputting audio samples in the training data set into the convolution unit to obtain audio features;

[0016] The audio features are input into the fully convolutional layer to obtain a corresponding first preset text.

[0017] The present invention provides a method for training a speech model for Nigerian Piruan. Nigerian Piruan is a minority language, and corresponding audio samples are relatively scarce. Because Nigerian Piruan and English are highly related, sharing a common alphabet and many words, adding English audio samples and their corresponding second text samples to the training dataset enriches the sample data and enhances the model's generalization capabilities.

[0018] In combination with the second embodiment of the first aspect, in the third embodiment of the first aspect, adjusting the parameters of the Nigerian Piruan speech model based on the first predicted text and the first text sample to determine the Nigerian Piruan speech model includes:

[0019] Extracting a text sample corresponding to the audio sample input into the convolution unit from the training data set;

[0020] A loss function is calculated based on the extracted text sample and the first predicted text, and the parameters of the convolution unit and the fully connected layer are adjusted to determine the Nigerian Piruan speech model.

[0021] In combination with the first aspect, or the first to third embodiments of the first aspect, in a fourth embodiment of the first aspect, the Nigerian soft speech model further includes a language model connected to the fully connected layer, and the training method further includes:

[0022] Fix the parameters of the convolution unit and the fully connected layer, and initialize the parameters of the language model;

[0023] Inputting the Nigerian Piruan audio sample into the Nigerian Piruan speech model to obtain a second predicted text;

[0024] Based on the second predicted text and the error of the second text sample, the parameters of the language model are adjusted to determine the Nigerian Piruan speech model.

[0025] The training method for the Nigerian Piruan speech model provided in an embodiment of the present invention further connects a language model after the fully connected layer to further optimize the predicted text output by the fully connected layer, thereby improving the accuracy of the Nigerian Piruan speech model.

[0026] In combination with the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect, inputting the Nigerian Piruan audio sample into the Nigerian Piruan speech model to obtain the second predicted text includes:

[0027] Performing character probability statistics on the predicted text output by the fully connected layer using the language units in the language model;

[0028] Decoding is performed based on the result of the character probability statistics to obtain the second predicted text.

[0029] The training method for the Nigerian Piruan speech model provided in the embodiment of the present invention uses a language model to optimize the decoding process of the prediction probability sequence, because Nigerian Piruan text resources are easier to obtain than audio resources, thereby further improving the final speech recognition prediction results.

[0030] According to a second aspect, an embodiment of the present invention further provides a method for recognizing Nigerian soft speech, comprising:

[0031] Obtaining a Nigerian Piruan speech model, where the Nigerian Piruan speech model is trained according to the Nigerian Piruan speech model training method described in the first aspect of the present invention or any embodiment of the first aspect;

[0032] The target Nigerian Piruan language audio is input into the Nigerian Piruan language speech model to determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

[0033] In the Nigerian Piruan speech recognition method provided in an embodiment of the present invention, the Nigerian Piruan speech model includes a convolutional unit and a fully connected layer connected to the convolutional unit. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information of an audio sequence without requiring additional features to record, the network model parameters can be designed to be relatively small, significantly reducing the required computing resources and enabling operation on mobile terminals.

[0034] According to a third aspect, an embodiment of the present invention further provides a training device for a Nigerian soft speech model, comprising:

[0035] A first acquisition module is used to acquire a Nigerian Piruan language audio sample and a corresponding first text sample;

[0036] a training module, configured to input the Nigerian Piruan audio sample into a convolutional unit of a Nigerian Piruan speech model to obtain audio features, and input the audio features into a fully connected layer to obtain a first predicted text;

[0037] An adjustment module is used to adjust the parameters of the convolution unit and the fully connected layer based on the first predicted text and the first text sample to determine a Nigerian Piruan language speech model.

[0038] The present invention provides a device for training a Nigerian Piruan speech model. The Nigerian Piruan speech model includes convolutional units and fully connected layers connected to the convolutional units. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information in an audio sequence without requiring additional features to record, the network model parameters can be designed to be relatively small, significantly reducing the computing resources required and enabling operation on mobile devices.

[0039] According to a fourth aspect, an embodiment of the present invention further provides a device for recognizing Nigerian soft speech, comprising:

[0040] A second acquisition module is configured to acquire a Nigerian Piruan speech model, wherein the Nigerian Piruan speech model is trained according to the Nigerian Piruan speech model training method described in the first aspect of the present invention or any one of the embodiments of the first aspect;

[0041] The recognition module is used to input the target Nigerian Piruan language audio into the Nigerian Piruan language speech model to determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

[0042] The Nigerian Piruan speech recognition device provided by the embodiment of the present invention can ensure the accuracy of the recognition result based on the accuracy of the Nigerian Piruan speech model.

[0043] According to the fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the training method of the Nigerian Pisan speech model described in the first aspect or any one embodiment of the first aspect, or the Nigerian Pisan speech recognition method described in the second aspect.

[0044] According to the sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the training method of the Nigerian Pisan speech model described in the first aspect or any one embodiment of the first aspect, or the recognition method of Nigerian Pisan speech described in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a flow chart of a method for training a Nigerian Piruan speech model according to an embodiment of the present invention;

[0047] Figure 2 is a flow chart of a method for training a Nigerian Piruan speech model according to an embodiment of the present invention;

[0048] Figure 3 is a flow chart of a method for training a Nigerian Piruan speech model according to an embodiment of the present invention;

[0049] Figure 4 2 is a processing architecture diagram of a method for training a Nigerian Piruan speech model according to an embodiment of the present invention;

[0050] Figure 5 is a flow chart of a method for recognizing Nigerian soft speech according to an embodiment of the present invention;

[0051] Figure 6 2 is a block diagram of a training device for a Nigerian Piruan speech model according to an embodiment of the present invention;

[0052] Figure 7 2 is a block diagram of a device for recognizing Nigerian Pisoft speech according to an embodiment of the present invention;

[0053] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0055] It should be noted that the Nigerian Piruan language described in the embodiments of the present invention is a transliteration of Nigerian Pidgin. Among them, the Chinese translation of pidgin is sometimes called Piruan language and sometimes called Pidgin language. There is no limitation on the specific name herein.

[0056] According to an embodiment of the present invention, an embodiment of a method for training a Nigerian Piruan speech model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0057] In this embodiment, a method for training a Nigerian soft speech model is provided, which can be used in electronic devices such as computers, mobile terminals, etc. Figure 1 : is a flow chart of a method for training a Nigerian soft speech model according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0058] S11, obtaining a Nigerian Piruan language audio sample and a corresponding first text sample.

[0059] The electronic device may obtain the Nigerian soft language audio sample and its corresponding first text sample from the outside, or may store them in the electronic device, or the electronic device may first obtain the original audio file and obtain the audio sample and its corresponding text sample by processing the original audio file and its corresponding original text file.

[0060] Nigerian Piruan is written in the same way as English, so text processing can be done in a similar way to English. First, the character set is determined, which has a total of 26 English letters.

[0061] When processing the original text, the electronic device first traverses each character in the original text, removes characters that do not belong to the 26 letters, and removes unnecessary spaces and line breaks. For Arabic numerals or abbreviations in the text, they must be converted into the corresponding complete Nigerian Piruan words. Finally, all characters in the text need to be converted to lowercase. In the Nigerian Piruan speech model described in the embodiment of the present invention, the case of letters is not distinguished.

[0062] Furthermore, when electronic devices process raw audio, in order to facilitate the training of the Nigerian Piruan speech model, the length of the audio needs to be constrained. Among them, based on daily communication habits, the length of a sentence is about 10 seconds. For example, electronic devices can use the VAD tool in the WebRTC library to perform endpoint detection on all audio, divide the continuous audio into smaller audio files containing only one sentence, and then perform duration statistics on each audio, removing audio that is less than 1 second and greater than 20 seconds. Secondly, there may be long silent segments in the audio, which brings redundant information. For example, the silence removal tool in the Librosa library can be used to remove the silence in the audio, retaining only the audio segments with human voices.

[0063] After processing the audio and corresponding text, the electronic device needs to align the audio and text. The audio is the input signal, and the text is the supervisory signal used to calculate the loss. Therefore, each audio file and text must have a one-to-one correspondence to ensure proper speech model training. If at least one pronunciation or word in the audio and text content does not match, the audio and text content will be discarded.

[0064] S12: Input the Nigerian Piruan audio sample into the convolution unit of the Nigerian Piruan speech model to obtain audio features, and input the audio features into the fully connected layer to obtain the first predicted text.

[0065] The convolution unit can include multiple convolution layers, or an activation function layer combined with a convolution layer. There is no restriction on the specific structure of the convolution unit, and it can be configured according to the actual situation, as long as it can achieve feature extraction.

[0066] The electronic device inputs the Nigerian Piruan audio sample into the convolution unit of the Nigerian Piruan speech model, uses the convolution unit to extract features, and obtains audio features; when the audio features are input into the fully connected layer, since the fully connected layer is used to classify characters, the fully connected layer can output the first predicted text.

[0067] S13: Based on the first predicted text and the first text sample, adjust the parameters of the convolution unit and the fully connected layer to determine a Nigerian Piruan speech model.

[0068] After obtaining the first predicted text, the electronic device calculates a loss function using the first predicted text and the first text sample. Based on the loss function calculation results, the parameters of the convolutional unit and the fully connected layer are updated. Through continuous learning and adjustment, the parameters of the convolutional unit and the fully connected layer in the Nigerian soft speech model can be determined. For example, a CTC loss function can be used. Of course, other loss functions can also be used, and this embodiment of the present invention does not impose any limitation on this.

[0069] The Nigerian Piruan speech model obtained through the above training is based on character modeling, which can get rid of the pronunciation dictionary and perform end-to-end training.

[0070] This embodiment provides a method for training a Nigerian Piruan speech model. The Nigerian Piruan speech model includes convolutional units and fully connected layers connected to the convolutional units. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information in an audio sequence without requiring additional features to record, the network model parameters can be designed to be relatively small, significantly reducing the computing resources required and enabling operation on mobile devices.

[0071] In this embodiment, a method for training a Nigerian soft speech model is provided, which can be used in electronic devices such as computers, mobile terminals, etc. Figure 2 : is a flow chart of a method for training a Nigerian soft speech model according to an embodiment of the present invention, such as Figure 2 As shown, the process includes the following steps:

[0072] S21, obtaining a Nigerian Piruan language audio sample and a corresponding first text sample.

[0073] For details, please see Figure 1 S11 of the illustrated embodiment will not be described in detail here.

[0074] S22: Input the Nigerian Piruan audio sample into the convolution unit of the Nigerian Piruan speech model to obtain audio features, and input the audio features into the fully connected layer to obtain a first predicted text.

[0075] The convolution unit includes at least one convolution sub-unit, and each of the convolution sub-units in the convolution unit is connected in a residual manner; the convolution sub-unit includes a connected depth-separable convolution layer, a normalization layer, and an activation function layer.

[0076] Specifically, the convolutional unit is built based on depthwise separable convolution. Each convolutional unit includes a depthwise separable convolution layer, a batch normalization layer, and a ReLU nonlinear activation function layer. Each convolutional unit is stacked using residual connections. Multiple stacked convolutional subunits form the convolutional unit used for feature extraction in the Nigerian Piruan speech recognition model, completing the extraction of high-dimensional features containing complex information.

[0077] The depth-wise separable convolution layer is used for feature extraction in each convolution sub-unit, which greatly reduces the amount of computation and improves the training efficiency.

[0078] Specifically, the above S22 includes the following steps:

[0079] S221: Acquire an English audio sample and its corresponding second text sample.

[0080] Nigerian Piruan is a minority language, and acquiring large amounts of audio and text data is very expensive. However, since Nigerian Piruan is highly related to English, sharing a common alphabet and many words, the two languages can be trained using English audio samples and their corresponding text samples.

[0081] S222: Form a training data set based on the English audio sample, the Nigerian Piruan audio sample, the corresponding second text, and the first text sample.

[0082] When training the Nigerian Piruan speech model, electronic devices can add some English data as augmented data to expand the data set and enhance the model's generalization ability.

[0083] The electronic device forms a training dataset with the acquired English audio sample and its corresponding second text sample, and the Nigerian Piruan audio sample and its first text sample, for subsequent training of the Nigerian Piruan speech model. The audio samples and text samples in the training dataset have a one-to-one correspondence.

[0084] S223, sequentially inputting audio samples in the training data set into the convolution unit to obtain audio features.

[0085] After obtaining the training data set, the electronic device sequentially inputs the audio samples in the training data set into the convolution unit, uses the convolution unit to perform feature extraction, and obtains audio features.

[0086] S224: Input the audio features into the fully connected layer to obtain the corresponding first preset text.

[0087] The electronic device then inputs the audio features into a fully connected layer, which classifies the audio features and outputs a first predicted text. Specifically, the electronic device inputs the high-dimensional features output by the convolutional unit into a fully connected layer for character classification prediction. The output is the probability of letters related to a predefined language. The first predicted text is obtained by concatenating the letters with the highest probability from each prediction result.

[0088] S23: Based on the first predicted text and the first text sample, adjust the parameters of the convolution unit and the fully connected layer to determine a Nigerian Piruan speech model.

[0089] Specifically, the above S23 may include the following steps:

[0090] S231: Extract text samples corresponding to the audio samples input into the convolution unit from the training data set.

[0091] The electronic device extracts a text sample corresponding to the audio sample in S223 from the training data set.

[0092] S232: Calculate a loss function based on the extracted text sample and the first predicted text, and adjust parameters of the convolution unit and the fully connected layer to determine a Nigerian Piruan speech model.

[0093] After obtaining the first predicted text, the electronic device calculates a loss function using the first predicted text and the extracted text sample. Based on the loss function calculation results, the parameters of the convolutional units and the fully connected layer are updated. Through continuous learning and adjustment, the parameters of the convolutional units and the fully connected layer in the Nigerian Piruan speech model can be determined. For example, a CTC loss function can be used. Of course, other loss functions may also be used, and this is not limited in any way by the present invention.

[0094] This embodiment provides a method for training a speech model for Nigerian Piruan. Nigerian Piruan is a minority language, and corresponding audio samples are relatively scarce. Because Nigerian Piruan and English are highly related, sharing a common alphabet and many words, adding English audio samples and their corresponding second text samples to the training dataset enriches the sample data and enhances the model's generalization capabilities.

[0095] In this embodiment, a method for training a Nigerian soft speech model is provided, which can be used in electronic devices such as computers, mobile terminals, etc. Figure 3 : is a flow chart of a method for training a Nigerian soft speech model according to an embodiment of the present invention, such as Figure 3 As shown, the process includes the following steps:

[0096] S31, obtaining a Nigerian Piruan language audio sample and a corresponding first text sample.

[0097] For details, please see Figure 2 S21 of the illustrated embodiment will not be described in detail here.

[0098] S32: Input the Nigerian Piruan audio sample into the convolution unit of the Nigerian Piruan speech model to obtain audio features, and input the audio features into the fully connected layer to obtain a first predicted text.

[0099] For details, please see Figure 2 S22 of the illustrated embodiment will not be described in detail here.

[0100] S33: Based on the first predicted text and the first text sample, adjust the parameters of the convolution unit and the fully connected layer to determine a Nigerian Piruan speech model.

[0101] For details, please see Figure 2 S23 of the illustrated embodiment will not be described in detail here.

[0102] S34, fix the parameters of the convolution unit and the fully connected layer, and initialize the parameters of the language model.

[0103] To further improve the accuracy of the trained Nigerian Piruan speech model, the Nigerian Piruan speech model described in this embodiment includes a language model in addition to the convolutional unit and fully connected layer described in S33. Specifically, the language model is connected after the fully connected layer. That is, the Nigerian Piruan speech model includes a convolutional unit, a fully connected layer, and a language model.

[0104] S35: Input the Nigerian Piruan audio sample into the Nigerian Piruan speech model to obtain a second predicted text.

[0105] Specifically, the above S35 may include the following steps:

[0106] S351, using the language units in the language model to perform character probability statistics on the predicted text output by the fully connected layer.

[0107] The electronic device inputs the Nigerian Piruan audio sample into the Nigerian Piruan speech model, outputs the predicted text from the fully connected layer of the Nigerian Piruan speech model, and then uses the language units in the language model to perform character probability statistics on it.

[0108] The language unit may adopt the N-Gam language model or other models, and there is no limitation here. It only needs to ensure that the language unit can perform character probability statistics on the predicted text output by the fully connected layer.

[0109] S352: Decode based on the result of character probability statistics to obtain a second predicted text.

[0110] A decoding unit is connected after the language unit and is used to decode the result of character probability statistics output by the language unit to obtain a second predicted text.

[0111] Since Nigerian Piruan text resources are easier to obtain than audio resources, the language model is used to optimize the decoding process of the predicted probability sequence, further improving the final speech recognition prediction results.

[0112] Since Nigerian Piruan text resources are easier to obtain than audio resources, the language model is used to optimize the decoding process of the prediction probability sequence, further improving the final speech recognition prediction results.

[0113] S36 , adjusting parameters of the language model based on the second predicted text and the error of the second text sample to determine a Nigerian Piruan speech model.

[0114] After obtaining the second predicted text, the electronic device calculates a loss function using the second predicted text and the second text sample, and adjusts the parameters of the language model based on the calculation result. Through continuous training and parameter adjustment, the parameters of the language model are determined, and the Nigerian Piruan speech model is ultimately determined.

[0115] As an optional implementation of this embodiment, the electronic device uses the N-Gam language model to further improve the accuracy of Nigerian Piruan speech recognition. Specifically, the predicted text output by the fully connected layer is subjected to a sliding window operation of size 4, forming a byte segment sequence of length 4, and the frequency of occurrence of each byte segment sequence is then counted. During the speech recognition process, the character probability sequence output by the N-Gam language model is subjected to Beam Search decoding. During the decoding process, the statistical byte segment frequencies of the language units are introduced to optimize the search process and further improve the accuracy of the recognition results.

[0116] The training method of the Nigerian Piruan speech model provided in this embodiment further connects a language model after the fully connected layer to further optimize the predicted text output by the fully connected layer, thereby improving the accuracy of the Nigerian Piruan speech model.

[0117] As a specific implementation of this embodiment, the training of the Nigerian soft speech model can be carried out as follows: Figure 4 Specifically, the training process can be divided into three aspects: data preprocessing to obtain audio samples and their corresponding text samples; an acoustic model, including convolutional units and fully connected layers connected to the convolutional units. During the training of this model, English data is added to Nigerian Piruan data as data augmentation to obtain a training dataset; and a language model, including language units and decoding units.

[0118] According to an embodiment of the present invention, an embodiment of a Nigerian Piruan speech recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0119] In this embodiment, a method for Nigerian soft speech recognition is provided, which can be used in electronic devices such as computers, mobile terminals, etc. Figure 5 : is a flow chart of a method for Nigerian soft speech recognition according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0120] S41, obtain the Nigerian Piruan speech model.

[0121] The Nigerian Piruan speech model is obtained by training according to the training method of the Nigerian Piruan speech model described in the embodiment.

[0122] For details, please refer to the description of the training method of the Nigerian soft language speech model described in the above embodiment, which will not be repeated here.

[0123] S42: Input the target Nigerian Piruan language audio into the Nigerian Piruan language speech model to determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

[0124] The electronic device inputs the target Nigerian Piruan audio into the Nigerian Piruan speech model, and after being processed by the model, the target Nigerian Piruan text can be output.

[0125] This embodiment provides a method for recognizing Nigerian Piruan speech. The Nigerian Piruan speech model includes convolutional units and fully connected layers connected to the convolutional units. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information in an audio sequence without requiring additional features, the network model parameters can be designed to be relatively small, significantly reducing the computing resources required and enabling operation on mobile devices.

[0126] In this embodiment, a training device for a Nigerian Piruan language speech model is also provided. Furthermore, in this embodiment, a recognition device for Nigerian Piruan language speech is also provided. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0127] This embodiment provides a training device for a Nigerian soft speech model. Figure 6 As shown, including:

[0128] A first acquisition module 51 is used to acquire a Nigerian Piruan language audio sample and a corresponding first text sample;

[0129] A training module 52 is configured to input the Nigerian Piruan audio sample into a convolutional unit of a Nigerian Piruan speech model to obtain audio features, and input the audio features into a fully connected layer to obtain a first predicted text;

[0130] The adjustment module 53 is configured to adjust the parameters of the convolution unit and the fully connected layer based on the first predicted text and the first text sample to determine a Nigerian Piruan speech model.

[0131] This embodiment provides a training device for a Nigerian Piruan speech model. The Nigerian Piruan speech model includes convolutional units and fully connected layers connected to the convolutional units. The resulting Nigerian Piruan speech model can also be understood as a fully convolutional network. Because a fully convolutional network can simultaneously observe both historical and future information in an audio sequence without requiring additional features to record, the network model parameters can be designed to be relatively small, significantly reducing the computing resources required and enabling operation on mobile devices.

[0132] This embodiment also provides a Nigerian soft speech recognition device, such as Figure 7 As shown, including:

[0133] A second acquisition module 61 is configured to acquire a Nigerian Piruan speech model, wherein the Nigerian Piruan speech model is trained according to the Nigerian Piruan speech model training method described in the above embodiment;

[0134] The recognition module 62 is configured to input the target Nigerian Piruan language audio into the Nigerian Piruan language speech model and determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

[0135] The Nigerian Piruan speech recognition device provided in this embodiment can ensure the accuracy of the recognition result based on the accuracy of the Nigerian Piruan speech model.

[0136] The training device for the Nigerian Piruan speech model or the Nigerian Piruan speech recognition device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0137] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0138] An embodiment of the present invention further provides an electronic device having the above Figure 6 The training device of the Nigerian soft language voice model shown in the figure, or the ... Figure 7 The device for recognizing Nigerian Piruan speech is shown.

[0139] See also Figure 8 , Figure 8 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 8As shown, the electronic device may include: at least one processor 71, such as a CPU (Central Processing Unit), at least one communication interface 73, a memory 74, and at least one communication bus 72. The communication bus 72 is used to realize the connection and communication between these components. The communication interface 73 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 73 may also include a standard wired interface and a wireless interface. The memory 74 may be a high-speed RAM memory (Random Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 74 may optionally be at least one storage device located away from the aforementioned processor 71. The processor 71 may be combined with Figure 6 or Figure 7 In the described apparatus, the memory 74 stores an application program, and the processor 71 calls the program code stored in the memory 74 to execute any of the above method steps.

[0140] The communication bus 72 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 72 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0141] Among them, the memory 74 may include a volatile memory (English: volatile memory), such as a random-access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 74 may also include a combination of the above types of memory.

[0142] The processor 71 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0143] The processor 71 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0144] Optionally, the memory 74 is also used to store program instructions. The processor 71 can call the program instructions to implement the Figures 1 to 3 The training method of the Nigerian soft language voice model shown in the embodiment, or Figure 4 The method for recognizing Nigerian Piruan speech shown in the embodiment.

[0145] An embodiment of the present invention further provides a non-transitory computer storage medium storing computer-executable instructions, which can execute the method for training a Nigerian Piruan language speech model or the method for recognizing Nigerian Piruan language speech in any of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0146] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for training a Nigerian soft speech model, characterized in that: include: Obtain a Nigerian Piruan audio sample and its corresponding first text sample; Inputting the Nigerian Piruan audio sample into a convolutional unit of a Nigerian Piruan speech model to obtain audio features, and inputting the audio features into a fully connected layer to obtain a first predicted text, including: obtaining an English audio sample and a corresponding second text sample; Forming a training data set based on the English audio sample, the Nigerian Piruan audio sample, the corresponding second text, and the first text sample; sequentially inputting audio samples in the training data set into the convolution unit to obtain audio features; Inputting the audio features into the fully connected layer to obtain a corresponding first predicted text; Adjusting parameters of the convolution unit and the fully connected layer based on the first predicted text and the first text sample to determine a Nigerian Piruan speech model; The Nigerian Piruan speech model also includes a language model connected to the fully connected layer, and the training method further includes: Fix the parameters of the convolution unit and the fully connected layer, and initialize the parameters of the language model; Performing character probability statistics on the predicted text output by the fully connected layer using the language units in the language model; Decoding is performed based on the result of the character probability statistics to obtain a second predicted text; Based on the second predicted text and the first text sample, the parameters of the language model are adjusted to determine the Nigerian Piruan speech model.

2. The training method according to claim 1, characterized in that The convolution unit includes at least one convolution subunit, and each of the convolution subunits in the convolution unit is connected in a residual manner; wherein the convolution subunit includes a connected depth-separable convolution layer, a normalization layer, and an activation function layer.

3. The training method according to claim 1, characterized in that The adjusting the parameters of the Nigerian Piruan speech model based on the first predicted text and the first text sample to determine the Nigerian Piruan speech model includes: Extracting a text sample corresponding to the audio sample input into the convolution unit from the training data set; A loss function is calculated based on the extracted text sample and the first predicted text, and the parameters of the convolution unit and the fully connected layer are adjusted to determine the Nigerian Piruan speech model.

4. A method for recognizing Nigerian soft speech, characterized in that: include: Obtaining a Nigerian Piruan speech model, wherein the Nigerian Piruan speech model is trained according to the Nigerian Piruan speech model training method according to any one of claims 1 to 3; The target Nigerian Piruan language audio is input into the Nigerian Piruan language speech model to determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

5. A training device for a Nigerian soft speech model, based on the method of claim 1, characterized in that: include: A first acquisition module is used to acquire a Nigerian Piruan language audio sample and a corresponding first text sample; a training module, configured to input the Nigerian Piruan audio sample into a convolutional unit of a Nigerian Piruan speech model to obtain audio features, and input the audio features into a fully connected layer to obtain a first predicted text; An adjustment module is used to adjust the parameters of the convolution unit and the fully connected layer based on the first predicted text and the first text sample to determine a Nigerian Piruan language speech model.

6. A device for recognizing Nigerian soft speech, characterized in that: include: A second acquisition module is used to acquire a Nigerian Piruan speech model, where the Nigerian Piruan speech model is trained according to the Nigerian Piruan speech model training method according to any one of claims 1 to 3; The recognition module is used to input the target Nigerian Piruan language audio into the Nigerian Piruan language speech model to determine the target Nigerian Piruan language text corresponding to the target Nigerian Piruan language audio.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for training the Nigerian Pirouan speech model according to any one of claims 1 to 3, or the method for recognizing Nigerian Pirouan speech according to claim 4, by executing the computer instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the training method for the Nigerian Piruan language speech model according to any one of claims 1 to 3, or the Nigerian Piruan language speech recognition method according to claim 4.

Images