Training method of speech recognition model based on next-generation reservoir computing, speech recognition method based on next-generation reservoir computing

Abstract

The present disclosure provides a next-generation reservoir computing-based speech recognition model training method, a next-generation reservoir computing-based speech recognition method, an apparatus, a device, and a storage medium. The next-generation reservoir computing-based speech recognition model training method comprises: obtaining sample speech data; performing feature extraction on the sample speech to obtain a plurality of sample speech features; inputting the plurality of sample speech features into an initial speech recognition model for processing to obtain a main speech feature, an auxiliary speech feature, and a non-repeated feature group, and output a predicted speech result based on the main speech feature, the auxiliary speech feature, and the non-repeated feature group; adjusting the output layer weight of the initial speech recognition model based on the predicted speech result and the sample speech label to obtain a target speech recognition model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a training method for a speech recognition model based on next-generation reservoir computing, a speech recognition method, apparatus, device, and storage medium based on next-generation reservoir computing. Background Technology

[0002] The next-generation reservoir is a variant of the traditional reservoir. It can directly use the input linear feature vector to generate multiple non-linear feature vectors to form the total feature vector to calculate the weights of the output layer and thus train the model. However, if the amount of feature vector data in the reservoir is too large, it will lead to excessive computation of the Kronecker product and the large scale of the output layer weights in the model, resulting in excessive hardware pressure and slow model training speed. Summary of the Invention

[0003] In view of this, the present disclosure provides a training method for a speech recognition model based on next-generation reservoir computing, a speech recognition method, apparatus, device, and storage medium based on next-generation reservoir computing.

[0004] One aspect of this disclosure provides a training method for a speech recognition model based on a next-generation reservoir computing, comprising: acquiring sample speech data, the sample speech data including sample speech and sample speech labels representing the text content of the sample speech; extracting features from the sample speech to obtain multiple sample speech features; inputting the multiple sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and a non-repetitive feature group, the non-repetitive feature group including self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and the auxiliary speech features; outputting a predicted speech result based on the main speech features, auxiliary speech features, and non-repetitive feature group, wherein the main speech features, auxiliary speech features, and non-repetitive feature group are stored in the reservoir of the initial speech recognition model; and adjusting the output layer weights of the initial speech recognition model based on the predicted speech result and the sample speech labels to obtain a target speech recognition model.

[0005] According to embodiments of this disclosure, the above-mentioned inputting the sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and a non-repetitive feature group obtained by combining the main speech features and auxiliary speech features includes: calculating the variance of the plurality of sample speech features to obtain a plurality of speech variance data; determining the sample speech features whose variance in the plurality of speech variance data is greater than a preset variance threshold as main speech features; determining the sample speech features whose variance in the plurality of speech variance data is less than a preset variance threshold as auxiliary speech features; combining the plurality of the main speech features and the auxiliary speech features to obtain a candidate feature group containing repetitive features; and selecting non-repetitive features from the candidate feature group to obtain the non-repetitive feature group.

[0006] According to embodiments of this disclosure, the preset optimization algorithm is regularized linear least squares, and the output layer weights are calculated using the following formula:

[0007]

[0008] Among them, W out Y represents the weights of the output layer mentioned above. d This represents the sample speech output matrix constructed based on the above sample speech tags; O total Let represent the sample speech input matrix constructed based on the above main speech features, the above auxiliary speech features, and the above non-repetitive feature groups; α represents the regularization parameter; and I represents the identity matrix.

[0009] According to embodiments of this disclosure, the above-mentioned feature extraction of the sample speech to obtain sample speech features includes: preprocessing the sample speech to obtain preprocessed sample speech; performing a fast Fourier transform on the preprocessed sample speech to obtain a speech domain signal, and obtaining a power spectrum based on the speech domain signal; using a Mel filter to filter the power spectrum to obtain frequency band data within the frequency range perceptible to the human ear; and performing a discrete cosine transform on the frequency band data to obtain the sample speech features.

[0010] According to embodiments of this disclosure, the preprocessing of the sample speech to obtain preprocessed sample speech includes: filtering the sample speech data using a high-pass filter to obtain filtered speech data; dividing the filtered speech data to obtain multiple short-time frame speech data, wherein every two adjacent short-time frame speech data overlap in the time dimension; and processing the short-time frame speech data using a preset window function to obtain the preprocessed sample speech.

[0011] Another aspect of this disclosure provides a speech recognition method based on next-generation reservoir computing, comprising: acquiring target speech; extracting features from the target speech to obtain target speech features; inputting the target speech features into a target speech recognition model to obtain a target speech recognition result, wherein the target speech recognition model is trained using the training method of the speech recognition model based on next-generation reservoir computing.

[0012] Another aspect of this disclosure provides a training apparatus for a speech recognition model based on a next-generation reservoir computing, comprising: an acquisition module for acquiring sample speech data, the sample speech data including sample speech and sample speech tags representing the text content of the sample speech; a feature extraction module for extracting features from the sample speech to obtain multiple sample speech features; a prediction module for inputting the multiple sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and a non-repetitive feature group, the non-repetitive feature group including self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and the auxiliary speech features, and outputting a predicted speech result based on the main speech features, auxiliary speech features, and non-repetitive feature group, wherein the main speech features, auxiliary speech features, and non-repetitive feature group are stored in the reservoir of the initial speech recognition model; and an adjustment module for adjusting the output layer weights of the initial speech recognition model based on the predicted speech result and the sample speech tags to obtain a target speech recognition model.

[0013] Another aspect of this disclosure provides a speech recognition device based on next-generation reservoir computing, comprising: a target speech acquisition module for acquiring target speech; a target speech feature extraction module for extracting features from the target speech to obtain target speech features; and a target model input module for inputting the target speech features into a target speech recognition model to obtain a target speech recognition result, wherein the target speech recognition model is trained using the training method of the speech recognition model based on next-generation reservoir computing.

[0014] Another aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the methods described above.

[0015] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the methods described above.

[0016] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, implement the methods described above.

[0017] According to the embodiments of this disclosure, by distinguishing between primary speech features and secondary speech features, only self-combined speech features obtained by self-combining primary speech features and mutually combined speech features obtained by combining primary speech features and secondary speech features are retained. This avoids the problem of too many vector features caused by combining all sample speech features in the prior art. By outputting the predicted speech results through primary speech features, secondary speech features and non-repetitive feature groups, the computational load and the scale of weights required for training can be reduced while ensuring the accuracy of the model, thereby reducing the pressure on hardware devices and improving model stability. Attached Figure Description

[0018] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 The illustration schematically depicts an application scenario of a training method and apparatus for a speech recognition model based on next-generation reservoir computing according to embodiments of the present disclosure.

[0020] Figure 2 A flowchart illustrating a method for training a speech recognition model based on next-generation reservoir computing according to an embodiment of the present disclosure is shown.

[0021] Figure 3 A schematic diagram of the NGRC network structure in related technologies is shown.

[0022] Figure 4 The diagram illustrates the relationship between nonlinear and linear eigenvectors.

[0023] Figure 5 A flowchart illustrating a speech recognition method based on next-generation reservoir computing according to an embodiment of the present disclosure is shown schematically.

[0024] Figure 6 The diagram illustrates a comparison of accuracy and weighting between a conventional NGRC speech recognition model and an improved NGRC speech recognition model according to embodiments of the present disclosure.

[0025] Figure 7 This diagram illustrates the relationship between the number of linear feature vectors and accuracy.

[0026] Figure 8 This schematically illustrates a structural block diagram of a training apparatus for a speech recognition model based on next-generation reservoir computing according to an embodiment of the present disclosure;

[0027] Figure 9 A schematic diagram illustrates a structural block diagram of a speech recognition device based on next-generation reservoir computing according to an embodiment of the present disclosure; and

[0028] Figure 10 A block diagram of an electronic device suitable for implementing a training method for a speech recognition model based on next-generation reservoir computing, according to an embodiment of the present disclosure, is shown schematically. Detailed Implementation

[0029] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0032] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0033] Next Generation Reservoir Computing (NGRC) is a variant of traditional reservoir computing. It directly uses the input linear feature vector to generate multiple low-order polynomials to form a nonlinear feature vector. The linear and nonlinear feature vectors constitute the total feature vector, which is used to train or infer the output layer of the model. This replaces the reservoir part in traditional reservoir computing networks, eliminating neurons in the network. NGRC avoids the randomness and difficulty of training a large number of hyperparameters in traditional reservoir computing networks, while retaining the advantage of only needing to train the output layer. It also makes the network more interpretable.

[0034] Next-generation reservoir computation for speech recognition requires the total feature vectors in the reservoir to calculate the output layer weights for model training. However, too many total feature vectors in the reservoir can lead to a large computational burden on the model hardware, resulting in slower model training.

[0035] In view of this, embodiments of the present disclosure provide a training method for a speech recognition model based on next-generation reservoir computing, characterized by comprising: acquiring sample speech data, the sample speech data including sample speech and sample speech tags representing the text content of the sample speech; extracting features from the sample speech to obtain multiple sample speech features; inputting the multiple sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and a non-repetitive feature group, the non-repetitive feature group including self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and auxiliary speech features; and outputting a predicted speech result based on the main speech features, auxiliary speech features, and non-repetitive feature group; and adjusting the output layer weights of the initial speech recognition model based on the predicted speech result and the sample speech tags to obtain a target speech recognition model.

[0036] Figure 1 The illustration schematically depicts an application scenario of a training method and apparatus for a speech recognition model based on next-generation reservoir computing according to embodiments of the present disclosure.

[0037] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0041] It should be noted that the training method for the speech recognition model based on next-generation reservoir computing provided in this embodiment can generally be executed by server 105. Correspondingly, the training device for the speech recognition model based on next-generation reservoir computing provided in this embodiment can generally be located in server 105. The training method for the speech recognition model based on next-generation reservoir computing provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the training device for the speech recognition model based on next-generation reservoir computing provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] The following will be based on Figure 1 The described scene, through Figures 2-4 The training method of the speech recognition model based on next-generation reservoir computing according to the disclosed embodiments is described in detail.

[0044] Figure 2 A flowchart illustrating a method for training a speech recognition model based on next-generation reservoir computing according to an embodiment of the present disclosure is shown.

[0045] like Figure 2 As shown, the training method of the speech recognition model based on next-generation reservoir computing in this embodiment includes operations S210 to S240.

[0046] Operate S210 to acquire sample speech data.

[0047] The sample speech data includes sample speech and sample speech tags that represent the text content of the sample speech.

[0048] In operation S220, feature extraction is performed on the sample speech to obtain multiple sample speech features.

[0049] According to embodiments of this disclosure, the Mel frequency cepstral coefficient algorithm can be used to extract features from the above-mentioned sample speech to obtain multiple sample speech features.

[0050] In operation S230, multiple sample speech features are input into the initial speech recognition model for processing to obtain main speech features, auxiliary speech features and non-repetitive feature groups, and the predicted speech result is output based on the main speech features, auxiliary speech features and non-repetitive feature groups.

[0051] The non-repetitive feature group includes self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and the auxiliary speech features.

[0052] According to the embodiments of this disclosure, the main speech features retain more high-dimensional information in the sample speech features compared to the auxiliary speech features. Therefore, the main speech features can be combined to form nonlinear features in NGRC, thereby ensuring the accuracy of the model based on the more high-dimensional information in the sample speech features during subsequent model adjustments.

[0053] In operation S240, the output layer weights of the initial speech recognition model are adjusted based on the predicted speech results and sample speech labels to obtain the target speech recognition model.

[0054] According to embodiments of this disclosure, the hardware device determines the number of threads based on the number of feature vectors in the reservoir. When the number of feature vectors in the reservoir decreases, the hardware device will allocate fewer threads to train the initial speech recognition model, thereby reducing the burden on the hardware device.

[0055] For example, sample speech data can be segments of a user clearly reading the numbers "one", "two", and "three". Each speech segment is labeled with a corresponding sample speech label, such as label 1 for the speech segment "one" and label 2 for the speech segment "two". Then, feature extraction is performed on each sample speech data to obtain sample speech features that can reflect information such as speech frequency and energy. The extracted sample speech features are input into the initial speech recognition model. Taking the speech segment "one" as an example, the main speech features may contain the most representative frequency and energy distribution information in the speech segment. This information can reflect the speech features of the number "one" well. The auxiliary speech features may contain some secondary feature information, such as background noise in the speech. Based on the input speech segment "one", the model may initially predict it as 1, and further adjust the output layer weights for cases where the model prediction is wrong.

[0056] According to the embodiments of this disclosure, by distinguishing between primary speech features and secondary speech features, only self-combined speech features obtained by self-combining primary speech features and mutually combined speech features obtained by combining primary speech features and secondary speech features are retained. This avoids the problem of too many vector features caused by combining all sample speech features in the prior art. By outputting the predicted speech results through primary speech features, secondary speech features and non-repetitive feature groups, the computational load and the scale of weights required for training can be reduced while ensuring the accuracy of the model, thereby reducing the pressure on hardware devices and improving model stability.

[0057] Figure 3 A schematic diagram of an NGRC network structure in related technologies is shown.

[0058] like Figure 3 As shown, the input sample speech features are linear features. Non-linear features are obtained by combining all linear features. The output layer weights are trained using the total features obtained by combining linear and non-linear features, thereby training the speech recognition model. For example, combining the input linear features [x,y,z,a,b,c] yields the non-linear features [x*x,x*y,x*z,x*a,x*b,x*c,y*x,y*y,y*z,y*a,y*b,y*c……c*c].

[0059] According to embodiments of this disclosure, the above-mentioned inputting sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and non-repetitive feature groups includes: calculating the variance of multiple sample speech features to obtain multiple speech variance data; determining the sample speech features whose variance in the multiple speech variance data is greater than a preset variance threshold as main speech features; determining the sample speech features whose variance in the multiple speech variance data is less than a preset variance threshold as auxiliary speech features; combining multiple main speech features and auxiliary speech features to obtain a candidate feature group containing repetitive features; and selecting non-repetitive features from the candidate feature group to obtain a non-repetitive feature group.

[0060] According to embodiments of this disclosure, the main speech features and auxiliary speech features can be determined by calculating the variance of linear features. Alternatively, the high-dimensional information content in the sample speech features can be calculated using other methods (e.g., calculating standard deviation, mean deviation, etc.) to determine the main speech features and auxiliary speech features. This disclosure does not impose any limitations on this. Furthermore, in the process of combining to obtain non-repeating feature groups, only the main speech features undergo self-intersection combination, while the auxiliary speech features are not self-intersection combined, which can reduce the number of features in the non-repeating feature groups.

[0061] For example, when the input sample speech features are [x,a,b,y,z,c], the corresponding variance ranking is x>y>z>a>b>c. Therefore, [x,y,z] can be used as the primary speech feature, and [a,b,c] as the secondary speech feature. Further, by combining the primary and secondary speech features, we obtain [x*x,x*y,x*z,x*a,x*b,x*c,y*x,y*y,y*z,y*a,y*b,y*c,z*x,z*y,z*z,z*a,z*b,z*c], where the self-combined speech features are [x*x, y*y,...]. The combined speech features are [x*y,x*z,x*a,x*b,x*c,y*x,y*z,y*a,y*b,y*c,z*x,z*y,z*a,z*b,z*c]. Non-repeating features are then selected from these features to obtain [x*x,x*y,x*z,x*a,x*b,x*c,y*y,y*z,y*a,y*b,y*c,z*z,z*a,z*b,z*c] as the non-repeating feature group.

[0062] Figure 4 The diagram illustrates the relationship between nonlinear and linear eigenvectors.

[0063] like Figure 4 As shown, with the increase of the dimension of the linear feature vector, the nonlinear feature vector exhibits an accelerated growth pattern.

[0064] According to embodiments of this disclosure, by using variance calculation to filter primary speech features and secondary speech features, it is possible to distinguish the more important features in speech recognition. Then, by combining and filtering the primary speech features and secondary speech features, non-repetitive feature groups can be obtained, which can optimize feature combination, reduce redundancy between features, thereby improving the efficiency and accuracy of the model and reducing the waste of computing resources.

[0065] According to the embodiments of this disclosure, the above-mentioned preset optimization algorithm is regularized linear least squares, and the output layer weights are calculated using the following formula (1):

[0066] (1)

[0067] Among them, W out Y represents the output layer weights. d This represents the sample speech output matrix constructed based on sample speech tags; O total Let represent the sample speech input matrix constructed based on the main speech features, auxiliary speech features, and non-repetitive feature groups; α represents the regularization parameter; and I represents the identity matrix.

[0068] According to embodiments of this disclosure, the above-mentioned feature extraction of sample speech to obtain sample speech features includes: preprocessing the sample speech to obtain preprocessed sample speech; performing a fast Fourier transform on the preprocessed sample speech to obtain a speech domain signal, and obtaining a power spectrum based on the speech domain signal; using a Mel filter to filter the power spectrum to obtain frequency band data within the frequency range perceptible to the human ear; and performing a discrete cosine transform on the frequency band data to obtain sample speech features.

[0069] According to embodiments of this disclosure, the Mel-scale Frequency Cepstral Coefficients (MFCC) algorithm can be used to calculate sample speech features. MFCC analyzes the spectrum of speech based on human auditory experiment results, transforming sample speech into sample speech features in frequencies perceptible to the human ear.

[0070] According to an embodiment of this disclosure, the above-mentioned preprocessing of sample speech to obtain preprocessed sample speech includes: filtering the sample speech data using a high-pass filter to obtain filtered speech data; dividing the filtered speech data to obtain multiple short-time frame speech, wherein every two adjacent short-time frame speech overlaps in the time dimension; and processing the short-time frame speech using a preset window function to obtain preprocessed sample speech.

[0071] According to embodiments of this disclosure, a pre-emphasis filter can be applied to sample speech to amplify high frequencies. The pre-emphasis filter can be, for example, a high-pass filter. After pre-emphasis, the filtered speech data needs to be divided into short-time frame speech. This is because speech is non-stationary, and it is meaningless to perform a Fourier transform on the entire long-time speech. Long-time speech will lose its frequency profile over time. However, speech is a short-time stationary signal, so a Fourier transform can be performed on the short-time frame speech. Furthermore, by connecting adjacent short-time frame speech, a good approximation of the signal frequency profile can be obtained. Therefore, each two adjacent short-time frame speech will have an overlap of about 1 / 2 or 1 / 3 in the time dimension.

[0072] According to embodiments of this disclosure, the aforementioned preset window function can be, for example, a Hamming window function. After obtaining multiple short-time frame speech, each short-time frame speech can be multiplied by a window function to increase the connectivity between the left and right ends of the short-time frame, thereby reducing spectral leakage.

[0073] According to embodiments of this disclosure, a Mel filter can be used to simulate the nonlinear perception of sound by the human ear, which is more discriminative at lower frequencies. Furthermore, discrete cosine transform can be used to decorrelate the frequency band data and perform mean normalization to balance the spectrum and improve the signal-to-noise ratio.

[0074] According to embodiments of this disclosure, by preprocessing the sample speech, noise can be effectively removed, signal distortion in the sample speech can be reduced, and the robustness of the sample speech can be enhanced. Furthermore, by using a Mel filter to simulate the human ear's sensitivity to different frequencies of sound, the extracted features are made to better match the characteristics of the human auditory system, thereby improving the naturalness and comprehensibility of the speech recognition process. By using discrete cosine filtering to extract more independent feature vectors, the computational complexity of subsequent processing is reduced.

[0075] Figure 5 A flowchart illustrating a speech recognition method based on next-generation reservoir computing according to an embodiment of the present disclosure is shown schematically.

[0076] like Figure 5 As shown, the speech recognition method based on next-generation reservoir computing in this embodiment includes operations S510 to S530.

[0077] When operating the S510, the target voice is acquired.

[0078] During operation of S520, feature extraction is performed on the target speech to obtain target speech features.

[0079] According to embodiments of this disclosure, the target speech features include primary target speech features, secondary target speech features, and a group of target non-repetitive features. The aforementioned group of target non-repetitive features includes target self-combined speech obtained by self-combining the primary target speech features and target mutual-combined speech obtained by mutual combination of the primary target speech features and the secondary target speech features.

[0080] In operation S530, the target speech features are input into the target speech recognition model to obtain the target speech recognition result. The target speech recognition model is trained using the training method of the speech recognition model based on the next-generation reservoir computing described above.

[0081] Figure 6 The diagram illustrates a comparison of accuracy and weighting between a conventional NGRC speech recognition model and an improved NGRC speech recognition model according to embodiments of the present disclosure.

[0082] like Figure 6 As shown, the improved NGRC speech recognition model in this embodiment has a recognition accuracy that is basically the same as that of the traditional NGRC speech recognition model, but the weight scale is reduced by 2.32 times.

[0083] Figure 7 The diagram illustrates the relationship between the number of linear feature vectors and accuracy.

[0084] like Figure 7 As shown, the red line represents the improved NGRC speech recognition model of this disclosure embodiment, and the black line represents the traditional NGRC speech recognition model. It can be seen that the improved NGRC speech recognition model according to this disclosure embodiment is less affected by the change in the number of linear feature vectors compared to the traditional NGRC speech recognition model, and the improved NGRC speech recognition model of this disclosure embodiment is more stable.

[0085] Based on the above-described training method for a speech recognition model based on next-generation reservoir computing, this disclosure also provides a training apparatus for a speech recognition model based on next-generation reservoir computing. The following will be combined with... Figure 8 The device is described in detail.

[0086] Figure 8 A schematic block diagram of a training apparatus for a speech recognition model based on next-generation reservoir computing according to an embodiment of the present disclosure is shown.

[0087] like Figure 8 As shown, the training device 800 for the speech recognition model based on next-generation reservoir computing in this embodiment includes an acquisition module 810, a feature extraction module 820, a prediction module 830, and an adjustment module 840.

[0088] The acquisition module 810 is used to acquire sample speech data, which includes sample speech and sample speech tags representing the text content of the sample speech. In one embodiment, the acquisition module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0089] The feature extraction module 820 is used to extract features from the sample speech to obtain multiple sample speech features. In one embodiment, the feature extraction module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0090] The prediction module 830 is used to input multiple sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and a non-repetitive feature set. Based on the main speech features, auxiliary speech features, and the non-repetitive feature set, it outputs a predicted speech result. The non-repetitive feature set includes self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by combining the main speech features and auxiliary speech features. In one embodiment, the prediction module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0091] The adjustment module 840 is used to adjust the output layer weights of the initial speech recognition model based on the predicted speech results and sample speech labels to obtain the target speech recognition model. In one embodiment, the adjustment module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0092] According to embodiments of this disclosure, any multiple modules among the acquisition module 810, feature extraction module 820, prediction module 830, and adjustment module 840 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 810, feature extraction module 820, prediction module 830, and adjustment module 840 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 810, feature extraction module 820, prediction module 830, and adjustment module 840 may be implemented at least partially as a computer program module that can perform corresponding functions when the computer program module is run.

[0093] Figure 9A schematic block diagram of a speech recognition device based on next-generation reservoir computing according to an embodiment of the present disclosure is shown.

[0094] like Figure 9 As shown, the speech recognition device 900 based on next-generation reservoir computing in this embodiment includes a target speech acquisition module 910, a target speech feature extraction module 920, and a target model input module 930.

[0095] The target speech acquisition module 910 is used to acquire target speech. In one embodiment, the target speech acquisition module 910 can be used to perform the operation S510 described above, which will not be repeated here.

[0096] The target speech feature extraction module 920 is used to extract features from the target speech to obtain target speech features. In one embodiment, the target speech feature extraction module 920 can be used to perform the operation S520 described above, which will not be repeated here.

[0097] The target model input module 930 is used to input the target speech into the target speech recognition model to obtain the target speech recognition result. The target speech recognition model is trained using the training method of the speech recognition model based on the next-generation reservoir computation described above. In one embodiment, the target model input module 930 can be used to perform the operation S530 described above, which will not be repeated here.

[0098] According to embodiments of this disclosure, any plurality of modules among the target speech acquisition module 910, the target speech feature extraction module 920, and the target model input module 930 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the target speech acquisition module 910, the target speech feature extraction module 920, and the target model input module 930 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the target speech acquisition module 910, the target speech feature extraction module 920, and the target model input module 930 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0099] Figure 10A block diagram of an electronic device suitable for implementing a training method for a speech recognition model based on next-generation reservoir computing, according to an embodiment of the present disclosure, is shown schematically.

[0100] like Figure 10 As shown, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage portion 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0101] RAM 1003 stores various programs and data required for the operation of electronic device 1000. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Processor 1001 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1002 and / or RAM 1003. It should be noted that the programs may also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0102] According to embodiments of this disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to a bus 1004. The electronic device 1000 may also include one or more of the following components connected to the input / output (I / O) interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output (I / O) interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0103] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0104] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1002 and / or RAM 1003 and / or one or more memories other than ROM 1002 and RAM 1003 described above.

[0105] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the training method for the speech recognition model based on next-generation reservoir computing provided in embodiments of this disclosure.

[0106] When the computer program is executed by the processor 1001, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0107] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1009, and / or installed from a removable medium 1011. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0108] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by processor 1001, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0109] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0112] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A training method for a speech recognition model based on next-generation reservoir computing, characterized in that, include: Acquire sample speech data, which includes sample speech and sample speech tags representing the text content of the sample speech; Multiple sample speech features are obtained by performing feature extraction on the sample speech; The multiple sample speech features are input into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features and non-repetitive feature groups. Based on the main speech features, auxiliary speech features and non-repetitive feature groups, the predicted speech result is output. The non-repetitive feature group includes self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and the auxiliary speech features. The output layer weights of the initial speech recognition model are adjusted based on the predicted speech results and the sample speech labels to obtain the target speech recognition model.

2. The method according to claim 1, characterized in that, The step of inputting the sample speech features into an initial speech recognition model for processing to obtain main speech features, auxiliary speech features, and non-repetitive feature groups includes: Variance calculation is performed on the multiple sample speech features to obtain multiple speech variance data; The speech features of samples whose variance is greater than a preset variance threshold from the multiple speech variance data are determined as the main speech features; Speech features of samples whose variance is less than a preset variance threshold from the plurality of speech variance data are identified as auxiliary speech features; Multiple primary speech features and secondary speech features are combined to obtain a candidate feature group containing repetitive features; The non-repeating feature group is obtained by filtering non-repeating features from the candidate feature group.

3. The method according to claim 1, characterized in that, The preset optimization algorithm is regularized linear least squares, and the output layer weights are calculated using the following formula: Among them, W out Y represents the output layer weights. d This represents the sample speech output matrix constructed based on the sample speech tags; O total The input matrix represents the sample speech matrix constructed based on the main speech features, the auxiliary speech features, and the non-repetitive feature group; α represents the regularization parameter; and I represents the identity matrix.

4. The method according to claim 1, characterized in that, The step of extracting features from the sample speech to obtain sample speech features includes: The sample speech is preprocessed to obtain preprocessed sample speech; The preprocessed sample speech is subjected to Fast Fourier Transform to obtain the audio domain signal, and the power spectrum is obtained based on the audio domain signal. The power spectrum is filtered using a Mel filter to obtain frequency band data within the range of frequencies perceptible to the human ear; The frequency band data is subjected to discrete cosine transform to obtain the sample speech features.

5. The method according to claim 4, characterized in that, The preprocessing of the sample speech to obtain preprocessed sample speech includes: The sample speech data is filtered using a high-pass filter to obtain filtered speech data. The filtered speech data is divided into multiple short-time frame speech data, and every two adjacent short-time frame speech data overlap in the time dimension. The short-time frame speech is processed using a preset window function to obtain the preprocessed sample speech.

6. A speech recognition method based on next-generation reservoir computing, characterized in that, The method includes: Acquire the target speech; The target speech is subjected to feature extraction to obtain target speech features; The target speech features are input into the target speech recognition model to obtain the target speech recognition result. The target speech recognition model is trained using the method described in any one of claims 1-5.

7. A training device for a speech recognition model based on next-generation reservoir computing, characterized in that, include: The acquisition module is used to acquire sample speech data, which includes sample speech and sample speech tags that represent the text content of the sample speech; The feature extraction module is used to extract features from the sample speech to obtain multiple sample speech features; The prediction module is used to input the multiple sample speech features into the initial speech recognition model for processing to obtain main speech features, auxiliary speech features and non-repetitive feature groups, and output the predicted speech result based on the main speech features, auxiliary speech features and non-repetitive feature groups. The non-repetitive feature groups include self-combined speech features obtained by self-combining the main speech features and mutually combined speech features obtained by mutual combination of the main speech features and the auxiliary speech features. The adjustment module is used to adjust the output layer weights of the initial speech recognition model based on the predicted speech results and the sample speech labels to obtain the target speech recognition model.

8. A speech recognition device based on next-generation reservoir computing, characterized in that, The method includes: The target speech acquisition module is used to acquire the target speech. The target speech feature extraction module is used to extract features from the target speech to obtain target speech features; The target model input module is used to input the target speech features into the target speech recognition model to obtain the target speech recognition result, wherein the target speech recognition model is trained using the method described in any one of claims 1-4.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 5.

Images