A speech recognition method, device, storage medium and electronic device

Abstract

This specification discloses a speech recognition method, device, storage medium and electronic device. In the speech recognition method provided in this specification, a sample speech and a labeling result corresponding to the sample speech are obtained; the sample speech is input into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer; the recognition model is trained according to the difference between the recognition result to be optimized and the labeling result; and the trained recognition model is used to recognize the target speech.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a speech recognition method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of deep learning, neural network technology has been widely used in many fields such as image recognition, speech recognition, 3D object detection, etc. These applications usually need to be deployed on edge devices. Taking speech recognition technology as an example, at present, the application of speech recognition is mainly based on raw waveforms and Mel filter-based neural network methods.

[0003] Among them, although the neural network method based on the original waveform has simple calculation steps and can realize end-to-end speech recognition, it has the disadvantages of large number of parameters and low recognition accuracy; although the method based on Mel filter has high recognition accuracy, the calculation steps of Mel filter are cumbersome, and the original audio signal needs to be short-time Fourier transformed first, and then Mel filtered, which is more complicated to implement.

[0004] Therefore, how to simplify the neural network used in speech recognition technology and improve the construction and application efficiency of speech recognition networks is an urgent problem to be solved. Summary of the invention

[0005] The present specification provides a speech recognition method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a speech recognition method, including:

[0008] Obtaining sample speech and the annotation result corresponding to the sample speech;

[0009] Inputting the sample speech into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer;

[0010] Training the recognition model according to the difference between the recognition result to be optimized and the labeling result;

[0011] The trained recognition model is used to recognize the target speech.

[0012] Optionally, obtaining a sample speech and a labeling result corresponding to the sample speech specifically includes:

[0013] Identify all target users that need to be identified;

[0014] The speech of the target user is collected as a sample speech, and the target user to which the sample speech belongs is determined as a labeling result of the sample speech.

[0015] Optionally, inputting the sample speech into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model specifically includes:

[0016] Inputting the sample speech into the resistive memory feature learning layer to obtain a conductance value sequence output by the resistive memory feature learning layer;

[0017] Inputting the conductance value sequence into the fully connected layer to obtain the intermediate features output by the fully connected layer;

[0018] Inputting the intermediate feature into the activation layer to obtain an activation function value output by the activation layer;

[0019] The activation function value is input into the output layer to obtain the recognition result to be optimized output by the output layer.

[0020] Optionally, the resistive random access memory feature learning layer outputs a conductance value sequence, specifically including:

[0021] Determine each moment in the time length included in the sample speech;

[0022] For each moment, the conductance value at that moment is determined according to the sample speech input at the previous moment and the conductance value output by the resistive random access memory feature learning layer;

[0023] The conductance values ​​at each moment are arranged in order from front to back in time to obtain a conductance value sequence of the sample speech.

[0024] Optionally, training the recognition model according to the difference between the recognition result to be optimized and the labeling result specifically includes:

[0025] The parameters of the recognition model are adjusted with the optimization goal of minimizing the difference between the recognition result to be optimized and the labeling result or converging the recognition model.

[0026] Optionally, using the trained recognition model to recognize the target speech specifically includes:

[0027] The target speech is input into the recognition model to obtain a recognition result output by the recognition model.

[0028] Optionally, the recognition model is built on hardware including a resistive random access memory.

[0029] This specification provides a speech recognition device, the device comprising:

[0030] An acquisition module, used to acquire sample speech and the annotation result corresponding to the sample speech;

[0031] An input module, used for inputting the sample speech into the recognition model to be trained, and obtaining the recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer;

[0032] A training module, used for training the recognition model according to the difference between the recognition result to be optimized and the labeling result;

[0033] The application module is used to recognize the target speech using the trained recognition model.

[0034] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned speech recognition method is implemented.

[0035] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned speech recognition method when executing the program.

[0036] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0037] In the speech recognition method provided in the present specification, a sample speech and a labeling result corresponding to the sample speech are obtained; the sample speech is input into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer; the recognition model is trained according to the difference between the recognition result to be optimized and the labeling result; and the trained recognition model is used to recognize the target speech.

[0038] When the speech recognition method provided in this specification is used to identify the identity of the speaker, a resistive memory feature learning layer based on resistive memory can be used to replace the traditional feature extraction network layer in the recognition model used to recognize the speech. The integrated storage and computing characteristics of the resistive memory can be utilized to effectively avoid the situation where data is frequently moved when the recognition model is running, simplify the construction and training process of the recognition model, and greatly improve the recognition efficiency of the recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0040] Figure 1 A flowchart of a speech recognition method in this specification;

[0041] Figure 2 A schematic diagram of the model structure of an identification model provided in this specification;

[0042] Figure 3 A schematic diagram of a mapping method for mapping a resistive memory feature learning layer to a resistive memory provided in this specification;

[0043] Figure 4 A schematic diagram of a mapping method for mapping a fully connected layer to a resistive random access memory provided in this specification;

[0044] Figure 5 A comparison chart of the effects between the recognition model provided in this manual and the traditional recognition model;

[0045] Figure 6 A schematic diagram of a speech recognition device provided in this specification;

[0046] Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0049] Figure 1 The following is a flow chart of a speech recognition method in this specification, which specifically includes the following steps:

[0050] S100: Acquire sample speech and the annotation result corresponding to the sample speech.

[0051] All steps in the speech recognition method provided in this specification can be implemented by any electronic device with computing functions, such as a terminal, a server, and the like.

[0052] The speech recognition method provided in this specification is mainly used to realize the efficient construction and application of speech recognition network. In this method, supervised training is adopted for the training of speech recognition network. Based on this, sample speech for training speech recognition network and the annotation result corresponding to the sample speech can be first obtained in this step. Among them, the sample speech can be any speech actually spoken by the user, including but not limited to various characters, words, sentences, etc.; the annotation result of a sample speech is the user to which the sample speech belongs, that is, the user who speaks the sample speech.

[0053] Generally, in the application scenario of speech recognition, it is necessary to first determine the user to be recognized. In this method, the user to be recognized is called the target user. On this basis, training samples can be purposefully obtained when training speech recognition. Specifically, all target users to be recognized can be determined; the speech of the target user can be collected as the sample speech, and the target user to whom the sample speech belongs can be determined as the annotation result of the sample speech.

[0054] For each target user, several sample voices can be obtained, and the number can be determined according to specific needs. All the sample voices obtained and the annotation results are stored together to construct a speaker data set for training the recognition model in the subsequent steps.

[0055] In a specific embodiment, speech recognition can be performed on speakers in the SITW dataset. The SITW database is a database designed for text-independent speaker recognition. The database consists of recordings of 299 speakers collected from open source media, with an average of 8 speeches per speaker. All recordings are collected under different conditions such as high-quality studio interviews, interviews, and acquisitions from original audio, including real noise, reverberation, intra-speaker variability, and compression artifacts. The speech contained in the SITW dataset can be directly used as the sample speech.

[0056] Specifically, 10 speakers can be selected from the SITW dataset, and a 10-second recording can be randomly selected from each speaker, so as to obtain 10 sample speech segments in total. First, each sample speech is divided into m segments, each segment is an n-frame audio signal, where m and n can be set according to specific needs. For example, m can be set to 128 and n to 1250, and finally a 1250×128 two-dimensional waveform signal can be formed. 1250 represents the number of all time frames of the 10 sample speech, and 128 represents the input feature dimension of each frame. 80% of the time frames are selected as the training set, and the remaining 20% ​​of the time frames are selected as the test set, that is, the number of samples in the training set is 1000, and the number of samples in the test set is 250.

[0057] S102: Inputting the sample speech into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer.

[0058] After obtaining the sample speech and the corresponding annotation results in step S100, the recognition model can be trained in this step. The model structure of the recognition model used in the speech recognition method provided in this specification may include at least a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer. In addition to the necessary network layers mentioned above, the recognition model may also include other network layers, such as an input layer, etc., and this specification does not impose specific restrictions on this.

[0059] Figure 2 This is a schematic diagram of the model structure of a recognition model provided in this specification. Figure 2 As shown, through each network layer included in the recognition model, the recognition model can output the recognition result to be optimized according to the input sample speech. Specifically, the sample speech can be input into the resistive memory feature learning layer to obtain the conductance value sequence output by the resistive memory feature learning layer; the conductance value sequence can be input into the fully connected layer to obtain the intermediate features output by the fully connected layer; the intermediate features can be input into the activation layer to obtain the activation function value output by the activation layer; the activation function value can be input into the output layer to obtain the recognition result to be optimized output by the output layer. Among them, the activation layer can be an activation layer including, for example, a sigmoid activation function, and the output layer can be an output layer such as softmax. Of course, since the recognition model at this time has not been trained, the output recognition result to be optimized is not accurate.

[0060] In the audio recognition method provided in this specification, the resistive random access memory is used as a network layer in the recognition network according to the drift-diffusion dynamic characteristics of the resistive random access memory. By adaptively adjusting the parameters of the resistive random access memory to learn the features of different audios based on the resistive random access memory, the generalization ability of the model can be effectively improved.

[0061] The RRAM feature learning layer is a network layer built on the RRAM. In simple terms, it can be understood as a network layer that uses the RRAM as a feature of speech extraction in the recognition model. The function of the RRAM feature learning layer is to extract and learn the features of the input sample speech. Generally, the output of the RRAM is a conductance value. Based on this, in this method, the conductance value output by the RRAM can be used as the feature output by the RRAM feature learning layer.

[0062] In most cases, the audio input into the recognition model is a temporally continuous audio signal. Therefore, when the input signal is continuous, the output of the resistive memory feature learning layer is not a single value, but a sequence of conductance values. Determine each moment in the time length contained in the sample speech; for each moment, determine the conductance value of the moment according to the sample speech input at the previous moment and the conductance value output by the resistive memory feature learning layer; arrange the conductance values ​​of the moments in time from front to back to obtain the conductance value sequence of the sample speech. Among them, each moment contained in the sample speech can be set according to demand. For example, the number of frames contained per second can be pre-set, and each frame of audio in the sample speech can be regarded as a moment, etc. This manual does not impose specific restrictions on this.

[0063] The specific implementation method of the resistive memory feature learning layer is that according to the characteristics of the resistive memory, as the number of pulses applied to the resistive memory increases, the conductivity value of the resistive memory tends to decay. The specific decay method of the resistive memory can be expressed as follows:

[0064]

[0065] Wherein, t represents the current moment, t-1 represents the previous moment; x represents input, y represents output; a and b represent the parameters of the resistive memory, that is, the parameters of the resistive memory feature learning layer, which can be adjusted during the training process. The input of the resistive memory feature learning layer is the voltage converted from the audio signal, and the output is the conductance value. For each sample audio, the sample audio can be mapped into a pulse sequence with different voltage amplitudes over a period of time, and the pulse sequence is applied to the resistive memory as input, and the conductance value output by the resistive memory at different voltages is read as the feature output by the resistive memory feature learning layer at different times. Furthermore, the conductance value output by the resistive memory can also be normalized, and the normalized conductance value is used as the feature output by the resistive memory feature learning layer.

[0066] The above formula shows that after the sample speech is input, the output of the resistive memory feature learning layer at each moment is related to its own input and output at the previous moment. In particular, when t = 0, that is, at the starting moment, when there is no previous moment, the output can be set to a constant 1.

[0067] S104: Training the recognition model according to the difference between the recognition result to be optimized and the labeling result.

[0068] In the process of training the recognition model, after obtaining the recognition result to be optimized output by the recognition model, the recognition model can be trained according to the recognition result to be optimized and the labeling result. Specifically, the difference between the recognition result to be optimized and the labeling result can be minimized or the recognition model converges as the optimization goal, and the parameters of the recognition model can be adjusted. In the process of adjusting the parameters, the parameters of all network layers included in the recognition model can be adjusted.

[0069] In the recognition model, the parameters and size of each network layer can be set according to the requirements. For example, the dimension of the input layer is 128; the parameters of the resistive memory feature learning layer are two values, a and b; the number of neurons in the fully connected layer is 16; the number of neurons in the output layer is 10, etc. This manual does not make specific restrictions on this.

[0070] In a specific embodiment, when training the recognition model, the audio signal of the training set of size 1000×128 can be input into the recognition model. For each input audio signal, its eigenvalue in the resistive memory feature learning layer can be obtained by the formula proposed in step S102, and then the eigenvalues ​​of all audio signals in the resistive memory feature learning layer can be obtained, and finally the eigenvalue of size 1000×128 is formed. Then the 1000×128 eigenvalue is input into the fully connected layer, and then the output of the recognition model is obtained through the sigmoid activation layer and the softmax output layer. The parameters a and b of the resistive memory feature learning layer and the parameters of other network layers are updated by the back propagation algorithm until the recognition model converges.

[0071] Additionally, for the trained recognition parameters, forward reasoning tests can be used to verify whether their functions meet expectations. In a specific embodiment, an audio signal of a test set of size 250×128 can be input into the recognition model. For each input feature dimension, the normalized conductance value of 128 moments can be obtained by the formula proposed in step S102. Then the normalized conductance value of size 250×128 is input into the fully connected layer to obtain the output value of the fully connected layer. Then, through the sigmoid activation function, the output value of the sigmoid activation function is used as the input of the softmax output layer. Finally, the output value of the softmax output layer, that is, the output value of the model, is obtained, and the index of the maximum value of the output layer is used as the category predicted by the network.

[0072] S106: Using the trained recognition model to recognize the target speech.

[0073] After the training of the recognition model is completed in step S104, the trained recognition model can be finally used in this step to realize the speech recognition function in the preset scenario. Specifically, the target speech can be input into the recognition model to obtain the recognition result output by the recognition model. The target speech is the speech spoken by the preset target user, that is, the object of collecting the sample speech in the sample speech acquisition stage.

[0074] When the speech recognition method provided in this specification is used to identify the identity of the speaker, a resistive memory feature learning layer based on resistive memory can be used to replace the traditional feature extraction network layer in the recognition model used to recognize the speech. The integrated storage and computing characteristics of the resistive memory can be utilized to effectively avoid the situation where data is frequently moved when the recognition model is running, simplify the construction and training process of the recognition model, and greatly improve the recognition efficiency of the recognition model.

[0075] Additionally, in the speech recognition method provided in this specification, the recognition model can be built on hardware including a resistive random access memory. Thus, the storage and computing characteristics of the resistive random access memory can be utilized, and the computing requirements of all network layers in the recognition model can be completed by relying on a single piece of resistive random access memory hardware. Specifically, in the process of processing audio by the recognition model, the parameters of each network layer can be mapped to the resistive random access memory in turn, and the computing functions of each network layer can be completed by the resistive random access memory.

[0076] This specification provides a specific embodiment for reference. For the resistive memory feature learning layer, since it is constructed based on the resistive memory, its parameters a and b are parameters corresponding to the resistive memory, so the parameters a and b can be directly used as the parameters a and b of the resistive memory, and the resistive memory realizes the calculation function of the resistive memory feature learning layer. Figure 3A schematic diagram of a mapping method for mapping a resistive memory feature learning layer to a resistive memory provided in this specification, such as Figure 3 As shown, the characteristic values ​​of each sample in all input feature dimensions are mapped into a pulse sequence with different voltage amplitudes over a period of time, the pulse sequence is applied to the resistive memory as input, and the normalized conductance values ​​of the resistive memory at different voltages are read as the characteristic values ​​of the resistive memory feature learning layer at different times.

[0077] Figure 4 A schematic diagram of a mapping method for mapping a fully connected layer to a resistive random access memory provided in this specification, such as Figure 4 As shown, V represents voltage value, G represents conductance value, and I represents current value. The specific mapping method is to map the output value of the resistive memory feature learning layer to the input voltage, map the trained parameters of the fully connected layer to the conductance value of the resistive memory array, and map the result of the fully connected operation to the current value output by the resistive memory.

[0078] The operation performed in the activation layer is to use the activation function to process the output of the fully connected layer. Therefore, the activation function used in the activation layer can be directly stored in the resistive random access memory, and the resistive random access memory completes the operation processing of the activation layer. For example, the sigmoid activation function can be pre-stored in the resistive random access memory, the output of the fully connected layer can be processed by the sigmoid activation function, and the processed output value can be used as the input of the output layer.

[0079] The mapping method of the output layer can be the same as the mapping method of the fully connected layer, and this specification will not be repeated here. According to the output result finally obtained by the output layer, the final classification result can be completed to obtain the identity recognition of the target speech.

[0080] Compared with the traditional speech recognition method, the speech recognition method provided in this specification can achieve a significant improvement in the accuracy of speech recognition. In a specific embodiment, the neural network structure based on the original waveform used in this specification is: input layer → convolution layer 1 → batch normalization layer 1 → RELU activation layer 1 → maximum pooling layer 1 → convolution layer 2 → batch normalization layer 2 → RELU activation layer 2 → maximum pooling layer 2 → convolution layer 3 → batch normalization layer 3 → RELU activation layer 3 → maximum pooling layer 3 → convolution layer 4 → batch normalization layer 4 → RELU activation layer 4 → maximum pooling layer 4 → convolution layer 5 → batch normalization layer 5 → RELU activation layer 5 → maximum pooling layer 5 → fully connected layer 1 → fully connected layer 2 → softmax output layer. The size of the convolution kernel is 3×3, the step size of convolution layer 1 is 3, and the step sizes of other convolution layers are all 1. The number of neurons in each layer is 128, 128, 128, 256, 256, 512, and 10 respectively; the neural network structure based on the Mel filter is: input layer → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → softmax output layer. Among them, the network input is the Mel cepstral coefficient after the original audio is Mel filtered, the number of Mel filters is 40, and the number of neurons in the three fully connected layers is 32. The above two traditional neural network models with different structures commonly used in speech recognition are compared with the recognition model provided in this manual.

[0081] Figure 5 This is a comparison chart of the effects of using the recognition model provided in this specification and the traditional recognition model. Figure 5 As shown, the parameter amount of the recognition model provided in this specification is 1 / 299 of the parameter amount of the neural network based on the original waveform, and the recognition accuracy of the test set is improved by about 5 percentage points compared with the neural network based on the original waveform. The parameter amount is 1 / 1.7 of the parameter amount of the neural network based on the Mel filter, and the recognition accuracy of the test set is improved by about 1 percentage point compared with the neural network based on the Mel filter. It can be seen that for the speaker recognition application of the SITW dataset, this application can achieve better results.

[0082] The above is the speech recognition method provided in this specification. Based on the same idea, this specification also provides a corresponding speech recognition device, such as Figure 6 shown.

[0083] Figure 6 A schematic diagram of a speech recognition device provided in this specification specifically includes:

[0084] The acquisition module 200 is used to acquire sample speech and the annotation result corresponding to the sample speech;

[0085] An input module 202, used for inputting the sample speech into the recognition model to be trained, and obtaining the recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer;

[0086] A training module 204, configured to train the recognition model according to the difference between the recognition result to be optimized and the labeling result;

[0087] The application module 206 is used to recognize the target speech using the trained recognition model.

[0088] Optionally, the acquisition module 200 is specifically configured to determine all target users that need to be identified; collect the speech of the target users as sample speech, and determine the target user to whom the sample speech belongs as the labeling result of the sample speech.

[0089] Optionally, the input module 202 is specifically used to input the sample speech into the resistive memory feature learning layer to obtain a conductance value sequence output by the resistive memory feature learning layer; input the conductance value sequence into the fully connected layer to obtain intermediate features output by the fully connected layer; input the intermediate features into the activation layer to obtain activation function values ​​output by the activation layer; input the activation function values ​​into the output layer to obtain the recognition result to be optimized output by the output layer.

[0090] Optionally, the input module 202 is specifically used to determine each moment in the time length contained in the sample speech; for each moment, determine the conductance value of the moment based on the sample speech input at the previous moment and the conductance value output by the resistive memory feature learning layer; arrange the conductance values ​​of the moments in time from front to back to obtain a conductance value sequence of the sample speech.

[0091] Optionally, the training module 204 is specifically configured to adjust the parameters of the recognition model with the optimization goal of minimizing the difference between the recognition result to be optimized and the labeling result or converging the recognition model.

[0092] Optionally, the application module 206 is specifically configured to input the target speech into the recognition model to obtain a recognition result output by the recognition model.

[0093] Optionally, the recognition model is built on hardware including a resistive random access memory.

[0094] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides speech recognition methods.

[0095] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0096] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0097] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0098] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0099] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0106] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0107] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0108] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0110] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A speech recognition method, characterized in that: include: Obtaining sample speech and the annotation result corresponding to the sample speech; Inputting the sample speech into a recognition model to be trained to obtain a recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer; Training the recognition model according to the difference between the recognition result to be optimized and the labeling result; Recognize the target speech using the trained recognition model; The step of inputting the sample speech into the recognition model to be trained to obtain the recognition result to be optimized output by the recognition model specifically includes: Inputting the sample speech into the resistive memory feature learning layer to obtain a conductance value sequence output by the resistive memory feature learning layer; Inputting the conductance value sequence into the fully connected layer to obtain the intermediate features output by the fully connected layer; Inputting the intermediate feature into the activation layer to obtain an activation function value output by the activation layer; Inputting the activation function value into the output layer to obtain the recognition result to be optimized output by the output layer; The resistive memory feature learning layer outputs a conductance value sequence, specifically including: Determine each moment in the time length included in the sample speech; For each moment, the conductance value at that moment is determined according to the sample speech input at the previous moment and the conductance value output by the resistive random access memory feature learning layer; The conductance values ​​at each moment are arranged in order from front to back in time to obtain a conductance value sequence of the sample speech.

2. The method according to claim 1, characterized in that Acquiring sample speech and the annotation results corresponding to the sample speech, specifically including: Identify all target users that need to be identified; The speech of the target user is collected as a sample speech, and the target user to which the sample speech belongs is determined as a labeling result of the sample speech.

3. The method according to claim 1, characterized in that According to the difference between the recognition result to be optimized and the labeling result, the recognition model is trained, specifically including: The parameters of the recognition model are adjusted with the optimization goal of minimizing the difference between the recognition result to be optimized and the labeling result or converging the recognition model.

4. The method according to claim 1, characterized in that The target speech is recognized by using the trained recognition model, specifically including: The target speech is input into the recognition model to obtain a recognition result output by the recognition model.

5. The method according to claim 1, characterized in that The recognition model is built on hardware including resistive random access memory.

6. A speech recognition device, characterized in that: include: An acquisition module, used to acquire sample speech and the annotation result corresponding to the sample speech; An input module, used for inputting the sample speech into the recognition model to be trained, and obtaining the recognition result to be optimized output by the recognition model, wherein the recognition model at least includes a resistive memory feature learning layer, a fully connected layer, an activation layer, and an output layer; A training module, used for training the recognition model according to the difference between the recognition result to be optimized and the labeling result; An application module, used for recognizing a target speech by using the trained recognition model; The input module is specifically used to input the sample speech into the resistive memory feature learning layer to obtain the conductance value sequence output by the resistive memory feature learning layer; input the conductance value sequence into the fully connected layer to obtain the intermediate features output by the fully connected layer; input the intermediate features into the activation layer to obtain the activation function value output by the activation layer; input the activation function value into the output layer to obtain the recognition result to be optimized output by the output layer; The input module is specifically used to determine each moment in the time length contained in the sample speech; for each moment, determine the conductance value of the moment according to the sample speech input at the previous moment and the conductance value output by the resistive memory feature learning layer; arrange the conductance values ​​of the moments in time from front to back to obtain the conductance value sequence of the sample speech.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images