Speech recognition method and device, equipment and storage medium
Through the self-attention mechanism, the query vector and update key vector are generated in speech recognition, which solves the problem of high and low efficiency of speech recognition memory in the prior art, and improves the accuracy and efficiency of speech recognition.
Patent Information
- Application Number
- CN202411237790.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-27
AI Technical Summary
When existing speech recognition technologies process long-term or complex speech, the memory requirements are high and inefficient, resulting in a large amount of memory occupancy and low efficiency during the recognition process.
The self-attention mechanism is adopted to generate query vectors and value vectors through the target speech recognition model and generate them at the first level, and the key vectors are updated on each layer, and the attention mechanism is optimized layer by layer to better capture the key information of speech data and reduce the Memory requirements and improve identification efficiency.
It significantly improves the accuracy and robustness of speech recognition, reduces additional storage space during speech recognition, reduces memory requirements, and improves the overall efficiency of speech recognition.
Smart Images
Figure CN119207378A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of financial technology, and in particular to a speech recognition method, a speech recognition device, a computer device, and a computer-readable storage medium. Background Art
[0002] Telephone customer service can be applied in multiple scenarios, such as intelligent customer service in the financial technology scenario. Telephone customer service interacts with customers by performing speech recognition on their voices. Currently, since the customer's voice signal is usually continuous time series data, each time step corresponds to an input vector. Therefore, when performing speech recognition, the entire voice signal needs to be cut into smaller fixed-length segments and a proper input representation needs to be constructed so that it can be processed.
[0003] The above segmentation process will bring certain memory overhead, and in the speech recognition process, since the length of the input sequence is usually long, more memory is required to store and process the relevant intermediate results, resulting in not only a large amount of memory occupied in the speech recognition process, but also low speech recognition efficiency. Summary of the invention
[0004] The present application provides a speech recognition method, a speech recognition apparatus, a computer device and a computer-readable storage medium, aiming to reduce the memory requirement in speech recognition and improve the efficiency of speech recognition.
[0005] To achieve the above purpose, the present application also provides a speech recognition method, comprising:
[0006] Acquire voice data to be recognized;
[0007] Extracting features from the speech data to obtain a speech feature vector;
[0008] The speech feature vector is converted by a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model;
[0009] Based on the query vector and value vector generated by the first layer and the key vector of the target layer, an attention calculation result of the target layer is obtained, and based on the attention calculation result, the speech data is recognized to obtain a speech recognition result.
[0010] To achieve the above objectives, the present application also provides a speech recognition device, comprising:
[0011] An acquisition module, the acquisition module is used to acquire voice data to be recognized;
[0012] A feature extraction module, wherein the feature extraction module is used to extract features from the speech data to obtain a speech feature vector;
[0013] A conversion processing module, the conversion processing module is used to convert the speech feature vector through a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model;
[0014] A speech recognition module, the speech recognition module is used to obtain the attention calculation result of the target layer based on the query vector and value vector generated by the first layer, and the key vector of the target layer, and recognize the speech data based on the attention calculation result to obtain a speech recognition result.
[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of any one of the speech recognition methods provided in the embodiments of the present application when executing the computer program.
[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the speech recognition method provided in any one of the embodiments of the present application.
[0017] The present application provides a speech recognition method, a speech recognition device, a computer device and a computer-readable storage medium, wherein the method comprises: obtaining speech data to be recognized, and performing feature extraction on the speech data to obtain a speech feature vector. Furthermore, the speech feature vector is converted and processed by a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple comprises: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model. Thus, the attention calculation result of the target layer can be obtained based on the query vector and the value vector generated at the first layer, and the key vector of the target layer, and the speech data is recognized based on the attention calculation result to obtain the speech recognition result. The present application can significantly improve the recognition accuracy by using the self-attention mechanism for speech recognition. First, the query vector and the value vector generated by the target speech recognition model can obtain the global features of the speech at the first layer. The key vectors generated by updating each layer enable the model to optimize the attention mechanism layer by layer and make more detailed adjustments to the attention of the voice data, which can not only better capture the key information in the voice data, but also improve the performance of the model when processing long or complex voices, thereby improving the accuracy and robustness of the overall voice recognition. Furthermore, since the query vector and value vector are generated in the first layer, the additional storage space in the voice recognition process can be reduced, the memory requirements in voice recognition are reduced, and the efficiency of voice recognition is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 is a scenario diagram of a speech recognition method provided in an embodiment of the present application;
[0020] Figure 2 It is a flowchart of a speech recognition method provided by an embodiment of the present application;
[0021] Figure 3 It is a schematic diagram of a flow chart of obtaining a target speech recognition model provided by an embodiment of the present application;
[0022] Figure 4 is a schematic block diagram of a speech recognition device provided in an embodiment of the present application;
[0023] Figure 5 It is a schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0025] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions. In addition, although the functional modules are divided in the device schematic, in some cases, the module division can be different from that in the device schematic.
[0026] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0027] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0028] like Figure 1 As shown, the speech recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The application environment includes a terminal device 110 and a server 120, wherein the terminal device 110 can communicate with the server 120 through a network. Specifically, the server 120 can obtain the voice data to be recognized; perform feature extraction on the voice data to obtain a voice feature vector; convert the voice feature vector through a target voice recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target voice recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target voice recognition model; the attention calculation result of the target layer is obtained based on the query vector and value vector generated at the first layer, and the key vector of the target layer, and the voice data is recognized based on the attention calculation result to obtain a voice recognition result, thereby sending the voice recognition result to the terminal device 110. Among them, the server 120 can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device 110 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0029] Please continue reading Figure 2 , Figure 2 is a flow chart of a speech recognition method provided by an embodiment of the present application. Figure 2 As shown, the speech recognition method can be implemented through steps S11 to S13.
[0030] Step S11: Acquire speech data to be recognized.
[0031] For example, speech data usually exists in the form of audio signals, which can be obtained through recording, real-time microphone input, telephone voice, etc. Among them, the speech signal can be digitized through sampling and quantization processes to form digital speech data that can be processed on a computer. Furthermore, speech data is usually represented as a one-dimensional time series, indicating the intensity change of sound in the time dimension.
[0032] Step S12: extracting features from the speech data to obtain a speech feature vector.
[0033] The purpose of feature extraction is to extract features from speech data that can effectively represent the speech content. Since speech data itself is complex and contains a lot of information, such as frequency, amplitude, pitch, etc., it is not conducive to the efficient processing of the model. Therefore, through feature extraction, the speech data is converted into a low-dimensional feature vector, which can reduce redundant information and retain key information.
[0034] For example, feature extraction may be performed using Mel-frequency cepstral coefficients, linear prediction cepstral coefficients, amplitude spectrum, phase spectrum, etc., which is not limited in the present application.
[0035] Step S13: The speech feature vector is converted through the target speech recognition model to obtain a triplet for self-attention calculation.
[0036] Among them, the triplet includes: query vector, key vector and value vector; the query vector and value vector are generated in the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated in each layer of the target speech recognition model.
[0037] Furthermore, the self-attention mechanism is a technique for capturing long-range dependencies in sequence data (such as speech or text). It enables the model to refer to other parts of the entire sequence while processing the current input, thereby better understanding and capturing key information in the speech.
[0038] For example, in the first layer of the target speech recognition model, that is, the first stage when the model starts to process the input feature vector, the query vector and the value vector are generated. In other words, the model processes the entire speech feature vector globally at the beginning, generates an initial set of query and information extraction mechanisms, and lays the foundation for the subsequent attention mechanism. In the self-attention mechanism, the model is processed in layers, and each layer updates its parameters based on the output of the previous layer. In this hierarchical structure, the key vector is updated and generated at each layer. This layer-by-layer update process enables the model to gradually optimize its attention mechanism and locate the most relevant information in the speech data more and more accurately.
[0039] For example, after obtaining the query vector, key vector, and value vector, the model determines the distribution of attention by calculating the similarity between the query vector and the key vector. Specifically, the model calculates the dot product of the query vector and all key vectors and normalizes the result into a probability distribution. These probability values are used to weight the value vector to obtain the final attention output.
[0040] In the above way, by converting the speech feature vector into query vector, key vector and value vector, and dynamically updating these vectors at each layer, the target speech recognition model can better capture and understand the complex dependencies in the speech data. The self-attention mechanism enables the model to refer to the contextual information of the entire speech sequence when processing the current speech segment, thereby improving the accuracy of speech recognition. Furthermore, since the query vector and value vector are generated in the first layer, the additional storage space in the speech recognition process can be reduced, which reduces the memory requirements in speech recognition and improves the efficiency of speech recognition.
[0041] Step S14: Based on the query vector and value vector generated by the first layer and the key vector of the target layer, the attention calculation result of the target layer is obtained, and the speech data is recognized based on the attention calculation result to obtain the speech recognition result.
[0042] For example, the model calculates the similarity between the query vector and the key vector of the target layer, and then converts these similarity values into weights to indicate the degree of attention paid by the target speech recognition model to each value vector in the current layer. Furthermore, after being processed by the self-attention mechanism, the output of the target speech recognition model at each layer contains a weighted summary of the global and local features of the speech data. These outputs are passed and updated layer by layer, and finally after several layers of attention calculation, a highly comprehensive information representation can be obtained. After completing the attention calculation of all levels, the final attention output will be input into one or more classification layers to convert these high-dimensional feature representations into specific recognition results.
[0043] In the above method, by using the query vector and value vector generated by the first layer and the key vector of the target layer, the self-attention results are optimized and calculated layer by layer, so that the target speech recognition model can comprehensively consider the global and local information of the speech data. Finally, the attention calculation results are used to recognize the speech data and obtain accurate speech recognition results. In this way, the target speech recognition model can effectively process complex speech input and improve the accuracy and robustness of recognition.
[0044] The present application provides a speech recognition method, including: obtaining speech data to be recognized, and performing feature extraction on the speech data to obtain a speech feature vector. Furthermore, the speech feature vector is converted and processed by a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model. Thus, the attention calculation result of the target layer can be obtained based on the query vector and the value vector generated at the first layer, and the key vector of the target layer, and the speech data is recognized based on the attention calculation result to obtain the speech recognition result. The present application can significantly improve the recognition accuracy by using a self-attention mechanism for speech recognition. First, the query vector and the value vector generated by the target speech recognition model can obtain the global features of the speech at the first layer. The key vectors generated by updating each layer enable the model to optimize the attention mechanism layer by layer and make more detailed adjustments to the attention of the voice data, which can not only better capture the key information in the voice data, but also improve the performance of the model when processing long or complex voices, thereby improving the accuracy and robustness of the overall voice recognition. Furthermore, since the query vector and value vector are generated in the first layer, the additional storage space in the voice recognition process can be reduced, the memory requirements in voice recognition are reduced, and the efficiency of voice recognition is improved.
[0045] Optionally, before performing feature extraction on speech data to obtain a speech feature vector, the method includes: performing a preprocessing operation on the speech data to obtain preprocessed speech data, wherein the preprocessing operation includes at least one of a noise reduction operation, a pre-emphasis operation, and a frame processing operation; performing feature extraction on the speech data to obtain a speech feature vector, including: performing feature extraction on the preprocessed speech data to obtain a speech feature vector.
[0046] For example, the preprocessing operation refers to some preliminary processing of the original speech signal before feature extraction of the speech data, the purpose of which is to improve the quality of the speech signal, so that the subsequent feature extraction is more effective. Feature extraction of the preprocessed speech data can effectively improve the quality of the speech feature vectors finally generated. These vectors can more accurately reflect the essential characteristics of the speech signal, thereby providing stronger support for the subsequent speech recognition process.
[0047] Optionally, feature extraction is performed on the preprocessed speech data to obtain a speech feature vector, including: processing the preprocessed speech data through a speech decoder in a target speech recognition model to obtain a speech coding vector output by a speech encoder; and feature extraction is performed on the speech coding vector to obtain a speech feature vector.
[0048] For example, the speech decoder is responsible for decoding the input speech signal and converting the original speech signal into an internal representation that the model can understand. Specifically, the preprocessed speech data is input into the speech decoder, which processes it and generates a speech coding vector. Among them, the speech coding vector is a low-dimensional representation extracted from the speech signal, which is suitable for subsequent speech recognition or processing tasks. After obtaining the speech coding vector, the speech coding vector can be further feature extracted to extract the important information in the speech coding vector and convert it into a speech feature vector that is more suitable for speech recognition.
[0049] Optionally, the speech feature vector is transformed through the target speech recognition model to obtain a triplet for self-attention calculation, including: transforming the speech feature vector through the filter in the target speech recognition model to obtain a query vector and a value vector for first-layer generation; performing an assignment operation at the target layer based on the speech feature vector to obtain a key vector of the target layer.
[0050] For example, the role of the filter is to convert the input speech feature vector and convert it into the form required for the next step of the model. Through the conversion process, the speech feature vector is converted into a query vector and a value vector, which are used to represent the "query" and "value" information in the input feature respectively. In the target layer, based on the query vector and value vector that have been generated, combined with the speech feature vector, an assignment operation can be performed. Among them, the assignment operation usually involves further processing of the query vector and the value vector with the speech feature vector, such as calculating a certain weighted sum or performing a nonlinear operation to generate a key vector.
[0051] In the above way, through the target speech recognition model, the speech feature vector is further processed and converted into query vector, key vector and value vector, which constitute the core triple in the self-attention mechanism. In the self-attention mechanism, the model uses these vectors to dynamically calculate the relationship between input features, thereby improving the accuracy and robustness of speech recognition.
[0052] Optionally, obtaining the attention calculation result of the target layer based on the query vector and the value vector generated by the first layer and the key vector of the target layer includes: obtaining the attention calculation result of the target layer by the following formula:
[0053] Y i =FFN(MHA(Q 1 ,K i ,V 1 ))
[0054] Among them, Y i is the attention calculation result of the target layer; Q 1 is the query vector generated by the first layer; V1 is the value vector generated by the first layer; K i is the key vector of the target layer.
[0055] Please continue reading Figure 3 , Figure 3 is a flow chart of a speech recognition method provided by an embodiment of the present application. Figure 3 As shown, the speech recognition method can be implemented through steps S21 to S23.
[0056] Step S21: Acquire a training data set and an initial speech recognition model, wherein the training data set includes a plurality of speech data.
[0057] Step S22: labeling the training data set to obtain labeling results, wherein the labeling results include speech recognition results corresponding to each speech data.
[0058] Step S23: Train the initial speech recognition model using the training data set and the annotation results to obtain a target speech recognition model.
[0059] Specifically, a training data set including a number of speech data and an initial speech recognition model can be obtained, and a number of speech data can be labeled in sequence to obtain speech recognition results corresponding to each speech data. The labeled results are then used as labels for the group of input data, and each group of labeled training data sets is input into the initial speech recognition model for supervised learning. When the training end conditions are met, such as when the number of training times reaches a number threshold or the output accuracy of the model reaches an accuracy threshold, the training ends, and the trained target speech recognition model is obtained.
[0060] It should be noted that the present application does not limit the type of target speech recognition model, for example, including BERT model, ALBERT model, RoBERTa model and XLNet model, etc., the present application does not limit this.
[0061] In the embodiment of the present application, the training data set and the annotation results can be input into the initial speech recognition model for supervised learning, and then the target speech recognition model can be trained to obtain the target speech recognition model. Thus, the recognition result corresponding to the speech to be recognized can be output based on the target speech recognition model.
[0062] Optionally, after the initial speech recognition model is trained using the training data set and the annotation results to obtain the target speech recognition model, the method includes: iteratively training the target speech recognition model based on the training data set and the annotation results to extract data features, and calculating the loss function; iteratively training the loss function using a preset method with the purpose of reducing the value of the loss function until the expected threshold requirement is met; and obtaining the iterative target speech recognition model based on the loss function after iterative training.
[0063] It can be understood that in order to train a target speech recognition model with higher accuracy, the target speech recognition model can be repeatedly trained iteratively so that the loss function is continuously reduced until the loss function meets the expected threshold requirement, and then a more accurate speech recognition result corresponding to the speech to be recognized can be obtained based on the iterated target speech recognition model.
[0064] It should be noted that the present application does not limit the above-mentioned preset method and expected threshold. For example, the preset method may be a gradient descent algorithm, a batch gradient descent algorithm, a stochastic gradient descent algorithm, etc. The present application takes the gradient descent algorithm as an example for explanation.
[0065] The purpose of the gradient descent algorithm is to find the minimum value of the loss function through iteration, or to converge to the minimum value. In a geometric sense, the gradient descent algorithm means that the gradient decreases fastest in the direction opposite to the vector where the function changes the fastest, so it is easier to find the minimum value of the function. Based on this, in an embodiment of the present application, the target speech recognition model can be repeatedly iterated and trained by the gradient descent algorithm so that the loss function is continuously reduced, thereby reducing the error of the calculation result.
[0066] In an embodiment of the present application, the target speech recognition model is repeatedly iteratively trained using a gradient descent algorithm so that the loss function is continuously reduced to obtain an iterative target speech recognition model, and then a more accurate speech recognition result corresponding to the speech to be recognized can be obtained based on the iterative target speech recognition model.
[0067] See also Figure 4 , Figure 4 1 is a schematic block diagram of a speech recognition device provided in an embodiment of the present application. The speech recognition device can be configured in a server to execute the aforementioned speech recognition method.
[0068] like Figure 4 As shown, the speech recognition device 200 includes: an acquisition module 201, a feature extraction module 202, a conversion processing module 203 and a speech recognition module 204.
[0069] An acquisition module 201 is used to acquire speech data to be recognized;
[0070] A feature extraction module 202 is used to extract features from the speech data to obtain a speech feature vector;
[0071] A conversion processing module 203 is used to convert the speech feature vector through a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model;
[0072] The speech recognition module 204 is used to obtain the attention calculation result of the target layer based on the query vector and value vector generated by the first layer and the key vector of the target layer, and recognize the speech data based on the attention calculation result to obtain the speech recognition result.
[0073] The feature extraction module 202 is also used to perform a preprocessing operation on the speech data to obtain preprocessed speech data, wherein the preprocessing operation includes at least one of a noise reduction processing operation, a pre-emphasis processing operation, and a frame processing operation; and perform feature extraction on the preprocessed speech data to obtain the speech feature vector.
[0074] The feature extraction module 202 is also used to convert the speech feature vector through the filter in the target speech recognition model to obtain the query vector and value vector generated for the first layer; and perform assignment operation according to the speech feature vector at the target layer to obtain the key vector of the target layer.
[0075] The conversion processing module 203 is further used to obtain the attention calculation result of the target layer through the following formula:
[0076] Y i =FFN(MHA(Q 1 ,K i ,V 1 ))
[0077] Among them, Y i is the attention calculation result of the target layer; Q 1 V is the query vector generated for the first layer; 1 is the value vector generated for the first layer; K i is the key vector of the target layer.
[0078] The acquisition module 201 is also used to obtain a training data set and an initial speech recognition model, wherein the training data set includes a number of speech data; annotate the training data set to obtain an annotation result, wherein the annotation result includes a speech recognition result corresponding to each speech data; and train the initial speech recognition model through the training data set and the annotation result to obtain the target speech recognition model.
[0079] The acquisition module 201 is also used to iteratively train the target speech recognition model based on the training data set and the annotation results to extract data features and calculate the loss function; iteratively train the loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; based on the loss function after iterative training, obtain the iterative target speech recognition model.
[0080] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0081] The method and apparatus of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.
[0082] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Figure 5 Runs on the computer device shown.
[0083] See also Figure 5 , Figure 5 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.
[0084] like Figure 5 As shown, the computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a volatile storage medium, a non-volatile storage medium and an internal memory.
[0085] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any speech recognition method.
[0086] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0087] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any speech recognition method.
[0088] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure of the computer device is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0089] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0090] Among them, in some embodiments, the processor is used to obtain speech data to be recognized; perform feature extraction on the speech data to obtain a speech feature vector; convert the speech feature vector through a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model; based on the query vector and value vector generated at the first layer, and the key vector of the target layer, the attention calculation result of the target layer is obtained, and the speech data is recognized based on the attention calculation result to obtain a speech recognition result.
[0091] In some embodiments, the processor is also used to perform a preprocessing operation on the speech data to obtain preprocessed speech data, wherein the preprocessing operation includes at least one of a noise reduction processing operation, a pre-emphasis processing operation, and a frame processing operation; and perform feature extraction on the preprocessed speech data to obtain the speech feature vector.
[0092] In some embodiments, the processor is further used to process the preprocessed speech data through the speech decoder in the target speech recognition model to obtain the speech coding vector output by the speech encoder; and perform feature extraction on the speech coding vector to obtain the speech feature vector.
[0093] In some embodiments, the processor is also used to convert the speech feature vector through the filter in the target speech recognition model to obtain the query vector and value vector generated for the first layer; and perform an assignment operation on the target layer according to the speech feature vector to obtain the key vector of the target layer.
[0094] In some implementations, the processor is further configured to obtain the attention calculation result of the target layer by using the following formula:
[0095] Y i =FFN(MHA(Q 1 ,K i ,V 1 ))
[0096] Among them, Y i is the attention calculation result of the target layer; Q 1 V is the query vector generated for the first layer; 1 is the value vector generated for the first layer; K i is the key vector of the target layer.
[0097] In some embodiments, the processor is also used to obtain a training data set and an initial speech recognition model, wherein the training data set includes a number of speech data; annotate the training data set to obtain an annotation result, wherein the annotation result includes a speech recognition result corresponding to each speech data; and train the initial speech recognition model using the training data set and the annotation result to obtain the target speech recognition model.
[0098] In some embodiments, the processor is also used to iteratively train the target speech recognition model based on the training data set and the annotation results to extract data features and calculate a loss function; iteratively train the loss function using a preset method with the purpose of reducing the value of the loss function until the expected threshold requirement is met; and obtain the iterative target speech recognition model based on the loss function after iterative training.
[0099] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the speech recognition methods provided in the embodiments of the present application is implemented.
[0100] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.
[0101] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like.
[0102] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A speech recognition method, characterized in that: The method comprises: Acquire voice data to be recognized; Extracting features from the speech data to obtain a speech feature vector; The speech feature vector is converted by a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model; Based on the query vector and value vector generated by the first layer and the key vector of the target layer, an attention calculation result of the target layer is obtained, and based on the attention calculation result, the speech data is recognized to obtain a speech recognition result.
2. The method according to claim 1, characterized in that: Before extracting features from the speech data to obtain a speech feature vector, the method includes: Performing a preprocessing operation on the voice data to obtain preprocessed voice data, wherein the preprocessing operation includes at least one of a noise reduction operation, a pre-emphasis operation, and a frame processing operation; The step of extracting features from the speech data to obtain a speech feature vector comprises: Feature extraction is performed on the preprocessed speech data to obtain the speech feature vector.
3. The method according to claim 2, characterized in that The step of extracting features from the preprocessed speech data to obtain the speech feature vector comprises: Processing the preprocessed speech data by a speech decoder in the target speech recognition model to obtain a speech coding vector output by the speech encoder; Feature extraction is performed on the speech coding vector to obtain the speech feature vector.
4. The method according to claim 1, characterized in that: The speech feature vector is converted by the target speech recognition model to obtain a triplet for self-attention calculation, including: The speech feature vector is converted by a filter in the target speech recognition model to obtain a query vector and a value vector for the first layer generation; An assignment operation is performed at the target layer according to the speech feature vector to obtain a key vector of the target layer.
5. The method according to claim 1, characterized in that The query vector and value vector generated based on the first layer, and the key vector of the target layer are used to obtain the attention calculation result of the target layer, including: The attention calculation result of the target layer is obtained by the following formula: Y i =FFN(MHA(Q 1 ,K i ,V 1 )) Among them, Y i is the attention calculation result of the target layer; Q 1 V is the query vector generated for the first layer; 1 is the value vector generated for the first layer; K i is the key vector of the target layer.
6. The method according to claim 1, characterized in that The method further comprises: Acquire a training data set and an initial speech recognition model, wherein the training data set includes a plurality of speech data; Annotating the training data set to obtain an annotation result, wherein the annotation result includes a speech recognition result corresponding to each speech data; The initial speech recognition model is trained using the training data set and the annotation results to obtain the target speech recognition model.
7. The method according to claim 6, characterized in that After the initial speech recognition model is trained by the training data set and the annotation result to obtain the target speech recognition model, the method includes: Iteratively train the target speech recognition model based on the training data set and the annotation results to extract data features, and calculate a loss function; Iteratively training the loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; Based on the loss function after iterative training, an iterative target speech recognition model is obtained.
8. A speech recognition device, characterized in that: The speech recognition device comprises: An acquisition module, the acquisition module is used to acquire voice data to be recognized; A feature extraction module, wherein the feature extraction module is used to extract features from the speech data to obtain a speech feature vector; A conversion processing module, the conversion processing module is used to convert the speech feature vector through a target speech recognition model to obtain a triple for self-attention calculation, wherein the triple includes: a query vector, a key vector and a value vector; the query vector and the value vector are generated at the first layer of the target speech recognition model, and the key vector in the attention mechanism is updated and generated at each layer of the target speech recognition model; A speech recognition module, the speech recognition module is used to obtain the attention calculation result of the target layer based on the query vector and value vector generated by the first layer, and the key vector of the target layer, and recognize the speech data based on the attention calculation result to obtain a speech recognition result.
9. A computer device, characterized in that: include: A memory and a processor; wherein the memory is connected to the processor and is used to store a program; the processor is used to implement the steps of the speech recognition method as described in any one of claims 1 to 7 by running the program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the speech recognition method according to any one of claims 1 to 7.