Speech recognition method, device, equipment, medium and product

Abstract

In the speech recognition method, apparatus, device, medium and product provided by the present application, the electronic device recognizes the target speech through the encoding layer of the end-to-end speech recognition model and the language model with a dictionary to obtain multiple candidate texts of the target speech; then, the multiple candidate texts are scored respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text; and the target text is selected from the multiple candidate texts according to the score of each candidate text; because the dictionary and text corpus in the language model can be adjusted according to the needs of a specific field, the method can not only adapt to general scenarios, but also adapt to the speech recognition needs of a specific field by adjusting the dictionary and text corpus.

Description

Technical Field

[0001] The present application relates to the field of speech recognition, and more specifically, to a speech recognition method, apparatus, device, medium and product. Background Art

[0002] With the development and commercial application of speech recognition technology, speech recognition technology is increasingly used in various scenarios, such as customer service, quality inspection, in-vehicle, mobile phone, speaker, home, etc. Among them, in general application scenarios, the end-to-end speech recognition model (i.e., the neural network model used for speech recognition) has a strong fitting ability and can achieve a more obvious speech recognition effect advantage, so it is widely used.

[0003] However, the inventors found that the fitting ability of the end-to-end speech recognition model also led to problems such as domain customization and immature speech functions, and then there were serious false triggering phenomena when recognizing speech in specific fields. Therefore, the end-to-end speech recognition model is difficult to adapt to the speech recognition needs in general scenarios and specific fields at the same time. Summary of the invention

[0004] In order to overcome at least one deficiency in the prior art, the present application provides a speech recognition method, apparatus, device and medium.

[0005] In a first aspect, an embodiment of the present application provides a speech recognition method, which is applied to an electronic device, and the method includes:

[0006] Get the target voice;

[0007] Recognize the acoustic recognition result of the target speech through a language model with a dictionary to obtain a plurality of candidate texts of the target speech;

[0008] Scoring the multiple candidate texts respectively through a decoding layer of an end-to-end speech recognition model to obtain a score for each candidate text;

[0009] Selecting a target text from the plurality of candidate texts according to the scores of the candidate texts;

[0010] The target text is used as a speech recognition result of the target speech.

[0011] In a second aspect, an embodiment of the present application provides a speech recognition device, which is applied to an electronic device, and the speech recognition device includes:

[0012] A speech acquisition module, used to acquire the target speech;

[0013] A speech processing module, used for recognizing the acoustic recognition result of the target speech through a language model with a dictionary, and obtaining a plurality of candidate texts of the target speech;

[0014] The speech processing module is further used to score the multiple candidate texts respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text;

[0015] A text output module, used for selecting a target text from the plurality of candidate texts according to the scores of the candidate texts;

[0016] The text output module is also used to use the target text as the speech recognition result of the target speech.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the speech recognition method is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the speech recognition method is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which implement the speech recognition method when executed by a processor.

[0020] Compared with the prior art, this application has the following beneficial effects:

[0021] In the speech recognition method, apparatus, device, medium and product provided in this embodiment, the electronic device recognizes the acoustic recognition result of the target speech through a language model with a dictionary to obtain multiple candidate texts of the target speech; then, the multiple candidate texts are scored respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text; and the target text is selected from the multiple candidate texts based on the score of each candidate text; because the dictionary and text corpus in the language model can be adjusted according to the needs of a specific field, this method can not only adapt to general scenarios, but also adapt to the speech recognition needs of a specific field by adjusting the dictionary and text corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the structure of a traditional hybrid speech recognition model provided in an embodiment of the present application;

[0025] Figure 3 A flowchart of the steps of the speech recognition method provided in the embodiment of the present application;

[0026] Figure 4 A schematic diagram of data flow between an end-to-end speech recognition model and a language model provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the training of an end-to-end speech recognition model provided in an embodiment of the present application;

[0028] Figure 6 A structural schematic diagram of a speech recognition device is provided for an embodiment of the present application.

[0029] Icons: 120 - memory; 130 - processor; 140 - communication device; 201 - voice acquisition module; 202 - voice processing module; 203 - text output module. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0032] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0033] In the description of the present application, it should be noted that the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. In addition, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0034] It should be understood that the operations of the flow chart may not be implemented in order, and steps without logical contextual relationships may be reversed in order or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flow chart, or may remove one or more operations from the flow chart under the guidance of the content of this application.

[0035] In the related art, since neural networks can provide strong fitting capabilities, end-to-end speech recognition models based on neural networks are widely used, for example, in the fields of voice input method, intelligent customer service, etc.

[0036] The inventors have found that in order to improve the general capabilities of the end-to-end speech recognition model, it is necessary to train it with a large number of paired samples of speech and text. This also leads to an obvious effect bottleneck problem when the end-to-end speech recognition model recognizes the speech in a specific field, among which there will be serious false triggering when recognizing the speech in a specific field. It can be understood that the recognition effect of the end-to-end speech recognition model is affected by the sample space. If the end-to-end speech recognition model is to be able to recognize the speech in a specific field, it is necessary to increase the proportion of samples in the specific field in the sample space and retrain the end-to-end speech recognition model. Therefore, it is difficult for the end-to-end speech recognition model to adapt to the speech recognition needs in both general scenarios and specific fields at the same time.

[0037] In addition, the inventors also found that, in contrast to the end-to-end speech recognition model, the traditional hybrid speech recognition model is applied to speech recognition in various specific domain scenarios. Among them, the traditional hybrid speech recognition model can conveniently use the text corpus in the specific domain scenario to optimize the language model, so as to improve the speech recognition effect in the specific domain scenario.

[0038] Based on the discovery of the above technical problems, the inventors have proposed the following technical solutions to solve or improve the above problems through creative work. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventors after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of the present application for the above problems below should all be the contributions made by the inventors to the present application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0039] In view of this, in order to at least partially solve the related technical problems, this embodiment provides a speech recognition method applied to an electronic device. In this method, by combining the language model in the traditional hybrid speech recognition model with the end-to-end speech recognition model, the method can not only adapt to general scenarios, but also adapt to the speech needs of specific fields.

[0040] The electronic device referred to in this embodiment may be a server or a user terminal. The server may be, but not limited to, a Web server, an FTP server, etc. In some other implementation scenarios, the electronic device may also be a user terminal. The user terminal may be, but not limited to, a smart phone, an intelligent voice assistant (e.g., an intelligent speaker), a personal computer (PC), a tablet computer, a personal digital assistant (PDA), a mobile Internet device (MID), etc.

[0041] This embodiment also provides a structural schematic diagram of the electronic device. Figure 1 As shown, the electronic device may include a memory 120 , a processor 130 , and a communication device 140 .

[0042] The memory 120, the processor 130 and the communication device 140 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0043] The memory 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction. The communication device 140 is used to send and receive data.

[0044] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0045] Considering that this embodiment involves speech recognition, in order to enable those skilled in the art to more clearly understand this solution, the following first explains the relevant terms involved in this embodiment.

[0046] First, the speech recognition method provided in this embodiment uses the language model in the traditional hybrid speech recognition model. Figure 2 As shown, the traditional hybrid speech recognition model usually includes three stages: feature extraction, acoustic model and decoding when recognizing the target speech. Among them, in the decoding stage, a language model with a dictionary is required to decode the acoustic features of the speech to be recognized. It is worth noting that the language model can be, but is not limited to, a general language model, a domain language model, an adaptive language model, etc. Those skilled in the art can choose adaptably according to needs.

[0047] Among them, in the feature extraction stage: directly recognizing the original waveform of the target speech cannot achieve good recognition results. Therefore, it is necessary to convert the speech waveform signal in the time domain into a frequency domain signal before performing feature extraction. The mainstream acoustic feature coefficients include Linear Predictive Cepstral Coefficient (LPCC), Mel Frequency Cepstrum Coefficient (MFCC), and Filter Bank.

[0048] Among them, the Linear Predictive Cepstral Coefficient starts from the vocal model and uses LPC (Linear Predictive Cepstral) technology to calculate the cepstral coefficients. The Mel Frequency Cepstrum Coefficient, on the other hand, simulates the auditory model, takes the output of the speech passing through the filter bank model as the acoustic feature, and then uses the Discrete Fourier Transform (DFT) for transformation.

[0049] Acoustic model stage: The acoustic model processes the acoustic features obtained in the feature extraction stage to obtain the pronunciation units of the target speech. Among them, common acoustic models include Hidden Markov Model, Gaussian Mixture Model, etc. It should be understood that the pronunciation of a word is composed of pronunciation units. For example, all pronunciations in English can be composed of 39 pronunciation units provided by Carnegie Mellon University; all pronunciations in Chinese can be represented by the pronunciation units of all initials and finals. Among them, the pronunciation units in Chinese include two situations: with tone and without tone.

[0050] Taking the pronunciation unit with tone as an example, an exemplary description is as follows. The pronunciation of "我 (wǒ)" is composed of two pronunciation units, "w" and "o_3"; the pronunciation of "爱 (ài)" is composed of one pronunciation unit, "ai_4". Among them, the "3" in "o_3" indicates the third tone, and the "4" in "ai_4" indicates the fourth tone.

[0051] And, assuming that the target speech is a Chinese speech, the pronunciation unit sequence including multiple pronunciation units in this embodiment can correspond to one or more Chinese characters, and this embodiment does not make specific limitations. For example, if the pronunciation units in the pronunciation unit sequence are "w, o_3", the pronunciation unit sequence can correspond to the Chinese character "我 (wǒ)"; if the pronunciation units in the pronunciation unit sequence are "w, o_3, ai_4", then the pronunciation unit sequence can correspond to the Chinese phrase "我爱 (wǒ ài)".

[0052] Decoding stage: Since pronunciation units are relatively independent of the entire target speech, and the same pronunciation units may have homophones; therefore, the language model needs to use a dictionary to decode relatively isolated and scattered pronunciation units to obtain a target text that matches the target speech.

[0053] In addition, this embodiment also involves hot words and dictionaries, wherein hot words refer to words that appear frequently in a specific field, or words that appear frequently in a specific time period.

[0054] For example, in the field of speech recognition, “Mel-frequency cepstral coefficients”, “acoustics”, “language model”, “acoustic model”, “Markov”, etc. are hot words in the field of speech recognition. Similarly, in the field of image recognition, “convolution”, “convolution kernel”, “convolution layer”, “training”, “sample”, “label”, etc. are hot words in the field of image recognition.

[0055] Dictionary: The dictionary contains a set of words that can be processed and is used to find the corresponding text based on the pronunciation units recognized by the acoustic model.

[0056] The following is a detailed description of the speech recognition model provided by this embodiment in conjunction with the introduction of the above-mentioned related terms. Figure 3 As shown, the speech recognition method includes:

[0057] Step S101, obtaining target speech.

[0058] It should be understood that the target speech referred to in this embodiment is the speech to be recognized, and the target speech may be, but is not limited to, Chinese speech, English speech, etc. In addition, this embodiment does not limit the speech duration of the target speech, and those skilled in the art may make adaptive adjustments as needed.

[0059] Step S102, recognizing the acoustic recognition result of the target speech by using a language model with a dictionary to obtain a plurality of candidate texts of the target speech.

[0060] The speech recognition model is a language recognition model in a traditional hybrid speech recognition model, and the acoustic recognition result represents the pronunciation unit recognized from the target speech. Therefore, the electronic device can analyze the pronunciation unit of the target speech through the language model with a dictionary to obtain multiple candidate texts. The present embodiment does not limit the specific number of multiple candidate texts, and those skilled in the art can make adaptive adjustments according to actual needs.

[0061] It should be noted that the traditional hybrid speech recognition model has good recognition effects in terms of hot words, word boundaries, language model adaptation, etc., and has been commercially applied. The language model in the traditional hybrid speech recognition model has a dictionary. Therefore, when facing the speech recognition needs in a specific field, the dictionary of the language model can be adaptively adjusted. For example, different dictionaries can be used for user groups that like to listen to music and user groups that like to listen to news.

[0062] As a possible implementation, the electronic device can receive a weight adjustment operation for a hot word in a dictionary; then, in response to the weight adjustment operation, the weight of the hot word in the dictionary is adjusted. Thus, the language model is introduced into the end-to-end speech recognition model, which can adapt to the speech techniques in a specific field and thus improve the speech recognition accuracy.

[0063] For example, assuming that the actual text of the target speech is "I want to sharpen a pen", the candidate texts recognized by the language model may have the following situations:

[0064] (I want to sharpen a pen, 0.4, 1.5);

[0065] (I want to buy a pen, 0.5, 1.8);

[0066] (I want to erase the pen, 0.7, 0.9);

[0067] (I need to sharpen my pencil, 0.7, 1.8).

[0068] Among them, in “(I want to sharpen a pen, 0.4, 1.5)”, “0.4” represents the acoustic score of “I want to sharpen a pen”, and “1.5” represents the language model score of “I want to sharpen a pen”.

[0069] Taking "(I want to erase the pen, 0.7, 0.9)" and "(I want to sharpen the pen, 0.7, 1.8)" as examples, the pronunciation unit sequences of the two are consistent, and the acoustic scores are consistent, but the text wording of "I want to erase the pen" is unreasonable, so its language model score is lower than the language model score of "I want to sharpen the pen".

[0070] Step S103, scoring the multiple candidate texts respectively through the decoding layer of the end-to-end speech recognition model to obtain a score for each candidate text.

[0071] The end-to-end speech recognition model in this embodiment is also called the Seq2Seq model, which includes an encoding layer and a decoding layer; wherein the encoding layer is used to encode the input sequence; and the decoding layer is used to decode the encoding vector output by the encoding layer to obtain an output sequence. In this way, the mapping relationship between the input sequence and the output sequence is constructed by the Seq2Seq model. For example, the end-to-end speech recognition model in this embodiment has an input sequence of the target speech and an output sequence of the target text corresponding to the target speech.

[0072] Among them, the working principle of the decoding layer can be expressed as:

[0073] y1=f(E);

[0074] y2=f(E,y1);

[0075] y3=f(E,y1,y2);

[0076] y n =f(E, y1, y2...y n );

[0077] In the formula, E represents the encoding vector output by the encoding layer, y1 represents a subsequence decoded by the decoding layer according to the encoding vector; when the decoding layer obtains the subsequence y2, it needs to rely on the encoding vector E and the subsequence y1; when the decoding layer obtains the subsequence y3, it needs to rely on the encoding vector E, the subsequence y1 and the subsequence y2; finally, the decoding layer combines the subsequences into "y1, y1, y1...y n ” for scoring.

[0078] Based on the above working principle of the decoding layer, each candidate text in this embodiment is used to replace the above “y1, y1, y1…y n ", so as to obtain the score of each candidate text. That is, the electronic device inputs multiple candidate texts into the decoding layer, and the decoding layer scores the multiple candidate texts respectively to obtain the score of each candidate text.

[0079] Step S104: selecting a target text from a plurality of candidate texts according to the scores of the candidate texts.

[0080] Exemplarily, the electronic device may select the candidate text with the highest score as the target text according to the score of each candidate text.

[0081] Step S105: taking the target text as the speech recognition result of the target speech.

[0082] Thus, the electronic device recognizes the acoustic recognition result of the target speech through a language model with a dictionary, and obtains multiple candidate texts of the target speech; then, the multiple candidate texts are scored respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text; and according to the score of each candidate text, the target text is selected from the multiple candidate texts; because the dictionary and text corpus in the language model can be adjusted according to the needs of specific fields, this method can not only adapt to general scenarios, but also adapt to the speech recognition needs of specific fields by adjusting the dictionary and text corpus.

[0083] In this embodiment, the acoustic recognition result identified by the language recognition model is derived from the end-to-end language recognition model. Figure 4 , this embodiment provides a data flow relationship diagram between a language recognition model and an end-to-end speech recognition model.

[0084] like Figure 4 As shown, the end-to-end speech recognition model also includes a coding layer and a connection time series classification layer. Before the electronic recognizes the acoustic recognition result of the target speech through a language model with a dictionary, the target speech is encoded through the coding layer to obtain the acoustic coding information of the target speech; the acoustic coding information is input into the connection time series classification layer for processing to obtain the acoustic recognition result of the target speech.

[0085] Among them, the Connectionist Temporal Classification (CTC) layer is used to classify the acoustic coding information to obtain the pronunciation units corresponding to the target speech, so that the language model can obtain the candidate text of the target speech according to the pronunciation units.

[0086] To improve the recognition accuracy of the end-to-end speech recognition model, please refer to Figure 4 An attention mechanism layer is also configured between the encoding layer and the decoding layer of the end-to-end speech recognition model. Based on the attention mechanism layer, the electronic device adjusts the acoustic coding information through the attention mechanism layer; then, the adjusted acoustic coding information is input into the decoding layer, so that the decoding layer scores the multiple candidate texts respectively according to the adjusted acoustic coding information, and obtains the score of each candidate text.

[0087] In addition, the end-to-end speech recognition model in this embodiment is a machine learning model, and therefore, this embodiment also provides a corresponding training method; and in this training method, the attention mechanism layer is constrained by connecting the acoustic loss of the temporal classification layer.

[0088] like Figure 5As shown, the electronic device obtains sample speech; inputs the sample speech into the configured model to be trained for recognition; finally, according to the acoustic loss and text loss of the sample speech recognized by the model to be trained, the model parameters of the speech recognition model to be trained are adjusted to obtain the above-mentioned end-to-end speech recognition model.

[0089] The model to be trained has the same model structure as the end-to-end speech recognition model, so it will not be described in detail. Figure 4 A detailed description of the end-to-end speech recognition model in . Figure 5 , the acoustic loss is calculated by connecting the predicted pronunciation output by the temporal classification layer, and the text loss is calculated by the predicted text output by the decoding layer. Therefore, the above acoustic loss can be obtained based on the connection temporal classification criterion, and the text loss can be obtained based on the cross entropy criterion.

[0090] The electronic device fuses the acoustic loss and the text loss to obtain the fused loss, and then adjusts the model parameters of the model to be trained by means of the reverse gradient propagation algorithm until the model to be trained meets the preset convergence conditions, thereby obtaining a pre-trained end-to-end speech recognition model. As an implementation method, the electronic device can use the preset weights of the acoustic loss and the text loss to perform weighted summation to obtain the fused loss. For example, assuming that the weight of the acoustic loss is 0.5, the weight of the text loss is also 0.5.

[0091] In addition, the above-mentioned preset convergence condition may be, but is not limited to, stopping iteration when the model loss value no longer decreases; stopping iteration when the number of iterations reaches a set number of times, or stopping iteration when the model loss value is lower than a set threshold. Therefore, those skilled in the art may make adaptive selections according to the training objectives.

[0092] Based on the same inventive concept as the speech recognition method, this embodiment also provides a speech recognition device, which is applied to an electronic device. The speech recognition device includes at least one functional module that can be stored in the memory 120 in the form of software. Figure 6 As shown, from the functional point of view, the speech recognition device may include:

[0093] The speech acquisition module 201 is used to acquire the target speech.

[0094] In this embodiment, the voice acquisition module 201 is used to implement Figure 3 For a detailed description of the voice acquisition module 201, please refer to the detailed description of step S101.

[0095] A speech processing module 202 is used to recognize the acoustic recognition result of the target speech through a language model with a dictionary to obtain multiple candidate texts of the target speech;

[0096] The speech processing module 202 is further used to score the multiple candidate texts respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text;

[0097] In this embodiment, the voice processing module 202 is used to implement Figure 3 For the detailed description of the speech processing module 202, please refer to the detailed description of steps S102-S103.

[0098] A text output module 203, used to select a target text from a plurality of candidate texts according to the scores of the candidate texts;

[0099] The text output module 203 is further used to use the target text as the speech recognition result of the target speech.

[0100] In this embodiment, the text output module 203 is used to implement Figure 3 For the detailed description of the text output module 203, please refer to the detailed description of steps S104-S105.

[0101] Optionally, the end-to-end speech recognition model further includes a coding layer and a connection time classification layer. Before recognizing the acoustic recognition result of the target speech by the language model with a dictionary, the speech processing module 202 is further used to:

[0102] Encoding the target speech through the encoding layer to obtain acoustic coding information of the target speech;

[0103] The acoustic coding information is input into the connection temporal classification layer for processing to obtain the acoustic recognition result of the target speech.

[0104] In addition, it should be noted that the speech recognition device may also include other software function modules for implementing other steps or sub-steps of the above-mentioned speech recognition method. Similarly, the above-mentioned speech acquisition module 201, speech processing module 202 and text output module 203 may also be used to implement other steps or sub-steps of the above-mentioned speech recognition method. This embodiment does not make specific limitations on this.

[0105] Optionally, this embodiment further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the speech recognition method provided in this embodiment is implemented.

[0106] This embodiment further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the speech recognition method provided in this embodiment is implemented.

[0107] This embodiment also provides a computer program product, including a computer program / instruction, and a speech recognition method implemented when the computer program / instruction is executed by a processor.

[0108] In summary, in the speech recognition method, apparatus, device, medium and product provided in the present embodiment, the electronic device recognizes the acoustic recognition result of the target speech through a language model with a dictionary to obtain multiple candidate texts of the target speech; then, the multiple candidate texts are scored respectively through the decoding layer of the end-to-end speech recognition model to obtain the score of each candidate text; and based on the score of each candidate text, the target text is selected from the multiple candidate texts; since the dictionary and text corpus in the language model can be adjusted according to the needs of a specific field, this method can not only adapt to general scenarios, but also adapt to the speech recognition needs of a specific field by adjusting the dictionary and text corpus.

[0109] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0111] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0112] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A speech recognition method, characterized in that: Applied to an electronic device, the electronic device is configured with an end-to-end speech recognition model, the end-to-end speech recognition model further includes a coding layer and a connection timing classification layer, the method includes: Get the target voice; Encoding the target speech through the encoding layer to obtain acoustic encoding information of the target speech; Inputting the acoustic coding information into the connection temporal classification layer for processing to obtain an acoustic recognition result of the target speech; Recognize the acoustic recognition result of the target speech through a language model with a dictionary to obtain a plurality of candidate texts of the target speech; Scoring the multiple candidate texts respectively through a decoding layer of an end-to-end speech recognition model to obtain a score for each candidate text; Selecting a target text from the plurality of candidate texts according to the scores of the candidate texts; The target text is used as a speech recognition result of the target speech.

2. The speech recognition method according to claim 1, characterized in that: The end-to-end speech recognition model further includes an attention mechanism layer, and the decoding layer of the end-to-end speech recognition model is used to score the multiple candidate texts respectively to obtain the score of each candidate text, including: Adjusting the acoustic encoding information through the attention mechanism layer; The adjusted acoustic coding information is input into the decoding layer, so that the decoding layer scores the multiple candidate texts respectively according to the adjusted acoustic coding information to obtain the score of each candidate text.

3. The speech recognition method according to claim 1, characterized in that: The electronic device is configured with a model to be trained corresponding to the end-to-end speech recognition model, and the method further comprises: Get sample speech; Inputting the sample speech into the model to be trained for recognition; According to the acoustic loss and text loss of the sample speech recognized by the model to be trained, the model parameters of the model to be trained are adjusted to obtain the end-to-end speech recognition model.

4. The speech recognition method according to any one of claims 1 to 3, characterized in that: The method further comprises: Receive the user's weight adjustment operation on the hot word; In response to the weight adjustment operation, the weight of the hot word in the dictionary is adjusted.

5. A speech recognition device, characterized in that: Applied to an electronic device, the electronic device is configured with an end-to-end speech recognition model, the end-to-end speech recognition model includes a coding layer and a connection time sequence classification layer, and the speech recognition device includes: A speech acquisition module, used to acquire the target speech; Encoding the target speech through the encoding layer to obtain acoustic encoding information of the target speech; Inputting the acoustic coding information into the connection temporal classification layer for processing to obtain an acoustic recognition result of the target speech; A speech processing module, used for recognizing the acoustic recognition result of the target speech through a language model with a dictionary, and obtaining a plurality of candidate texts of the target speech; The speech processing module is further used to score the multiple candidate texts respectively through the decoding layer of the end-to-end speech recognition model to obtain a score for each candidate text; A text output module, used for selecting a target text from the plurality of candidate texts according to the scores of the candidate texts; The text output module is also used to use the target text as the speech recognition result of the target speech.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory stores a computer program, and when the computer program is executed by the processor, the speech recognition method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the speech recognition method according to any one of claims 1 to 4 is implemented.

8. A computer program product, characterized in that The invention comprises a computer program / instruction, which, when executed by a processor, implements the speech recognition method according to any one of claims 1 to 4.

Images