Speech recognition system re-scoring method, device, electronic device and storage medium

Abstract

The present application relates to a re-scoring method, device, electronic device and storage medium for a speech recognition system, the method comprising: determining whether the text corresponding to the speech to be recognized is a long text or a short text; if the judgment result is that the speech to be recognized is a long text, segmenting the long text to obtain multiple short texts; determining the re-scoring result of each short text in the multiple short texts respectively; summing the re-scoring result of each short text to obtain the re-scoring result of the long text. The present application determines whether the current word in the text corresponding to the speech to be recognized is the end of a sentence during the instant decoding process in the re-scoring stage, if the current word is the end of a sentence, segmenting the long text to obtain multiple short texts, determining the re-scoring result of each short text respectively, and then determining the re-scoring result of the long text based on the re-scoring result of the short text, thereby obtaining a more accurate language model score, improving the accuracy of the re-scoring, and further improving the recognition rate of the entire recognition system.

Description

Technical Field

[0001] The present application relates to the technical field of re-scoring of speech recognition systems, and in particular to a re-scoring method, device, electronic device and storage medium for speech recognition systems. Background Art

[0002] The re-scoring rescore technology using language models is a very popular practice in the industry. The language model will recalculate the joint probability of recognizing the text string based on the Nbest results obtained in the initial decoding, perform a second round of rescore, and return the optimal result.

[0003] The current language model re-scoring technology treats the recognized text string as a sentence and updates the score in real time as the decoding progresses. However, in long audio recognition, the recognition result is composed of many sentences. If it is still regarded as a sentence for scoring, it is not appropriate, especially at the sentence boundary, the probability jump will appear very abrupt and unreasonable, which will affect the system's re-scoring effect and the final recognition accuracy.

[0004] for example," <s>I'm not sure what the weather will be like today.< / s> "From a linguistic perspective, this sentence is not as good as " <s> What's the weather like today?< / s> <s> I'm not sure yet< / s> "More appropriate. Among them, <s> Indicates the beginning of a sentence.< / s> Indicates the end of the sentence. For long audio recognition, the above problem of calculating the probability of long text during re-scoring has not yet been studied and solved in the industry. Summary of the invention

[0005] Based on the above problems, the present application provides a re-scoring method, device, electronic device and storage medium for a speech recognition system.

[0006] In a first aspect, an embodiment of the present application provides a re-scoring method for a speech recognition system, comprising:

[0007] Determine whether the text corresponding to the speech to be recognized is a long text or a short text;

[0008] If the judgment result is that the speech to be recognized is a long text, the long text is segmented to obtain multiple short texts;

[0009] Determine the re-scoring result of each short text in the plurality of short texts respectively;

[0010] The re-scoring results of each short text are summed up to get the re-scoring result of the long text.

[0011] Furthermore, in the above-mentioned re-scoring method of a speech recognition system, segmenting a long text into multiple short texts includes:

[0012] During the instant decoding process of the rescoring phase, the rescoring result of each word in the long text is determined;

[0013] Determine the difference between the re-scoring result and a first preset threshold;

[0014] If the re-scoring result of the current character is greater than a first preset threshold, determining that the current character is a position for sentence segmentation;

[0015] The long text is segmented according to the segmentation positions to obtain multiple short texts.

[0016] Furthermore, in the above-mentioned re-scoring method of a speech recognition system, before segmenting the long text into multiple short texts according to the positions of the segmentation, the method further includes:

[0017] A start symbol and an end symbol are added at the start position and the end position of each short text in each short text.

[0018] Furthermore, in the above-mentioned re-scoring method for a speech recognition system, the re-scoring result of each short text is summed up to obtain the re-scoring result of the long text, which is determined by calculating the joint probability of each short text string.

[0019] Furthermore, in the above-mentioned re-scoring method for a speech recognition system, determining whether the text corresponding to the speech to be recognized is a long text or a short text includes:

[0020] Determine the difference between the number of characters in the text corresponding to the speech to be recognized and a second preset threshold;

[0021] If the number of characters in the speech to be recognized is greater than or equal to the second preset threshold, the result of the determination is that the speech to be recognized is a long text;

[0022] If the number of characters in the speech to be recognized is less than a second preset threshold, the judgment result is that the speech to be recognized is a short text;

[0023] Furthermore, in the above-mentioned re-scoring method for a speech recognition system, the re-scoring result of each short text in the plurality of short texts is determined respectively by the coefficient of the language model, the language model score and the decoded score.

[0024] Furthermore, the above-mentioned re-scoring method for a speech recognition system further includes:

[0025] During decoding, the coefficients of the language model are updated.

[0026] In a second aspect, an embodiment of the present application further provides a re-scoring device for a speech recognition system, comprising:

[0027] Judgment module: used to determine whether the text corresponding to the speech to be recognized is a long text or a short text;

[0028] Sentence segmentation module: used to segment the long text into multiple short texts if the judgment result is that the speech to be recognized is a long text;

[0029] Determination module: used for respectively determining the re-scoring result of each short text in the plurality of short texts;

[0030] Summation module: used to sum the re-scoring results of each short text to obtain the re-scoring result of the long text.

[0031] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor and a memory;

[0032] The processor is used to execute the above-mentioned re-scoring method for a speech recognition system by calling the program or instruction stored in the memory.

[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instruction, and the program or instruction enables the computer to implement the above-mentioned re-scoring method for a speech recognition system.

[0034] The advantages of the embodiments of the present application are: the present application relates to a re-scoring method, device, electronic device and storage medium for a speech recognition system, the method comprising: determining whether the text corresponding to the speech to be recognized is a long text or a short text; if the judgment result is that the speech to be recognized is a long text, segmenting the long text to obtain multiple short texts; determining the re-scoring result of each short text in the multiple short texts respectively; summing the re-scoring results of each short text to obtain the re-scoring result of the long text. The present application determines whether the current word in the text corresponding to the speech to be recognized is the end of a sentence during the instant decoding process of the re-scoring stage, if the current word is the end of a sentence, segmenting the long text to obtain multiple short texts, determining the re-scoring result of each short text respectively, and then determining the re-scoring result of the long text based on the re-scoring result of the short text, thereby obtaining a more accurate language model score, improving the accuracy of the re-scoring, and further improving the recognition rate of the entire recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 1 ;

[0037] Figure 2 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 2 ;

[0038] Figure 3 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 3 ;

[0039] Figure 4 A schematic diagram of a re-scoring device for a speech recognition system provided in an embodiment of the present application;

[0040] Figure 5 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0043] Figure 1 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 1 .

[0044] In a first aspect, the present application provides a method for rescoring a speech recognition system, combining Figure 1 , including four steps from S101 to S104:

[0045] S101: Determine whether the text corresponding to the speech to be recognized is a long text or a short text.

[0046] Specifically, in the embodiment of the present application, the speech to be recognized can be any sentence of speech, which can be Chinese speech or English speech. This is not limited here, and the protection scope of the present application is not limited by this. The number of characters in the text corresponding to the speech to be recognized is determined to determine whether the text corresponding to the speech to be recognized is a long text or a short text.

[0047] S102: If the judgment result is that the text corresponding to the to-be-recognized speech is a long text, the long text is segmented into multiple short texts.

[0048] Specifically, in an embodiment of the present application, if the judgment result is that the text corresponding to the speech to be recognized is a long text, the long text is segmented to obtain multiple short texts. The following describes the detailed steps of segmenting the long text to obtain multiple short texts in combination with specific steps.

[0049] S103: Determine the re-scoring result of each short text in the plurality of short texts respectively.

[0050] Specifically, in an embodiment of the present application, the re-scoring result of each short text in a plurality of short texts is determined by determining the re-scoring result of each character, and then accumulating the re-scoring result of each character to obtain the re-scoring result of the short text, which is described in detail below in conjunction with a specific formula.

[0051] S104: summing up the re-scoring results of each short text to obtain the re-scoring result of the long text.

[0052] Specifically, in the embodiment of the present application, the re-scoring result of each short text is determined separately and then the re-scoring result of the long text is determined based on the re-scoring result of the short text, so that a more accurate language model score can be obtained, the accuracy of the re-scoring can be improved, and the recognition rate of the entire recognition system can be improved.

[0053] Figure 2 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 2 .

[0054] Furthermore, in the above-mentioned re-scoring method of a speech recognition system, a long text is segmented to obtain multiple short texts, and combined with Figure 2 , including four steps from S201 to S204:

[0055] S201: During the instant decoding process of the re-scoring stage, the re-scoring result of each word in the long text is determined.

[0056] S202: Determine the difference between the re-scoring result and a first preset threshold.

[0057] S203: If the judgment result is that the re-scoring result of the current character is greater than the first preset threshold, determine that the current character is a position for sentence segmentation.

[0058] S204: Segment the long text according to the segmentation positions to obtain a plurality of short texts.

[0059] Specifically, in an embodiment of the present application, during the instant decoding process of the re-typing stage, a preset threshold is used to determine whether the current word is the end of a sentence, and the current word is determined to be the position for sentence segmentation. According to the position of the sentence segmentation, the long text is segmented to obtain multiple short texts.

[0060] Furthermore, in the above-mentioned re-scoring method of a speech recognition system, before segmenting the long text into multiple short texts according to the positions of the segmentation, the method further includes:

[0061] A start symbol and an end symbol are added at the start position and the end position of each short text in each short text.

[0062] Specifically, in the embodiment of the present application, a start symbol is added to the start position and the end position of each short text in each short text. <s> and end symbol< / s> , <s> and< / s> They respectively represent the beginning of a sentence (BOS, Begin Of Sentence) and the end of a sentence (EOS, End Of Sentence).

[0063] Furthermore, in the above-mentioned re-scoring method for a speech recognition system, the re-scoring result of each short text is summed up to obtain the re-scoring result of the long text, which is determined by calculating the joint probability of each short text string.

[0064] Specifically, in the embodiment of the present application, for example, the re-scoring result of a long text in the prior art is logP( <s> I'm not sure what the weather will be like today.< / s> ), this application is divided into two short texts after sentence segmentation, logP( <s> What's the weather like today?< / s> )+logP( <s> I'm not sure yet< / s> ), thereby obtaining the re-scoring result of the long text, determining the re-scoring result of each short text separately, and then determining the re-scoring result of the long text based on the re-scoring result of the short text, so as to obtain a more accurate language model score, improve the accuracy of the re-scoring, and thus improve the recognition rate of the entire recognition system.

[0065] Figure 3 A method for rescoring a speech recognition system provided in an embodiment of the present application is shown in FIG. Figure 3 .

[0066] Furthermore, in the above-mentioned re-scoring method of a speech recognition system, it is determined whether the text corresponding to the speech to be recognized is a long text or a short text, combined with Figure 2 , including two steps S301 to S302:

[0067] S301: Determine the difference between the number of characters in the text corresponding to the speech to be recognized and a second preset threshold.

[0068] S302: If the number of characters in the text corresponding to the speech to be recognized is greater than or equal to a second preset threshold, the judgment result is that the speech to be recognized is a long text; if the number of characters in the speech to be recognized is less than the second preset threshold, the judgment result is that the speech to be recognized is a short text.

[0069] Specifically, in an embodiment of the present application, the second preset threshold value can be a number such as 10 or 12, and can be flexibly set according to actual conditions. By judging the difference between the number of characters in the text corresponding to the speech to be recognized and the second preset threshold value, it is determined whether the text corresponding to the speech to be recognized is a long text or a short text; if the number of characters in the text corresponding to the speech to be recognized is 15 and greater than 10, it is determined that the text corresponding to the speech to be recognized is a long text; if the number of characters in the text corresponding to the speech to be recognized is 8 and less than 10, it is determined that the text corresponding to the speech to be recognized is a short text.

[0070] Furthermore, in the above-mentioned re-scoring method for a speech recognition system, the re-scoring result of each short text in the plurality of short texts is determined respectively by the coefficient of the language model, the language model score and the decoded score.

[0071] Specifically, in the embodiment of the present application, the re-scoring result of each short text in the plurality of short texts is determined by first determining the re-scoring result of each word through the following formula:

[0072] final_score=model_score+lm_scale*ngram_score

[0073] Among them, model_score is the decoded score, ngram_score is the score of the language model N-gram LM, and lm_scale is the coefficient of the language model, which can be adjusted according to its importance. The value range is 0.0 to 1.0. Then, the re-scoring results of each word are accumulated to obtain the re-scoring results of each short text.

[0074] Furthermore, the above-mentioned re-scoring method for a speech recognition system further includes:

[0075] During decoding, the coefficients of the language model are updated.

[0076] Specifically, in the embodiment of the present application, the coefficients of the language model will be updated for the current word, and the coefficients of the language model represent the current re-scoring status. As the decoding proceeds, the coefficients of the language model will be continuously updated.

[0077] Figure 4 A schematic diagram of a re-scoring device for a speech recognition system provided in an embodiment of the present application.

[0078] In a second aspect, the present application also provides a re-scoring device for a speech recognition system, Figure 4 ,include:

[0079] Determination module 401: used to determine whether the text corresponding to the speech to be recognized is a long text or a short text.

[0080] Specifically, in the embodiment of the present application, the speech to be recognized can be any sentence of speech, which can be Chinese speech or English speech. This is not limited here, and the protection scope of the present application is not limited by this. The judgment module 401 determines whether the text corresponding to the speech to be recognized is a long text or a short text by judging the number of words in the text corresponding to the speech to be recognized.

[0081] Sentence segmentation module 402: used for segmenting the long text into multiple short texts if the judgment result is that the speech to be recognized is a long text.

[0082] Specifically, in an embodiment of the present application, if the judgment result is that the text corresponding to the speech to be recognized is a long text, the segmentation module 402 segments the long text to obtain multiple short texts. The above has introduced the detailed steps of segmenting the long text to obtain multiple short texts in combination with specific steps.

[0083] The determination module 403 is used to determine the re-scoring result of each short text in the plurality of short texts.

[0084] Specifically, in the embodiment of the present application, the determination module 403 determines the re-scoring result of each short text in the multiple short texts by determining the re-scoring result of each character, and then accumulating the re-scoring result of each character to obtain the re-scoring result of the short text, which has been introduced in detail above in conjunction with the specific formula.

[0085] The summing module 404 is used to sum the re-scoring results of each short text to obtain the re-scoring result of the long text.

[0086] Specifically, in the embodiment of the present application, the re-scoring result of each short text is determined separately and then the summation module 404 sums the re-scoring results of multiple short texts to determine the re-scoring result of the long text, thereby obtaining a more accurate language model score, improving the accuracy of the re-scoring, and further improving the recognition rate of the entire recognition system.

[0087] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor and a memory;

[0088] The processor is used to execute the above-mentioned re-scoring method for a speech recognition system by calling the program or instruction stored in the memory.

[0089] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instruction, and the program or instruction enables the computer to implement the above-mentioned re-scoring method for a speech recognition system.

[0090] Figure 5 It is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure.

[0091] like Figure 5 As shown, the electronic device includes: at least one processor 501, at least one memory 502 and at least one communication interface 503. The various components in the electronic device are coupled together through a bus system 504. The communication interface 503 is used to transmit information between external devices. It can be understood that the bus system 504 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 504 .

[0092] It can be understood that the memory 502 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0093] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system and application programs.

[0094] Among them, the operating system includes various system programs, such as the framework layer, the core library layer, the driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. A program that implements any method in the re-scoring method of a speech recognition system provided in an embodiment of the present application can be included in the application program.

[0095] In an embodiment of the present application, the processor 501 calls a program or instruction stored in the memory 502, specifically, a program or instruction stored in an application, and the processor 501 is used to execute the steps of each embodiment of a re-scoring method for a speech recognition system provided in an embodiment of the present application.

[0096] Determine whether the text corresponding to the speech to be recognized is a long text or a short text;

[0097] If the judgment result is that the speech to be recognized is a long text, the long text is segmented to obtain multiple short texts;

[0098] Determine the re-scoring result of each short text in the plurality of short texts respectively;

[0099] The re-scoring results of each short text are summed up to get the re-scoring result of the long text.

[0100] Any of the methods in the re-scoring method of a speech recognition system provided in an embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 501 or instructions in the form of software. The above-mentioned processor 501 can be a general-purpose processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0101] The steps of any method in the re-scoring method for a speech recognition system provided in an embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software units in a decoding processor. The software unit can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and completes the steps of a re-scoring method for a speech recognition system in combination with its hardware.

[0102] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0103] Those skilled in the art will appreciate that the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0104] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A re-scoring method for a speech recognition system, characterized in that: include: Determine whether the text corresponding to the speech to be recognized is a long text or a short text; If the judgment result is that the speech to be recognized is a long text, the long text is segmented to obtain multiple short texts, including: During the instant decoding process of the rescoring phase, the rescoring result of each word in the long text is determined; Determine the difference between the re-scoring result and a first preset threshold; If the re-scoring result of the current character is greater than a first preset threshold, determining that the current character is a position for sentence segmentation; Segment a long text according to the position of the segment to obtain multiple short texts; Determining the re-scoring result of each short text in the plurality of short texts respectively; The re-scoring results of each short text are summed up to obtain the re-scoring result of the long text.

2. A re-scoring method for a speech recognition system according to claim 1, characterized in that: Before segmenting the long text according to the positions of the segmentation to obtain a plurality of short texts, the method further includes: A start symbol and an end symbol are added to a start position and an end position of each of the plurality of short texts.

3. The re-scoring method for a speech recognition system according to claim 1, characterized in that: The re-scoring result of the long text is obtained by summing up the re-scoring results of each short text, which is determined by calculating the joint probability of each short text string.

4. The re-scoring method for a speech recognition system according to claim 1, characterized in that: The step of determining whether the text corresponding to the speech to be recognized is a long text or a short text comprises: Determine the difference between the number of characters in the text corresponding to the speech to be recognized and a second preset threshold; If the number of characters in the speech to be recognized is greater than or equal to the second preset threshold, the judgment result is that the speech to be recognized is a long text; If the number of characters in the speech to be recognized is less than the second preset threshold, the judgment result is that the speech to be recognized is a short text.

5. The re-scoring method for a speech recognition system according to claim 1, characterized in that: The re-scoring result of each short text in the plurality of short texts is determined respectively by using the coefficient of the language model, the language model score and the decoded score.

6. A re-scoring method for a speech recognition system according to claim 5, characterized in that: The method further comprises: During the decoding process, the coefficients of the language model are updated.

7. A re-scoring device for a speech recognition system, characterized in that: include: Judgment module: used to determine whether the text corresponding to the speech to be recognized is a long text or a short text; Sentence segmentation module: used for segmenting the long text to obtain multiple short texts if the judgment result is that the speech to be recognized is a long text, including: During the instant decoding process of the rescoring phase, the rescoring result of each word in the long text is determined; Determine the difference between the re-scoring result and a first preset threshold; If the re-scoring result of the current character is greater than a first preset threshold, determining that the current character is a position for sentence segmentation; Segment a long text according to the position of the segment to obtain multiple short texts; A determination module: used to respectively determine the re-scoring result of each short text in the plurality of short texts; A summing module is used to sum the re-scoring results of each short text to obtain the re-scoring result of the long text.

8. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute the re-scoring method for a speech recognition system as claimed in any one of claims 1 to 6 by calling the program or instruction stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the re-scoring method for a speech recognition system as claimed in any one of claims 1 to 6.

Images