Text verification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a text verification method and device, electronic equipment and a storage medium. The method comprises the steps that a first text and first audio data obtained after audio conversion is conducted on the first text are obtained; converting the first audio data into a second text; obtaining a tone-removed first pinyin sequence of the first text and a tone-removed second pinyin sequence of the second text; according to the first text, the second text, the first pinyin sequence and the second pinyin sequence, identifying the first text with an audio conversion error; the first audio data corresponding to the first text with the wrong audio conversion has at least one of more reading, less reading and wrong reading compared with the first text with the wrong audio conversion.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular to a text verification method, device, electronic device and storage medium. Background Art

[0002] In the field of natural language processing, it is often necessary to convert text into audio data, and train language processing models based on text and audio data obtained by text conversion. For example, in the field of speech synthesis, you can ask a professional reader to read the text, and use the text and the audio data obtained by reading the text to train a text-to-speech model. It can be seen that it is crucial to accurately convert text into audio data. At present, considering the situation of too many or too few words when converting text into audio data, whether the text is correctly converted into audio data is mainly verified by manual verification, and the data verification efficiency is low. Summary of the invention

[0003] The embodiments of the present disclosure provide a text verification method, device, electronic device and storage medium, which can improve the verification efficiency of whether the text is correctly converted into audio data.

[0004] In a first aspect, an embodiment of the present disclosure provides a text verification method, comprising:

[0005] Acquire a first text and first audio data obtained by performing audio conversion on the first text;

[0006] Converting the first audio data into a second text;

[0007] Acquire a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text;

[0008] According to the first text, the second text, the first pinyin sequence and the second pinyin sequence, a first text with audio conversion error is identified; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0009] In a second aspect, an embodiment of the present disclosure provides a text verification device, including:

[0010] A first acquisition unit, used to acquire a first text and first audio data obtained by performing audio conversion on the first text;

[0011] An audio conversion unit, used for converting the first audio data into a second text;

[0012] A second acquisition unit, configured to acquire a first pinyin sequence of the first text without tones and a second pinyin sequence of the second text without tones;

[0013] A text recognition unit is used to identify a first text with audio conversion error based on the first text, the second text, the first pinyin sequence and the second pinyin sequence; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0016] In one or more embodiments of the present disclosure, first, a first text and first audio data obtained after audio conversion of the first text are obtained, then, the first audio data is converted into a second text, then, a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text are obtained, and finally, the first text with audio conversion errors is identified based on the first text, the second text, the first pinyin sequence and the second pinyin sequence, and the first audio data corresponding to the first text with audio conversion errors has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion errors. It can be seen that through this embodiment, the first text with audio conversion errors can be automatically identified based on the first text, the second text, the first pinyin sequence without tones of the first text and the second pinyin sequence without tones of the second text, so as to avoid the problem of low data verification efficiency through manual verification in the scenario of verifying whether the text is correctly converted into audio data, thereby improving data verification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0018] Figure 1 A flowchart of a text verification method provided by an embodiment of the present disclosure;

[0019] Figure 2A flowchart of a text verification method provided by another embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of the structure of a text verification device provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present disclosure.

[0023] In order to solve the problem of whether the text is correctly converted into audio data, the embodiment of the present disclosure provides a text verification method, which can avoid the problem of low efficiency of data verification through manual verification and improve the verification efficiency of whether the text is correctly converted into audio data.

[0024] Figure 1 A flowchart of a text verification method provided by an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the method includes:

[0025] Step S102, acquiring a first text and first audio data obtained by performing audio conversion on the first text;

[0026] Step S104, converting the first audio data into a second text;

[0027] Step S106, obtaining a first pinyin sequence of the first text without tones and a second pinyin sequence of the second text without tones;

[0028] Step S108, identifying the first text with audio conversion error based on the first text, the second text, the first pinyin sequence and the second pinyin sequence; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0029] In this embodiment, first, a first text and first audio data obtained after audio conversion of the first text are obtained, then, the first audio data is converted into a second text, then, a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text are obtained, and finally, the first text with audio conversion errors is identified according to the first text, the second text, the first pinyin sequence and the second pinyin sequence, and the first audio data corresponding to the first text with audio conversion errors has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion errors. It can be seen that through this embodiment, the first text with audio conversion errors can be automatically identified according to the first text, the second text, the first pinyin sequence without tones of the first text and the second pinyin sequence without tones of the second text, so as to avoid the problem of low data verification efficiency through manual verification in the scenario of verifying whether the text is correctly converted into audio data, and improve the data verification efficiency.

[0030] Moreover, in the present embodiment, text verification is achieved without comparing tones, the first text with audio conversion errors is identified, the computational complexity of text verification is reduced, and the first text, the second text, the pinyin sequence of the first text without tones, and the pinyin sequence of the second text without tones are combined for text recognition, thereby improving the accuracy of text verification.

[0031] In the above step S102, a first text is obtained. The first text may be any text that needs to be converted into audio data, such as a paragraph, an article, etc. After the first text is converted into audio data, the first text and the audio data corresponding to the first text may be used to train a variety of language models, such as training a speech synthesis (TTS, Text To Speech) model. The speech synthesis model is also called a text-to-speech model.

[0032] In the above step S102, the first audio data obtained by performing audio conversion on the first text is obtained. In this embodiment, the first audio data can be obtained by performing audio conversion on the first text in a variety of ways. For example, the first audio data can be obtained by performing audio conversion on the first text by manually reading the first text aloud, or the first text can be input into the model, and the first audio data can be obtained by performing audio conversion on the first text using the model.

[0033] In the above step S104, the first audio data is converted into the second text. In one embodiment, the first audio data can be input into an ASR (Automatic Speech Recognition) system, and the first audio data is converted into the second text through the automatic speech recognition system.

[0034] In the above step S106, a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text are obtained. In one embodiment, the pinyin sequence of the first text can be obtained by table lookup or model processing, and the pinyin sequence of the first text records the pinyin with tones of each word in the first text. For polyphones, one or more pinyin with tones of the polyphone can be recorded in the pinyin sequence of the first text, and for non-polyphones, one pinyin with tones of the polyphone can be recorded in the pinyin sequence of the first text. Then, the tones of each pinyin are removed in the pinyin sequence of the first text, and the pinyin sequence of the first text without tones is obtained as the first pinyin sequence.

[0035] Similarly, in one embodiment, the pinyin sequence of the second text can be obtained by table lookup or model processing, and the pinyin sequence of the second text records the pinyin with tones of each character in the second text. For polyphones, one or more pinyin with tones of the polyphone can be recorded in the pinyin sequence of the second text, and for non-polyphones, one pinyin with tones of the polyphone can be recorded in the pinyin sequence of the second text. Then, the tones of each pinyin are removed in the pinyin sequence of the second text, and the pinyin sequence of the second text without tones is obtained as the second pinyin sequence.

[0036] In the above step S108, the first text with audio conversion error is identified based on the first text, the second text, the first pinyin sequence and the second pinyin sequence.

[0037] When there are multiple first texts, it is inevitable that some of the first texts are correctly converted to audio, while some of the first texts are incorrectly converted to audio. Correct audio conversion means that the pronunciation of each word in the first audio data obtained by converting the first text is consistent with the pronunciation of each word in the first text, and the first audio data corresponding to the first text with correct audio conversion does not have the phenomenon of over-reading, under-reading or wrong reading compared with the first text with correct audio conversion. Incorrect audio conversion means that the pronunciation of each word in the first audio data obtained by converting the first text is not completely consistent with the pronunciation of each word in the first text, and the first audio data corresponding to the first text with audio conversion errors has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion errors.

[0038] In one embodiment, identifying the first text with an audio conversion error according to the first text, the second text, the first pinyin sequence and the second pinyin sequence includes:

[0039] If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error:

[0040] The number of words in the first text is different from the number of words in the corresponding second text;

[0041] The first pinyin sequence of the first text is different from the corresponding second pinyin sequence of the second text.

[0042] In this embodiment, since each first text corresponds to a first audio data, and each first audio data corresponds to a second text, each first text corresponds to a second text. Based on this, for each first text, if the number of words in the first text is different from the number of words in the corresponding second text, it is determined that the first text is not correctly converted into audio data, and the first text is determined to be a first text with audio conversion error.

[0043] Alternatively, for each first text, if the first pinyin sequence of the first text is different from the second pinyin sequence of the corresponding second text, it is determined that the first text is not correctly converted into audio data and is determined to be a first text with audio conversion errors.

[0044] Alternatively, for each first text, if the number of words in the first text is different from the number of words in the corresponding second text, and the first pinyin sequence of the first text is different from the second pinyin sequence of the corresponding second text, it is determined that the first text is not correctly converted into audio data, and the first text is determined to be a first text with audio conversion errors.

[0045] In another embodiment, the characters in the first text and the characters in the second text have a corresponding relationship, and the corresponding characters in the first text and the second text have the same sequence number in the first text as the sequence number in the second text. The pinyin in the first pinyin sequence and the pinyin in the second pinyin sequence have a corresponding relationship, and the corresponding pinyin in the first pinyin sequence and the corresponding characters in the second text have the same sequence number in the first text and the second text. Among them, the sequence number of the character can be a sequence number encoded from 0 for each character starting from the first character of the text.

[0046] In this embodiment, the first pinyin sequence of the first text is the same as the second pinyin sequence of the corresponding second text, which means that each pinyin in the first pinyin sequence is the same as the corresponding pinyin in the second pinyin sequence. The first pinyin sequence of the first text is different from the second pinyin sequence of the corresponding second text, which means that there are one or more pinyins in the first pinyin sequence that are different from the corresponding pinyins in the second pinyin sequence.

[0047] It can be seen that through this embodiment, it is possible to determine whether the first text is correctly converted into audio data based on whether the number of words in the first text is the same as the number of words in the second text, and whether the first pinyin sequence is the same as the second pinyin sequence. The determination method is simple and efficient.

[0048] In the scenario where the first text is read aloud by a professional reader to obtain the first audio data, considering that a professional reader usually does not mispronounce a character after correctly recognizing it, in this case, the pinyin and tone of this character in the first text and the second text are the same. The common errors are reading a few more characters, reading a few less characters, or misrecognizing the characters. In this case, after excluding the tones, there are also different pinyins in the first text and the second text. Therefore, in this embodiment, it is not necessary to separately compare whether the pinyin tones of the first text and the second text are the same. By comparing the first pinyin sequence without tones of the first text with the second pinyin sequence without tones of the second text, it is possible to detect the situation of reading more, reading less, or misreading, so as to determine whether the first text is correctly read, improving the efficiency of identifying the first text with audio conversion errors.

[0049] Considering the situation where there are polyphonic characters in the first text, if the audio data conversion of the first text is correct, then the polyphonic character will also exist in the second text obtained by converting the first audio data. In the first pinyin sequence and the second pinyin sequence, there may be multiple pinyins without tones for this polyphonic character. For example, in the first pinyin sequence and the second pinyin sequence, the pinyins without tones corresponding to each pronunciation of the polyphonic character are recorded. Based on this, in one embodiment, the above method further includes:

[0050] When there is a polyphonic character in the first text, if any one of the pinyins of the polyphonic character in the first pinyin sequence is the same as any one of the pinyins of the polyphonic character in the second pinyin sequence, it is determined that the pinyin of the polyphonic character in the first pinyin sequence is the same as the pinyin of the polyphonic character in the second pinyin sequence.

[0051] In this embodiment, when there is a polyphonic character in the first text, the first pinyin sequence records the pinyins without tones corresponding to one or more pronunciations of the polyphonic character. If the audio data conversion of the first text is correct, then the polyphonic character will also exist in the second text obtained by converting the first audio data. In the second pinyin sequence, the pinyins without tones corresponding to one or more pronunciations of the polyphonic character will also be recorded. If any one of the pinyins of the polyphonic character in the first pinyin sequence is the same as any one of the pinyins of the polyphonic character in the second pinyin sequence, it is determined that the pinyin of the polyphonic character in the first pinyin sequence is the same as the pinyin of the polyphonic character in the second pinyin sequence.

[0052] For example, when there is a polyphonic character "dou" in the first text, the first pinyin sequence records one pronunciation "dou" of this polyphonic character, and the second pinyin sequence records two pronunciations "dou, du" of this polyphonic character. Since "dou" is recorded in both the first pinyin sequence and the second pinyin sequence for this polyphonic character, it is determined that the pinyin of the polyphonic character in the first pinyin sequence is the same as the pinyin of the polyphonic character in the second pinyin sequence.

[0053] It can be understood that, for a non-polyphonetic character, if the only pinyin of the non-polyphonetic character in the first pinyin sequence is the same as the only pinyin of the non-polyphonetic character in the second pinyin sequence, then it is determined that the pinyin of the non-polyphonetic character in the first pinyin sequence is the same as the pinyin of the non-polyphonetic character in the second pinyin sequence.

[0054] It can be seen that the present embodiment also takes into account the situation where the first text contains polyphones. When any identical pinyin of the polyphone is recorded in the first pinyin sequence and the second pinyin sequence, it can be determined that the pinyin of the polyphone in the first pinyin sequence is the same as the pinyin of the polyphone in the second pinyin sequence, thereby improving the accuracy and efficiency of verifying polyphones.

[0055] In one embodiment, identifying the first text with an audio conversion error according to the first text, the second text, the first pinyin sequence and the second pinyin sequence includes:

[0056] Modify the nasal sounds in the first phonetic sequence and the nasal sounds in the second phonetic sequence into non-nasal sounds;

[0057] If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error:

[0058] The number of words in the first text is different from the number of words in the corresponding second text;

[0059] The modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text.

[0060] In this embodiment, the nasal sounds in the first pinyin sequence and the nasal sounds in the second pinyin sequence are modified into non-nasal sounds. For example, "ang, eng, ing, ong" in the first pinyin sequence are respectively modified into "an, en, in, on", that is, the "g" in the nasal sounds are removed; and "ang, eng, ing, ong" in the second pinyin sequence are respectively modified into "an, en, in, on", that is, the "g" in the nasal sounds are removed.

[0061] Next, it is determined whether the number of characters in the first text is the same as the number of characters in the corresponding second text, and whether the modified first pinyin sequence of the first text is the same as the modified second pinyin sequence of the corresponding second text.

[0062] In this embodiment, since each first text corresponds to a first audio data, and each first audio data corresponds to a second text, each first text corresponds to a second text. Based on this, for each first text, if the number of words in the first text is different from the number of words in the corresponding second text, it is determined that the first text is not correctly converted into audio data, and the first text is determined to be a first text with audio conversion error.

[0063] Alternatively, for each first text, if the modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text, it is determined that the first text is not correctly converted into audio data and is determined to be a first text with audio conversion errors.

[0064] Alternatively, for each first text, if the number of words in the first text is different from the number of words in the corresponding second text, and the modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text, it is determined that the first text is not correctly converted into audio data, and the first text is determined to be a first text with audio conversion errors.

[0065] In another embodiment, the characters in the first text and the characters in the second text have a corresponding relationship, and the corresponding characters in the first text and the second text have the same sequence number in the first text as the sequence number in the second text. The pinyin in the first pinyin sequence and the pinyin in the second pinyin sequence have a corresponding relationship, and the corresponding pinyin in the first pinyin sequence and the corresponding characters in the second text have the same sequence number in the first text and the second text. Among them, the sequence number of the character can be a sequence number encoded from 0 for each character starting from the first character of the text.

[0066] In this embodiment, the modified first pinyin sequence of the first text is the same as the modified second pinyin sequence of the corresponding second text, which means that each pinyin in the modified first pinyin sequence is the same as the corresponding pinyin in the modified second pinyin sequence. The modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text, which means that there are one or more pinyins in the modified first pinyin sequence that are different from the corresponding pinyins in the modified second pinyin sequence.

[0067] It can be seen that through this embodiment, not only can it be judged whether the first text is correctly converted into audio data based on whether the number of words in the first text is the same as that in the second text, but it also takes into account the problem that the front and rear nasal sounds are easily converted incorrectly when converting text into audio, and the problem that the front and rear nasal sounds may have similar pronunciations when manually reading the first text. The nasal sounds are uniformly removed, and it is judged whether the first text is correctly converted into audio data based on the pinyin sequence without the nasal sounds, thereby eliminating the interference of the front and rear nasal sounds and improving the accuracy of recognizing the first text.

[0068] Similar to the previous example, in the present embodiment, in the scenario where a professional reader reads aloud the first text to obtain the first audio data, it is considered that after a professional reader correctly recognizes a certain word, he or she usually will not misread the word. In this case, the pinyin and pinyin tone of the word in the first text and the second text are the same. Common errors are reading a few more words, reading a few less words or misrecognizing the text. In this case, after excluding the pinyin tone, there are also different pinyins in the first text and the second text. Therefore, in the present embodiment, there is no need to separately compare whether the pinyin tone of the first text and the pinyin tone of the second text are consistent. By comparing the first pinyin sequence of the first text without the tone with the second pinyin sequence of the second text without the tone, it is possible to find out whether the first text is read too much, too little or wrongly. This can determine whether the first text is read correctly, thereby improving the efficiency of identifying the first text with audio conversion errors.

[0069] Considering the situation that there are polyphones in the first text, if the audio data of the first text is converted correctly, the polyphone will also exist in the second text obtained by converting the first audio data. In the first pinyin sequence and the second pinyin sequence, the polyphone may have multiple pinyins without tones. For example, in the first pinyin sequence and the second pinyin sequence, the pinyins without tones corresponding to each pronunciation of the polyphone are recorded. Based on this, in one embodiment, the above method further includes:

[0070] When there are polyphones in the first text, if any pinyin of the polyphone in the modified first pinyin sequence is the same as any pinyin of the polyphone in the modified second pinyin sequence, then it is determined that the pinyin of the polyphone in the modified first pinyin sequence is the same as the pinyin of the polyphone in the modified second pinyin sequence.

[0071] In the present embodiment, when there are polyphones in the first text, the first pinyin sequence records the pinyins without tones corresponding to one or more pronunciations of the polyphones. If the audio data of the first text is converted correctly, the polyphone will also exist in the second text obtained by converting the first audio data, and the pinyins without tones corresponding to one or more pronunciations of the polyphone will also be recorded in the second pinyin sequence. After the nasal sounds in the first pinyin sequence and the nasal sounds in the second pinyin sequence are modified to non-nasal sounds, if any pinyin of the polyphone in the modified first pinyin sequence is the same as any pinyin of the polyphone in the modified second pinyin sequence, it is determined that the pinyin of the polyphone in the modified first pinyin sequence is the same as the pinyin of the polyphone in the modified second pinyin sequence.

[0072] For example, when there is a polyphonic character "dou" in the first text, one pronunciation "dou" of this polyphonic character is recorded in the modified first pinyin sequence, and two pronunciations "dou, du" of this polyphonic character are recorded in the modified second pinyin sequence. Since "dou" is recorded in both the modified first pinyin sequence and the modified second pinyin sequence for this polyphonic character, it is determined that the pinyin of this polyphonic character in the modified first pinyin sequence is the same as the pinyin of this polyphonic character in the modified second pinyin sequence.

[0073] It can be understood that for a non - polyphonic character, if the only pinyin of this non - polyphonic character in the modified first pinyin sequence is the same as the only pinyin of this non - polyphonic character in the modified second pinyin sequence, it is determined that the pinyin of this non - polyphonic character in the modified first pinyin sequence is the same as the pinyin of this non - polyphonic character in the modified second pinyin sequence.

[0074] It can be seen that in this embodiment, the situation where the first text contains polyphonic characters is also considered. When any one of the same pinyins of this polyphonic character is recorded in the modified first pinyin sequence and the modified second pinyin sequence, it can be determined that the pinyin of this polyphonic character in the modified first pinyin sequence is the same as the pinyin of this polyphonic character in the modified second pinyin sequence, which improves the accuracy and efficiency of verifying polyphonic characters.

[0075] In one embodiment, for the polyphonic character in the first text, if the pinyins of two different pronunciations of this polyphonic character are respectively recorded in the first pinyin sequence and the second pinyin sequence, it is also determined that the pinyin of this polyphonic character in the first pinyin sequence is the same as the pinyin of this polyphonic character in the second pinyin sequence. And, for the polyphonic character in the first text, if the pinyins of two different pronunciations of this polyphonic character are respectively recorded in the first pinyin sequence after removing the nasal sound and the second pinyin sequence after removing the nasal sound, it is also determined that the pinyin of this polyphonic character in the first pinyin sequence after removing the nasal sound is the same as the pinyin of this polyphonic character in the second pinyin sequence after removing the nasal sound.

[0076] Through the above process, the first text with audio conversion errors can be initially identified. Considering that in the above process, when converting the first audio data into the second text, the audio - to - text technology has a certain error rate, so in the identified first text with audio conversion errors, there may be a first text with correct audio conversion. Based on this, Figure 2 is a schematic flowchart of the text verification method provided by another embodiment of the present disclosure. As Figure 2 shown, this process further includes:

[0077] Step S110, according to the first text with audio conversion errors and the first audio data corresponding to the first text with audio conversion errors, determine whether to correct the first text with audio conversion errors into the first text with correct audio conversion.

[0078] In this embodiment, by Figure 1 After the process in the foregoing preliminarily identifies and obtains one or more first texts with audio conversion errors, for each first text with audio conversion error, a determination is made as to whether the first text with audio conversion error should be corrected to a first text with correct audio conversion based on the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error.

[0079] It can be seen that through this embodiment, the first text with audio conversion errors can be corrected to the first text with correct audio conversion through secondary recognition, thereby achieving the effect of improving the accuracy of text verification.

[0080] In one embodiment, judging whether to correct the first text with audio conversion error to the first text with audio conversion error according to the first audio data corresponding to the first text with audio conversion error includes:

[0081] Determine the confidence of each word in the first text with audio conversion errors according to the first text with audio conversion errors and the first audio data corresponding to the first text with audio conversion errors; the confidence indicates the degree of confidence that the word is correctly pronounced in the first audio data;

[0082] According to the confidence of each word in the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion.

[0083] In this embodiment, for each first text with audio conversion error, the confidence of each word in the first text with audio conversion error is determined based on the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error. The confidence indicates the degree of confidence that the word is correctly pronounced in the first audio data. The higher the confidence, the more likely it is that the word is correctly pronounced in the first audio data.

[0084] Next, for each first text with audio conversion errors, according to the confidence of each word in the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion. For example, if the confidence of each word in the first text with audio conversion errors is greater than the confidence threshold, the first text with audio conversion errors is corrected to the first text with correct audio conversion.

[0085] It can be seen that through this embodiment, the confidence of each character in the first text with audio conversion error can be determined. The confidence represents the confidence level of the correct pronunciation of the character in the first audio data. According to the confidence of each character in the first text with audio conversion error, it is determined whether to correct the first text with audio conversion error to the first text with correct audio conversion, so as to achieve the effect of secondary recognition of the first text with audio conversion error and avoid misidentifying the first text with correct audio conversion as the first text with audio conversion error.

[0086] In one embodiment, for each first text with audio conversion error, according to the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error, determining the confidence of each character in the first text with audio conversion error includes:

[0087] Determining a third pinyin sequence according to the first audio data corresponding to the first text with audio conversion error; at least one pinyin of each character in the first text with audio conversion error is recorded in the third pinyin sequence;

[0088] Determining the matching degree value between each character in the first text with audio conversion error and each pinyin of each character recorded in the third pinyin sequence;

[0089] According to the matching degree value, selecting a corresponding pinyin for each character among each pinyin of each character; the matching degree value corresponding to the selected pinyin is the confidence of the character corresponding to the selected pinyin.

[0090] In this embodiment, any first text with audio conversion error is taken as an example for illustration. Determining a third pinyin sequence according to the first audio data corresponding to the first text with audio conversion error, at least one pinyin of each character in the first text with audio conversion error is recorded in the third pinyin sequence, and each pinyin of each character represents a possible pronunciation of the character, and the pinyin can have a tone or no tone.

[0091] Next, each character in the first text with audio conversion error is compared with each pinyin of each character recorded in the third pinyin sequence to determine the matching degree value between each character in the first text with audio conversion error and each pinyin of each character recorded in the third pinyin sequence. For example, there is a character '天' in the first text with audio conversion error, and the pinyins recorded in the third pinyin sequence for this character include 'tian' and 'tan'. It is determined that the matching degree value between this character and the pinyin 'tian' is 9.8, and the matching degree value between this character and the pinyin 'tan' is 5.6.

[0092] Then, according to the matching degree value, selecting the pinyin with the highest matching degree value as the pinyin of each character among each pinyin of each character, and the matching degree value corresponding to the selected pinyin is the confidence of the character corresponding to the selected pinyin.

[0093] It can be seen that through this embodiment, the pinyin of each character in the first text with audio conversion errors and the matching degree value between the pinyin of each character and the corresponding character can be determined. The higher the matching degree value, the better the match between the pinyin of the character and the character, that is, the higher the confidence level that the character is correctly pronounced in the first audio data, thereby accurately identifying the first text with correct audio conversion.

[0094] The above process of determining whether to correct the first text with audio conversion errors to the first text with correct audio conversion can be implemented through a grapheme-to-phoneme alignment system such as a G2P (Grapheme-to-Phoneme) model. In a specific embodiment, the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error are input into the G2P model. The G2P model first determines a third pinyin sequence according to the first audio data corresponding to the first text with audio conversion error. The third pinyin sequence records at least one pinyin of each word in the first text with audio conversion error. Then the G2P model compares each word in the first text with audio conversion error with each pinyin of each word recorded in the third pinyin sequence to determine the matching degree value of each word in the first text with audio conversion error and each pinyin of each word recorded in the third pinyin sequence. The matching degree value can be determined based on the graph path shortest search algorithm. Then, the G2P model selects the pinyin with the highest matching degree value as the pinyin of each word from each pinyin of each word according to the matching degree value. The matching degree value corresponding to the selected pinyin is the confidence of the word corresponding to the selected pinyin, so that the G2P model outputs the confidence of each word in the first text with audio conversion error. If the confidence of each word in the first text with audio conversion errors is higher than the confidence threshold, the first text with audio conversion errors is corrected to the first text with correct audio conversion.

[0095] Of course, the above process of determining whether to correct the first text with audio conversion errors to the first text with correct audio conversion can be implemented by a character-phonetic alignment system other than the G2P model. This embodiment does not limit the character-phonetic alignment system, and any character-phonetic alignment system that can obtain the confidence of each character can be used.

[0096] In summary, through the above-mentioned text verification method, the first text with audio conversion errors can be automatically identified based on the first text, the second text, the first pinyin sequence without tones of the first text, and the second pinyin sequence without tones of the second text. Therefore, in the scenario of verifying whether the text is correctly converted into audio data, the problem of low data verification efficiency through manual verification is avoided, the data verification efficiency is improved, and text and correct audio data corresponding to the text are provided for the field of natural language processing, and a basis is provided for secondary processing of text with audio conversion errors, such as re-conversion of audio.

[0097] Figure 3 A structural diagram of a text verification device provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the device comprises:

[0098] A first acquisition unit 31 is used to acquire a first text and first audio data obtained by performing audio conversion on the first text;

[0099] An audio conversion unit 32, configured to convert the first audio data into a second text;

[0100] A second acquiring unit 33 is used to acquire a first pinyin sequence of the first text without tones and a second pinyin sequence of the second text without tones;

[0101] The text recognition unit 34 is used to identify the first text with audio conversion error based on the first text, the second text, the first pinyin sequence and the second pinyin sequence; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0102] Optionally, the text recognition unit 34 is specifically used for:

[0103] If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error:

[0104] The number of words in the first text is different from the number of words in the corresponding second text;

[0105] The first pinyin sequence of the first text is different from the corresponding second pinyin sequence of the second text.

[0106] Optionally, the method further comprises a first polyphonetic character recognition unit, which is used to:

[0107] When there are polyphones in the first text, if any pinyin of the polyphone in the first pinyin sequence is the same as any pinyin of the polyphone in the second pinyin sequence, it is determined that the pinyin of the polyphone in the first pinyin sequence is the same as the pinyin of the polyphone in the second pinyin sequence.

[0108] Optionally, the text recognition unit 34 is specifically used for:

[0109] Modify the nasal sounds in the first pinyin sequence and the nasal sounds in the second pinyin sequence into non-nasal sounds;

[0110] If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error:

[0111] The number of words in the first text is different from the number of words in the corresponding second text;

[0112] The modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text.

[0113] Optionally, a second polyphonetic character recognition unit is further included, which is used to:

[0114] When there are polyphones in the first text, if any pinyin of the polyphone in the modified first pinyin sequence is the same as any pinyin of the polyphone in the modified second pinyin sequence, then it is determined that the pinyin of the polyphone in the modified first pinyin sequence is the same as the pinyin of the polyphone in the modified second pinyin sequence.

[0115] Optionally, a post-processing unit is further included, for:

[0116] According to the first text with audio conversion errors and the first audio data corresponding to the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion.

[0117] Optionally, the post-processing unit is specifically used for:

[0118] Determine the confidence of each word in the first text with audio conversion error according to the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error; the confidence indicates the confidence level that the word is correctly pronounced in the first audio data;

[0119] According to the confidence of each word in the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion.

[0120] Optionally, the post-processing unit is further specifically used for:

[0121] Determine a third pinyin sequence according to the first audio data corresponding to the first text with audio conversion error; the third pinyin sequence records at least one pinyin of each word in the first text with audio conversion error;

[0122] Determine a matching degree value between each character in the first text with the audio conversion error and each pinyin of each character recorded in the third pinyin sequence;

[0123] According to the matching degree value, a corresponding pinyin is selected for each character from each pinyin of each character; the matching degree value corresponding to the selected pinyin is the confidence of the character corresponding to the selected pinyin.

[0124] The text verification device in the embodiment of the present disclosure can implement each process of the above-mentioned text verification method embodiment and achieve the same effect and function, which will not be repeated here.

[0125] An embodiment of the present disclosure further provides an electronic device, Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 401 and memory 402, and one or more applications or data may be stored in the memory 402. Among them, the memory 402 may be a temporary storage or a permanent storage. The application stored in the memory 402 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 401 may be configured to communicate with the memory 402 to execute a series of computer executable instructions in the memory 402 on the electronic device. The electronic device may also include one or more power supplies 403, one or more wired or wireless network interfaces 404, one or more input or output interfaces 405, one or more keyboards 406, etc.

[0126] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process:

[0127] Acquire a first text and first audio data obtained by performing audio conversion on the first text;

[0128] Converting the first audio data into a second text;

[0129] Acquire a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text;

[0130] According to the first text, the second text, the first pinyin sequence and the second pinyin sequence, a first text with audio conversion error is identified; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0131] The text verification device in the embodiment of the present disclosure can implement each process of the above-mentioned text verification method embodiment and achieve the same effect and function, which will not be repeated here.

[0132] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented:

[0133] Acquire a first text and first audio data obtained by performing audio conversion on the first text;

[0134] Converting the first audio data into a second text;

[0135] Acquire a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text;

[0136] According to the first text, the second text, the first pinyin sequence and the second pinyin sequence, a first text with audio conversion error is identified; the first audio data corresponding to the first text with audio conversion error has at least one of over-reading, under-reading and wrong reading compared with the first text with audio conversion error.

[0137] The storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned text verification method embodiment and achieve the same effect and function, which will not be repeated here.

[0138] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0140] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0141] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0142] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0143] Those skilled in the art will appreciate that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0147] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0148] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0149] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0150] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A text verification method, characterized in that: include: Acquire a first text and first audio data obtained by performing audio conversion on the first text; Converting the first audio data into a second text; Acquire a first pinyin sequence without tones of the first text and a second pinyin sequence without tones of the second text; Identify the first text with audio conversion error according to the first text, the second text, the first pinyin sequence and the second pinyin sequence; The first audio data corresponding to the first text with the audio conversion error has at least one of over-reading, under-reading, and wrong reading compared with the first text with the audio conversion error.

2. The method according to claim 1, characterized in that The step of identifying the first text with an audio conversion error according to the first text, the second text, the first pinyin sequence, and the second pinyin sequence includes: If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error: The number of words in the first text is different from the number of words in the corresponding second text; The first pinyin sequence of the first text is different from the corresponding second pinyin sequence of the second text.

3. The method according to claim 2, characterized in that The method further comprises: When there are polyphones in the first text, if any pinyin of the polyphone in the first pinyin sequence is the same as any pinyin of the polyphone in the second pinyin sequence, it is determined that the pinyin of the polyphone in the first pinyin sequence is the same as the pinyin of the polyphone in the second pinyin sequence.

4. The method according to claim 1, characterized in that: The step of identifying the first text with an audio conversion error according to the first text, the second text, the first pinyin sequence, and the second pinyin sequence includes: Modify the nasal sounds in the first pinyin sequence and the nasal sounds in the second pinyin sequence into non-nasal sounds; If the first text satisfies one of the following conditions, the first text is determined to be the first text with audio conversion error: The number of words in the first text is different from the number of words in the corresponding second text; The modified first pinyin sequence of the first text is different from the modified second pinyin sequence of the corresponding second text.

5. The method according to claim 4, characterized in that The method further comprises: When there are polyphones in the first text, if any pinyin of the polyphone in the modified first pinyin sequence is the same as any pinyin of the polyphone in the modified second pinyin sequence, then it is determined that the pinyin of the polyphone in the modified first pinyin sequence is the same as the pinyin of the polyphone in the modified second pinyin sequence.

6. The method according to claim 1, characterized in that The method further comprises: According to the first text with audio conversion errors and the first audio data corresponding to the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion.

7. The method according to claim 6, characterized in that The step of judging whether to correct the first text with audio conversion error to the first text with audio conversion error according to the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error includes: Determine the confidence of each word in the first text with audio conversion error according to the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error; the confidence indicates the confidence level that the word is correctly pronounced in the first audio data; According to the confidence of each word in the first text with audio conversion errors, it is determined whether to correct the first text with audio conversion errors to the first text with correct audio conversion.

8. The method according to claim 7, characterized in that The step of determining the confidence of each word in the first text with audio conversion error according to the first text with audio conversion error and the first audio data corresponding to the first text with audio conversion error comprises: Determine a third pinyin sequence according to the first audio data corresponding to the first text with audio conversion error; the third pinyin sequence records at least one pinyin of each word in the first text with audio conversion error; Determine a matching degree value between each character in the first text with the audio conversion error and each pinyin of each character recorded in the third pinyin sequence; According to the matching degree value, a corresponding pinyin is selected for each character from each pinyin of each character; the matching degree value corresponding to the selected pinyin is the confidence of the character corresponding to the selected pinyin.

9. A text verification device, characterized in that: include: A first acquisition unit, used to acquire a first text and first audio data obtained by performing audio conversion on the first text; An audio conversion unit, used for converting the first audio data into a second text; A second acquisition unit, configured to acquire a first pinyin sequence of the first text without tones and a second pinyin sequence of the second text without tones; A text recognition unit, configured to recognize a first text having an audio conversion error according to the first text, the second text, the first pinyin sequence, and the second pinyin sequence; The first audio data corresponding to the first text with the audio conversion error has at least one of over-reading, under-reading, and wrong reading compared with the first text with the audio conversion error.

10. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 8.

Images