A method and apparatus for processing a speech corpus

Abstract

Embodiments of the present application provide a speech corpus processing method and device, relating to the field of speech recognition in the field of artificial intelligence, which can efficiently and accurately determine the usability of the speech corpus. The method comprises: obtaining a speech corpus to be processed, the speech corpus to be processed comprising text of the speech corpus and speech of the speech corpus; and performing standardized processing on the text of the speech corpus to obtain standardized text of the speech corpus, the standardized text comprising one or more candidate texts; then generating a structured description of a feasible phonetic sequence of the standardized text; performing speech recognition on the speech of the speech corpus to obtain a phonetic probability matrix of the speech corpus; and performing alignment processing on the structured description of the feasible phonetic sequence of the standardized text and the phonetic probability matrix of the speech corpus to determine the usefulness of the speech corpus.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech recognition, and in particular, to a speech corpus processing method and device. BACKGROUND

[0002] Automatic speech recognition (ASR) is increasingly widely used, and the accuracy of speech recognition is increasingly required.

[0003] Speech recognition is a process of converting speech into corresponding text based on a speech recognition model. In current speech recognition methods based on an ASR model, a large amount of speech corpus (including speech and corresponding text) is used as training samples to train the ASR model, so that the ASR model is used to recognize speech to be recognized and obtain the text corresponding to the speech to be recognized.

[0004] However, when the text in the training sample has a reading ambiguity (for example, special punctuation, numbers, and coordinated pronunciation), the speech and the text in the training sample cannot be completely matched, the performance of the ASR model based on such training samples is poor, and the accuracy of speech recognition is reduced. SUMMARY

[0005] Embodiments of the present application provide a speech corpus processing method and device, which can efficiently and accurately determine the usability of the speech corpus.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a speech corpus processing method, which includes: obtaining a speech corpus to be processed, the speech corpus to be processed including text of the speech corpus and speech of the speech corpus; and performing normalization processing on the text of the speech corpus to obtain normalized text of the speech corpus, the normalized text including one or more candidate texts; and generating a structured description of a feasible phonetic sequence of the normalized text; then performing speech recognition on the speech of the speech corpus to obtain a phonetic probability matrix of the corpus; and finally performing alignment processing on the structured description of the feasible phonetic sequence of the normalized text and the phonetic probability matrix of the speech corpus to determine the usefulness of the speech corpus.

[0008] Since the structured description of the feasible phonetic spelling sequence generated after the standardization of the speech corpus is a standard and comprehensive phonetic spelling sequence, not all phonetic spellings of the text of the speech corpus are traversed, and the alignment of the structured description of the feasible phonetic spelling sequence of the standardized text and the phonetic spelling probability matrix can reduce the time complexity and space complexity of the speech corpus processing, therefore, the speech corpus processing method provided in the embodiments of the present application can efficiently and accurately determine the availability of the speech corpus.

[0009] In a possible implementation, the method for standardizing the text of the speech corpus to obtain the standardized text of the speech corpus comprises: segmenting the text of the speech corpus to be processed to obtain at least one segmented text; and structurally describing the at least one segmented text to obtain the standardized text.

[0010] In a possible implementation, the method for segmenting the text of the speech corpus to be processed comprises: segmenting the text according to the symbol feature in the text and / or the number feature in the text. Specifically, the method for segmenting the text according to the symbol feature in the text is: deleting the first type of symbol in the text, and segmenting the text according to the second type of symbol in the text to obtain at least one segmented text. The first type of symbol includes, but is not limited to, the book name mark, the quotation mark (which can include double quotation marks or single quotation marks), and the bracket; and the second type of symbol includes, but is not limited to, the comma, the period, the semicolon, the colon, the comma, the ellipsis, the dash, the exclamation mark, and the question mark. The method for segmenting the text according to the number feature in the text is: dividing the text that can be matched to the date (including the year, the month, and the day) or the arithmetic expression into one segmented text.

[0011] Optionally, the text that can be matched to the proper noun in the text can also be divided into one segmented text, and the text that can be matched to the decimal, the fraction, the percentage, etc. in the text can also be divided into one segmented text.

[0012] It should be noted that if the text contains continuous Chinese characters and does not contain punctuation marks, the continuous Chinese characters are divided into one segment.

[0013] In a possible implementation, the structured description of the feasible phonetic spelling sequence of the standardized text comprises a finite state transducer (FST) description.

[0014] In a possible implementation, the structured description of the segmented text comprises a Backus-Naur Form (BNF) description. Other methods for structurally describing the text can also be used to obtain the standardized text, and the embodiments of the present application are not limited in this regard.

[0015] In a possible implementation, when the normalized text is composed of continuous characters (without punctuation marks), for example, the normalized text is composed of continuous Chinese characters, the generating the phonetic sequence of the normalized text comprises: performing word segmentation on the normalized text to obtain at least one word segmentation result of the normalized text; and based on a preset phonetic dictionary, phoneticizing the at least one word segmentation result to obtain the phonetic sequence of the normalized text.

[0016] In the embodiments of the present application, the text of the voice corpus is normalized, and a structured description of the feasible phonetic sequence of the normalized text is generated, instead of traversing all possible phonetic sequences of the text. In this way, after the text is normalized and then phoneticized, the phoneticization of the text can be obtained with lower time complexity and space complexity.

[0017] In a possible implementation, the method for performing voice recognition on the voice of the voice corpus to obtain a phonetic probability matrix of the voice corpus comprises: performing voice recognition on the voice of the voice corpus based on a connectionist temporal classification (CTC) acoustic model to obtain the phonetic probability matrix of the voice corpus.

[0018] In the embodiments of the present application, the phonetic probability matrix obtained based on the CTC acoustic model can indicate the probability of the phonetic unit corresponding to each frame of the signal in the phonetic unit list, for example, an element P t,v represents the probability of taking the v th phonetic unit in the phonetic unit list at the t th time frame.

[0019] In the embodiments of the present application, the alignment processing is performed according to the above method, the alignment processing can be completed with lower time complexity, and the unusable voice corpus can be efficiently screened out. Since the text of the voice corpus is normalized, the alignment processing based on the normalized text can reduce the mis-screening of the voice corpus.

[0020] In a possible implementation, the method for processing the voice corpus provided in the embodiments of the present application further comprises: using the plurality of usable voice corpora as training samples, and learning an automatic speech recognition (ASR) model by using an artificial intelligence (AI) algorithm, the ASR model being used to convert the voice into text.

[0021] The method for processing the voice corpus provided in the embodiments of the present application can process a large amount of voice corpora, screen out unusable voice corpora, and then use the plurality of usable voice corpora as training samples to learn an ASR model with better performance (the ASR model is used to convert the voice into text) by using an AI algorithm, thereby improving the accuracy of speech recognition.

[0022] In a second aspect, an embodiment of the present application provides a speech corpus processing apparatus, comprising an obtaining module, a normalization processing module, a phonetic transcription module, a speech recognition module, and an alignment module. The obtaining module is configured to obtain a speech corpus to be processed, the speech corpus to be processed comprising text of the speech corpus and speech of the speech corpus. The normalization processing module is configured to perform normalization processing on the text of the speech corpus to obtain normalized text of the speech corpus, the normalized text comprising one or more candidate texts. The phonetic transcription module is configured to generate a structured description of a feasible phonetic transcription sequence of the normalized text. The speech recognition module is configured to perform speech recognition on the speech of the speech corpus to obtain a phonetic transcription probability matrix of the speech corpus. The alignment module is configured to perform alignment processing on the structured description of the feasible phonetic transcription sequence of the normalized text and the phonetic transcription probability matrix of the speech corpus to determine usefulness of the speech corpus.

[0023] In a possible implementation, the normalization processing module is specifically configured to perform segmentation processing on the text of the speech corpus to obtain at least one segmented text, and perform structured description on the at least one segmented text to obtain the normalized text.

[0024] In a possible implementation, the normalization processing module is specifically configured to segment the text according to a symbol feature in the text and / or a number feature in the text.

[0025] In a possible implementation, the structured description of the feasible phonetic transcription sequence of the normalized text comprises a finite state transducer (FST) description.

[0026] In a possible implementation, the structured description of the segmented text comprises a Backus-Naur Form (BNF) description.

[0027] In a possible implementation, the speech recognition module is specifically configured to perform speech recognition on the speech of the speech corpus based on a connectionist temporal classification (CTC) acoustic model to obtain the phonetic transcription probability matrix of the speech corpus.

[0028] In a possible implementation, the speech corpus processing apparatus provided by an embodiment of the present application further comprises a model training module. The model training module is configured to use a plurality of available speech corpora as training samples, and learn an automatic speech recognition (ASR) model by using an artificial intelligence algorithm, the ASR model being configured to convert speech into text.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory coupled to the processor. The memory is configured to store computer instructions. When the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method in any one of the first aspect and possible implementation manners thereof.

[0030] In a fourth aspect, the present application provides a computer readable storage medium, such as a computer non-transitory readable storage medium. A computer program (or instructions) is stored on the computer readable storage medium, and when the computer program (or instructions) is run on an electronic device, the electronic device is caused to perform the method of any one of the first aspect and the optional implementation manners thereof.

[0031] In a fifth aspect, the present application provides a computer program product, which when run on a computer, causes the method of any one of the first aspect and the optional implementation manners thereof to be performed.

[0032] In a sixth aspect, the present application provides a chip system, comprising: a processor, the processor being configured to invoke and run a computer program stored in a memory, and perform the method of any one of the first aspect and the possible implementation manners thereof.

[0033] It can be understood that any one of the electronic device, the computer storage medium, the computer program product or the chip system provided above can be applied to the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A hardware schematic diagram of an electronic device provided by an embodiment of the present application;

[0035] Figure 2 A processing method of a speech corpus provided by an embodiment of the present application Figure 1 ;

[0036] Figure 3 A processing method of a speech corpus provided by an embodiment of the present application Figure 2 ;

[0037] Figure 4 A structure of FST data of a phonetic sequence provided by an embodiment of the present application;

[0038] Figure 5 A flowchart of a processing method of a speech corpus provided by an embodiment of the present application Figure 1 ;

[0039] Figure 6 A flowchart of a processing method of a speech corpus provided by an embodiment of the present application Figure 2 ;

[0040] Figure 7 A structure of a processing device of a speech corpus provided by an embodiment of the present application Figure 1 ;

[0041] Figure 8A structure diagram of a speech corpus processing device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0042] The term "and / or" in the present application is merely used to describe an associated relationship with associated objects, and means that three relationships can exist, for example, A and / or B can mean that three cases of A alone, A and B together, and B alone exist.

[0043] In the embodiments of the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design schemes. Rather, the words such as "exemplary" or "for example" are intended to present the relevant concept in a specific manner.

[0044] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. For example, a plurality of processing units means two or more processing units; a plurality of systems means two or more systems.

[0045] First, some concepts involved in the speech corpus processing method and device provided in the embodiments of the present application are explained.

[0046] Speech corpus: composed of speech and text corresponding to the speech. Generally, there are two methods to generate a speech corpus, one is to read the known text to obtain the speech; the other is to recognize the known speech to obtain the text. It should be understood that the process of obtaining speech from text is text-to-speech (TTS), and the process of obtaining text from speech is speech recognition (ASR). The above-mentioned conversion of text to speech or speech to text can be realized based on machine, or realized by artificial, or realized by machine and artificial cooperation.

[0047] Pinyin sequence: a sequence composed of pinyin units, the pinyin sequence can express the pronunciation of the characters contained in the text that need to be pronounced, and the text corresponding to the speech can be obtained by reading according to the pinyin sequence of the text.

[0048] Phonetic unit: A phonetic symbol used to represent the pronunciation of text. Exemplarily, the pinyin of a Chinese character is a phonetic unit. For example, the phonetic unit of the Chinese character "你" is "ni3", where 3 represents the third tone in tones. In the embodiments of the present application, "0", "1", "2", "3", "4" are used to represent the tones in Chinese pinyin, "0" represents the light tone, "1" represents the first tone, "2" represents the second tone, "3" represents the third tone, and "4" represents the fourth tone. The phonetic notation of an English word is also a phonetic unit. For example, the phonetic unit of the English word "nice" is "[nais]".

[0049] In summary, for the text "你吃饭了没?", the phonetic sequence of this text is "ni3 chi1 fan4 le0mei2". It should be noted that the punctuation mark "?" in the text is not pronounced. For the text "nice to meet you!", the phonetic sequence of this text is "[nais][tu:][mi:t][ju:]". Similarly, the punctuation mark "!" in the text is not pronounced.

[0050] Word segmentation: Refers to splitting the character sequence in a text into individual words one by one. Word segmentation is the process of recombining the continuous character sequence into a word sequence according to certain specifications. For example, a possible result of word segmentation for the text "杭州西湖风景很好,是旅游胜地" is "杭州|西湖|风景|很|好|,|是|旅游|胜地|", where "|" is the segmentation delimiter. For example, in this text, "杭州" is segmented into one word, and "西湖" is segmented into one word.

[0051] It should be understood that in the field of speech recognition, common word segmentation methods include but are not limited to word segmentation methods based on string matching, understanding-based word segmentation methods, and statistics-based word segmentation methods. For specific references, see existing relevant content. Exemplarily, common Chinese word segmentation tools include but are not limited to jieba segmentation, SnowNLP, THULAC (THU Lexical Analyzer for Chinese), and NLPIR word segmentation system.

[0052] Phonetic dictionary: A mapping dictionary between words and their phonetic sequences. Through the phonetic dictionary, the phonetic sequence of a word can be found.

[0053] It should be noted that the phonetic dictionary in the embodiments of the present application is not an authoritative dictionary or lexicon of various languages (such as Chinese dictionaries, idiom dictionaries, English dictionaries, etc.). The phonetic dictionary is a tool developed by various developers in the field of speech recognition for phonetically annotating text.

[0054] Phonetic unit list / dictionary: Includes a list of all phonetic units. It should be noted that the phonetic dictionary is different from the phonetic unit list.

[0055] Finite state transducer (FST): a mathematical operation that integrates several rules into a single large rule for a one-way pass to improve the efficiency of a rule-based system, which is widely used in the fields of speech recognition and natural language search and processing.

[0056] Based on the problems existing in the background art, the embodiment of the present application provides a speech corpus processing method and device, which is used for processing equipment (such as a computer) to perform normalization processing on the text of the obtained speech corpus to be processed, to obtain a normalized text of the speech corpus; and to generate a structured description of a feasible pinyin sequence of the normalized text; then the processing equipment performs speech recognition on the speech of the speech corpus to obtain a pinyin probability matrix of the speech corpus; and finally, the structured description of the feasible pinyin sequence of the normalized text and the pinyin probability matrix of the speech corpus are aligned to determine the usefulness of the speech corpus. Through the technical scheme provided in the embodiment of the present application, the unusable speech corpus can be efficiently and accurately screened out.

[0057] Optionally, the speech corpus processing method provided in the embodiment of the present application can determine whether a piece of speech corpus is usable. In the embodiment of the present application, the plurality of usable speech corpora determined based on the method are used to train an ASR model, or to train a TTS model, or to be used in other scenarios, which are not limited in the embodiment of the present application.

[0058] The speech corpus processing method provided in the embodiment of the present application can be applied to an electronic device, such as a server or a smart terminal, etc. Figure 1 is a hardware schematic diagram of the electronic device provided in the embodiment of the present application.

[0059] Reference Figure 1 The electronic device includes a processor 101, a memory 102, a network interface 103 and a bus 104. Among them, the processor 101, the memory 102 and the network interface 103 can be connected through the bus 104, or connected with each other in other ways.

[0060] The processor 101 is the control center of the electronic device, and the processor 101 can be a general-purpose central processing unit (CPU), or other general-purpose processors, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc. Illustratively, the processor 101 can include one or more CPUs. The CPU is a single-core CPU (single-CPU) or a multi-core CPU (multi-CPU).

[0061] The memory 102 includes, but is not limited to, a random access memory (RAM), a read only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or an optical memory, a magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0062] In a possible implementation, the memory 102 can exist independently of the processor 101. The memory 102 can be connected to the processor 101 through the bus 104, for storing data, instructions or program codes. When the processor 101 invokes and executes the instructions or program codes stored in the memory 102, the voice corpus processing method provided by the embodiments of the present application can be implemented.

[0063] In another possible implementation, the memory 102 can also be integrated with the processor 101.

[0064] The network interface 103 is a wired interface (port), such as a fiber distributed data interface (FDDI), a gigabit Ethernet (GE) interface. Alternatively, the network interface 103 is a wireless interface. It should be understood that the network interface 103 includes a plurality of physical ports, and the network interface 103 can be used to receive or send a voice.

[0065] Optionally, the electronic device further includes an input / output interface 105, which is used to connect with an input device and receive information input by a user through the input device. The input device includes, but is not limited to, a keyboard, a touch screen, a microphone, and the like. The input / output interface 105 is also used to connect with an output device and output a processing result (for example, a voice corpus is available or unavailable) of the processor 101. The output device includes, but is not limited to, a display, a printer, and the like.

[0066] The bus 104 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation,Figure 1 In the drawings, like reference numerals refer to like elements throughout the several views, although the same reference numeral can be re-used in different figures to denote similar or like elements. The drawings are not to scale.

[0067] It should be noted that, Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, except Figure 1 In addition to the components shown, the electronic device can include more or less components, or combine some components, or different component arrangements.

[0068] It should be understood that the speech corpus processing method provided by the embodiments of the present application can be used for speech corpora of different languages, such as Chinese, English, Japanese, etc. For the sake of understanding, the speech corpus processing method is described in detail by taking processing of a Chinese speech corpus as an example.

[0069] As shown in the figure, Figure 2 The speech corpus processing method provided by the embodiments of the present application includes steps 201 to 205.

[0070] Step 201, obtaining a speech corpus to be processed, the speech corpus to be processed including text of the speech corpus and speech of the speech corpus.

[0071] In the embodiments of the present application, the speech of the speech corpus is an audio, and the text of the speech corpus is a Chinese character expression of the speech, for example, the text of a piece of speech is "How is the weather today?".

[0072] In actual application, the text of a speech corpus can only include characters, for example, "I hit", "I accidentally", "Learning to say is not a kind of ability", etc. The text of a speech corpus can also include Chinese characters, symbols, numbers, or combinations of at least two of Chinese characters, symbols or numbers, etc., for example, "2001", "T-62 tank", "P2P", "6:30", "I like the book 'Analects of Confucius', do you like it?" etc.

[0073] It should be noted that the speech and the text of the speech corpus to be processed are not necessarily completely matched, and the speech and the text can not match, for example, the speech expresses "he is in school", and the text is "he is laughing", if the speech corpus is used to train an ASR model, it will result in poor accuracy of the ASR model.

[0074] Through the following embodiments, it can be determined whether the speech and the text of the speech corpus match, i.e., whether the speech corpus is available. It should be understood that when the speech and the text of the speech corpus match, it is determined that the speech corpus is available; when the speech and the text of the speech corpus do not match, it is determined that the speech corpus is not available.

[0075] Step 202: Standardize the text of the speech corpus to obtain the standardized text of the speech corpus.

[0076] In this embodiment of the application, the normalized text obtained after the above normalization process contains one or more candidate texts.

[0077] Table 1 below shows several types of text and their normalized versions.

[0078] Table 1

[0079]

[0080] As shown in Table 1, the standardized text of the text "2001" contains 2 candidate texts; the standardized text of the text "6:30" contains 5 candidate texts; and the standardized text of texts composed of continuous Chinese characters, such as "I took a taxi", contains only one candidate text.

[0081] Optionally, combined Figure 2 ,like Figure 3 As shown, step 202 can be implemented through steps 2021 to 2022.

[0082] Step 2021: Segment the text of the speech corpus to obtain at least one segmented text.

[0083] Optionally, in one implementation, the above-described method for segmenting the text of the speech corpus may include, but is not limited to, segmenting the text according to the symbolic features and / or the numerical features in the text.

[0084] Specifically, the method for segmenting text according to the symbolic features in the text is as follows: delete the first type of symbols in the text, and segment the text according to the second type of symbols in the text to obtain at least one segmented text. Among them, the first type of symbols includes, but is not limited to, book title marks, quotation marks (which can include double quotation marks or single quotation marks), and parentheses; the second type of symbols includes, but is not limited to, commas, periods, semicolons, colons, pause marks, ellipses, dashes, exclamation marks, and question marks.

[0085] For example, after segmenting the text "You are not at home today, did you go to the Analects discussion meeting?", we get two segmented texts: "You are not at home today" and "Did you go to the Analects research meeting?"

[0086] If the text does not contain any first-class symbols, the action of deleting first-class symbols is not performed (skipped), and the text is segmented according to the second-class symbols.

[0087] The method of segmenting text based on numerical features is to divide the text into segments where each segment matches a date (including year, month, and day) or an arithmetic expression. For example, in the text "2001 year", "2001" is divided into one segment; the text "1+1" is an arithmetic expression, so "1+1" is divided into another segment.

[0088] Alternatively, text that matches proper nouns can be segmented into a single text segment. For example, "T-62" in the text "T-62 tank" is a proper noun, so "T-62" can be segmented into a single text segment.

[0089] Alternatively, text that matches decimals, fractions, percentages, etc., can be divided into a single segment. For example, in the text "1.25 times", "1.25" is a decimal, so "1.25" in that text can be divided into a single segment.

[0090] It should be noted that if the text consists of continuous Chinese characters without punctuation, the continuous Chinese characters are divided into one segment. For example, the text "Learning to speak is not a skill" remains the same text after being segmented. If the text consists of continuous English words without punctuation, each English word is a segment of text. For example, the text "you are beautiful" becomes three segments: "you", "are", and "beautiful".

[0091] In summary, for example, the text "8:00~12:15 free taxi ride" can be segmented into four parts: "8:00", "~", "12:15", and "free taxi ride".

[0092] Step 2022: Perform a structured description on at least one segmented text to obtain normalized text.

[0093] Optionally, in the embodiments of this application, the structured description of the segmented text can be described using Backus's Normal Form (BNF). BNF refers to starting with a symbol (called the start sign) and then giving rules for replacing the preceding symbols. The language defined by BNF syntax is a set of strings.

[0094] In the embodiments of the present application, for each of the at least one segmented text, the segmented text is described using BNF according to all possible pronunciations of the segmented text. For example, the segmented text "1+1" has two pronunciations "one plus one" or "one plus one", and the structured description of the segmented text is "one plus [one]", wherein [one] indicates that the pronunciation is optional, i.e., it can be read or not. Or the structured description of the segmented text "1+1" can also be expressed as "one plus one | one plus one", and the expression form of the structured description of the segmented text after the BNF description is not limited in the embodiments of the present application.

[0095] Alternatively, other methods for structurally describing the text can be used to obtain the structured text, for example, regular description, and the embodiments of the present application are not limited.

[0096] Still taking the text "8:00~12:15 call free" as an example, the structured description of the four segmented texts "8:00", "~", "12:15", and "call free" obtained by segmentation is shown in Table 2.

[0097] Table 2

[0098]

[0099] It should be understood that (**|**) indicates that one result is selected from the two sides of "|", for example, in Table 2, the result selected from the two sides of "| " in (point | hour) is used as the pronunciation of ":" in "8:00". [**|**] indicates optional information, i.e., the content in "[]" can be selected or not selected, and if the content in "[]" is selected, one result is selected from the two sides of "|", for example, in Table 2, [whole | clock] is optional information.

[0100] Step 203, generating a structured description of the feasible phonetic sequence of the structured text.

[0101] In the embodiments of the present application, the phonetic sequence of the structured text is the phonetic sequence corresponding to one or more candidate texts contained in the structured text, and the phonetic sequence corresponding to the one or more candidate texts is referred to as the feasible phonetic sequence of the structured text.

[0102] Alternatively, when the structured text is composed of continuous characters without punctuation marks, for example, the structured text is composed of continuous Chinese characters, the step 203 can be implemented by steps 2031 and 2032.

[0103] Step 2031, performing word segmentation on the structured text to obtain at least one word segmentation result of the structured text.

[0104] Step 2032: Based on a preset phonetic dictionary, phonetically annotate at least one word segmentation result to obtain a feasible phonetic sequence of the normalized text, and generate a structured description of the feasible phonetic sequence of the normalized text.

[0105] In the embodiments of the present application, by performing word segmentation on the normalized text, the phonetic annotation of at least one word segmentation result can be queried from the phonetic dictionary, so as to obtain a feasible phonetic sequence of the normalized text, and then a structured description of the feasible phonetic sequence of the normalized text is performed. For example, for the above text "8:00~12:15 taking a taxi is free", according to Step 2031 and Step 2032, the phonetic sequence shown in Table 3 can be obtained. Among them, after word segmentation of "taking a taxi is free", two word segmentation results are obtained, namely "taking a taxi" and "free".

[0106] Table 3

[0107]

[0108] It should be noted that in the process of generating a feasible phonetic sequence of the normalized text, the pronunciation of a word segmentation result can include one or more pronunciations, because there may be polyphonic words, co-articulation or habitual mispronunciation in the text. For example, in Table 3 above, "taking a taxi" in "taking a taxi is free" is a polyphonic word, and based on the phonetic dictionary, the phonetic annotation results of "taking a taxi" are "da 3di1" and "da3de0".

[0109] Another example is that when "you" and "hao" form "ni hao", "you" and "hao" will form co-articulation. The original pronunciation of "you" is "ni3". By performing word segmentation and determining that "ni hao" is a word, phonetically annotating "ni hao" gives the phonetic annotation result as (ni2|ni3)hao3. Another example is that the phonetic annotation result of "foot washing water" is (xi3|xi2)(jiao3|jiao2)shui3.

[0110] Another example is that there is a habitual mispronunciation in the daily pronunciation of "dang" in "file". The correct pronunciation of "dang" is "dang4", and it is usually habitually mispronounced as "dang3". Therefore, for the case of habitual mispronunciation, it should also be considered. Based on this, the phonetic annotation result of "file" is (dang3|dang4)an4.

[0111] Combined with the above description, for various texts in Table 1, their feasible phonetic sequences are shown in Table 4.

[0112] Table 4

[0113]

[0114] In the embodiments of the present application, after obtaining the normalized text feasible annotation sequence, the normalized text feasible annotation sequence is described in a structured manner. Optionally, a structured description method satisfying a connectionist temporal classification (CTC) rule can be used, for example, a finite state transducer (FST) is used to describe the normalized text feasible annotation sequence in a structured manner (i.e., FST description), and FST data is obtained by describing in a structured manner through the FST, Figure 4 The above is a schematic diagram of the structure of the FST data of the annotation sequence in Table 3. In the embodiments of the present application, the normalized text feasible annotation sequence is described as FST data, which facilitates subsequent alignment processing of the text.

[0115] In the embodiments of the present application, the CTC rule refers to: the same annotation unit predicted on adjacent time frames should be considered as the same annotation unit; and a null tag is a special annotation unit, indicating that the time frame is silent, noise or repetition. For details of the CTC rule, refer to related prior art. The FST is a prior art tool, and for details, refer to related prior art. The embodiments of the present application will not be described in detail.

[0116] In the embodiments of the present application, the text of the speech corpus is normalized, and a structured description of the normalized text feasible annotation sequence is generated, instead of traversing all possible annotation sequences of the text. In this way, after normalizing the text, annotation is performed, and annotation of the text can be obtained with low time complexity and space complexity.

[0117] In step 204, speech recognition is performed on the speech of the speech corpus to obtain an annotation probability matrix of the speech corpus.

[0118] In the embodiments of the present application, the speech of the speech corpus is recognized based on a connectionist temporal classification (CTC) acoustic model to obtain an annotation probability matrix of the speech corpus. The annotation probability matrix is a probability matrix of annotation units of the speech corpus, which means the prediction of the acoustic model on the pronunciation unit of each time frame of the speech. It should be understood that all possible annotation sequences predicted by the acoustic model are implied in the annotation probability matrix of the speech corpus. One or more (top N) feasible annotation sequences can be obtained by decoding the annotation probability matrix according to the CTC rule. The probabilities of the annotation units in the top N annotation sequences are greater than a preset probability threshold.

[0119] Specifically, the above annotation probability matrix can indicate the probability of the annotation unit corresponding to each frame signal of the above speech in the annotation unit list, for example, an element P t,v represents the probability of taking the vth annotation unit in the annotation unit list in the tth time frame.

[0120] Optionally, the phonetic transcription probability matrix can also be a phonetic transcription probability matrix of a word in the speech corpus, and the specific type of the phonetic transcription probability matrix can be determined according to actual requirements, which is not limited in the embodiments of the present application.

[0121] In the embodiments of the present application, all signals of the speech of the speech corpus to be processed can include non-blank frames (Non-Blank frames) and blank frames (Blank frames), wherein the information corresponding to the non-blank frames is the speech information of the speech.

[0122] The following Table 5 shows an example of the probability of the phonetic transcription unit corresponding to the non-blank frame in the speech recognition result of the speech corpus to be processed.

[0123] Table 5

[0124]

[0125] It should be noted that the several phonetic transcription units and the probabilities of the phonetic transcription units corresponding to the non-blank frames with relatively large probabilities (greater than 0.001) are shown in Table 5, and the phonetic transcription units and the probabilities of the phonetic transcription units with relatively small probabilities are not shown.

[0126] In combination with Table 5, the phonetic transcription sequence of the speech corpus obtained by speech recognition is described as: ba1(dian3|dian4)(dao4|dao3)shi2 er4(dian3|deng3)(yi1|yu2)(ke4|ke1)(da3|da2|da4)(di1|di3|de0)mian3 fei4.

[0127] In step 205, the structured description of the feasible phonetic transcription sequence of the normalized text and the phonetic transcription probability matrix of the speech corpus are aligned to determine the usefulness of the speech corpus.

[0128] In the embodiments of the present application, the FST decoding alignment algorithm is used to align the structured description of the feasible phonetic transcription sequence of the normalized text and the phonetic transcription probability matrix of the speech corpus. Specifically, the structured description of the feasible phonetic transcription sequence of the normalized text can indicate the feasible phonetic transcription sequence of the normalized text, the phonetic transcription probability matrix obtained by recognizing the speech can indicate the feasible phonetic transcription sequence of the speech corpus, and the FST decoding method is used for alignment to realize text alignment and phonetic transcription sequence alignment.

[0129] In essence, the alignment process described above determines whether one or more candidate texts in the normalized text can be aligned with the text corresponding to the speech recognition result, and whether the feasible phonetic sequence of the normalized text can be aligned with the phonetic sequence indicated by the phonetic probability matrix. In this embodiment, alignment means whether the normalized text contains one or more texts corresponding to the speech recognition result, and whether the feasible phonetic sequence of the normalized text contains one or more phonetic sequences corresponding to the speech recognition result. If the normalized text contains one or more texts corresponding to the speech recognition result, and the feasible phonetic sequence of the normalized text contains one or more phonetic sequences corresponding to the speech recognition result, then the speech corpus is determined to be usable; otherwise, the speech corpus is unusable.

[0130] For example, in combination with the above Figure 4 Based on the FST data shown and the probabilities of the phonetic units shown in Table 5 above, it can be determined that the normalized text contains the phrase "8 o'clock to 12 o'clock 15 minutes ago" from the speech recognition results, thus confirming that the text can be aligned.

[0131] Furthermore, based on the probability of the phonetic unit indicated by the phonetic probability matrix, each phonetic unit is matched with... Figure 4 The phonetic units in the phonetic sequence of the FST format shown are aligned, that is, according to the node order in the FST format. Figure 4 The sequence numbered 1 to 24 in the FST data represents 24 nodes, where 1 is the start node and 24 is the end node. The matching is performed sequentially. For all Chinese characters in the speech corpus, if the feasible phonetic unit corresponding to each Chinese character in the FST data contains one or more phonetic units corresponding to the speech recognition result, and the phonetic probability of the phonetic unit corresponding to the speech recognition result is greater than the probability threshold of the phonetic unit, then the feasible phonetic sequence of the normalized text can be aligned with the phonetic sequence corresponding to the speech recognition result; otherwise, the two cannot be aligned.

[0132] Optionally, the probability thresholds of all phonetic units in the corpus to be identified can be the same or different. In this embodiment, it is assumed that the probability thresholds of all phonetic units are the same, for example, the probability threshold is 0.001.

[0133] For example, referring to Table 5 (it should be understood that the phonetic units in the examples in Table 5 are phonetic units with probability values ​​greater than the probability threshold of the phonetic unit), the specific process of aligning the FST data of the feasible phonetic sequence of the normalized text with the phonetic probability matrix of the speech corpus is as follows:

[0134] 1. For the phonetic unit "ba1" with a probability value of 1.000000, combined with... Figure 4 It can match the phonetic unit "ba1" in the phonetic sequence of the normalized text.

[0135] 2、 For the phonetic unit "dian3" with a probability value of 0.997685, combined with Figure 4 , the next state "dian3" of "ba1" can be matched, that is, the phonetic unit "ba1" in the phonetic sequence of the normalized text can be matched; and the phonetic unit "dian4" with a probability value of 0.002315 cannot be aligned with the next state of "ba1" in Figure 4 .

[0136] 3、 For the phonetic unit "dao4" with a probability value of 0.997783, in Figure 4 , the next state "dao4" of "dian3" can be matched, that is, the phonetic unit "dao4" in the phonetic sequence of the normalized text can be matched; and the phonetic unit "dao3" with a probability value of 0.002217 cannot be aligned with the next state of "dian3" in Figure 4 .

[0137] 4、 For the phonetic unit "shi2" with a probability value of 1.000000, combined with Figure 4 , the next state "shi2" of "dao4" can be matched, that is, the phonetic unit "shi2" in the phonetic sequence of the normalized text can be matched.

[0138] 5、 For the phonetic unit "er4" with a probability value of 1.000000, combined with Figure 4 , the next state "er4" of "shi2" can be matched, that is, the phonetic unit "er4" in the phonetic sequence of the normalized text can be matched.

[0139] 6、 For the phonetic unit "dian3" with a probability value of 0.998562, combined with Figure 4 , the next state "dian3" of "er4" can be matched, that is, the phonetic unit "dian3" in the phonetic sequence of the normalized text can be matched; and the phonetic unit "deng3" with a probability value of 0.001438 cannot be aligned with the next state of "er4" in Figure 4 .

[0140] 7、 For the phonetic unit "yi1" with a probability value of 0.991636, combined with Figure 4 , the next state "yi1" of "dian3" can be matched, that is, the phonetic unit "yi1" in the phonetic sequence of the normalized text can be matched; and the phonetic unit "deng3" with a probability value of 0.004628 cannot be aligned with the next state of "er4" in Figure 4 .

[0141] 8、 For the phonetic unit "ke4" with a probability value of 0.997313, combined withFigure 4 , the phonetic unit "ke4" with a probability value of 0.002687 cannot be aligned with the next state of "yi1" in Figure 4 .

[0142] 9. For the phonetic unit "da3" with a probability value of 0.997261, combined with Figure 4 , the phonetic unit "da3" with a probability value of 0.997261 can be matched to the next state "da3" of "ke4", i.e. to the phonetic unit "da3" in the phonetic sequence of the normalized text; the phonetic unit "da2" with a probability value of 0.001566 cannot be aligned with the next state of "ke4" in Figure 4 ; and the phonetic unit "da4" with a probability value of 0.001173 also cannot be aligned with the next state of "ke4" in Figure 4 .

[0143] 10. For the phonetic unit "di1" with a probability value of 0.647850, combined with Figure 5 , the phonetic unit "di1" with a probability value of 0.647850 can be matched to the next state "di1" of "da3", i.e. to the phonetic unit "di1" in the phonetic sequence of the normalized text; for the phonetic unit "de0" with a probability value of 0.044220, the phonetic unit "de0" can be matched to the next state "de0" of "da3" in Figure 6 ; and the phonetic unit "di3" with a probability value of 0.188438 cannot be aligned with the next state of "da3" in Figure 7 .

[0144] 11. For the phonetic unit "mian3" with a maximum probability value of 1.000000, combined with Figure 8 , the phonetic unit "mian3" with a maximum probability value of 1.000000 can be matched to the next state "mian3" of "di1", i.e. to the phonetic unit "mian3" in the phonetic sequence of the normalized text.

[0145] 12. For the phonetic unit "fei4" with a probability value of 1.000000, combined with Figure 8 , the phonetic unit "fei4" with a probability value of 1.000000 can be matched to the next state "fei4" of "mian3", i.e. to the phonetic unit "fei4" in the phonetic sequence of the normalized text.

[0146] It can be known in combination with the above alignment process that all the phonetic units in the phonetic sequence corresponding to the above phonetic result can be matched to the phonetic sequence of the normalized text, and the matched phonetic sequence is: ba1 dian3 dao4 shi2 er4 dian3yi1 ke4 da3(di1|de0)mian3 fei4, and the matched text is "eight points to twelve points one o'clock free".

[0147] In summary, the structured description of the feasible phonetic sequence of the normalized text of the voice corpus of the above example can be aligned with the phonetic probability matrix of the voice corpus, and therefore it is determined that the voice corpus is available.

[0148] In an implementation manner, if multiple phonetic units of a character in the voice recognition structure can be aligned with multiple phonetic units in the phonetic sequence of the normalized text, the phonetic unit with the largest probability value is usually taken as the correct phonetic unit of the character. For example, both the two phonetic results "di1" and "de0" of "de" in the above "dai de" can be aligned with the phonetic units in the phonetic sequence of the normalized text, and then "di1" with the larger probability value is considered as the correct phonetic unit.

[0149] For example, in a case, it is assumed that the probability distribution of the 12th frame is "ba3 0.56000, ba1 0.44000", that is, the probability value of the actual phonetic unit "ba3" is 0.56000, and the probability value of the actual phonetic unit "ba1" is 0.44000", in combination with the above alignment process Figure 8 It can be known that the "ba1" with the smaller probability value (0.44000) can be matched with the phonetic unit of the normalized text, but the "ba3" with the larger probability value (0.56000) cannot be matched with the phonetic unit of the normalized text. It can be seen that the maximum probability value predicted by the acoustic model is wrong, but the correct phonetic "ba1" can still be obtained by integrating the probability distribution in the subsequent "FST decoding alignment algorithm".

[0150] For example, in a case, it is assumed that when the execution reaches the 5th step, for the phonetic unit "er4" with the probability value of 1.000000, if the next state of "shi2" in the phonetic sequence of the FST data is "er3", it can be seen that "er4" is not matched, and the alignment process ends, and it is determined that the voice corpus is not available.

[0151] It can be understood that determining whether the voice corpus is available can also be understood as proofreading the voice corpus, so as to screen out available voice corpora and unavailable voice corpora.

[0152] In this embodiment of the application, the alignment process performed according to the above method can be completed with low time complexity, which can efficiently filter out unusable speech data. Furthermore, since the text of the above speech data is normalized text, the alignment process based on the normalized text can reduce the false screening of speech data.

[0153] Optionally, refer to Figure 1 The speech corpus processing method provided in the embodiments of this application can process a large amount of speech corpus, filter out unusable speech corpus, and delete unusable speech corpus (the process of deleting unusable speech corpus from a large amount of speech corpus can be understood as the process of correcting speech corpus). Then, multiple usable speech corpus are used as training samples, and artificial intelligence (AI) algorithms are used to learn a better-performing automatic speech recognition (ASR) model (the ASR model is used to convert speech into text), which can improve the accuracy of speech recognition.

[0154] In summary, you can refer to the following: Figure 1 The flowchart shown illustrates the speech corpus processing method provided in the embodiments of this application.

[0155] The speech corpus processing method provided in this application can normalize the text of the acquired speech corpus to obtain normalized text; generate a structured description of feasible phonetic sequences of the normalized text; then perform speech recognition on the speech of the speech corpus to obtain a phonetic probability matrix of the speech corpus; and finally align the structured description of feasible phonetic sequences of the normalized text with the phonetic probability matrix of the speech corpus to determine the usefulness of the speech corpus. Since the structured description of feasible phonetic sequences generated after normalizing the speech corpus is a normalized and comprehensive phonetic sequence, rather than traversing all phonetic symbols in the text of the speech corpus, and aligning the structured description of feasible phonetic sequences of the normalized text with the phonetic probability matrix can reduce the time and space complexity of speech corpus processing, the speech corpus processing method provided in this application can efficiently and accurately determine the usability of the speech corpus.

[0156] The embodiment of the present application provides a speech corpus processing device, which is used for executing steps in the speech corpus processing method, the embodiment of the present application can divide functional modules of the speech corpus processing device according to the method examples, for example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware or in the form of a software functional module. The division of the module in the embodiment of the present application is illustrative, and is only a logical function division, and another division mode can be used in actual implementation.

[0157] In the case of dividing each functional module according to each function, Figure 1 A possible structure schematic diagram of the speech corpus processing device involved in the above embodiment is shown. As shown in the figure, ​ The speech corpus processing device includes an acquisition module 701, a normalization processing module 702, a phonetic transcription module 703, a speech recognition module 704 and an alignment module 705.

[0158] The acquisition module 701 is used for supporting the speech corpus processing device to acquire a speech corpus to be processed, for example, step 201 in the method embodiment.

[0159] The normalization processing module 702 is used for supporting the speech corpus processing device to perform normalization processing on text of the speech corpus, to obtain normalized text of the speech corpus, for example, step 202 (including steps 2021 to 2022) in the method embodiment.

[0160] The phonetic transcription module 703 is used for supporting the speech corpus processing device to generate a structured description of a feasible phonetic transcription sequence of the normalized text, for example, step 203 (including steps 2031 to 2032) in the method embodiment.

[0161] The speech recognition module 704 is used for supporting the speech corpus processing device to perform speech recognition on speech of the speech corpus, to obtain a phonetic transcription probability matrix of the speech corpus, for example, step 204 in the method embodiment.

[0162] The alignment module is used for supporting the speech corpus processing device to perform alignment processing on the structured description of the feasible phonetic transcription sequence of the normalized text and the phonetic transcription probability matrix of the speech corpus, to determine usefulness of the speech corpus, for example, step 205 in the method embodiment.

[0163] All related contents of each step in the method embodiment can be referred to the function description of the corresponding functional module, and will not be repeated here.

[0164] Of course, the voice corpus processing apparatus provided by the embodiments of the present application includes but is not limited to the above-mentioned modules, for example: the voice corpus processing apparatus can further include a model training module. The model training module can be used to use multiple available voice corpora as training samples, learn an automatic speech recognition (ASR) model using an artificial intelligence algorithm, and use the ASR model to convert speech into text.

[0165] In the case of using an integrated unit, a structural diagram of the voice corpus processing apparatus provided by the embodiments of the present application is as shown in ​ In ​ , the voice corpus processing apparatus includes a processing module 801 and a communication module 802. The processing module 801 is used to control and manage the actions of the voice corpus processing apparatus, for example, to execute the steps performed by the above-mentioned normalization processing module 702, the phonetic transcription module 703, the speech recognition module 704, and the alignment module 705, and / or to execute other processes of the technology described herein. The communication module 802 is used to support the interaction between the voice corpus processing apparatus and other devices, for example, to receive speech, etc. As ​ shown, the voice corpus processing apparatus can further include a storage module 803, which is used to store the program code and data of the voice corpus processing apparatus.

[0166] The processing module 801 can be a processor or a controller, for example ​ the processor 101 in . The communication module 802 can be a transceiver, an RF circuit, or a communication interface, etc., for example ​ the network interface 103 in . The storage module 803 can be a memory, for example ​ the memory 102 as shown.

[0167] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, all or part generates the processes or functions in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, magnetic disk, magnetic tape), optical media (such as digital video disc (DVD)), or semiconductor media (such as solid state drive (SSD)) and the like.

[0168] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0169] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0171] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0172] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various program code storage media.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of processing a speech corpus, characterized by, The method comprises: acquiring a to-be-processed speech corpus, the to-be-processed speech corpus comprising text of the speech corpus and speech of the speech corpus; performing normalization processing on the text of the speech corpus to obtain normalized text of the speech corpus, the normalized text comprising one or more candidate texts; generating a structured description of a feasible phonetic spelling sequence of the normalized text; performing speech recognition on the speech of the speech corpus to obtain a phonetic spelling probability matrix of the speech corpus; performing alignment processing on the structured description of the feasible phonetic spelling sequence of the normalized text and the phonetic spelling probability matrix of the speech corpus to determine usefulness of the speech corpus.

2. The method of claim 1, wherein, The method further comprises: performing segmentation processing on the text of the speech corpus to obtain at least one segmented text; performing structured description on the at least one segmented text to obtain the normalized text.

3. The method of claim 2, wherein, The method further comprises: segmenting the text according to symbol features in the text and / or numerical features in the text.

4. The method of claim 2 or 3, wherein: the structured description of the segmented text comprises a Backus-Naur Form (BNF) description.

5. The method of any one of claims 1 to 3, wherein: the structured description of the feasible phonetic spelling sequence of the normalized text comprises a Finite State Transducer (FST) description.

6. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: performing speech recognition on the speech of the speech corpus based on a Connectionist Temporal Classification (CTC) acoustic model to obtain the phonetic spelling probability matrix of the speech corpus.

7. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: using a plurality of available speech corpora as training samples to learn an Automatic Speech Recognition (ASR) model using an artificial intelligence algorithm, the ASR model being used to convert speech into text.

8. A speech corpus processing apparatus characterized by comprising: The apparatus comprises an acquisition module, a normalization processing module, a phonetic spelling module, a speech recognition module, and an alignment module; the acquisition module is configured to acquire a to-be-processed speech corpus, the to-be-processed speech corpus comprising text of the speech corpus and speech of the speech corpus; the normalization processing module is configured to perform normalization processing on the text of the speech corpus to obtain normalized text of the speech corpus, the normalized text comprising one or more candidate texts; the phonetic spelling module is configured to generate a structured description of a feasible phonetic spelling sequence of the normalized text; the speech recognition module is configured to perform speech recognition on the speech of the speech corpus to obtain a phonetic spelling probability matrix of the speech corpus; the alignment module is configured to perform alignment processing on the structured description of the feasible phonetic spelling sequence of the normalized text and the phonetic spelling probability matrix of the speech corpus to determine usefulness of the speech corpus.

9. The apparatus of claim 8, wherein: The normalization processing module is specifically configured to: segment the text of the speech corpus to obtain at least one segmented text; and perform structured description on the at least one segmented text to obtain the normalized text.

10. The apparatus of claim 9, wherein, The normalization processing module is specifically configured to segment the text according to a symbol feature in the text and / or a number feature in the text.

11. The apparatus of claim 9 or 10, wherein, The structured description of the segmented text includes a Backus-Naur Form (BNF) description.

12. The apparatus of any one of claims 8 to 10, wherein, The structured description of the feasible phonetic spelling sequence of the normalized text includes a Finite State Transducer (FST) description.

13. The apparatus of any one of claims 8 to 10, wherein, The speech recognition module is specifically configured to perform speech recognition on the speech of the speech corpus based on a Connectionist Temporal Classification (CTC) acoustic model to obtain a phonetic spelling probability matrix of the speech corpus.

14. The apparatus of any one of claims 8 to 10, wherein, The processing apparatus further includes a model training module; The model training module is configured to use an artificial intelligence algorithm to learn an Automatic Speech Recognition (ASR) model by taking a plurality of available speech corpora as training samples, and the ASR model is used to convert speech into text.

15. An electronic device, comprising: The electronic device includes a processor and a memory coupled to the processor; the memory is configured to store computer instructions; when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device performs the method of any one of claims 1 to 7.

Images