Voice recognition model additional learning apparatus, voice recognition model additional learning method, and program

Abstract

To provide a technique capable of reducing additional learning costs for making a voice recognition model recognize unknown words.SOLUTION: A voice recognition model additional learning apparatus according to one aspect of the present disclosure, comprises: a voice synthesis unit for using a text including unknown words to generate a synthetic voice that reads the text; and a model learning unit for additionally learning the learned voice recognition model using the text as training data so as to minimize an error between the result of recognizing the synthetic voice using the learned voice recognition model and the training data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a speech recognition model additional training device, a speech recognition model additional training method, and a program. [Background technology]

[0002] End-to-end speech recognition models realized by neural networks have been known for some time, and various techniques have been proposed to improve the accuracy of the models. For example, Patent Document 1 describes a technique that can train a highly accurate end-to-end speech recognition model with a small amount of training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-39218 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, in order to correctly recognize unknown words such as technical terms, it was necessary to prepare speech containing the unknown words and its transcribed text, and then perform additional training (tuning) on ​​the speech recognition model, which resulted in high costs for additional training.

[0005] The present disclosure has been made in consideration of the above points, and provides a technology that can reduce the additional learning cost required to make a speech recognition model recognize unknown words. [Means for solving the problem]

[0006] A speech recognition model additional training device according to one aspect of the present disclosure includes a speech synthesis unit that uses text including unknown words to generate synthetic speech reading the text, and a model training unit that additionally trains the trained speech recognition model using the text as training data to minimize an error between the training data and the result of recognizing the synthetic speech using a trained speech recognition model. Effect of the Invention

[0007] A technique is provided that can reduce the additional training cost required to make a speech recognition model recognize unknown words. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a functional configuration of a speech recognition model additional learning device according to a first embodiment. [Diagram 2] 5 is a flowchart illustrating an example of a voice recognition model additional learning process according to the first embodiment. [Diagram 3] FIG. 11 is a diagram illustrating an example of a functional configuration of a speech recognition model additional learning device according to a second embodiment. [Figure 4] 13 is a flowchart illustrating an example of a voice recognition model additional learning process according to a second embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of a functional configuration of a speech recognition model additional learning device according to a third embodiment. [Figure 6] 13 is a flowchart illustrating an example of a voice recognition model additional learning process according to a third embodiment. [Figure 7] FIG. 13 is a diagram illustrating an example of a functional configuration of a speech recognition model additional learning device according to a fourth embodiment. [Figure 8] 13 is a flowchart illustrating an example of a voice recognition model additional learning process according to a fourth embodiment. [Figure 9] FIG. 13 is a diagram illustrating an example of a functional configuration of a speech recognition model additional learning device according to a fifth embodiment. [Figure 10] 13 is a flowchart illustrating an example of a voice recognition model additional learning process according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the first to fifth embodiments will be described as an embodiment of the present invention. In each of the following embodiments, a speech recognition model additional learning device 10 that can reduce the additional learning cost compared to the conventional method when additional learning (tuning) is performed to allow an end-to-end speech recognition model realized by a neural network to correctly recognize unknown words will be described. Here, the unknown words refer to words (words, phrases, phrases, etc.) that the speech recognition model cannot correctly recognize, or words (words, phrases, phrases, etc.) that were not included in the learning data used when training the end-to-end speech recognition model. In the following, unknown words mainly refer to words that are used only in specific fields, areas, industries, contexts, etc. and are not common. Typical examples of unknown words include technical terms, industry terms, proper nouns (product names, service names, personal names, organization names, etc.), slang, abbreviations, etc. that are used only in specific fields, areas, industries, contexts, etc. Note that additional learning may be called re-learning.

[0010] [First embodiment] The first embodiment will be described below.

[0011] <Functional configuration example of the voice recognition model additional training device 10 according to the first embodiment> FIG. 1 shows an example of a functional configuration of a voice recognition model additional training device 10 according to the first embodiment. As shown in FIG. 1, the voice recognition model additional training device 10 according to the first embodiment includes a text extraction unit 101, a voice synthesis unit 102, and a model training unit 103. These units are realized, for example, by processing in which one or more programs installed in the voice recognition model additional training device 10 are executed by various arithmetic units such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). In addition, the voice recognition model additional training device 10 according to the first embodiment includes a voice recognition model storage unit 201. The voice recognition model storage unit 201 is realized, for example, by a storage area of ​​various storage devices such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory.

[0012] The text extraction unit 101 extracts text (sentences) from a document provided to the speech recognition model additional learning device 10. Here, it is assumed that the document contains unknown words, but it does not necessarily have to contain unknown words. Specific examples of documents include documents related to voice dialogue in a conference using a Web conference system or the like (for example, presentation materials, explanatory materials, reports, etc.). However, the document may be any document as long as it contains at least text, and may contain, for example, charts and the like in addition to text. Furthermore, the document may be a sentence with some structure (a structured document) or a document without any structure.

[0013] The method by which the text extraction unit 101 extracts text from a document is not limited to a specific method, but for example, if the document has a structure, it is possible to extract text for each structural unit. Specifically, for example, if the document is composed of multiple objects (figures, tables, text boxes, paragraphs, etc.), the text extraction unit 101 may extract text from each object. On the other hand, for example, if the document does not have a structure, it is possible to extract text for each sentence separated by a comma, a period, etc.

[0014] Hereinafter, a set of documents provided to the speech recognition model additional training device 10 (hereinafter also referred to as a document set) is defined as D={D i Let |i=1,...,|D|}. Also, document D i The text extracted from S j (i) (j=1,...,N (i) ), and the set of texts is S = {S j (i) |i=1,...,|D|,j=1,...,N (i)} where |D| is the number of documents in the document set D, and N (i) Document D i Represents the amount of text extracted from

[0015] The speech synthesis unit 102 generates synthetic speech from the text extracted by the text extraction unit 101 using a speech synthesis model. j (i) ∈S is used to synthesize speech T j (i) where T j (i) is text S j (i) The set of synthetic speeches is represented as T = {T j (i) |i=1,...,|D|,j=1,...,N (i)}.

[0016] The speech synthesis model is a machine learning model that takes text (more precisely, a language vector obtained by vectorizing the information of the text) as input, and generates and outputs synthetic speech (more precisely, information representing the characteristics of the synthetic speech (e.g., a mel spectrogram, etc.)) that reads the text. The speech synthesis unit 102 may be realized, for example, by any existing speech synthesis software (or a speech synthesis module, etc.), and may generate synthetic speech from text using a trained speech synthesis model included in the speech synthesis software. However, there is also existing speech synthesis software that realizes speech synthesis using a method other than the trained speech synthesis model, and the speech synthesis unit 102 may be realized by the speech synthesis method of the speech synthesis software.

[0017] The model learning unit 103 additionally learns (tunes) the voice recognition model stored in the voice recognition model storage unit 201 by a known supervised learning method using the text extracted by the text extraction unit 101 and the synthetic voice generated by the voice synthesis unit 102. j (i) ∈S is used as training data, and the synthetic speech T j (i) The trainable parameters of the speech recognition model are trained (updated) using a known supervised learning method so as to minimize the error between the text representing the results of recognition by the speech recognition model and the training data.

[0018] The voice recognition model storage unit 201 stores an end-to-end voice recognition model realized by a neural network.

[0019] <Speech Recognition Model Additional Learning Process According to the First Embodiment> The speech recognition model additional training process according to the first embodiment will be described with reference to Fig. 2. In the following, it is assumed that a document set D is provided to the speech recognition model additional training device 10.

[0020] The text extraction unit 101 extracts each document D i(i=1, ,|D|) to text S j (i) (j=1,...,N (i) ) are extracted (step S101). j (i) |i=1,...,|D|,j=1,...,N (i)} is obtained.

[0021] Next, the speech synthesis unit 102 synthesizes the text S extracted in step S101. j (i) (i=1,...,|D|,j=1,...,N (i) ) to generate synthesized speech T j (i) (Step S102). This results in a synthetic speech set T={T j (i) |i=1,...,|D|,j=1,...,N (i)} is obtained.

[0022] Then, the model learning unit 103 extracts the text S extracted in step S101. j (i) (i=1,...,|D|,j=1,...,N (i) ) and the synthetic speech T generated in step S102 above. j (i) (i=1,...,|D|,j=1,...,N (i) ) and performs additional learning (tuning) of the voice recognition model stored in the voice recognition model storage unit 201 by a known supervised learning method (step S103). j (i) Using the training data, the synthetic speech T j (i) The trainable parameters of the speech recognition model are trained (updated) using a known supervised learning method so as to minimize the error between the text representing the results of recognition by the speech recognition model and the training data. This results in a speech recognition model capable of recognizing unknown words present in the document set D.

[0023] As described above, the speech recognition model additional learning device 10 according to the first embodiment extracts text from a document containing unknown words, and generates a voice reading the text using a speech synthesis model. This makes it possible to obtain text as training data and voice as input to the speech recognition model at low cost, and therefore makes it possible to realize additional learning (tuning) at low cost to make the speech recognition model correctly recognize unknown words.

[0024] Note that the document set D is provided to the speech recognition model additional training device 10 at, for example, any timing. Therefore, for example, a person (user) who wishes to additionally train a speech recognition model can provide the document set D or documents constituting the document set D to the speech recognition model additional training device 10 at any timing. As a result, for example, when a speech recognition model is additionally trained once a day, the document set D is made up of the documents provided in a day. Similarly, for example, when a speech recognition model is additionally trained every time a meeting is held, the document set D is made up of documents provided for each meeting.

[0025] [Second embodiment] The second embodiment will be described below. Since the unknown words are technical terms that are used only in a specific field, area, industry, context, etc., the text S j (i) If there is an unknown word in the speech, the speech synthesis unit 102 generates a correct synthetic speech T j (i) However, there is a possibility that unknown words and their pronunciations cannot be generated. On the other hand, some speech synthesis software can generate synthetic speech that reads out words exactly as they are pronounced by registering the words and their pronunciations in a dictionary. In the second embodiment, therefore, a case will be described in which the speech synthesis unit 102 is realized by speech synthesis software that can use a dictionary, and unknown words and their pronunciations are registered in the dictionary.

[0026] In the second embodiment, differences from the first embodiment will be mainly described, and descriptions of the parts that may be similar to the first embodiment will be omitted or simplified.

[0027] <Example of functional configuration of the voice recognition model additional training device 10 according to the second embodiment> FIG. 3 shows an example of a functional configuration of the speech recognition model additional learning device 10 according to the second embodiment. As shown in FIG. 3, the speech recognition model additional learning device 10 according to the second embodiment includes a text extraction unit 101, a speech synthesis unit 102, a model learning unit 103, an unknown word extraction unit 104, and a dictionary registration unit 105. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition model additional learning device 10 are executed by various computing devices such as a CPU and a GPU. In addition, the speech recognition model additional learning device 10 according to the second embodiment includes a speech recognition model storage unit 201 and a dictionary storage unit 202. Each of these storage units is realized, for example, by a storage area of ​​various storage devices such as an HDD, an SSD, and a flash memory.

[0028] The unknown word extraction unit 104 extracts unknown words from the text extracted by the text extraction unit 101. The method by which the unknown word extraction unit 104 extracts unknown words is not limited to a specific method, but may extract unknown words by, for example, named entity extraction. In addition to this, for example, if it is known that the unknown word is a word expressed in alphabetic characters, a character string of successive alphabetic characters may be extracted as the unknown word. Also, for example, if it is known that the unknown word is a word expressed in katakana, a character string of successive katakana characters may be extracted as the unknown word.

[0029] The dictionary registration unit 105 registers pairs of unknown words extracted by the unknown word extraction unit 104 and the readings of the unknown words in the dictionary storage unit 202. At this time, the readings of the unknown words may be provided by a user, or a reading of a notation (morpheme) corresponding to the unknown word may be obtained using a morphological analysis dictionary created in advance, or a document including the notation of the unknown word and its reading may be searched for, and the reading of the notation corresponding to the unknown word may be obtained from the document obtained by the search using rules such as pattern matching. Here, a dictionary is a collection of information represented by pairs of words (words, phrases, etc.) and their readings.

[0030] The dictionary registration unit 105 may register an unknown word extracted by the unknown word extraction unit 104 and its reading in the dictionary storage unit 202 only if the pair is not registered in the dictionary storage unit 202. The dictionary registration unit 105 may also decompose the text extracted by the text extraction unit 101 into morphemes by morphological analysis, and then register the pair of the morpheme (word) and its reading in the dictionary storage unit 202 if the pair is not registered in the dictionary storage unit 202.

[0031] The speech synthesis unit 102 uses the dictionary stored in the dictionary storage unit 202 to generate synthetic speech from the text extracted by the text extraction unit 101 using a speech synthesis model. At this time, if a word included in the text is registered in the dictionary, the speech synthesis unit 102 generates synthetic speech that reads out the word in a pronunciation corresponding to the word.

[0032] The dictionary storage unit 202 stores a dictionary represented by a set of pairs of words and their readings. i , and its reading is r i Then, {(w i ,r i )|i=1,...,M}, where M is the number of pairs of words and their readings registered in the dictionary.

[0033] <Speech Recognition Model Additional Learning Process According to the Second Embodiment> The speech recognition model additional training process according to the second embodiment will be described with reference to Fig. 4. In the following, it is assumed that a document set D is provided to the speech recognition model additional training device 10.

[0034] The text extraction unit 101 extracts each document D i (i=1, ,|D|) to text S j (i) (j=1,...,N (i) ) are extracted (step S201). j(i) |i=1,...,|D|,j=1,...,N (i)} is obtained.

[0035] Next, the unknown word extraction unit 104 extracts the text S extracted in step S101. j (i) (i=1,...,|D|,j=1,...,N (i) ), unknown words are extracted from w1, . . . , w2 (step S202). Hereinafter, it is assumed that L types of unknown words are extracted in this step, and these unknown words are denoted by w1, . . . , w L Let us assume that.

[0036] Next, the dictionary registration unit 105 extracts the unknown word w i (i=1, ,L) and its reading r i With (w i ,r i ) is registered in the dictionary storage unit 202 (step S203).

[0037] Next, the speech synthesis unit 102 uses the dictionary stored in the dictionary storage unit 202 to synthesize the text S extracted in the above step S201. j (i) (i=1,...,|D|,j=1,...,N (i) ) to generate synthesized speech T j (i) In other words, the speech synthesis unit 102 generates the text S j (i) If a word in the dictionary is included in the set, a synthetic voice is generated that reads the word in the corresponding pronunciation. j (i) |i=1,...,|D|,j=1,...,N (i)} is obtained.

[0038] Then, the model learning unit 103 performs the same processing as in step S103 of FIG. 2 on the text S extracted in step S201 above. j (i) (i=1,...,|D|,j=1,...,N(i) ) and the synthetic speech T generated in step S204 above. j (i) (i=1,...,|D|,j=1,...,N (i) ) is used to additionally train (tune) the speech recognition model stored in the speech recognition model storage unit 201 by a known supervised learning method (step S205). As a result, a speech recognition model capable of recognizing unknown words present in the document set D is obtained.

[0039] As described above, the speech recognition model additional training device 10 according to the second embodiment, in addition to the configuration described in the first embodiment, extracts unknown words from text and registers the unknown words and their readings in a dictionary used by the speech synthesis model. This makes it possible to generate synthetic speech that reads the text more accurately, and to obtain a speech recognition model with higher accuracy than that of the first embodiment.

[0040] [Third embodiment] A third embodiment will be described below. Some voice synthesis software allows various parameters (for example, a parameter indicating which speaker model to use, a parameter indicating speech rate, a parameter indicating intonation, etc.) to be changed when generating synthetic voice. Therefore, by changing these parameters, it is possible to generate various synthetic voices using the same text as training data. In other words, by changing the parameters, it is possible to augment the synthetic voice (data expansion). Therefore, in the third embodiment, a case will be described in which augmentation (data expansion) of synthetic voice using the same text as training data is performed by changing the parameters of the voice synthesis software.

[0041] In the third embodiment, differences from the second embodiment will be mainly described, and descriptions of the parts that may be similar to the second embodiment will be omitted or simplified.

[0042] <Example of functional configuration of the voice recognition model additional learning device 10 according to the third embodiment> FIG. 5 shows an example of a functional configuration of the speech recognition model additional learning device 10 according to the third embodiment. As shown in FIG. 5, the speech recognition model additional learning device 10 according to the third embodiment includes a text extraction unit 101, a model learning unit 103, an unknown word extraction unit 104, a dictionary registration unit 105, and an augmented speech synthesis unit 106. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition model additional learning device 10 are executed by various computing devices such as a CPU and a GPU. In addition, the speech recognition model additional learning device 10 according to the third embodiment includes a speech recognition model storage unit 201 and a dictionary storage unit 202. Each of these storage units is realized, for example, by a storage area of ​​various storage devices such as an HDD, an SSD, and a flash memory.

[0043] The augmented speech synthesis unit 106, like the speech synthesis unit 102 according to the third embodiment, generates synthetic speech from the text extracted by the text extraction unit 101 using a dictionary stored in the dictionary storage unit 202, with a speech synthesis model. At this time, the augmented speech synthesis unit 106 generates multiple synthetic speeches from the same text by changing parameters of speech synthesis software that realizes the augmented speech synthesis unit 106. Here, examples of parameters of the speech synthesis software include a parameter indicating which speaker model is to be used, a parameter indicating speech rate, and a parameter indicating intonation. Note that there is no particular limitation on how the parameters are changed, and examples include comprehensively changing each parameter within the range of values ​​that the parameter can take.

[0044] As an example, the number of parameter changes is K (where K≧2), and text S j (i) Let K synthetic speeches be generated from T jk (i) (k=1, ,K). Let the set of synthetic speech be T'={T jk (i) |i=1,...,|D|,j=1,...,N (i) ,k=1,···,K}.

[0045] The model learning unit 103 learns the text S j (i) ∈S is used as training data, and the synthetic speech T jk (i) The trainable parameters of the speech recognition model are trained (updated) using a known supervised learning method so as to minimize the error between the text representing the results of recognition by the speech recognition model and the training data.

[0046] <Speech Recognition Model Additional Learning Process According to the Third Embodiment> The speech recognition model additional training process according to the third embodiment will be described with reference to Fig. 6. In the following, it is assumed that a document set D is provided to the speech recognition model additional training device 10.

[0047] Steps S301 to S303 in FIG. 6 are similar to steps S201 to S203 in FIG. 4, respectively, and therefore will not be described.

[0048] The padded voice synthesis unit 106 uses the dictionary stored in the dictionary storage unit 202 while changing the parameters of the voice synthesis software to pad the text S extracted in the above step S301. j (i) (i=1,...,|D|,j=1,...,N (i) ) to generate synthesized speech T jk (i) (k=1, . . . , K) are generated (step S304).

[0049] Then, the model learning unit 103 extracts the text S extracted in the above step S301. j (i) (i=1,...,|D|,j=1,...,N (i) ) and the synthetic speech T generated in step S304 above. jk (i) (i=1,...,|D|,j=1,...,N (i), k=1, . . . , K), the speech recognition model stored in the speech recognition model storage unit 201 is additionally trained (tuned) by a known supervised learning method (step S305). jk (i) For (k=1, ,K), text S j (i) Please note that we use the training data as training data. This results in a speech recognition model that can recognize unknown words in the document set D.

[0050] As described above, the speech recognition model additional training device 10 according to the third embodiment generates multiple synthetic voices from the same text by changing parameters of the speech synthesis software in addition to the configuration described in the second embodiment. This makes it possible to generate a variety of synthetic voices from the same text, and to obtain a speech recognition model with higher accuracy than the second embodiment.

[0051] [Fourth embodiment] The fourth embodiment will be described below. By superimposing noise on the synthetic speech, it is possible to further augment (data augment) the synthetic speech using the same text as training data, and it is also possible to obtain a speech recognition model that is robust against noise. Therefore, in the fourth embodiment, a case will be described in which the synthetic speech using the same text as training data is further augmented (data augmented) by superimposing noise on the synthetic speech.

[0052] In the fourth embodiment, differences from the third embodiment will be mainly described, and descriptions of the same points as in the third embodiment will be omitted or simplified.

[0053] <Functional configuration example of the voice recognition model additional training device 10 according to the fourth embodiment> FIG. 7 shows an example of a functional configuration of the speech recognition model additional learning device 10 according to the fourth embodiment. As shown in FIG. 7, the speech recognition model additional learning device 10 according to the fourth embodiment includes a text extraction unit 101, a model learning unit 103, an unknown word extraction unit 104, a dictionary registration unit 105, an augmented speech synthesis unit 106, and a noise superimposition unit 107. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition model additional learning device 10 are executed by various computing devices such as a CPU and a GPU. In addition, the speech recognition model additional learning device 10 according to the fourth embodiment includes a speech recognition model storage unit 201 and a dictionary storage unit 202. Each of these storage units is realized, for example, by a storage area of ​​various storage devices such as an HDD, an SSD, and a flash memory.

[0054] The noise superimposing unit 107 superimposes the synthetic voice T generated by the augmented voice synthesizing unit 106. jk (i) In this case, the noise superimposing unit 107 superimposes noise on one synthetic speech T jk (i) It is also possible to generate a plurality of noise-superimposed synthetic speeches from the above. Note that noise superimposing unit 107 may generate noise using an existing noise generation method and superimpose it on the synthetic speech.

[0055] As an example, the synthetic voice T jk (i) Let R (where R ≥ 1) noise-added synthetic speeches be generated from T jkr (i) (r = 1, , R).

[0056] As in the third embodiment, the model learning unit 103 learns the text S j (i) ∈S is used as training data, and the synthetic speech T jk (i)In order to minimize the error between the text representing the result of recognition by the speech recognition model and the teaching data, the model learning unit 103 learns (updates) the trainable parameters of the speech recognition model by a known supervised learning method. j (i) ∈S is used as training data, and the noise-added synthetic speech T jkr (i) The trainable parameters of the speech recognition model are trained (updated) using a known supervised learning method so as to minimize the error between the text representing the results of recognition by the speech recognition model and the training data.

[0057] <Speech Recognition Model Additional Learning Process According to the Fourth Embodiment> The speech recognition model additional training process according to the fourth embodiment will be described with reference to Fig. 8. In the following, it is assumed that a document set D is provided to the speech recognition model additional training device 10.

[0058] Steps S401 to S404 in FIG. 8 are similar to steps S301 to S304 in FIG. 6, respectively, and therefore will not be described.

[0059] The noise superimposing unit 107 superimposes each of the synthetic speeches T jk (i) (i=1,...,|D|,j=1,...,N (i) , k=1, , K) to obtain noise-added synthetic speech T jkr (i) (r=1, . . . , R) are generated (step S405).

[0060] Then, the model learning unit 103 extracts the text S extracted in step S401. j (i) (i=1,...,|D|,j=1,...,N (i) ) and the synthetic speech T generated in step S404 above. jk (i) (i=1,...,|D|,j=1,...,N (i), k=1, . . . , K) and the noise-added synthetic speech T jkr (i) (i=1,...,|D|,j=1,...,N (i) , k=1, . . . , K, r=1, . . . , R) to additionally train (tune) the speech recognition model stored in the speech recognition model storage unit 201 by a known supervised learning method (step S406). j (i) Using the training data, the synthetic speech T jk (i) The error between the text that represents the result of recognition by the speech recognition model and the training data is minimized, and the noise-added synthetic speech T jkr (i) The trainable parameters of the speech recognition model are trained (updated) by a known supervised learning method so as to minimize the error between the text representing the result of recognition by the speech recognition model and the training data. jk (i) (k=1, ,K) and each noise-added synthetic speech T jkr (i) For text S j (i) Please note that we use as training data. This results in a speech recognition model that can recognize unknown words in the document set D and is robust to noise.

[0061] As described above, the speech recognition model additional training device 10 according to the fourth embodiment generates noise-superimposed synthetic speech by superimposing noise on synthetic speech, in addition to the configuration described in the third embodiment. This makes it possible to generate a variety of synthetic speeches and noise-superimposed synthetic speeches from the same text, and makes it possible to obtain a speech recognition model that is more accurate than the third embodiment and is robust against noise.

[0062] [Fifth embodiment] The fifth embodiment will be described below. In the first to fourth embodiments, a document that may contain unknown words is provided to the speech recognition model additional training device 10, but an unknown word may be provided instead of a document. Therefore, in the fifth embodiment, a case in which an unknown word is provided to the speech recognition model additional training device 10 will be described.

[0063] In the fifth embodiment, differences from the second embodiment will be mainly described, and descriptions of the parts that may be similar to the second embodiment will be omitted or simplified.

[0064] <Functional configuration example of the voice recognition model additional training device 10 according to the fifth embodiment> FIG. 9 shows an example of a functional configuration of a speech recognition model additional learning device 10 according to the fifth embodiment. As shown in FIG. 9, the speech recognition model additional learning device 10 according to the fifth embodiment includes a text extraction unit 101, a speech synthesis unit 102, a model learning unit 103, a dictionary registration unit 105, and a document search unit 108. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition model additional learning device 10 are executed by various computing devices such as a CPU and a GPU. In addition, the speech recognition model additional learning device 10 according to the fifth embodiment includes a speech recognition model storage unit 201 and a dictionary storage unit 202. Each of these storage units is realized, for example, by a storage area of ​​various storage devices such as an HDD, an SSD, and a flash memory.

[0065] The document search unit 108 uses an unknown word given to the speech recognition model additional learning device 10 to search for documents containing the unknown word. The document search unit 108 may search for documents from a predetermined document database, for example, or may search for documents on the Web using a technique such as Web crawling. In this way, documents containing the given unknown word are searched for. As a specific example, when an unknown word such as a service name is given, documents such as news releases and pamphlets related to the service can be obtained by searching. Note that one unknown word or multiple unknown words may be given to the speech recognition model additional learning device 10. When multiple unknown words are given, documents containing at least one of the unknown words are searched for.

[0066] Hereinafter, the set of documents retrieved by the document retrieval unit 108 will be referred to as D={D i Here, the text extraction unit 101 extracts each document D searched by the document search unit 108. i (i=1, ,|D|) to text S j (i) (j=1,...,N (i) ) are extracted respectively.

[0067] The dictionary registration unit 105 registers a pair of an unknown word provided to the speech recognition model additional training device 10 and the reading of the unknown word in the dictionary storage unit 202. The reading of the unknown word may be acquired in the same manner as in the second embodiment.

[0068] <Speech recognition model additional learning process according to the fifth embodiment> The speech recognition model additional training process according to the fifth embodiment will be described with reference to Fig. 10. In the following, it is assumed that one or more unknown words are given to the speech recognition model additional training device 10.

[0069] The document search unit 108 searches for documents that contain at least one unknown word among one or more given unknown words (step S501). i|i=1,···,|D|} is obtained.

[0070] Step S502 in FIG. 10 is similar to step S201 in FIG. 4, and therefore the description thereof will be omitted.

[0071] Following step S502, the dictionary registration unit 105 registers the pair of the given unknown word and its reading in the dictionary storage unit 202 (step S503).

[0072] Steps S504 to S505 in FIG. 10 are similar to steps S204 to S205 in FIG. 4, respectively, and therefore will not be described.

[0073] As described above, the speech recognition model additional training device 10 according to the fifth embodiment searches for documents containing a given unknown word. This eliminates the need to prepare documents containing unknown words, making it possible to additionally train (tune) a speech recognition model at a lower cost than in the second embodiment, for example.

[0074] [Variations] Variation 1 The above-described embodiments may be appropriately combined. For example, any one of the first to fourth embodiments may be combined with the fifth embodiment. For example, the noise superimposing unit 107 according to the fourth embodiment may be combined with the first, second, and fifth embodiments.

[0075] Variation 2 Data expansion may be performed by transforming the text extracted by the text extraction unit 101. For example, data expansion may be performed by inputting text and an unknown word (the unknown word may be given or may be an unknown word extracted by the unknown word extraction unit 104) and generating text including the unknown word. Alternatively, data expansion may be performed by inputting an unknown word (the unknown word may be given or may be an unknown word extracted by the unknown word extraction unit 104) into a known language model or the like, or inputting an unknown word and text into a known language model or the like to generate text including the unknown word. At this time, when an instruction to generate text is given to the language model, it may be given by a prompt or the like using a dialogue-style interface or the like. Alternatively, data expansion may be performed by inputting text and an unknown word (the unknown word may be given or may be an unknown word extracted by the unknown word extraction unit 104) and replacing a word of the same part of speech as the unknown word in the text with the unknown word. Alternatively, for example, data expansion may be performed by inputting text containing unknown words and changing the grammatical conjugation of the unknown words in the text, or by paraphrasing the unknown words in the text with words having the same or similar meaning, etc. Furthermore, data expansion may be performed on the text generated in this way by changing the word order, converting between colloquial and literary styles, etc.

[0076] Variation 3 When the number of texts and synthetic speech containing an unknown word is small, additional training of the speech recognition model using those texts and synthetic speech may lead to a decrease in the accuracy of the entire model. Therefore, it may be possible to determine whether training using texts and synthetic speech containing the unknown word is necessary.

[0077] For example, the number of occurrences of each unknown word in each text may be counted, and if the number of occurrences is less than a predetermined threshold, the text and synthetic speech containing the unknown word may not be used for additional training of the speech recognition model. Alternatively, the number of texts containing the unknown word may be counted, and if the number of texts is less than a predetermined threshold, the text and synthetic speech reading it may not be used for additional training of the speech recognition model.

[0078] Also, for example, the number of occurrences of an unknown word contained in each text may be counted for each unknown word, and if the number of occurrences is less than a predetermined threshold, texts containing the unknown word may be collected or generated so that the number of occurrences is equal to or greater than the threshold. Similarly, for example, the number of texts containing the unknown word may be counted for each unknown word, and if the number of texts is less than a predetermined threshold, texts containing the unknown word may be collected or generated so that the number of texts is equal to or greater than the threshold. This makes it possible to prevent a decrease in accuracy of the speech recognition model due to additional learning, even if the number of texts and synthetic speeches containing an unknown word is small.

[0079] Variation 4 Although the text extraction unit 101 is assumed to extract text (sentences) from documents, the text extraction unit 101 may also extract text from sources other than documents. For example, the text extraction unit 101 may extract text from images (including images in documents) and the like, excluding speech. Note that the text extraction unit 101 may use technology such as OCR (Optical Character Recognition) when extracting text from an image. This makes it possible, for example, to use text in an image for additional learning of a speech recognition model.

[0080] In addition to the above, the text extraction unit 101 may extract text from various information that can be collected from a Web conference system (for example, names and nicknames of conference participants, group names, team names, conference names, event names, etc.). This makes it possible to use the text extracted from such information for additional training of the speech recognition model.

[0081] [summary] As described above, in the speech recognition model additional training device 10 according to each of the above-mentioned embodiments, the speech recognition model is additionally trained (tuned) using the text that may contain unknown words and the synthetic voice obtained by reading the text by the speech synthesis technology as training data. This makes it possible to create training data used for training the speech recognition model to recognize unknown words at low cost.

[0082] Therefore, for example, in one usage scenario, when one wishes to create minutes using a speech recognition model from voice dialogue during a meeting using a Web conferencing system or the like, it is possible to obtain at low cost a speech recognition model that can recognize unknown words, such as technical terms, contained in the meeting materials.

[0083] The present invention is not limited to the above-mentioned embodiments specifically disclosed, and various modifications, changes, combinations with known technologies, etc. are possible without departing from the scope of the claims. [Explanation of symbols]

[0084] 10. Speech recognition model additional learning device 101 Text Extraction Unit 102 Voice synthesis unit 103 Model Learning Department 104 Unknown word extraction part 105 Dictionary Registration Section 106 Inflated Voice Synthesis Unit 107 Noise Superimposition Section 108 Document Search Unit 201 Speech recognition model storage unit 202 Dictionary storage unit

Claims

1. a speech synthesis unit that uses a text including unknown words to generate a synthetic speech that reads the text; a model training unit that additionally trains the trained speech recognition model using the text as training data so as to minimize an error between a result of recognizing the synthetic speech using the trained speech recognition model and the training data; A speech recognition model additional learning device having the above configuration.

2. The method further comprises a noise superimposing unit for generating a noise-superimposed synthetic speech by superimposing noise on the synthetic speech, The voice synthesis unit generating a plurality of synthetic speeches from the text by varying values ​​of parameters representing at least one of a speaker reading the text, a speech rate at which the text is read, and an intonation at which the text is read; The model learning unit; 2. The speech recognition model additional learning device according to claim 1, further training the trained speech recognition model so as to minimize an error between a result of recognizing the synthetic speech using the trained speech recognition model and the training data, and to minimize an error between a result of recognizing the noise-added synthetic speech using the trained speech recognition model and the training data.

3. a text extraction unit configured to, when a document including unknown words is provided, extract text from the document for each structure if the document is structured, and to extract text from the document for each sentence if the document is not structured; The voice synthesis unit 3. The speech recognition model additional training device according to claim 1, wherein the synthetic speech is generated using the text extracted by the text extraction unit.

4. a speech synthesis step of generating synthetic speech by reading a text including unknown words; a model training step of additionally training the trained speech recognition model using the text as training data so as to minimize an error between a result of recognizing the synthetic speech by the trained speech recognition model and the training data; The computer then runs a speech recognition model with additional learning methods.

5. a speech synthesis step of generating synthetic speech by reading a text including unknown words; a model training step of additionally training the trained speech recognition model using the text as training data so as to minimize an error between a result of recognizing the synthetic speech by the trained speech recognition model and the training data; A program that causes a computer to execute the following.

Images