Reading estimation device and reading estimation method

Abstract

This reading estimation device includes: a reading candidate generation unit configured to generate a plurality of reading candidates with respect to text transcribed from speech data; a speech recognition unit configured to generate a speech recognition result relating to the speech data; and a selection unit configured to select, on the basis of the degree of similarity between the reading candidates and the speech recognition result, a reading of the text from among the plurality of reading candidates. The likelihood of obtaining a correct reading for transcribed text is thereby increased.

Description

Reading estimation device and reading estimation method

[0001] The present invention relates to a reading estimation device and a reading estimation method.

[0002] Speech and its transcription data are extremely useful for training models for speech recognition and speech synthesis.

[0003] However, commonly available data is often text that combines kana and kanji, and numeric strings in particular are often written in conventional notation that includes symbols. For example, even if an address is written as "Chiyoda-ku 1-1," it is often not read literally as "ichi haifun ichi," but rather as "ichi no ichi" or "ichibanchiichi," and the reading is often ambiguous. Similarly, when a numeric string is written as "8598777," depending on whether the context is a phone number or a number with digits, the reading will vary depending on whether the digits are read individually or including the digits, as in "happyakugojukyumanhassen 7,0 ...

[0004] Conventionally, dictionary-based (+co-occurrence, etc.) methods have been commonly used to estimate the pronunciation of words (see, for example, Patent Document 1).

[0005] Patent No. 6599219

[0006] Co-occurrence makes it possible to estimate readings that take context into account to some extent, but because there are many different ways to express number strings, it is difficult to create dictionaries and rules, and it is difficult to confirm whether a certain reading is correct.

[0007] The present invention has been made in view of the above points, and aims to increase the likelihood that a correct reading will be obtained from transcribed text.

[0008] In order to solve the above problem, the reading estimation device has a reading candidate generation unit configured to generate multiple reading candidates for text transcribed from audio data, a speech recognition unit configured to generate speech recognition results for the audio data, and a selection unit configured to select a reading for the text from the multiple reading candidates based on the similarity between the reading candidates and the speech recognition results.

[0009] This can increase the likelihood of obtaining a correct reading of the transcribed text.

[0010] FIG. 1 is a diagram illustrating an example of a hardware configuration of a reading estimation device 10 according to an embodiment of the present invention; FIG. 2 is a diagram illustrating an example of a functional configuration of the reading estimation device 10 according to an embodiment of the present invention; FIG. 3 is a flowchart illustrating an example of a processing procedure executed by the reading estimation device 10; FIG. 4 is a diagram illustrating a first example of a question sentence and a response sentence; FIG. 5 is a diagram illustrating a second example of a question sentence and a response sentence; and FIG. 6 is a diagram illustrating a third example of a question sentence and a response sentence.

[0011] An embodiment of the present invention will now be described with reference to the accompanying drawings. Fig. 1 is a diagram showing an example of the hardware configuration of a reading estimation device 10 according to an embodiment of the present invention. The reading estimation device 10 shown in Fig. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, all of which are interconnected via a bus B.

[0012] The program that realizes the processing in the reading estimation device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0013] When an instruction to start the program is received, the memory device 103 reads and stores the program from the auxiliary storage device 102. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the reading estimation device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0014] 2 is a diagram showing an example of the functional configuration of a reading estimation device 10 according to an embodiment of the present invention. In FIG. 1, the reading estimation device 10 includes a reading candidate generation unit 11, a speech recognition unit 12, and a selection unit 13. Each of these units is realized by a process executed by a processor 104 of one or more programs installed in the reading estimation device 10.

[0015] The reading candidate generation unit 11 inputs the transcribed text d1 and generates multiple reading candidates (hereinafter referred to as "reading candidates") for the transcribed text d1. The reading candidate generation unit 11 outputs the generated reading candidates. The transcribed text d1 is text data containing a mixture of kana and kanji characters, which is transcribed by a user who listened to the audio data d2 and estimated the content of the utterance. The reading candidate generation unit 11 may generate multiple reading candidates using a large language model (LLM (Large Language Models)). In this case, the reading candidate generation unit 11 inputs a question (prompt) requesting the generation of readings for the transcribed text d1 to the LLM and obtains reading candidates from the output (response) from the LLM. When using an LLM, if the reading candidates included in the response from the LLM include characters other than the readings (for example, numbers, alphabets, or kanji), the reading candidate generation unit 11 may instruct the LLM to regenerate reading candidates for the parts other than the readings. Alternatively, the reading candidate generating unit 11 may generate multiple reading candidates for the transcribed text d1 on a dictionary basis (using a dictionary that indicates the correspondence between kanji, numbers, alphabets, etc. and readings).

[0016] The speech recognition unit 12 generates a speech recognition result for the speech data d2. For example, the speech recognition unit 12 inputs the speech data d2 and applies speech recognition to the speech data d2 to generate a phoneme sequence. Any phoneme system may be used. Alternatively, the speech recognition unit 12 may generate a pronunciation sequence for the speech data d2. In this case, the pronunciation sequence is a character string in which all elements other than the pronunciation are guaranteed to be pronunciation.

[0017] The selection unit 13 calculates an index (hereinafter referred to as "similarity") for evaluating the similarity between each of the multiple reading candidates generated by the reading candidate generation unit 11 and the speech recognition result (phoneme string or kana pronunciation string) output by the speech recognition unit 12. For example, the selection unit 13 may calculate an edit distance using dynamic programming and use the edit distance as the similarity, or may calculate the similarity using another method. When the speech recognition result is a phoneme string, the selection unit 13 converts each of the multiple reading candidates generated by the reading candidate generation unit 11 into a phoneme string and then calculates the similarity. The conversion from reading to a phoneme string can be performed using a general tool. When the speech recognition result is a kana pronunciation string, the selection unit 13 calculates the similarity between each of the multiple reading candidates generated by the reading candidate generation unit 11 as is and the speech recognition result.

[0018] The selection unit 13 selects the reading candidate with the highest similarity (highest similarity) from among the multiple reading candidates as the correct reading of the transcription text d1, and outputs a set of {audio data d2: transcription text: correct reading (phoneme sequence)}. By using the similarity to match the reading candidate with the audio data d2, it is possible to select reading candidates with high accuracy.

[0019] A threshold value may be set for the similarity. In this case, the reading candidate generation unit 11 may input a question that points out mispronunciations (reading candidates below the threshold) into the LLM and regenerate reading candidates, thereby reacquiring a variety of reading candidates excluding the mispronunciations. The selection unit 13 may calculate the similarity of the regenerated reading candidates, and output a set of {audio data: transcribed text: correct reading (phoneme sequence)}, with the reading candidate with the highest similarity as the correct reading.

[0020] The following is a specific description of the processing procedure executed by the reading estimation device 10. FIG. 3 is a flowchart illustrating an example of the processing procedure executed by the reading estimation device 10.

[0021] In step S101, the speech recognition unit 12 applies speech recognition to the speech data d2 to generate a speech recognition result. Here, it is assumed that a phoneme string is generated as the speech recognition result.

[0022] Next, the reading candidate generation unit 11 generates a plurality of (for example, a predetermined number of) reading candidates for the transcribed text d1 (S102). Here, an example will be described in which the transcribed text d1 is "Yokosuka City 3-51" and reading candidates are generated using an LLM. In this case, the reading candidate generation unit 11 inputs, for example, the question q1 in FIG. 4 into the LLM and obtains a response sentence a1. The question q1 is a prompt requesting three readings for "Yokosuka City 3-51." All questions generated by the reading candidate generation unit 11 may be generated by matching individual character strings to templates prepared in advance.

[0023] The response sentence a1 includes the following three reading candidates: (1) "Yokosukashi sango goichi" (2) "Yokosukashi 3choume 51ban" (3) "Yokosukashi san no goichi" Therefore, the reading candidate generation unit 11 acquires the above three reading candidates (1) to (3). Note that the numbers in parentheses (1) to (3) are added for convenience.

[0024] Next, the reading candidate generation unit 11 determines whether or not there are any reading candidates that include characters other than the pronunciation of kana among the acquired reading candidates (S103). Among the reading candidates (1) to (3) above, (2) includes numbers ("3" and "51"). In this case (Yes in S103), the reading candidate generation unit 11 generates multiple (e.g., a predetermined number) reading candidates for the reading candidates that include characters other than the pronunciation of kana (S104). For example, the reading candidate generation unit 11 inputs the question sentence q2 in FIG. 5 into the LLM and obtains a response sentence a2. The question sentence q2 is a prompt requesting three readings for "3-choume 51ban."

[0025] Response sentence a2 contains three reading candidates, all of which are as follows (2'): (2') "yokosukashi sanchoume gojuuichiban" Therefore, at this point, three reading candidates (1), (2'), and (3) have been obtained.

[0026] Next, the reading candidate generation unit 11 determines whether or not there are any reading candidates that include characters other than kana readings for all reading candidates ((1), (2'), and (3)) currently obtained (S103). If there are any applicable reading candidates (Yes in S103), step S104 is repeated. If there are no applicable reading candidates (No in S103), the selection unit 13 converts all reading candidates obtained at this point into phoneme strings (S105). Any phoneme system may be selected. For example, if based on the IPA (International Phonetic Alphabet), the three reading candidates (1), (2'), and (3) are converted into the following phoneme strings: (1) "Yokosukashi sangougoichi" --> yokosuka sh isa N gougoi ch i (2') "Yokosukashi sanchoumegojuichiban" --> yokosuka sh isa N ch oumegoju: i ch iba N (3) "Yokosukashi sannogoichi" --> yokosuka sh isa N ogoi ch i Next, for each reading candidate, the selection unit 13 calculates the similarity between the phoneme sequence associated with the reading candidate and the phoneme sequence of the speech recognition result in step S101 (S106). Here, it is assumed that the speech recognition result is as follows: Speech recognition result --> yokosuka sh isa N ch oumegoju: i ch iba N (yokosuka sh isa N ch oumegoju: i ch iba N) Note that the string in parentheses ("yokosuka sh isa N") does not constitute the speech recognition result, but is a representation of the speech recognition result written in hiragana for convenience.

[0027] Next, the selection unit 13 determines whether the similarity of all the reading candidates is equal to or greater than a threshold (S107). If the similarity of all the reading candidates is equal to or greater than the threshold (Yes in S107), the selection unit 13 determines the reading candidate with the highest similarity as the correct reading and outputs {audio data d2: transcribed text: correct reading (phoneme sequence)} (S109). In the above example, (2') "yokosukashi sanchoume gojuuichiban" is determined to be the correct reading. As a result, the user can obtain the audio, the kana-kanji mixed text, and the correct reading kana.

[0028] If no threshold value is set, the selection unit 13 may execute step S109 without making the determination in step S107.

[0029] On the other hand, if there is a reading candidate whose similarity is less than the threshold (No in S107), the reading candidate generation unit 11 regenerates reading candidates (S108). That is, in this case, taking into consideration the possibility that the reading is incorrect, an additional question sentence is input to the LLM to obtain other reading candidates. By providing the incorrect reading candidate to the LLM and regenerating it, a large number of candidates can be generated efficiently.

[0030] For example, suppose the speech recognition result is as follows, and (3) above is less than the threshold. Speech recognition result --> yokosuka sh isa N ogoju: i ch i (Yokosukashisangojuichi) In this case, the reading candidate generation unit 11 inputs, for example, question sentence q3 in FIG. 6 into the LLM and obtains response sentence a3. Question sentence q3 is a prompt requesting three readings for "Yokosukashi 3-51" excluding "Yokosukashisangoichi."

[0031] Response sentence a3 includes the following three reading candidates: (4) "Yokosukashi sango gojuichi" (5) "Yokosukashi sango gojuichi" (6) "Yokosukashi san gojuichi" Next, steps S103 and onwards are repeated. In step S105 executed at this time, the reading candidates are converted into phoneme strings as follows: (4), (5) "Yokosukashi sango gojuichi" --> yokosuka sh isa N gougoju: i ch i (6) yokosuka sh san gojuichi --> yokosuka sh isa N ogoju: i ch i As shown above, (4) and (5) are converted into the same phoneme string.

[0032] Steps S106 and thereafter are as already explained.

[0033] The judgment condition in step S107 may be changed to one or more reading candidates whose similarity is equal to or greater than a threshold. In this case, if there is no reading candidate whose similarity is equal to or greater than the threshold, step S108 may be executed to exclude all reading candidates that have already been obtained. Alternatively, two thresholds, α1 and α2 (α1 > α2), may be set. In this case, if there are reading candidates whose similarity is equal to or greater than α2 but less than α1, some of the candidates may be corrected, and step S108 may be executed to exclude reading candidates whose similarity is less than α2, and the reading candidate with the highest similarity among the reading candidates whose similarity is equal to or greater than α1 may be selected as the correct reading.

[0034] Although the above describes an example in which reading candidates are generated using an LLM, reading candidates may also be generated using a dictionary.

[0035] As described above, this embodiment can increase the likelihood of obtaining a correct reading for the transcribed text. The speech recognition results obtained by the speech recognition unit 12 may contain errors. The transcribed text d1 is generally correct, but due to the mixture of kana and kanji, the exact reading may not be clear from the notation. Therefore, it is possible to identify a reading that is likely to be correct from the reading variations (candidate readings) of the transcribed text d1 and the speech recognition results.

[0036] Furthermore, by using the combination of high-precision speech data d2, mixed kanji and kana text (transcribed text d1), and readings to train a speech recognition model that outputs phonemes from speech, or to train a model that converts the output of a speech recognition system that outputs phonemes into mixed kanji and kana text, it is possible to expect improvements in the performance of these models.

[0037] Furthermore, by using this set to train a speech recognition model that outputs text containing both kana and kanji characters from speech data, it is expected that the performance of the model will improve.

[0038] Furthermore, with regard to the speech synthesis model, it is expected that a more natural pronunciation can be achieved by learning the pronunciation of text containing both kana and kanji characters obtained by this embodiment.

[0039] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0040] 10 Reading estimation device 11 Reading candidate generation unit 12 Speech recognition unit 13 Selection unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device B Bus

Claims

1. A reading estimation device comprising: a reading candidate generation unit configured to generate multiple reading candidates for text transcribed from audio data; a speech recognition unit configured to generate a speech recognition result for the audio data; and a selection unit configured to select a reading for the text from the multiple reading candidates based on the similarity between the reading candidates and the speech recognition result.

2. The reading estimation device according to claim 1, characterized in that the reading candidate generation unit is configured to regenerate reading candidates excluding the reading candidates whose similarity with respect to the text is less than a threshold value.

3. The reading estimation device according to claim 1 or 2, characterized in that the reading candidate generation unit is configured to generate the reading candidates using a large-scale language model.

4. A reading estimation method characterized in that a computer executes the following steps: a reading candidate generation step for generating multiple reading candidates for text transcribed from audio data; a speech recognition step for generating a speech recognition result for the audio data; and a selection step for selecting the reading of the text from the multiple reading candidates based on the similarity between the reading candidates and the speech recognition result.

Images