Speech synthesis device, speech synthesis method, and program

Abstract

To efficiently provide linguistic analysis and accent estimation of high level for speech synthesis differing in accent from a standard language.SOLUTION: A speech synthesis device according to an embodiment comprises a standard language linguistic analysis unit, a conversion unit and a speech synthesis unit. The standard language linguistic analysis unit generates a first phonogram character string having standard language accent. The conversion unit converts accent information on an accent phrase of the first phonogram character string into accent information corresponding to a first accent rule different from the standard language accent rule, and generates a second phonogram character string corresponding to the first accent rule from the accent information corresponding to the first accent rule. The speech synthesis unit synthesizes a speech from the second phonogram character string. The conversion unit holds, as an accent conversion rule, a neural network learnt from pair data on the first phonogram character string and the second phonogram character string, and then converts the accent information according to the accent conversion rule.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a speech synthesis device, a speech synthesis method, and a program. [Background technology]

[0002] There is text-to-speech synthesis technology that takes text as input and synthesizes speech that corresponds to the text. In order to synthesize natural-sounding speech in Japanese text-to-speech synthesis, it is necessary to estimate the appropriate accent for the input text. Creating an accent dictionary for accent estimation typically requires a large amount of accurately annotated text data. Since standard Japanese has well-established language resources such as corpora and dictionaries, it is relatively easy to create an accent dictionary. On the other hand, when synthesizing speech that corresponds to dialects such as Kansai dialect, it is not easy to create an accent dictionary because there are few well-established language resources. Furthermore, when dialect text is input into a language analyzer for standard Japanese, the appropriate accent corresponding to the dialect is not assigned, resulting in unnatural synthesized speech. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3681111 [Patent Document 2] Japanese Patent Application Laid-Open No. 2001-75585 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to efficiently develop a highly accurate language analysis and accent estimation system for speech synthesis of accents different from standard accents. [Means for solving the problem]

[0005] A speech synthesis device according to an embodiment includes a standard language analysis unit, a conversion unit, and a speech synthesis unit. The standard language analysis unit estimates reading information and accent information of accent phrases obtained by segmenting input text using a standard language analysis dictionary, and generates a first phonetic string with a standard accent from the reading information and accent information. The conversion unit converts accent information of the accent phrase of the first phonetic string into accent information corresponding to a first accent rule that is different from the standard language accent rule, and generates a second phonetic string corresponding to the first accent rule from the accent information corresponding to the first accent rule. The speech synthesis unit synthesizes speech from the second phonetic string. The conversion unit retains a neural network trained from pair data of the first phonetic string and the second phonetic string as accent conversion rules, and converts accent information based on the accent conversion rules. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of the functional configuration of a speech synthesis device according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the functional configuration of the conversion unit according to the first embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of the speech synthesis method according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing an example of the learning method according to the first embodiment. [Figure 5] FIG. 5 is a flowchart illustrating an example of processing by the conversion unit of the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of context information according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a phonetic character string according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing examples of pitch accents in the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of accent pitch information according to the first embodiment. [Figure 10]FIG. 10 is a diagram illustrating an example of the structure of the DNN model according to the first embodiment. [Figure 11] FIG. 11 is a flowchart showing Example 1 (in the case of mora units) of the process of step S23 by the determination unit of the first embodiment. [Figure 12] FIG. 12 is a flowchart showing Example 2 (in the case of phoneme units) of the process of step S23 by the determination unit of the first embodiment. [Figure 13] FIG. 13 is a diagram for explaining an example of the process of step S31-2 by the determination unit of the first embodiment. [Figure 14] FIG. 14 is a flowchart showing an example of the process of step S31-2 by the determination unit of the first embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of the functional configuration of the speech synthesis unit according to the first embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of the functional configuration of a speech synthesis device according to the second embodiment. [Figure 17] FIG. 17 is a diagram showing an example of phrase boundary correction rules according to the second embodiment. [Figure 18A] FIG. 18A is a flowchart illustrating an example of processing by the correction unit of the second embodiment. [Figure 18B] FIG. 18B is a diagram for explaining an example of processing by the correction unit of the second embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of a word dictionary in the accent correction dictionary storage unit of the second embodiment. [Figure 20A] FIG. 20A is a flowchart showing an example of processing by the accent correction unit of the second embodiment. [Figure 20B] FIG. 20B is a diagram for explaining an example of processing by the accent correction unit of the second embodiment. [Figure 21] FIG. 21 is a diagram illustrating an example of a hardware configuration of the speech synthesis device according to the first and second embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, embodiments of a speech synthesis device, a speech synthesis method, and a program will be described in detail with reference to the accompanying drawings.

[0008] (First embodiment) First, an example of the functional configuration of the speech synthesis device 1 according to the first embodiment will be described.

[0009] [Example of functional configuration of a speech synthesizer] 1 is a diagram showing an example of the functional configuration of a speech synthesizer 1 according to the first embodiment. In the speech synthesizer 1 according to the first embodiment, after processing by a standard language analysis unit 2 using a standard language dictionary, processing by a conversion unit 3 for converting to a desired accent rule is performed. This converts the accent according to standard Japanese to a desired accent rule such as a Kansai dialect accent, enabling text-to-speech synthesis with a natural accent.

[0010] Specifically, the speech synthesis device 1 of the first embodiment includes a standard language analysis unit 2, a conversion unit 3, a speech synthesis unit 4, a standard language dictionary storage unit 5, and a conversion rule storage unit 6.

[0011] The standard language analysis unit 2 uses the standard language dictionary storage unit 5 to estimate reading information and accent information for each accent phrase obtained by dividing the input text, and generates a first phonetic string with a standard accent.

[0012] The conversion unit 3 acquires a trained deep neural network (DNN) model from the conversion rule storage unit 6, and generates, from the first phonetic string, a second phonetic string corresponding to the first accent rule that is different from the standard accent rule.

[0013] The first accent rule is an accent rule that is different from the accent rule for standard Japanese. For example, the first accent rule is an accent rule for a specific dialect (such as Kansai dialect), a speaker with a distinctive speaking tone, or a character with a distinctive speaking tone. The accent of a character with a distinctive speaking tone represents a unique accent for a desired character expression, such as "~dawan" (a dog character) or "~da-nyan" (a cat character).

[0014] The speech synthesis unit 4 synthesizes speech from the second phonetic string.

[0015] The standard language dictionary storage unit 5 stores text analysis rules and word accent information in standard Japanese.

[0016] The conversion rule storage unit 6 stores accent conversion rules learned by the DNN.

[0017] Here, a phonetic string is a string that represents sounds. For example, a phonetic string is a string in a predetermined format that contains reading (pronunciation) information and accent information used for speech synthesis, obtained by analyzing input text containing kana and kanji characters. For example, JEITA's "IT-4006 Symbols for Japanese Text-to-Speech Synthesis" or speech synthesis symbols defined by each speech synthesis system can be used as a string in a predetermined format. A phonetic string does not have to be a readable string as long as it is data that indicates reading information and accent information. Therefore, in this specification, the term "phonetic string" also refers to information that indicates the reading and accent of input text expressed as electronic data.

[0018] Next, an example of the functional configuration of the conversion unit 3, which is a main component of the first embodiment, will be described with reference to FIG.

[0019] [Example of functional configuration of the conversion unit] 2 is a diagram showing an example of the functional configuration of the conversion unit 3 of the first embodiment. The conversion unit 3 of the first embodiment includes an estimation unit 31 and a determination unit 32. The conversion unit 3 converts a phonetic character string using a conversion rule storage unit 6.

[0020] The conversion rule storage unit 6 stores the DNN model of the accent conversion rules learned by the learning device 7.

[0021] The learning device 7 trains the DNN using paired data of phonetic strings based on standard accent and phonetic strings based on the first accent rule, and outputs the trained DNN model. For example, if the first accent rule is an accent rule for the Kansai dialect, the phonetic string based on standard accent is a phonetic string including an erroneous accent estimation, estimated from a sentence in the Kansai dialect by standard language analysis. In this case, the phonetic string based on the first accent rule means a phonetic string with an accent appropriate for the Kansai dialect.

[0022] That is, the learning device 7 uses, as paired data for learning, a phonetic string automatically estimated according to the standard accent rules and a phonetic string that is correct answer data according to the first accent rules, and performs learning of the DNN.

[0023] The accent conversion is performed by an estimation unit 31 and a determination unit 32.

[0024] The estimation unit 31 estimates accent pitch information for each phonetic syllable using a neural network from the first phonetic string and context information for each phonetic syllable divided into units that determine the pitch of the phonetic syllable. Specifically, the estimation unit 31 acquires a trained DNN model from the conversion rule storage unit 6 and estimates pitch information for each phonetic syllable from the first phonetic string.

[0025] The determination unit 32 determines accent information corresponding to the first accent rule from the accent pitch information, thereby generating a second phonetic string corresponding to the first accent rule. Specifically, the determination unit 32 determines accent from pitch information for each phoneme, and updates the accent of each accent phrase in the first phonetic string, thereby generating a second phonetic string corresponding to the first accent rule.

[0026] As shown in Figure 2, by dividing the configuration of the conversion unit 3 into two units, an estimation unit 31 and a determination unit 32, it is possible to clarify the roles of the units and to easily modify and change the modules. Furthermore, by having the estimation unit 31 output pitch information for single sounds, flexible conversion that is not limited to existing accent types becomes possible. Furthermore, the determination unit 32 can generate a second phonetic string that matches the input format of the speech synthesis unit 4.

[0027] The learning device 7 may be provided in the speech synthesis device 1 as a learning unit.

[0028] [Example of voice synthesis method] 3 is a flowchart showing an example of the speech synthesis method of the first embodiment. First, the standard language analysis unit 2 accepts input of text data (step S1).

[0029] Next, the standard language analysis unit 2 uses the standard language dictionary storage unit 5 to divide the text data input by the processing of step S1 into accent phrases, estimate reading information and accent information, and generate a first phonetic string using standard accent (step S2).

[0030] Next, the conversion unit 3 converts the accent information of the first phonetic character string generated by the processing of step S2 into accent information corresponding to the first accent rule, and generates a second phonetic character string according to the first accent rule (step S3).

[0031] Next, the speech synthesis unit 4 synthesizes speech using the second phonetic character string generated by the process of step S3 (step S4).

[0032] [Example of learning method] 4 is a flowchart showing an example of the learning method of the first embodiment. First, the learning device 7 receives input of pair data of a phonetic string based on a standard accent and a phonetic string based on the first accent rule (step S11).

[0033] Next, the learning device 7 generates (extracts) context information from the phonetic string with standard accent (step S12), and then estimates accent information for each phoneme from the phonetic string with standard accent (step S13).

[0034] Next, the learning device 7 extracts phonetic accent information from the phonetic string of the first accent rule (step S14). Next, the learning device 7 updates the DNN model using the estimation error of the phonetic accent information, and learns accent conversion rules (step S15).

[0035] Here, a phone is a unit of speech for which an accent is to be estimated, such as a phoneme, a mora, or a syllable. Specifically, a phone is a kana character corresponding to a mora, a vowel corresponding to a syllable, or a unit consisting of a consonant and a vowel.

[0036] The context information is information used to estimate accent conversion for each phone. For example, the context information includes at least one of the phone's phonological information, phonetic information before and after the phone, accent pitch information, the position of an accent phrase, and subsequent symbol information (e.g., whether or not there is a punctuation mark, whether or not there is an exclamation mark, whether or not there is a question mark, etc.). By using the context information, it becomes possible to estimate pitch information from the context information before and after the phone.

[0037] 5 is a flowchart showing an example of processing by the conversion unit of the first embodiment. First, the estimation unit 31 receives input of a first phonetic string with a standard accent and context information for each syllable separated into units that determine the pitch of the sound (step S21).

[0038] Next, the estimation unit 31 acquires the trained DNN model from the conversion rule storage unit 6 and estimates the pitch of each phonetic syllable (step S22). Next, the determination unit 32 generates a second phonetic character string corresponding to the first accent rule from the pitch of each phonetic syllable and the first phonetic character string based on the standard accent (step S23).

[0039] 6 is a diagram showing an example of context information in the first embodiment. The context information includes, for example, phonetic information for each phone and the phonemes before and after it, accent pitch information, position information within an accent phrase, and subsequent symbols (for example, periods, commas, exclamation marks, question marks, etc.). By inputting context information for each phone to the estimation unit 31, it is possible to focus on more detailed information than before.

[0040] The estimation unit 31 may use at least one of the context information in FIG. 6, or may use other feature amounts such as parts of speech.

[0041] Fig. 7 is a diagram showing an example of a phonetic character string in the first embodiment. The example in Fig. 7 shows an example of a first phonetic character string 102 in standard accent of an input text 101, which is a sentence in the Kansai dialect, and a second phonetic character string 103 corresponding to the first accent rule (Kansai dialect).

[0042] Accents in Japanese are expressed by the pitch of sounds, and in the case of the phonetic string in Figure 7, they are expressed by the pitch accent of each mora corresponding to one syllable (such as ``a,'' ``ga,'' or ``kya'') as shown in Figure 8.

[0043] FIG. 8 is a diagram showing an example of pitch accent in the first embodiment. The position where a high tone changes to a low tone is called an accent nucleus, and is represented by "'" in the phonetic string. Furthermore, since the accent nucleus of the second phonetic string 103 is in the third mora, the accent type is Type 3. When L represents a low tone and H represents a high tone, they are expressed as [H,L,L,L,L,L,L] and [L,H,H,L,L,L], respectively. According to the accent rules for standard Japanese, if the accent type is Type 1, the first letter is H, and otherwise the first letter is L, followed by H, and at the accent nucleus position, H changes to L. Therefore, the information on the position of the accent nucleus is used as the accent type to classify the pitch information.

[0044] The determination unit 32 generates a second phonetic string corresponding to the first accent rule by determining, as the position of the accent kernel, the position where the accent pitch of each phone estimated by the estimation unit 31 changes from a high to a low phone. The processing method of the determination unit 32 is used as a processing method using accent rules that are often seen, for example, in the Tokyo dialect.

[0045] Fig. 9 is a diagram showing an example of accent pitch information in the first embodiment. The example in Fig. 9 shows the degree of pitch of each single sound output by the estimation unit 31. Here, the degree of pitch is handled as an example in a range from 0 to 1, with the value closer to 1 indicating a higher pitch and the value closer to 0 indicating a lower pitch.

[0046] Fig. 10 is a diagram showing an example of the structure of a DNN model according to the first embodiment. The example in Fig. 10 shows a model structure in which a two-layer bidirectional LSTM network is used to output one value for each monotone sound, taking into account contextual information before and after the sound. Here, a sigmoid function is used for the value output from the two-layer bidirectional LSTM network, limiting the range of values ​​indicating the degree of pitch to between 0 and 1.

[0047] In the example of Figure 10, a DNN model based on LSTM (Long Short Term Memory) is used as the DNN model structure. However, the estimation unit 31 may be constructed using any neural network model, such as a feedforward neural network, a recurrent neural network, a convolutional neural network, or a transformer neural network. In the example of Figure 10, the accent pitch information for each monophonic note is a continuous value representing the degree of pitch. In this case, by using a continuous value such as 0 to 1, it can be treated as the likelihood or probability that the note is "high." The pitch information estimated by this neural network need only be information for estimating the pitch of each monophonic note, and is not limited to a continuous value representing the degree of pitch. Discrete symbol estimation, probability values ​​for each class in which pitch is classified into several levels, or direct pitch estimation may also be used.

[0048] In the example of FIG. 10, the degree of high and low is converted into a continuous value between 0 and 1 using a sigmoid function, but any conversion other than the sigmoid function may be used.

[0049] Fig. 11 is a flowchart showing Example 1 (in the case of mora units) of the process of step S23 of the determination unit 32. An example will be described in which the output of the estimation unit 31 is the accent pitch information shown in Fig. 9. First, the determination unit 32 receives input of accent pitch information (the degree of pitch of each sound) and the first phonetic character string 102 (step S31).

[0050] Next, the determination unit 32 converts the pitch degrees [0.1, 0.7, 0.8, 0.4, 0.1, 0.1] into pitch accent labels [L, H, H, L, L, L] (step S32). In the example of Fig. 11, pitch degrees of 0.5 or more are considered high-pitched (H), and other cases are considered low-pitched (L).

[0051] Next, the determining unit 32 determines whether or not there is one or more portions in the accent phrase where the pitch accent label changes from H to L (step S33).

[0052] If there is one or more parts where the H changes to L (step S33, Yes), the determination unit 32 determines the accent type based on the change points (step S34). If change points are found in multiple places, the determination unit 32 determines the first change point found as the accent nucleus position. In the example of Figure 11, since the change point is the third mora, the determination unit 32 determines the third mora as the accent nucleus and determines the accent type to be type 3.

[0053] If there is no part that changes from H to L (No at step S33), the determining unit 32 determines the accent type to be type 0 (step S35).

[0054] Finally, the determination unit 32 determines the accent of the second phonetic character string 103 by updating the accent of the first phonetic character string 102 (step S36, see FIG. 7).

[0055] In the example of FIG. 11, a mora unit is used as an example of a unit of a single phone, but a phoneme may also be used as a unit of a single phone.

[0056] Fig. 12 is a flowchart showing a second example (in the case of phoneme units) of the process of step S23 by the determination unit 32 of the first embodiment. In the example of Fig. 12, a process of step S31-2 is added to match the unit of the length of the pitch degree of each phoneme to the first phonetic character string 102.

[0057] Fig. 13 is a diagram for explaining an example of the processing of step S31 by the determination unit S31-2 of the first embodiment. The example of Fig. 13 shows the mora-based monophones (delimited by dotted lines in Fig. 13) and the phoneme-based monophones (lower part of Fig. 13) of the first phonetic character string 102. As in the example of Fig. 13, the first phonetic character string 102 usually uses mora-based monophones, which are different from phonemes in terms of monophone units. Therefore, in the example of the flowchart of Fig. 12, a process of matching units (step S31-2) is required.

[0058] 14 is a flowchart showing an example of the process of step S31-2 by the determination unit 32 of the first embodiment. First, the determination unit 32 receives input of the pitch of each phonetic syllable 104 and the first phonetic character string 102 (step S41). Next, the determination unit 32 matches the pitch of each phonetic syllable 104 with the first phonetic character string 102, as in the example of FIG. 13 (step S42).

[0059] Next, the determination unit 32 calculates the pitch degree corresponding to each character of the first phonetic string 102 by calculating the average value of the pitch degree corresponding to each character of the first phonetic string 102 (step S43). For example, the pitch degree of the first character in Fig. 13 is calculated from the pitch degree of the single sound n and the average value of the pitch degree of the single sound a.

[0060] In the example of step S43 in FIG. 14, the pitch of each phoneme is matched with the unit of the first phonetic string by calculating the average value, but any method may be used, such as calculating the maximum value.

[0061] 11 and 12, the pitch level is converted into a pitch accent label, and the accent kernel and accent type are determined. However, other accent determination methods may be used. For example, a method may be used in which the point with the greatest change in pitch level is used as the accent kernel. Also, for example, other accent determination methods that do not use an accent kernel may be used.

[0062] [Example of functional configuration of speech synthesis unit] 15 is a diagram showing an example of the functional configuration of the speech synthesis unit 4 of the first embodiment. The speech synthesis unit 4 of the first embodiment includes a phonogram character string analysis unit 41, a prosody generation unit 42, and a waveform generation unit 43.

[0063] The speech synthesis unit 4 synthesizes speech based on the second phonetic character string 103 generated by the conversion unit 3. The speech synthesis process is roughly divided into a phonetic character string analysis unit 41, a prosody generation unit 42, and a waveform generation unit 43.

[0064] The phonetic string analysis unit 41 interprets the input second phonetic string 103 and generates context information used for generating prosody and waveform.

[0065] The prosody generation unit 42 generates prosody information such as the length (duration) and pitch (inflection) of sounds for each phonetic unit from the context information obtained by the phonetic string analysis unit 41. In this way, prosody is generated based on the first accent rule in accordance with the second phonetic string 103.

[0066] The waveform generation unit 43 generates a speech waveform in accordance with the prosody and context information generated by the prosody generation unit 42, and outputs synthetic speech. The waveform generation unit 43 can also use data created from a recorded speech of a speaker based on a first accent rule, such as Kansai dialect, as speech synthesis data representing the voice characteristics used for speech synthesis. This makes it possible to obtain natural synthetic speech that conforms to the first accent rule.

[0067] As described above, in the speech synthesis device 1 of the first embodiment, the standard language analysis unit 2 estimates reading information and accent information of accent phrases obtained by segmenting input text using the standard language analysis dictionary 5, and generates a first phonetic character string 102 with a standard accent from the reading information and accent information. The conversion unit 3 converts the accent information of the accent phrase of the first phonetic character string 102 into accent information corresponding to a first accent rule that is different from the standard accent rule, and generates a second phonetic character string 103 corresponding to the first accent rule from the accent information corresponding to the first accent rule. The speech synthesis unit 4 then synthesizes speech from the second phonetic character string 103. The conversion unit 3 stores a neural network trained from pair data of the first phonetic character string 102 and the second phonetic character string 103 as accent conversion rules, and converts the accent information based on the accent conversion rules.

[0068] As a result, the speech synthesis device 1 of the first embodiment can efficiently implement highly accurate language analysis and accent estimation for speech synthesis with an accent different from that of standard Japanese.

[0069] For example, accent conversion rules are generated by a DNN model trained using paired data of a first phonetic string 102 with a standard accent and a second phonetic string 103 with a first accent rule that is different from the standard accent, and taking into account contextual information before and after. By applying accent conversion based on the generated accent conversion rules to the results of analyzing text data 101 using a standard language analysis dictionary 5, it becomes possible to synthesize speech in accordance with the first accent rule without requiring a large amount of language resources that cover the entire vocabulary.

[0070] The speech synthesizer 1 of the first embodiment can estimate dialect accents by converting the analysis results of a standard language analyzer using DNN-based access conversion rules trained from a relatively small number of correct examples compared to dialect-specific language analysis. That is, the speech synthesizer 1 of the first embodiment makes it possible to automatically create accent conversion rules using the conversion unit 3 that uses a neural network. Furthermore, the combination of the standard language analysis unit 2 and the conversion unit 3 makes it possible to use language analysis processing based on existing language resources.

[0071] (Second embodiment) Next, a second embodiment will be described. In the description of the second embodiment, the same description as in the first embodiment will be omitted, and only the differences from the first embodiment will be described.

[0072] [Example of functional configuration of a speech synthesizer] 16 is a diagram showing an example of the functional configuration of a speech synthesizer 1-2 according to the second embodiment. The speech synthesizer 1-2 according to the second embodiment further includes a boundary correction unit 8, an accent correction unit 9, a phrase boundary correction rule storage unit 10, and an accent correction dictionary storage unit 11 in addition to the configuration of the speech synthesizer 1 according to the first embodiment. This makes it possible in the second embodiment to correct analysis errors in the standard language analysis unit 2 and conversion errors in the conversion unit 3.

[0073] The boundary correction unit 8 corrects the boundaries of accent phrases included in the first phonetic string. The boundary correction unit 8 generates a corrected phonetic string in which the boundaries of accent phrases included in the first phonetic string have been corrected based on the rules for correcting accent phrase boundaries, and inputs the corrected phonetic string to the conversion unit 3. Specifically, the boundary correction unit 8 obtains phrase boundary correction rules from the phrase boundary correction rule storage unit 10, and corrects the accent phrase boundaries of the first phonetic string 102 using standard accent.

[0074] In the second embodiment, estimation errors can be reduced and the impact of phrase boundary estimation errors can be mitigated by adding a new boundary correction unit 8. By providing the boundary correction unit 8, errors in accent phrase boundary estimation can be reduced and control can be performed as intended by the user.

[0075] The accent correction unit 9 identifies an accent correction target included in the second phonetic character string based on an accent correction dictionary in which the pronunciation of a word is associated with an accent corresponding to the first accent rule of the word, and corrects the accent of the accent correction target. Specifically, the accent correction unit 9 obtains a word dictionary used as the accent correction dictionary from the word dictionary storage unit 11, and corrects the accent of the second phonetic character string 103 output from the conversion unit 3.

[0076] In the second embodiment, the accent correction unit 9 is added, which makes it easier to control accent conversion.

[0077] The phrase boundary correction rule storage unit 10 stores phrase boundary correction rules that associate a character string to be corrected at a phrase boundary with a character string after the phrase boundary has been corrected.

[0078] The accent correction dictionary storage unit 11 stores words to be corrected for accent and the corrected words.

[0079] FIG. 17 is a diagram showing an example of phrase boundary correction rules in the second embodiment. It shows an example in which a phrase boundary erroneously divided into "hon nara" and "ya ro" is corrected by joining "hon nara" and "ya ro" together. FIG. 18A is a flowchart showing an example of processing by the boundary correction unit 8 in the second embodiment. FIG. 18B is a diagram for explaining an example of processing by the boundary correction unit 8 in the second embodiment. In the examples of FIGS. 17 to 18B, consider a case in which input text data 101 is analyzed by the standard language analysis unit 2 and a first phonetic character string 102 is output. In this case, first, the boundary correction unit 8 receives input of the first phonetic character string 102 (step S51).

[0080] Next, the correcting unit 8 obtains a list of character strings to be corrected, including the correction targets 105 and 106 (see FIG. 17), from the phrase boundary correction rule storage unit 10 (step S52). Next, the boundary correcting unit 8 obtains the correction target 105 from the list of character strings to be corrected, and deletes the correction target 105 from the list (step S53).

[0081] Next, the boundary correction unit 8 confirms that the correction target 105 is included in the first phonetic character string 101 (step S54, Yes).

[0082] Next, the boundary correction unit 8 replaces the correction target 105 in the first phonetic character string 102 with the temporary symbol "#Symbol1#" (step S55). Similarly, the boundary correction unit 8 replaces the correction target 106 in the phonetic character string 107 being corrected with the temporary symbol "#Symbol2#", thereby obtaining the phonetic character string 108 being corrected.

[0083] Then, when the boundary correction unit 8 confirms that the list of character strings to be corrected is empty (step S56, Yes), it outputs the corrected phonetic character string 109 with the phrase boundary corrected by replacing the provisional symbol with the corrected character string (see Figure 17) (step S57).

[0084] Fig. 19 is a diagram showing an example of a word dictionary in the accent correction dictionary storage unit 11 of the second embodiment. Fig. 20A is a flowchart showing an example of processing by the accent correction unit 9 of the second embodiment. Fig. 20B is a diagram for explaining an example of processing by the accent correction unit 9 of the second embodiment.

[0085] First, the accent correction unit 9 receives the input of the second phonetic character string 103 (see FIG. 20B) of the first accent rule (step S61). Next, the accent correction unit 9 acquires a list of character strings to be corrected, including the character string to be corrected 110, from the accent correction dictionary storage unit 11 (step S62).

[0086] Next, the accent correction unit 9 acquires the correction target 110 from the correction target character string list and deletes the correction target 110 from the list (step S63). Next, the accent correction unit 9 confirms that the correction target 110 is included in the second phonetic character string 103 (step S64). Next, the accent correction unit 9 replaces the correction target 110 in the second phonetic character string 103 with the temporary symbol "#Symbol1#" (step S65).

[0087] Then, when the accent correction unit 9 confirms that the list of character strings to be corrected is empty (step S66, Yes), it outputs the corrected phonetic character string 112 with the accent corrected by replacing the temporary symbol with the corrected character string (see Figure 19) (step S67).

[0088] As described above, according to the speech synthesis device 1-2 of the second embodiment, phrase boundary estimation errors occurring in the standard language analysis unit 2 can be corrected by the boundary correction unit 8, and accent estimation errors occurring in the conversion unit 3 can be corrected by the accent correction unit 9.

[0089] Finally, an example of the hardware configuration of the speech synthesis device 1 (1-2) according to the first and second embodiments will be described. The speech synthesis device 1 (1-2) according to the first and second embodiments can be realized by using, for example, any computer device as basic hardware.

[0090] [Example of hardware configuration] 21 is a diagram showing an example of the hardware configuration of the speech synthesizer 1 (1-2) of the first and second embodiments. The speech synthesizer 1 (1-2) of the first and second embodiments includes a processor 201, a main storage device 202, an auxiliary storage device 203, a display device 204, an input device 205, and a communication device 206. The processor 201, the main storage device 202, the auxiliary storage device 203, the display device 204, the input device 205, and the communication device 206 are connected via a bus 210.

[0091] The speech synthesizer 1 (1-2) may not have some of the above components. For example, if the speech synthesizer 1 (1-2) can use the input function and display function of an external device, the speech synthesizer 1 (1-2) may not have the display device 204 and the input device 205.

[0092] The processor 201 executes a program read from the auxiliary storage device 203 to the main storage device 202. The main storage device 202 is a memory such as a ROM and a RAM. The auxiliary storage device 203 is a hard disk drive (HDD), a memory card, or the like.

[0093] The display device 204 is, for example, a liquid crystal display. The input device 205 is an interface for operating the speech synthesizer 1 (1-2). The display device 204 and the input device 205 may be realized by a touch panel or the like having a display function and an input function. The communication device 206 is an interface for communicating with other devices.

[0094] For example, the program executed by the speech synthesizer 1 (1-2) is provided as a computer program product in the form of a file in an installable or executable format recorded on a computer-readable storage medium such as a memory card, hard disk, CD-RW, CD-ROM, CD-R, DVD-RAM, or DVD-R.

[0095] Furthermore, for example, the program executed by the speech synthesizer 1 (1-2) may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network.

[0096] Alternatively, for example, the program executed by the speech synthesizer 1 (1-2) may be provided via a network such as the Internet without being downloaded. Specifically, the risk assessment process may be performed by a cloud service of, for example, an ASP (Application Service Provider) type.

[0097] Also, for example, the program for the speech synthesizer 1 (1-2) may be provided in a state where it is pre-installed in a ROM or the like.

[0098] The program executed by the speech synthesizer 1 (1-2) has a modular configuration that includes functions that can be realized by the program among the above-mentioned functional configurations. As for each function, as actual hardware, the processor 201 reads the program from a storage medium and executes it, and the above-mentioned functional blocks are loaded onto the main memory device 202. In other words, the above-mentioned functional blocks are generated on the main memory device 202.

[0099] Note that some or all of the above-described functions may be realized by hardware such as an integrated circuit (IC) rather than by software.

[0100] Furthermore, each function may be realized using a plurality of processors 201, and in this case, each processor 201 may realize one of the functions, or may realize two or more of the functions.

[0101] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0102] 1. Speech synthesizer 2 Standard Language Language Analysis Department 3. Conversion section 4. Speech synthesis unit 5 Standard language dictionary storage section 6 Conversion rule memory 7 Learning Device 8 Boundary correction section 9 Accent Correction Section 10 Phrase boundary modification rule storage 11 Accent correction dictionary memory section 31 Estimation part 32 Decision Section 41 Phonetic string analysis section 42 Prosody generation section 43 Waveform generator 201 processor 202 Main storage 203 Auxiliary storage device 204 Display device 205 Input Device 206 Communication Equipment 210 Bus

Claims

1. a standard language analysis unit that estimates pronunciation information and accent information of accent phrases obtained by dividing the input text using a standard language analysis dictionary, and generates a first phonetic string with a standard accent from the pronunciation information and accent information; a conversion unit that converts accent information of an accent phrase of the first phonetic character string into accent information corresponding to a first accent rule that is different from the standard accent rule, and generates a second phonetic character string corresponding to the first accent rule from the accent information corresponding to the first accent rule; a speech synthesis unit that synthesizes speech from the second phonogram character string, the conversion unit holds a neural network trained from pair data of the first phonetic character string and the second phonetic character string as accent conversion rules, and converts accent information based on the accent conversion rules; The conversion unit an estimation unit that estimates accent pitch information for each phonetic sound using the neural network from the first phonetic string and context information for each phonetic sound divided into units that determine the pitch of the sound; a determination unit that determines accent information corresponding to the first accent rule from the pitch information; A speech synthesis device comprising:

2. the unit of sound is a phoneme, a mora, or a syllable, The context information includes at least one of phonetic information of the phone, phonetic information before and after the phone, accent pitch information, the position of an accent phrase, the presence or absence of a punctuation mark, the presence or absence of an exclamation mark, and the presence or absence of a question mark. The speech synthesis device according to claim 1 .

3. the accent pitch information for each single note includes a continuous value representing the degree of pitch of the note; The speech synthesis device according to claim 1 .

4. The neural network is a neural network trained using a feedforward, recurrent, transformer, or bidirectional long short term memory (LSTM) model. The speech synthesis device according to claim 1 .

5. the determination unit determines a position where the accent pitch of each phone estimated by the estimation unit changes from a high pitch phone to a low pitch phone as a position of an accent kernel, thereby generating a second phonetic string corresponding to the first accent rule. The speech synthesis device according to claim 1 .

6. The first accent rule is the accent rule for a particular dialect accent, a speaker with a distinctive accent, or a character with a distinctive accent. The speech synthesis device according to claim 1 .

7. a boundary correcting unit that corrects a boundary of an accent phrase included in the first phonetic string, the boundary correction unit generates a corrected phonetic character string in which a boundary of a prosodic phrase included in the first phonetic character string is corrected based on a rule for correcting a boundary of a prosodic phrase, and inputs the corrected phonetic character string to the conversion unit. The speech synthesis device according to claim 1 .

8. an accent correction unit that identifies an accent correction target included in the second phonetic string based on an accent correction dictionary in which the pronunciation of a word is associated with an accent corresponding to a first accent rule of the word, and corrects the accent of the accent correction target; The speech synthesizer of claim 1 , further comprising:

9. a step in which a standard language analysis unit estimates reading information and accent information of an accent phrase obtained by dividing the input text using a standard language analysis dictionary, and generates a first phonetic string with a standard accent from the reading information and accent information; a conversion unit converting accent information of an accent phrase of the first phonetic character string into accent information corresponding to a first accent rule different from the standard accent rule, and generating a second phonetic character string corresponding to the first accent rule from the accent information corresponding to the first accent rule; a speech synthesis unit synthesizing speech from the second phonetic character string; the step of generating the second phonetic character string includes a step of retaining a neural network trained from pair data of the first phonetic character string and the second phonetic character string as accent conversion rules, and converting accent information based on the accent conversion rules; The step of converting the accent information includes: a step of estimating accent pitch information for each phonetic sound by using the neural network from the first phonetic string and context information for each phonetic sound divided into units that determine the pitch of the sound; determining accent information corresponding to the first accent rule from the pitch information; A speech synthesis method comprising:

10. Computer, a standard language analysis unit that estimates pronunciation information and accent information of accent phrases obtained by dividing the input text using a standard language analysis dictionary, and generates a first phonetic string with a standard accent from the pronunciation information and accent information; a conversion unit that converts accent information of an accent phrase of the first phonetic character string into accent information corresponding to a first accent rule that is different from the standard accent rule, and generates a second phonetic character string corresponding to the first accent rule from the accent information corresponding to the first accent rule; a speech synthesis unit that synthesizes speech from the second phonogram character string; the conversion unit holds a neural network trained from pair data of the first phonetic character string and the second phonetic character string as accent conversion rules, and converts accent information based on the accent conversion rules; The conversion unit an estimation unit that estimates accent pitch information for each phonetic sound using the neural network from the first phonetic string and context information for each phonetic sound divided into units that determine the pitch of the sound; a determination unit that determines accent information corresponding to the first accent rule from the pitch information; A program that includes:

Images