Emphasis position prediction device, emphasis position prediction method, and program

Abstract

In the present invention, an emphasis position for voice synthesis is automatically predicted not in units of sentences but from a document. To this end, an emphasis position prediction device according to the present disclosure comprises a document division section, a feature value extraction section, a positioning identification section, and a plurality of emphasis prediction sections. The document division section divides a document into prediction units, which are units for emphasis prediction. The feature value extraction section extracts a feature value of each prediction unit. The positioning identification section identifies what kind of component of the document each prediction unit is, and selects one of the plurality of emphasis prediction sections on the basis of identified component information. The emphasis position prediction device inputs the feature value to the emphasis prediction section selected by the positioning identification section, and outputs the presence or absence of emphasis in each prediction unit and an emphasis position.

Description

[Technical Field]

[0001] When synthesizing speech from a given text, adding prosodic emphasis at appropriate positions can make the synthesized speech more expressive. The disclosed technology relates to a technique for automatically predicting the positions of the emphasis based on the linguistic information of the text. [Background technology]

[0002] The naturalness of speech synthesis for news and informational texts has been improved, and in practice There is a demand for expanding the scope of application of speech synthesis, and research is being conducted to synthesize speech with a natural tone of voice (expressive speech synthesis) for various situations where speech is spoken to a person, such as in automated telephone answering and entertainment.

[0003] Synthesized speech still contains local imperfections, such as stressed or phrase-final intonation, which are characteristic of expressive speech synthesis. It has been shown that introducing labels indicating the location of stressed or phrase-final intonation into the context information of hidden Markov model-based speech synthesis methods can improve the synthesized speech. However, a technology for automatically predicting the location of stressed or phrase-final intonation from text has yet to be established.

[0004] Non-Patent Document 1 discloses a method for automatically predicting emphasis positions from text. Figure 1 shows a schematic diagram of the disclosure of Non-Patent Document 1. A sentence is input to the emphasis position prediction device 101 . The feature extraction unit 102 breaks down a sentence into accent phrases, and extracts the appearance form and part of speech of the main content word in the accent phrase and / or the appearance form and part of speech of the main function word as features. The emphasis prediction unit 103 receives the above feature amount as an input and predicts whether or not the accent phrase is emphasized.

[0005] An accent phrase is a tone unit when several words are spoken as a group, and consists of one or more words, with the entire accent phrase having at most one accent nucleus. For example, the sentence "I didn't give Jiro my glasses" can be broken down into three accent phrases: [Jiro ni] [Megane wo] [Agamenai]. Content words and function words are broadly classified into two categories: content words are words that express actual content, such as nouns, adjectives, verbs, and adverbs, while function words are words that modify or connect content words, such as particles, auxiliary verbs, and conjunctions. Furthermore, the emphasis prediction unit in Non-Patent Document 1 is constructed by machine learning using a support vector machine so as to function as a classifier for determining whether or not an accent phrase is emphasized.

[0006] The output of the emphasis position prediction device is input together with the sentence to a speech synthesizer 104, which outputs synthetic speech. The automatically predicted emphasis positions in the text are used as context information for a speech synthesis method based on a hidden Markov model, for example. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Hideharu Nakajima et al., "Prediction of Emphasized Accent Phrases from Advertising Texts for Expressive Text-to-Speech Synthesis," Transactions of Information Processing Society of Japan, Vol. 56, No. 12, pp. 2384-2394, 2015 Summary of the Invention [Problem to be solved by the invention]

[0008] The method in Non-Patent Document 1 predicts emphasis positions on a sentence-by-sentence basis, and is unable to make predictions that take into account the preceding and following sentences or the document structure. For example, the same word or word sequence may appear in both the title and body of a document, and may be unemphasized in the title but emphasized in the body. Unless input that can adequately distinguish between emphasis and non-emphasis is provided, the system may fluctuate between the two prediction targets, emphasis and non-emphasis, and learning may not converge. As the scale of computers has increased, it has become possible to input all the sentences that make up a document at once, but this method relies solely on the machine learning, which is not very reliable. [Means for solving the problem]

[0009] In order to solve the above problem, an emphasis position prediction device according to the disclosed technique includes a document division unit, a feature extraction unit, a positioning identification unit, and a plurality of emphasis prediction units. The document division unit divides the document into prediction units, which are units of emphasis prediction. The feature extraction unit extracts features of the prediction unit. The positioning specification unit specifies what kind of constituent element of the document the prediction unit is, and selects one of the plurality of emphasis prediction units based on the specified constituent element information. The emphasis position prediction device then inputs the feature amount to the emphasis prediction section selected by the positioning specification section, and outputs the presence or absence of emphasis in the prediction unit and the emphasis position. [Effects of the Invention]

[0010] According to the disclosed technology, by utilizing the structure of a document, such as whether it is a title or not, whether it is a bulleted list or not, etc., it is possible to make emphasis predictions using not only a sentence but also the preceding and following sentences in the document and the document structure as input. For example, if the same word or word sequence appears in both the title and the body of a document and is unemphasized in the title but emphasized in the body, if input sufficient to distinguish between emphasis and non-emphasis is not provided, the prediction may fluctuate between the two prediction targets, emphasis and non-emphasis, and learning may not converge. However, by identifying whether the part to be predicted is the title or the body and using different prediction models depending on whether it is the title or the body, it is possible to make predictions that make appropriate distinctions. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram schematically illustrating the technique disclosed in Non-Patent Document 1. [Figure 2] FIG. 1 is a functional block diagram of an emphasis position prediction device according to a first embodiment. [Figure 3] FIG. 6 is a flowchart showing the operation of a positioning identification unit according to the first embodiment. [Figure 4] FIG. 10 is a flowchart showing the operation of a positioning identification unit according to the second embodiment. [Figure 5] FIG. 11 is a flowchart showing a detailed example of a step of determining the relationship between a title and a body according to the second embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the functional configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the disclosed technology will be described in detail. Note that components having the same functions are assigned the same numbers, and duplicated descriptions will be omitted.

[0013] [First embodiment] FIG. 2 is a functional block diagram of an emphasis position prediction device 201 according to the first embodiment. The emphasis position prediction device includes a document division unit 202, a positioning identification unit 203, a feature extraction unit 204, and first emphasis prediction units 205 to Nth emphasis prediction units 207, and takes a document as input and outputs the prediction result of the emphasis position. The emphasis prediction unit is selectively used depending on what kind of constituent element in the document the prediction unit is. In the first embodiment, N=5 will be described.

[0014] <Document division section> The document segmentation unit 202 segments an input document into prediction units. Note that prediction units extracted from a document are generally sentences, but prediction units extracted from the document title, table of contents, headings, itemized items, contact information at the end of the document, etc., include phrases, clauses, and other forms that do not form complete sentences.

[0015] <Positioning Identification Section> The positioning identification unit 203 identifies what kind of document component the prediction unit is (title, table of contents, heading, body text, itemized items, contact information at the end of the document, etc.). FIG. 3 is a flowchart illustrating an example of the operation of the positioning identification unit 203. First, the positioning identification unit 203 acquires a prediction unit from the document division unit 202 (step S301). Next, the prediction unit is identified as a component of the document. The identification is performed, for example, by sequentially determining whether or not it corresponds to a specific element, but the order of the determination is not limited to the example below. Furthermore, the elements to be determined as corresponding or not are not limited to the example below.

[0016] First, it is determined whether the prediction unit is a title (step S302). This can be determined, for example, from the position within the document. If the result is Yes, the positioning identification unit 203 outputs an instruction to select the first emphasis prediction unit (step S306). In this case, the first emphasis prediction unit is equipped with an emphasis prediction model for titles. If the result is No, the process proceeds to determining the next element.

[0017] Next, it is determined whether the prediction unit is "a normal sentence of the main text" (step S303). This can be determined, for example, by whether the prediction unit ends with the final form of a verbal suffix or a specific final particle such as "ne" or "ka". If the result is Yes, the positioning specification unit 203 outputs a selection instruction for the second emphasis prediction unit (step S307). In this case, the second emphasis prediction unit is equipped with an emphasis prediction model for the main text. If the result is No, the process proceeds to determining the next element.

[0018] Next, it is determined whether the prediction unit is a "bullet point" (step S304). For example, an existing tool such as text2html can be used to convert an input document whose structure is not clearly stated into an HTML document, or to extract document structure information contained in a document file created with word processing software, and then analyze the information to determine whether it is a bullet point. For example, in an HTML document, a bullet is placed at the beginning as a tag to represent a bullet point. , with a number at the beginning These specify the range of the bullet points, and each bullet point is These can be used to determine whether a section is a bulleted item. If the result is Yes, the positioning identification unit 203 outputs an instruction to select the third emphasis prediction unit (step S308). In this case, the third emphasis prediction unit is provided with an emphasis prediction model for itemization. If the result is No, the process proceeds to determining the next element.

[0019] Next, it is determined whether the prediction unit is "contact information at the end of the document" (step S305). This can be determined, for example, from the presence or absence of keywords such as "contact information" or "contact details," or from addresses and telephone numbers that can be extracted using named entity extraction technology. Details of named entity extraction technology are described, for example, in "Digital Language Processing Encyclopedia," edited by the Association for Natural Language Processing, p. 354, Kyoritsu Shuppan, 2010" (Reference 1). If the result is Yes, the positioning identification unit 203 outputs an instruction to select the fourth emphasis prediction unit (step S309). In this case, the fourth emphasis prediction unit includes an emphasis prediction model for contact information at the end of the document.

[0020] If the result of step S305 is No, the positioning identification unit 203 outputs an instruction to select the "fifth emphasis prediction unit" equipped with an emphasis prediction model for "other elements" (step S310).

[0021] <Feature Extraction Unit> The feature extraction unit 204 extracts features of the prediction unit. For example, following Non-Patent Document 1, the prediction unit is decomposed into accent phrases, and the appearance form and part of speech of the main content word in the accent phrase and / or the appearance form and part of speech of the main function word may be extracted as features.

[0022] <Emphasis prediction section> For each prediction unit, the feature extracted by the feature extraction unit 204 is input to an emphasis prediction unit selected by the positioning specification unit 203, and the presence or absence and position of an emphasis part in the prediction unit are output. The emphasis prediction unit, for example, breaks down a prediction unit (such as a sentence) into accent phrases and outputs a prediction result of whether or not emphasis is applied to each accent phrase.

[0023] The learning model provided in each emphasis prediction unit may be constructed by machine learning using a support vector machine, as in Non-Patent Document 1, using specific components of a document as training data, or may use another two-class pattern classifier. In the first embodiment, an example was shown in which feature values ​​were extracted by decomposing a document into accent phrases and then used to predict the emphasis position, as in Non-Patent Document 1, but the feature extraction method and emphasis position prediction method are not limited to this. Features may also be extracted and emphasis position prediction may be performed for word units smaller than accent phrases.

[0024] [Second embodiment] In the first embodiment, the prediction unit was identified as a component of the document (title, table of contents, main text, bullet points, contact information at the end of the document, etc.), and an emphasis prediction unit optimized for the component was selected and used. For example, the title of a document may be completely unrelated to the main text, or it may be a good summary of the main text. Also, bullet points may be the main message of the document, or they may be merely a list of data that does not convey the intended message of the document. In this way, it is not necessarily appropriate to group prediction units that are actually quite different in nature together based solely on their positioning within a document, such as "title" or "bullet points," and apply them to the same emphasis prediction model. Therefore, in the second embodiment, prediction units are further classified according to the characteristics and roles of the components identified for the prediction units, and an emphasis prediction unit optimized for the classification results is selected and used.

[0025] The functional block diagram of the emphasis position prediction device according to the second embodiment is similar to that of the first embodiment, but differs in the operation of the positioning identification unit 203. Therefore, the following description will mainly focus on the operation of the positioning identification unit 203. In addition, the second embodiment will be described assuming that N=7 for the Nth emphasis prediction unit.

[0026] <Positioning Identification Section> FIG. 4 is a flowchart illustrating an example of the operation of the positioning identification unit 203 according to the second embodiment. As in the first embodiment, the elements are identified by sequentially determining whether or not they correspond to a specific element, but the order of the determination is not limited to the example below, and the viewpoint of the determination is not limited to the example below. Step S301 is the same as in the first embodiment.

[0027] If the result of step S302, which determines whether the prediction unit is a title, is Yes, in the second embodiment, the relationship between the prediction unit and the main text is further determined (step S401). If the result of step S401 is No (if the relationship between the title and the main text is not recognized), the positioning identification unit 203 outputs a selection instruction for the first emphasis prediction unit equipped with an emphasis prediction model for titles equivalent to that of the first embodiment. If the result of step S401 is Yes (if the relationship between the title and the main text is recognized), the positioning identification unit 203 outputs a selection instruction for the sixth emphasis prediction unit that makes predictions using a model according to the relationship with the main text.

[0028] Steps S303 and S307 are the same as in the first embodiment.

[0029] If the result of step S304, which is used to determine whether the itemized section is a bulleted section, is Yes, in the second embodiment, it is further determined whether the prediction unit is a main message (step S402). If the result of step S402 is No (if the contents of the bulleted section are not recognized as the main message of the document), the positioning identification unit 203 outputs a selection instruction to a third emphasis prediction unit equipped with an emphasis prediction model for bulleted sections equivalent to that of the first embodiment. If the result of step S402 is Yes (if the contents of the bulleted section are recognized as the main message of the document), the positioning identification unit 203 outputs a selection instruction to a seventh emphasis prediction unit that predicts using a model for the main message.

[0030] Steps S305, S309, and S310 are the same as those in the first embodiment.

[0031] <Additional information about step S401> Note that step S401, which further determines the relationship between the prediction unit and the main text, may be a multi-branch decision rather than a binary decision, and one of a plurality of prediction models may be selected depending on the degree of relationship between the title and the main text. For example, the more appropriate the title is as a summary of the main text (there is no more information in the main text than the title), the more likely it is that a prediction model that tends to emphasize the title will be selected, and the narrower the range written in the main text is compared to the range predicted from the title, the more likely it is that a prediction model that tends to not emphasize the title will be selected.

[0032] The determination of the relationship between the title and the text and the selection of the emphasis position prediction model to be used can be made based on, for example, a comparison of the title with the term at the end of the title or the summary result of the text other than the title. FIG. 5 is a flowchart showing a specific example of step S401 for determining the relationship between the body of the text and a prediction unit determined to be a "title" in the document structure. The positioning identification unit 203 inputs the text into a summarizer (step S501). Details of the summarizer are described, for example, in "Digital Natural Language Processing Encyclopedia," edited by the Association for Natural Language Processing, Kyoritsu Shuppan, 2010, p. 340 (Reference 1). Next, first, it is determined whether the structure of the prediction unit (title) is phrase-form and ends with a keyword indicating the theme or the reader, such as "about" or "to all" (step S502). If yes, it is considered that specific information is contained in the main text, so a non-emphasis model is specified (the selection of the first emphasis prediction unit is instructed) (step S503). If the result of step S502 is No, the positioning identification unit 103 compares the text summary result of step S501 with the prediction unit (title) and determines whether the content word matching rate exceeds a predetermined rate (step S504). Note that the predetermined rate can be set to an appropriate value such as "70%." If the result of step S502 is Yes, the title is considered to contain sufficient information about the text, so an emphasis model is specified (selection of the sixth emphasis prediction unit is instructed) (step S505). If the answer is No in step S504, it is determined that the relationship between the prediction unit (title) and the main text is weak, and a non-emphasis model is specified (selection of the first emphasis prediction unit is instructed) (step S506).

[0033] <Additional information about step 402> There are various possible methods and criteria for determining whether the prediction unit is the main message in step 402. For example, based on the temporal information obtained from predicates such as "will be held" and the date and time information written in the bullet points, it is determined whether the event written in the bullet points is to be held in the future, and if the answer is Yes, the prediction unit is determined to be the main message. If the answer is No, the content of the bullet points is considered to be a report of a past event, so the prediction unit is determined not to be the main message. For example, keywords related to "advice" such as "do it" and "don't do it" are defined, and it is determined whether the prediction unit contains the keyword. If yes, it is determined to be the main message, and if no, it is determined to be not the main message.

[0034] [Uses of emphasis positions other than speech synthesis] The emphasis positions predicted in the above embodiments can be used as context information for speech synthesis, and can also be used to decorate text (bold, color, larger font, underline, etc.) and visually present them to the reader.

[0035] [Programs, recording media] The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 6 and operating the control unit 2010, input unit 2030, output unit 2040, etc.

[0036] The program describing the processing contents can be recorded on a computer-readable recording medium, which may be, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, or any other suitable recording medium.

[0037] The program may be distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and then transferred from the server computer to another computer via a network, thereby distributing the program.

[0038] A computer that executes such a program may first temporarily store the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, the computer may execute the process in accordance with the received program each time a program is transferred from a server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function by issuing an execution instruction and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that is not a direct instruction to the computer but has properties that define computer processing).

[0039] Furthermore, in this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

Claims

1. An apparatus for predicting emphasis positions in a document, comprising: a document segmentation unit that segments a document into prediction units that are units of emphasis prediction; a feature extraction unit that extracts features of the prediction unit; a plurality of enhancement prediction units; a positioning specification unit that specifies what kind of component of a document the prediction unit is, and selects one of the plurality of emphasis prediction units based on the specified component information, The feature amount is input to the emphasis prediction unit selected by the positioning specification unit, and the presence or absence of emphasis in the prediction unit and the emphasis position are output. Emphasis position prediction device.

2. 2. The emphasis position prediction device according to claim 1, The positioning specification unit classifies the prediction units from a viewpoint that is predetermined for each of the components, and selects the enhancement prediction unit based on the classification. Emphasis position prediction device.

3. 3. The emphasis position prediction device according to claim 2, The classification is performed based on the relationship between the content of the prediction unit and the main content of the document. Emphasis position prediction device.

4. 1. A method for predicting emphasis locations in a document using a plurality of emphasis predictors, comprising: A document division unit divides a document into prediction units, which are units of emphasis prediction; a step in which a positioning identification unit identifies what kind of component of a document the prediction unit is, and selects one of the plurality of emphasis prediction units based on the identified component information; a feature extraction unit extracting features of the prediction units; inputting the feature amount to the emphasis prediction unit selected by the positioning specification unit, and outputting the presence or absence of emphasis in the prediction unit and the emphasis position; The emphasis position prediction method includes:

5. A program for causing a computer to function as the emphasis position prediction device according to any one of claims 1 to 3.

Images