Voice processing support device, voice processing support method, and voice processing support program

Abstract

To provide parameters related to transition of emotions changing on a time axis that can be set in detail.SOLUTION: A voice processing support device 10 includes an input receiving unit 20B and a recording unit 20F. The input receiving unit 20B accepts input of parameters including at least a plurality of emotion types that differ from each other and a mixing ratio of the plurality of emotion types during playback of voice data to be edited. The recording unit 20F records the input-accepted parameter while being associated with playback timing at which the input of the parameter is accepted in the voice data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a voice processing support device, a voice processing support method, and a voice processing support program. [Background technology]

[0002] There is a known technology for synthesizing a voice with a different voice quality from an existing voice by mixing (morphing) voice data. For example, a technology has been disclosed for generating synthetic voice data by synthesizing multiple voice data based on a pre-specified morphing ratio.

[0003] However, with conventional techniques, although it is possible to set parameters such as a morphing ratio for the entire voice data, it is difficult to set detailed parameters relating to the transition of emotions that change on the time axis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 9-50295 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem that the present invention aims to solve is to provide a voice processing support device, a voice processing support method, and a voice processing support program that can provide detailed settings for parameters related to the progression of emotions that change over time. [Means for solving the problem]

[0006] According to an embodiment, a voice processing support device includes an input receiving unit and a recording unit. The input receiving unit receives input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of voice data to be edited. The recording unit records the input of the input parameters in association with a playback timing of the voice data at which the input of the parameters was received. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a voice processing support device according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram of an example of a display screen. [Diagram 3] FIG. 3 is a schematic diagram of an example of a display screen. [Figure 4] FIG. 4 is a schematic diagram of an example of a display screen. [Diagram 5] FIG. 5 is a schematic diagram of an example of a display screen. [Figure 6] FIG. 6 is a flowchart showing an example of the flow of information processing executed by the voice processing support device of this embodiment. [Figure 7] FIG. 7 is a diagram showing the hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a voice processing support device, a voice processing support method, and a voice processing support program will be described in detail with reference to the accompanying drawings.

[0009] FIG. 1 is a diagram showing an example of a voice processing support device 10 according to the present embodiment.

[0010] The voice processing support device 10 is an information processing device that supports the processing of voice data.

[0011] The voice processing support device 10 includes a communication unit 12, a UI (user interface) unit 14, a storage unit 16, and a processing unit 20. The communication unit 12, the UI unit 14, the storage unit 16, and the processing unit 20 are communicatively connected via a bus 18.

[0012] The communication unit 12 communicates with other external information processing devices via a network, etc. The UI unit 14 includes a display unit 14A and an input unit 14B.

[0013] The display unit 14A displays various types of information and is, for example, a display such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, a projection device, or the like.

[0014] The input unit 14B receives user operations. The input unit 14B is, for example, a pointing device such as a digital pen, a mouse, or a trackball, or an input device such as a keyboard. The display unit 14A and at least a part of the input unit 14B may be integrally provided as a touch panel. In this embodiment, the input unit 14B includes a first operation unit 14B1 and a second operation unit 14B2.

[0015] The first operation unit 14B1 is an input device for inputting an operation direction and an operation amount. The first operation unit 14B1 is, for example, an input device that allows input of an operation direction and an operation amount according to a tilt direction and a tilt angle by tilting a lever (stick). Such a first operation unit 14B1 may be called a joystick. The first operation unit 14B1 may be, for example, a touch pad. In this embodiment, a form in which the first operation unit 14B1 is a joystick will be described as an example.

[0016] The second operation unit 14B2 is an input device for inputting an amount of operation. The second operation unit 14B2 is, for example, a pedal-type or button-type input device that is pressed by the user's leg or the like. In this embodiment, the second operation unit 14B2 is described as an example of a pedal-type input device that inputs an amount of operation by pressing the user's leg or the like.

[0017] The audio output unit 14C is a speaker that outputs audio.

[0018] The storage unit 16 stores various data. The storage unit 16 is, for example, a semiconductor memory element such as a random access memory (RAM), a flash memory, a hard disk, an optical disk, or the like. The storage unit 16 may be a storage device provided outside the voice processing support device 10. The storage unit 16 may also be a storage medium. Specifically, the storage medium may store or temporarily store programs and various information downloaded via a local area network (LAN) or the Internet. The storage unit 16 may also be composed of multiple storage media.

[0019] Next, a description will be given of the processing unit 20. The processing unit 20 executes various types of information processing. The processing unit 20 includes a display control unit 20A, an input receiving unit 20B, a setting unit 20C, an acquisition unit 20D, a playback unit 20E, and a recording unit 20F.

[0020] The display control unit 20A, the input receiving unit 20B, the setting unit 20C, the acquisition unit 20D, the playback unit 20E, and the recording unit 20F are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), that is, by hardware. Each of the above units may be realized by using both software and hardware. When multiple processors are used, each processor may realize one of the units, or may realize two or more of the units.

[0021] At least one of the above units and at least a part of the information stored in the storage unit 16 may be mounted on a cloud server or the like that executes processing on the cloud.

[0022] The display control unit 20A displays various display screens on the UI unit 14. The display screens will be described in detail later.

[0023] The input receiving unit 20B receives operation inputs from the UI unit 14 by the user.

[0024] For example, assume that a user wishes to generate synthetic voice data using certain voice data. More specifically, assume that the user wishes to process and edit the voice data so as to generate synthetic voice data that includes a predetermined emotion at a predetermined intensity along a time axis.

[0025] In this case, the user operates the input unit 14B to set voice dictionary data corresponding to the type of emotion used for setting the voice data. In this embodiment, the input receiving unit 20B receives an input by the user via a display screen displayed on the display unit 14A.

[0026] 2 is a schematic diagram of an example of a display screen 30A. The display screen 30A is an example of the display screen 30 displayed on the display unit 14A.

[0027] Display screen 30A includes an emotion map M, an emotion setting section 40A, and a voice dictionary data setting section 40B.

[0028] The emotion map M is a map that represents the correlation of multiple types of emotions. For example, the emotion map M is a diagram that allows the expression of complex emotions in which emotions are mixed together by expressing the types of emotions with colors and combining them three-dimensionally. In the emotion map M, for example, the regions of each of the eight basic emotions, joy, trust, fear, surprise, sadness, disgust, anger, and expectation, are arranged so that they radiate in different directions from each other, centered on the emotionless region. In addition, in the emotion map M, the regions of the opposing types of emotions are arranged 180° opposite each other through the emotionless region arranged in the center. In addition, in the emotion map M, the eight basic emotions are classified into three groups, positive, negative, and neutral, and the regions of the types of emotions belonging to each group are mapped so that they are arranged in adjacent positions. In addition, the closer the region of each type of emotion shown in the emotion map M is to the emotionless region arranged in the center, the weaker the emotion, and the farther it is from the emotionless region, the stronger the emotion.

[0029] It should be noted that the emotion map M is not limited to the form shown in FIG. 2 as long as it is a map that indicates the correlation between a plurality of types of emotions.

[0030] Emotion setting field 40A is an input field for the type of emotion used to set the voice data. For example, a user operates input unit 14B while viewing emotion map M on display screen 30A to input the type of emotion desired to be added to the voice data into emotion setting field 40A. Because display screen 30A includes emotion map M, a user can easily input the type of emotion into emotion setting field 40A by viewing emotion map M included in display screen 30A.

[0031] The voice dictionary data setting field 40B is an input field for setting voice dictionary data corresponding to the type of emotion input in the emotion setting field 40A.

[0032] The speech dictionary data is an acoustic model for deriving acoustic features from linguistic features. The speech dictionary data is created in advance. The linguistic features are language features extracted from the text of the speech uttered by the speaker. For example, the linguistic features include surrounding phonemes, information on pronunciation, phrase-final position, sentence length, accent phrase length, mora length, mora position, accent type, part of speech, and dependency information. The linguistic features are sometimes referred to as linguistic information. The acoustic features are speech or acoustic features extracted from speech data. For example, the acoustic features may be acoustic features used in HMM (Hidden Markov Model) speech synthesis. For example, the acoustic features include Mel-cepstral coefficients, Mel-LPC coefficients, Mel-LSP coefficients, which represent phonemes and tone, Mel-LPC coefficients, Mel-LSP coefficients ...

[0033] In this embodiment, it is assumed that the storage unit 16 stores in advance a plurality of speech dictionary data for outputting acoustic features of one or a plurality of speakers each uttered with a different emotion.

[0034] The user operates the input unit 14B while viewing the display screen 30A to set the voice dictionary data to be used for the type of emotion input in the emotion setting field 40A. Specifically, for example, the user operates the input unit 14B to select voice dictionary data that is considered to correspond to the corresponding type of emotion from among a plurality of voice dictionary data stored in the storage unit 16, and sets the voice dictionary data in the voice dictionary data setting field 40B.

[0035] 2 shows an example of voice dictionary data with a file name including the name of a corresponding emotion type. However, the file name of the voice dictionary data may be in a form that does not include the name of a corresponding emotion type.

[0036] It is assumed that the speech dictionary data corresponding to the emotion type "neutral" is preset. The speech dictionary data corresponding to neutral is speech dictionary data for outputting acoustic features of a speaker speaking without any emotion.

[0037] Through these input operations by the user, the input receiving unit 20B of the processing unit 20 receives the setting of the type of emotion to be used for setting the voice data and the voice dictionary data corresponding to the type of emotion.

[0038] Returning to Figure 1, we will continue with the explanation.

[0039] The setting unit 20C sets the voice dictionary data corresponding to each type of emotion received via the display screen 30A as voice dictionary data to be used for editing the voice data. For example, the setting unit 20C stores the voice dictionary data corresponding to each type of emotion received via the display screen 30A in a specific storage area of ​​the storage unit 16.

[0040] The acquisition unit 20D acquires the voice data to be edited. The user operates the UI unit 14 to specify the voice data to be edited that is stored in the storage unit 16 or an external information processing device. The acquisition unit 20D acquires the voice data specified by the user's operation instruction on the UI unit 14 as the voice data to be edited.

[0041] In this embodiment, the input receiving unit 20B receives an input specifying the audio data to be edited via the display screen 30 displayed on the display unit 14A.

[0042] 3 is a schematic diagram of an example of the display screen 30B. The display screen 30B is an example of the display screen 30. The display screen 30B is the display screen 30B that is displayed on the UI unit 14 when voice data is designated and input of parameters is accepted.

[0043] When the voice dictionary data is set by a user's operation instruction on the input unit 14B, the display control unit 20A displays the display screen 30B on the display unit 14A.

[0044] Display screen 30B includes an audio data file name input display field 40C, a play button 40D, an audio waveform display field 40E, an emotion map M, a pointer 40F, a speech rate adjustment button 40G, a gain adjustment button 40H, an edit button 40J, a synthetic voice playback button 40K, and a save button 40L.

[0045] The audio data file name input / display field 40C is an input and display field for the file name of the audio data to be edited. The user operates the input unit 14B while viewing the display screen 30B to input the file name of the audio data to be edited in the audio data file name input / display field 40C. The acquisition unit 20D acquires the audio data with the input file name as the audio data to be edited. The user may also specify the audio data to be edited that is stored in the storage unit 16 or the like by operating the input unit 14B while viewing the display screen 30B. In this case, the acquisition unit 20D acquires the specified audio data as the audio data to be edited.

[0046] The user may input the file name of the text data to be edited by operating the input unit 14B while viewing the display screen 30B. The user may also specify the text data to be edited stored in the storage unit 16 or the like by operating the input unit 14B while viewing the display screen 30B.

[0047] In this case, the acquisition unit 20D acquires the voice data to be edited by generating voice data using the text data represented by the input file name or the specified text data and the voice dictionary data corresponding to the emotion type "unemotional" in a well-known manner.

[0048] In this embodiment, the processing unit 20 of the voice processing support device 10 accepts input of parameters during playback of voice data to be edited.

[0049] The parameters are parameters related to the transition of emotions that change on the time axis when synthetic voice data is generated from the voice data to be edited. In detail, the parameters include at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions. The parameters may further include at least one of the intensity of emotions, speech speed, and sound pressure level. In this embodiment, a form will be described as an example in which the parameters include a plurality of different types of emotions, a mixing ratio of the plurality of types of emotions, and the intensity, speech speed, and sound pressure level of each of the plurality of emotions.

[0050] When the user designates the audio data to be edited, he / she operates the playback button 40D to instruct playback of the audio data. When the input receiving unit 20B receives a playback instruction signal input by operating the playback button 40D, the playback unit 20E starts playing the audio data. Playing the audio data means outputting the audio represented by the audio data from the audio output unit 14C.

[0051] When the reproduction of the audio data is started, the display control section 20A preferably displays a waveform representing the volume of the audio data in the audio waveform display field 40E.

[0052] When the playback of the audio data starts and the output of the audio of the audio data from the audio output unit 14C starts, the user operates the input unit 14B while viewing the display screen 30B to input parameters for a desired playback timing. The playback timing means each timing during the playback of the audio data played back along the time axis. That is, the input receiving unit 20B receives the input of parameters at each playback timing during the playback of the audio data to be edited.

[0053] In detail, the user operates the input unit 14B while gazing at at least one of the pointer 40F on the emotion map M displayed on the display screen 30B, the speech rate adjustment button 40G, and the gain adjustment button 40H, thereby inputting parameters for the desired playback timing of the audio data being played back.

[0054] Emotion map M included in display screen 30B is similar to the above-mentioned emotion map M. A pointer 40F is shown on emotion map M. Pointer 40F indicates a point on emotion map M designated by the user.

[0055] For example, the user adjusts the position of the pointer 40F on the emotion map M by operating the first operation unit 14B1 which is a joystick. Specifically, for example, when the tilt direction and tilt angle of the joystick which is the first operation unit 14B1 are adjusted, the position of the pointer 40F shown on the emotion map M displayed on the display screen 30B moves in the tilt direction of the joystick by an amount according to the tilt angle. The user adjusts the position of the pointer 40F displayed on the emotion map M to a position corresponding to the type of desired emotion, the mixture ratio of the desired type of emotion, and the strength of the desired emotion by operating the first operation unit 14B1. The input receiving unit 20B receives the type of emotion, the mixture ratio of the multiple types of emotions, and the strength of the emotion, which are represented by the position of the pointer 40F on the emotion map M.

[0056] The speech rate and sound pressure level are adjusted by the respective positions of a speech rate adjustment button 40G and a gain adjustment button 40H included in the display screen 30B. The user adjusts the respective positions of the speech rate adjustment button 40G and the gain adjustment button 40H on the display screen 30B by, for example, operating a pedal-type second operation unit 14B2 with the user's leg or the like.

[0057] For example, second operation unit 14B2 includes a pedal corresponding to speech rate adjustment button 40G and a pedal corresponding to gain adjustment button 40H.

[0058] When the user adjusts the depression amount of the pedal that is the second operation unit 14B2 corresponding to the speech speed adjustment button 40G, the position of the speech speed adjustment button 40G displayed on the display screen 30B moves in the direction to increase or decrease the speech speed. The input receiving unit 20B receives an input of the speech speed according to the depression amount of the second operation unit 14B2 corresponding to the speech speed adjustment button 40G.

[0059] Similarly, when the user adjusts the depression amount of the pedal that is the second operation unit 14B2 corresponding to the gain adjustment button 40H, the position of the gain adjustment button 40H displayed on the display screen 30B moves in the direction to increase or decrease the sound pressure (gain). The input receiving unit 20B receives an input of the sound pressure according to the depression amount of the second operation unit 14B2 corresponding to the gain adjustment button 40H.

[0060] In this way, the user operates at least one of the first operation unit 14B1 and the second operation unit 14B2 at each desired playback timing during playback of the voice data, thereby inputting parameters including at least one of the type of desired emotion, the mixing ratio of multiple emotions, the intensity of the emotion, the speech rate, and the sound pressure level for the playback timing. Also, the input receiving unit 20B receives input of parameters at each playback timing during playback of the voice data.

[0061] Returning to Figure 1, we will continue with the explanation.

[0062] The recording unit 20F records the inputted parameters in association with the playback timing of the inputted parameters in the voice data when the inputted parameters are received. In detail, for example, the recording unit 20F stores the inputted parameters in association with a time stamp indicating the playback timing of the inputted parameters in the voice data when the inputted parameters are received. The recording unit 20F may record the inputted parameters in association with a position corresponding to the playback timing of the inputted parameters in the voice data when the inputted parameters are received.

[0063] The reproducing unit 20E generates synthetic voice data based on voice dictionary data corresponding to emotions according to parameters associated with each of the reproduction timings for the voice data to be edited.

[0064] Specifically, the reproducing unit 20E obtains an acoustic feature corresponding to each type of emotion by inputting the linguistic feature (linguistic information) at the reproduction timing of the voice data to be edited into the voice dictionary data corresponding to each of the types of emotions included in the parameters set for the reproduction timing. Then, the reproducing unit 20E obtains a first mixed acoustic feature by mixing the obtained acoustic feature corresponding to each type of emotion according to the mixing ratio of the emotions included in the parameters set for the reproduction timing. Furthermore, the reproducing unit 20E obtains a second acoustic feature corresponding to neutral emotion by inputting the linguistic feature at the reproduction timing of the voice data to be edited into the voice dictionary data corresponding to neutral emotion. Then, the reproducing unit 20E obtains the second mixed acoustic feature by mixing the second acoustic feature corresponding to neutral emotion and the first mixed acoustic feature in a ratio according to the intensity of the emotion included in the parameters set for the reproduction timing. In detail, the reproducing unit 20E obtains a second mixed acoustic feature by increasing the ratio of the second acoustic feature corresponding to neutral emotion as the intensity of the emotion is lower, and increasing the ratio of the first mixed acoustic feature as the intensity of the emotion is stronger. Then, the reproduction unit 20E generates synthetic speech, which is a speech waveform represented by the second mixed acoustic feature, as synthetic speech data at the reproduction timing.

[0065] The playback unit 20E generates synthetic voice data by synthesizing the voice data in accordance with the parameters by performing the above-mentioned process using the parameters set for each of the multiple playback timings along the time axis included in the voice data.

[0066] Then, recording unit 20F records the parameters used to generate the synthetic voice at each reproduction timing, in association with each reproduction timing in the generated synthetic voice data.

[0067] The playback unit 20E plays the synthetic voice data when the synthetic voice playback button 40K on the display screen 30B is operated by the user in response to an instruction from the input unit 14B. The playback unit 20E plays the synthetic voice data by outputting the generated synthetic voice data to the audio output unit 14C. Note that the playback unit 20E may generate and play the synthetic voice data when the synthetic voice playback button 40K is operated by the user in response to an instruction from the input unit 14B.

[0068] 4 is a schematic diagram of an example of display screen 30C. Display screen 30C is an example of display screen 30. Display screen 30C is the display screen 30 displayed on display unit 14A when synthetic voice data is played back. When a user operates and instructs input unit 14B to operate synthetic voice play button 40K, display control unit 20A displays display screen 30C on display unit 14A.

[0069] In addition to the display screen 30B, the display screen 30C includes a playback timing image 40I. The playback timing image 40I is an image that represents the current playback timing in the waveform representing the synthetic voice data displayed in the voice waveform display field 40E. Therefore, the display control unit 20A moves the display position of the playback timing image 40I so that it is displayed at a position corresponding to the current playback timing in the waveform representing the synthetic voice data as time passes due to the playback of the synthetic voice data.

[0070] In addition, it is preferable that the display control unit 20A adjusts the positions of the pointer 40F, the speech rate adjustment button 40G, and the gain adjustment button 40H on the display screen 30C so that the display positions correspond to the parameters set for each playback timing of the synthetic voice data.

[0071] As described above, the recording unit 20F records the parameters used to generate the synthetic voice at each playback timing in association with each playback timing in the synthetic voice data. During playback of the synthetic voice data, the display control unit 20A displays a pointer 40F at a position on the emotion map M that represents the type of emotion, the mixture ratio, and the intensity of the emotion represented by the parameters recorded in association with the current playback timing. During playback of the synthetic voice data, the display control unit 20A also displays a speech speed adjustment button 40G and a gain adjustment button 40H at positions that represent the speech speed and gain represented by the parameters recorded in association with the current playback timing.

[0072] A user may wish to edit a parameter. In this case, the user operates the input unit 14B to instruct the edit button 40J on the display screen 30B. When the edit button 40J is instructed, the input reception unit 20B starts receiving the parameter edit. As described above, the input unit 14B is an input device such as a pointing device such as a digital pen, a mouse, or a trackball, or a keyboard. The input unit 14B may include a first operation unit 14B1 such as a joystick, and a second operation unit 14B2 such as a pedal. For this reason, the operation input of the parameter by the user is not limited to the first operation unit 14B1 such as a joystick, and the second operation unit 14B2 such as a pedal, but may be input by simultaneously operating one or more of a pointing device such as a mouse, a digital pen, or a dragball, and a keyboard.

[0073] In detail, the user operates the input unit 14B to select an editing point to be edited in the waveform representing the voice data displayed in the voice waveform display field 40E included in the display screen 30B. Then, the user operates the input unit 14B while gazing at at least one of the pointer 40F on the emotion map M displayed on the display screen 30B, the speech rate adjustment button 40G, and the gain adjustment button 40H, thereby editing the parameters associated with the editing point. The operations for editing these parameters are the same as those for inputting parameters for voice data.

[0074] That is, the user operates at least one of the first operation unit 14B1 and the second operation unit 14B2 to edit at least one parameter of the desired emotion type, the mixing ratio of multiple emotions, the emotion intensity, the speech rate, and the sound pressure level for the selected editing point. Also, the input receiving unit 20B receives an input for editing the parameters at the selected editing point.

[0075] The recording unit 20F records the parameters for which editing has been accepted, attached to the selected editing point in the synthetic voice data. In detail, for example, the recording unit 20F stores the parameters for which editing input has been accepted, in association with a timestamp representing the selected editing point in the synthetic voice data. Note that the recording unit 20F may record the parameters for which editing input has been accepted, in association with a position in the synthetic voice data that corresponds to the selected editing point.

[0076] The reproducing unit 20E then regenerates the synthetic voice data according to the edited parameters. The reproducing unit 20E may generate synthetic voice data according to the edited parameters in the same manner as generating synthetic voice data according to voice data with parameters. The recording unit 20F records the regenerated synthetic voice data in association with the parameters set for each playback timing.

[0077] When the user has finished inputting the parameters, he or she operates the save button 40L. The save button 40L is a button that is operated by the user when instructing storage of the synthetic voice data generated from the voice data for which the parameters have been set in the storage unit 16. When the save button 40L is operated, the input receiving unit 20B receives the save instruction.

[0078] When input accepting unit 20B accepts the save instruction, display control unit 20A displays display screen 30 on display unit 14A for accepting input of text information for the synthetic voice data.

[0079] 5 is a schematic diagram of an example of a display screen 30D. The display screen 30D is an example of the display screen 30. The display screen 30D is the display screen 30 that is displayed when the save button 40L is operated by the user.

[0080] When the user operates the save button 40L in response to an instruction from the input unit 14B, the display control unit 20A displays on the display unit 14A a display screen 30D in which a character information input field 40M is superimposed on the display screen 30C. The character information input field 40M is an input field for character information to be added to the synthetic voice data. For example, the user operates the input unit 14B to input character information such as an explanation for the synthetic voice data into the character information input field 40M.

[0081] Input receiving unit 20B receives character information for synthetic voice data via character information input field 40M. Recording unit 20F stores the received input character information in storage unit 16 in association with the synthetic voice data.

[0082] Through these processes, text information that explains one piece of synthetic speech data is stored in association with the synthetic speech data. This allows users of the synthetic speech data to check the text information, enabling effective reuse of the synthetic speech data. In addition, by using the synthetic speech data and the text information added to the synthetic speech data as learning data, it is possible to generate a learning model for outputting the text information from the synthetic speech data as a correct label.

[0083] Next, an example of the flow of information processing executed by the voice processing support device 10 of the present embodiment will be described.

[0084] FIG. 6 is a flowchart showing an example of the flow of information processing executed by the voice processing support device 10 of the present embodiment.

[0085] The display control unit 20A displays the display screen 30A on the display unit 14A (step S100). For example, when a signal indicating the start of editing of voice data is input by a user's operation instruction on the input unit 14B, the display control unit 20A displays the display screen 30A for accepting settings of the type of emotion and voice dictionary data on the display unit 14A.

[0086] The user operates the input unit 14B while viewing the display screen 30A to set the type of emotion to be used for the voice data and the voice dictionary data to be used for the type of emotion. The input receiving unit 20B receives the setting of the type of emotion to be used for the voice data and the voice dictionary data corresponding to the type of emotion (step S102).

[0087] The setting unit 20C sets the voice dictionary data corresponding to each of the types of emotions received via the display screen 30A as voice dictionary data to be used for editing the voice data (step S104).

[0088] The display control unit 20A displays the display screen 30B for accepting parameter settings for the voice data on the display unit 14A (step S106). By the process of step S106, the display screen 30B shown in FIG.

[0089] The user operates the input unit 14B while viewing the display screen 30B to input the file name of the audio data to be edited in the audio data file name input / display field 40C. The acquisition unit 20D acquires the audio data with the input file name as the audio data to be edited (step S108).

[0090] The user operates the play button 40D to instruct playback of the audio data. When the input receiving unit 20B receives a playback instruction signal input by operating the play button 40D, the playback unit 20E starts playing the audio data acquired in step S108 (step S110).

[0091] When the reproduction of the voice data is started, the user, while listening to the reproduced voice, watches at least one of the pointer 40F, the speech speed adjustment button 40G, and the gain adjustment button 40H on the emotion map M displayed on the display screen 30B, and simultaneously operates the first operation unit 14B1 such as a joystick, the second operation unit 14B2 such as a pedal type, the input unit 14B such as a mouse, etc., to input parameters for a desired reproduction timing in the reproduced voice data. That is, while listening to the reproduced voice, the user inputs parameters such as the type of desired emotion, the mixture ratio of multiple emotions, the intensity of the emotion, the speech speed, and the sound pressure level for one reproduction timing all at once. Then, the user performs the above operation in response to the reproduction of the voice data, and the parameters including the type of desired emotion, the mixture ratio of multiple emotions, the intensity of the emotion, the speech speed, and the sound pressure level are input all at once for each reproduction timing of the voice data.

[0092] During the reproduction of the audio data, the input receiving unit 20B judges whether or not the input of the parameter is received from the input unit 14B (step S112). If the judgment in step S112 is negative (step S112: No), the process proceeds to step S116 described later. If the judgment in step S112 is positive (step S112: Yes), the process proceeds to step S114.

[0093] In step S114, the recording unit 20F records the parameter whose input was accepted in step S112 in association with the reproduction timing in the audio data at which the input of the parameter was accepted (step S114).

[0094] Next, the reproducing unit 20E judges whether the reproduction of the audio data has ended (step S116). For example, the reproducing unit 20E makes the judgment in step S116 by determining whether the reproduction has ended up to the final timing on the time axis of the audio data that started to be reproduced in step S110. If the judgment in step S116 is negative, the process returns to step S112. If the judgment in step S116 is positive (step S116: Yes), the process proceeds to step S118.

[0095] The reproducing unit 20E generates synthetic voice data by synthesizing the voice data acquired in step S108 with voice dictionary data corresponding to emotions according to parameters associated with each of the reproduction timings (step S118).

[0096] Recording unit 20F associates and records the parameters used to generate the synthetic voice at each reproduction timing in the generated synthetic voice data (step S120).

[0097] The user can also set or edit the parameters for each playback timing again while listening to the synthetic voice data. As described above, processing unit 20 may accept a selection of an editing point in the waveform of the synthetic voice data displayed on display screen 30B, and accept editing of the parameters of the editing point, thereby recording the edited parameters in association with a position in the synthetic voice data that corresponds to the selected editing point.

[0098] Display control unit 20A displays display screen 30D on which text information input field 40M is superimposed. Input receiving unit 20B receives text information input into text information input field 40M in response to a user's operation instruction of input unit 14B (step S122).

[0099] Recording unit 20F records the text information received in step S138 in association with the synthetic voice data generated in step S132 (step S124), and then ends this routine.

[0100] As described above, the voice processing support device 10 of the present embodiment includes the input receiving unit 20B and the recording unit 20F. The input receiving unit 20B receives input of parameters including at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions during playback of the voice data to be edited. The recording unit 20F records the received input parameters in association with the playback timing of the voice data at which the input of the parameters was received.

[0101] Here, in the conventional technology, it is possible to set parameters such as a morphing ratio for the entire voice data, but it is difficult to set detailed parameters related to the transition of emotions that change on the time axis.

[0102] On the other hand, the voice processing support device 10 of the present embodiment accepts input of parameters including at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions during playback of the voice data to be edited. Then, the voice processing support device 10 records the inputted parameters in association with the playback timing of the voice data at which the input of the parameters was accepted.

[0103] Therefore, in the voice processing support device 10 of the present embodiment, it is possible to set parameters including at least the types of emotions and the mixture ratio of emotions for each playback timing along the time axis of the voice data.

[0104] Therefore, the voice processing support device 10 of the present embodiment can provide parameters related to the transition of emotions that change on the time axis that can be set in detail.

[0105] Furthermore, in the voice processing support device 10 of this embodiment, parameters that enable dynamic voice expression in which emotions and the intensity of emotions change on the time axis can be set for voice data.

[0106] In addition, the voice processing support device 10 of the present embodiment accepts input of parameters at each playback timing when playing back voice data, so that it is possible to set parameters for each playback timing in real time simultaneously with the playback of voice data. In addition, in the voice processing support device 10 of the present embodiment, the user can set parameters for each playback timing by operating the input unit 14B. Therefore, the voice processing support device 10 of the present embodiment can reduce the load on the user in inputting and editing, and enable detailed setting of parameters that change on the time axis.

[0107] Next, a hardware configuration of the voice processing support device 10 of the present embodiment will be described.

[0108] FIG. 7 is a diagram showing an example of a hardware configuration of the voice processing support device 10 according to the present embodiment.

[0109] The voice processing support device 10 of this embodiment includes a control device such as a CPU 10A, storage devices such as a ROM (Read Only Memory) 10B and a RAM (Random Access Memory) 10C, a HDD (Hard Disk Drive) 10D, an I / F 10E that connects to a network and communicates, and a bus 10F that connects each part.

[0110] The programs executed by the voice processing support device 10 of this embodiment are provided in a state that they are pre-installed in the ROM 10B or the like.

[0111] The program executed by the voice processing support device 10 of this embodiment may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disc).

[0112] Furthermore, the program executed by the voice processing support device 10 of the present embodiment may be stored in a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the voice processing support device 10 of the present embodiment may be provided or distributed via a network such as the Internet.

[0113] The program executed by the voice processing support device 10 of the present embodiment can cause a computer to function as each unit of the above-mentioned voice processing support device 10. In this computer, the CPU 10A can read the program from a computer-readable storage medium onto a main storage device and execute the program.

[0114] In the above embodiment, the voice processing support device 10 has been described as being configured as a single device. However, the voice processing support device 10 may be configured as a plurality of devices that are physically separated and connected to each other so as to be able to communicate with each other via a network or the like.

[0115] Furthermore, the voice processing support device 10 of the above embodiment may be realized as a virtual machine operating on a cloud system.

[0116] Although the embodiment of the present invention has been described above, the embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0117] 10. Audio processing support device 20A Display control unit 20B Input reception section 20C setting section 20D Acquisition Department 20E Playback section 20F Recording section

Claims

1. an input receiving unit that receives input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of the voice data to be edited; a recording unit that records the input of the parameter in association with a playback timing of the input of the parameter in the voice data; A voice processing support device comprising:

2. The parameters are: and further including at least one of emotional intensity, speech rate, and sound pressure level. The voice processing support device according to claim 1.

3. a display control unit that displays a display screen including an emotion map that shows correlations between a plurality of types of emotions; The input receiving unit is receiving, as the parameter, at least one of the type of emotion, the mixture ratio, and the intensity of emotion corresponding to a point specified on the emotion map by a user; The voice processing support device according to claim 1.

4. The input receiving unit is receiving settings of speech dictionary data corresponding to types of emotions to be used for setting the speech data; The voice processing support device according to claim 1.

5. a playback unit that plays back, for the voice data, synthetic voice data based on voice dictionary data corresponding to the type of emotion according to the parameters associated with each of the playback timings; The voice processing support device according to claim 1 .

6. The input receiving unit is Accepting edits of the parameters; The recording unit is storing the edited parameters in association with the selected edit point in the synthetic voice data; The voice processing support device according to claim 5.

7. The input receiving unit is Accepting input of character information for the synthetic voice data; The recording unit is recording the character information in association with the synthetic voice data; The voice processing support device according to claim 5.

8. A voice processing support method executed by a voice processing support device, an input receiving step of receiving input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of the voice data to be edited; a recording step of recording the inputted parameter in association with a reproduction timing at which the inputted parameter was received in the voice data; A voice processing support method comprising:

9. an input receiving step of receiving input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of the voice data to be edited; a recording step of recording the inputted parameter in association with a reproduction timing at which the inputted parameter was received in the voice data; A voice processing support program for causing a computer to execute the above.

Images