Speech processing assistance device, speech processing assistance method, and speech processing assistance program

Abstract

A speech processing assistance device (10) comprises: an input reception unit (20B); and a recording unit (20F). The input reception unit (20B) receives input of parameters including at least a plurality of mutually different emotion types and a mixing ratio of a plurality of types of emotions during reproduction of speech data to be edited. The recording unit (20F) records the parameters, which have been received as the input, in association with a reproduction timing when the input of the parameters has been received in the speech data.

Description

Voice processing support device, voice processing support method, and voice processing support program

[0001] An embodiment of the present invention relates to a voice processing support device, a voice processing support method, and a voice processing support program.

[0002] There are known techniques for synthesizing voices with different voice qualities from existing voices by mixing (morphing) voice data. For example, a technique has been disclosed for generating synthetic voice data by synthesizing multiple voice data based on a predetermined morphing ratio.

[0003] However, with conventional techniques, although it is possible to set parameters such as morphing ratios for the entire audio data, it is difficult to set detailed parameters relating to the transition of emotions that change on the time axis.

[0004] Japanese Patent Application Publication No. 9-50295

[0005] The problem that the present invention aims to solve is to provide a voice processing support device, a voice processing support method, and a voice processing support program that can provide detailed settings for parameters related to the progression of emotions that change over time.

[0006] According to an embodiment, a voice processing support device includes an input receiving unit and a recording unit. The input receiving unit receives input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of voice data to be edited. The recording unit records the received input parameters in association with playback timings of the voice data at which the input of the parameters was received.

[0007] FIG. 1 is a diagram showing an example of a voice processing support device according to an embodiment. FIG. 2 is a schematic diagram of an example of a display screen. FIG. 3 is a schematic diagram of an example of a display screen. FIG. 4 is a schematic diagram of an example of a display screen. FIG. 5 is a schematic diagram of an example of a display screen. FIG. 6 is a flowchart showing an example of the flow of information processing executed by the voice processing support device according to the embodiment. FIG. 7 is a hardware configuration diagram.

[0008] A voice processing support device, a voice processing support method, and a voice processing support program will be described in detail below with reference to the accompanying drawings.

[0009] FIG. 1 is a diagram showing an example of a voice processing support device 10 according to the present embodiment.

[0010] The voice processing support device 10 is an information processing device that supports the processing of voice data.

[0011] The voice processing support device 10 includes a communication unit 12, a UI (user interface) unit 14, a storage unit 16, and a processing unit 20. The communication unit 12, the UI unit 14, the storage unit 16, and the processing unit 20 are communicatively connected via a bus 18.

[0012] The communication unit 12 communicates with other external information processing devices via a network, etc. The UI unit 14 includes a display unit 14A and an input unit 14B.

[0013] The display unit 14A displays various types of information and is, for example, a display such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or a projection device.

[0014] The input unit 14B accepts user operations. The input unit 14B is, for example, a pointing device such as a digital pen, a mouse, or a trackball, or an input device such as a keyboard. Note that the display unit 14A and at least a part of the input unit 14B may be integrated into a touch panel. In this embodiment, the input unit 14B includes a first operation unit 14B1 and a second operation unit 14B2.

[0015] The first operation unit 14B1 is an input device for inputting an operation direction and an operation amount. The first operation unit 14B1 is, for example, an input device that allows input of an operation direction and an operation amount by tilting a lever (stick) according to a tilt direction and tilt angle. Such a first operation unit 14B1 may be called a joystick. The first operation unit 14B1 may also be, for example, a touchpad. In this embodiment, an example in which the first operation unit 14B1 is a joystick will be described.

[0016] The second operation unit 14B2 is an input device for inputting an operation amount. The second operation unit 14B2 is, for example, a pedal-type or button-type input device that is pressed by the user's leg or the like. In this embodiment, a pedal-type input device that inputs an operation amount by the user pressing the second operation unit 14B2 with the user's leg or the like will be described as an example.

[0017] The audio output unit 14C is a speaker that outputs audio.

[0018] The storage unit 16 stores various types of data. The storage unit 16 is, for example, a random access memory (RAM), a semiconductor memory element such as a flash memory, a hard disk, an optical disk, or the like. The storage unit 16 may be a storage device provided outside the voice processing assistance device 10. The storage unit 16 may also be a storage medium. Specifically, the storage medium may store or temporarily store programs and various information downloaded via a local area network (LAN) or the Internet. The storage unit 16 may also be composed of multiple storage media.

[0019] Next, we will explain the processing unit 20. The processing unit 20 executes various types of information processing. The processing unit 20 includes a display control unit 20A, an input receiving unit 20B, a setting unit 20C, an acquisition unit 20D, a playback unit 20E, and a recording unit 20F.

[0020] The display control unit 20A, the input receiving unit 20B, the setting unit 20C, the acquisition unit 20D, the playback unit 20E, and the recording unit 20F are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software. Each of the above units may be realized by a processor such as a dedicated IC (Integrated Circuit), i.e., by hardware. Each of the above units may be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

[0021] Furthermore, at least one of the above-mentioned units and at least a part of the information stored in the storage unit 16 may be installed on a cloud server or the like that executes processing on the cloud.

[0022] The display control unit 20A displays various display screens on the UI unit 14. Details of the display screens will be described later.

[0023] The input receiving unit 20B receives operation inputs from the UI unit 14 by the user.

[0024] For example, consider a situation in which a user wishes to generate synthetic voice data using certain voice data, specifically, a situation in which the user wishes to process and edit the voice data so that the synthetic voice data contains a predetermined emotion at a predetermined intensity along a time axis.

[0025] In this case, the user operates the input unit 14B to set voice dictionary data corresponding to the type of emotion to be used for setting voice data. In this embodiment, the input accepting unit 20B accepts input by the user via the display screen displayed on the display unit 14A.

[0026] 2 is a schematic diagram of an example of the display screen 30A. The display screen 30A is an example of the display screen 30 displayed on the display unit 14A.

[0027] The display screen 30A includes an emotion map M, an emotion setting field 40A, and a voice dictionary data setting field 40B.

[0028] The emotion map M is a map that represents the correlation between multiple types of emotions. For example, the emotion map M is a diagram that represents types of emotions with colors and combines them three-dimensionally to represent complex emotions, even those in which emotions are mixed together. For example, the emotion map M has regions for eight basic emotions, namely, joy, trust, fear, surprise, sadness, disgust, anger, and anticipation, arranged in such a way that they radiate in different directions from a neutral region at the center. Furthermore, the emotion map M also has regions for opposing types of emotions arranged 180° opposite the central neutral region. The emotion map M also classifies the eight basic emotions into three groups, positive, negative, and neutral, and maps the regions for types of emotions belonging to each group so that they are adjacent to each other. Furthermore, the regions for each type of emotion shown in the emotion map M indicate weaker emotions as they approach the central neutral region, and stronger emotions as they move away from the neutral region.

[0029] The emotion map M is not limited to the form shown in FIG. 2 as long as it is a map that shows the correlation between a plurality of types of emotions.

[0030] The emotion setting field 40A is an input field for the type of emotion used to set the voice data. For example, the user operates the input unit 14B while visually checking the emotion map M on the display screen 30A to input the type of emotion they wish to add to the voice data into the emotion setting field 40A. Because the display screen 30A includes the emotion map M, the user can easily input the type of emotion into the emotion setting field 40A by visually checking the emotion map M included on the display screen 30A.

[0031] The voice dictionary data setting field 40B is an input field for setting voice dictionary data corresponding to the type of emotion input in the emotion setting field 40A.

[0032] Speech dictionary data is an acoustic model for deriving acoustic features from linguistic features. The speech dictionary data is created in advance. Linguistic features are linguistic features extracted from text of speech uttered by a speaker. For example, linguistic features include surrounding phonemes, pronunciation information, phrase-final position, sentence length, accent phrase length, mora length, mora position, accent type, part of speech, and dependency information. Linguistic features are sometimes referred to as linguistic information. Acoustic features are speech or acoustic features extracted from speech data. For example, acoustic features used in HMM (Hidden Markov Model) speech synthesis may be used. Examples of acoustic features include Mel-cepstral coefficients, Mel-LPC coefficients, Mel-LSP coefficients, which represent phonemes and tone, fundamental frequency (F0), which represents voice pitch, and aperiodicity index (BAP), which represents the ratio of periodic and aperiodic components of speech. These coefficients of acoustic features are expressed by speech waveforms expressed by frequency, etc.

[0033] In this embodiment, the storage unit 16 is assumed to have stored in advance a plurality of speech dictionary data for outputting acoustic features of speeches from one or more speakers each conveying different emotions.

[0034] The user operates the input unit 14B while viewing the display screen 30A to set the voice dictionary data to be used for the type of emotion input in the emotion setting field 40A. Specifically, for example, the user operates the input unit 14B to select voice dictionary data that is thought to correspond to the corresponding type of emotion from among the multiple voice dictionary data stored in the storage unit 16, and sets the voice dictionary data in the voice dictionary data setting field 40B.

[0035] 2 shows an example of voice dictionary data with file names including the names of the types of corresponding emotions. However, the file names of the voice dictionary data may be in a form that does not include the names of the types of corresponding emotions.

[0036] It is assumed that the speech dictionary data corresponding to the emotion type "neutral" is set in advance. The speech dictionary data corresponding to neutral is speech dictionary data for outputting acoustic features of a speaker speaking without any emotion.

[0037] Through these input operations by the user, the input receiving unit 20B of the processing unit 20 receives the type of emotion to be used for setting the voice data and the setting of the voice dictionary data corresponding to the type of emotion.

[0038] Returning to FIG. 1, the explanation will be continued.

[0039] The setting unit 20C sets the voice dictionary data corresponding to each of the types of emotions received via the display screen 30A as voice dictionary data to be used for editing the voice data. For example, the setting unit 20C stores the voice dictionary data corresponding to each of the types of emotions received via the display screen 30A in a specific storage area of ​​the storage unit 16.

[0040] The acquisition unit 20D acquires the audio data to be edited. The user operates the UI unit 14 to specify the audio data to be edited that is stored in the storage unit 16 or an external information processing device, etc. The acquisition unit 20D acquires the audio data specified by the user's operation instruction on the UI unit 14 as the audio data to be edited.

[0041] In this embodiment, the input receiving unit 20B receives an input specifying the audio data to be edited via the display screen 30 displayed on the display unit 14A.

[0042] 3 is a schematic diagram of an example of a display screen 30B. The display screen 30B is an example of the display screen 30. The display screen 30B is the display screen 30B that is displayed on the UI unit 14 when specifying voice data and accepting input of parameters.

[0043] When the voice dictionary data is set by a user's operation instruction on the input unit 14B, the display control unit 20A displays a display screen 30B on the display unit 14A.

[0044] The display screen 30B includes an audio data file name input display field 40C, a play button 40D, an audio waveform display field 40E, an emotion map M, a pointer 40F, a speech rate adjustment button 40G, a gain adjustment button 40H, an edit button 40J, a synthetic voice playback button 40K, and a save button 40L.

[0045] The audio data file name input / display field 40C is a field for inputting and displaying the file name of the audio data to be edited. The user operates the input unit 14B while viewing the display screen 30B to input the file name of the audio data to be edited in the audio data file name input / display field 40C. The acquisition unit 20D acquires the audio data with the input file name as the audio data to be edited. The user may also operate the input unit 14B while viewing the display screen 30B to specify the audio data to be edited that is stored in the storage unit 16 or the like. In this case, the acquisition unit 20D acquires the specified audio data as the audio data to be edited.

[0046] The user may input the file name of the text data to be edited by operating the input unit 14B while viewing the display screen 30B. The user may also specify the text data to be edited that is stored in the storage unit 16 or the like by operating the input unit 14B while viewing the display screen 30B.

[0047] In this case, the acquisition unit 20D acquires the audio data to be edited by generating audio data using the text data represented by the input file name or the specified text data and audio dictionary data corresponding to the emotion type "unemotional" in a known manner.

[0048] In this embodiment, the processing unit 20 of the voice processing support device 10 accepts input of parameters while playing back voice data to be edited.

[0049] The parameters are parameters related to the transition of emotions that changes over time when synthetic speech data is generated from speech data to be edited. Specifically, the parameters include at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions. The parameters may also include at least one of emotional intensity, speech rate, and sound pressure level. In this embodiment, an example will be described in which the parameters include a plurality of different types of emotions, a mixing ratio of the plurality of types of emotions, and the intensity, speech rate, and sound pressure level of each of the plurality of emotions.

[0050] When the user designates the audio data to be edited, he / she operates the playback button 40D to instruct playback of the audio data. When the input receiving unit 20B receives a playback instruction signal input by operating the playback button 40D, the playback unit 20E starts playing the audio data. Playing audio data means outputting the audio represented by the audio data from the audio output unit 14C.

[0051] When the reproduction of the audio data is started, the display control unit 20A preferably displays a waveform representing the volume of the audio data in the audio waveform display field 40E.

[0052] When playback of the audio data begins and output of the audio data from the audio output unit 14C begins, the user operates the input unit 14B while viewing the display screen 30B to input parameters for the desired playback timing. The playback timing refers to each timing during playback of the audio data played back along the time axis. In other words, the input accepting unit 20B accepts input of parameters for each playback timing during playback of the audio data to be edited.

[0053] In detail, the user operates the input unit 14B while gazing at at least one of the pointer 40F on the emotion map M displayed on the display screen 30B, the speech rate adjustment button 40G, and the gain adjustment button 40H, thereby inputting parameters for the desired playback timing of the audio data being played.

[0054] The emotion map M included in the display screen 30B is the same as the above-described emotion map M. A pointer 40F is shown on the emotion map M. The pointer 40F indicates a point on the emotion map M designated by the user.

[0055] For example, the user adjusts the position of pointer 40F on emotion map M by operating first operation unit 14B1, which is a joystick. Specifically, for example, adjusting the tilt direction and tilt angle of first operation unit 14B1, or the joystick, moves the position of pointer 40F shown on emotion map M displayed on display screen 30B in the tilt direction of the joystick by an amount corresponding to the tilt angle. By operating first operation unit 14B1, the user adjusts the position of pointer 40F displayed on emotion map M to a position corresponding to the desired type of emotion, the mixture ratio of the desired type of emotion, and the intensity of the desired emotion. Input receiving unit 20B receives the type of emotion, the mixture ratio of the multiple types of emotions, and the intensity of the emotion represented by the position of pointer 40F on emotion map M.

[0056] The speech rate and sound pressure level are adjusted by changing the positions of a speech rate adjustment button 40G and a gain adjustment button 40H included on the display screen 30B. The user adjusts the positions of the speech rate adjustment button 40G and the gain adjustment button 40H on the display screen 30B, for example, by operating a pedal-type second operation unit 14B2 using the user's leg or the like.

[0057] For example, the second operation unit 14B2 includes a pedal corresponding to the speech rate adjustment button 40G and a pedal corresponding to the gain adjustment button 40H.

[0058] When the user adjusts the depression amount of the pedal that is the second operation unit 14B2 corresponding to the speech speed adjustment button 40G, the position of the speech speed adjustment button 40G displayed on the display screen 30B moves in the direction to increase or decrease the speech speed. The input receiving unit 20B receives an input of the speech speed according to the depression amount of the second operation unit 14B2 corresponding to the speech speed adjustment button 40G.

[0059] Similarly, when the user adjusts the depression amount of the pedal that is the second operation unit 14B2 corresponding to the gain adjustment button 40H, the position of the gain adjustment button 40H displayed on the display screen 30B moves in the direction to increase or decrease the sound pressure (gain). The input receiving unit 20B receives an input of the sound pressure according to the depression amount of the second operation unit 14B2 corresponding to the gain adjustment button 40H.

[0060] In this way, the user operates at least one of the first operation unit 14B1 and the second operation unit 14B2 at each desired playback timing during playback of the audio data to input parameters including at least one of the type of desired emotion, the mixing ratio of multiple types of emotions, the intensity of the emotion, the speech rate, and the sound pressure level for that playback timing. Also, the input accepting unit 20B accepts input of parameters at each playback timing during playback of the audio data.

[0061] Returning to FIG. 1, the explanation will be continued.

[0062] The recording unit 20F records the input parameters in association with the playback timing in the voice data at which the input of the parameters was received. In particular, for example, the recording unit 20F stores the input parameters in association with a timestamp indicating the playback timing in the voice data at which the input of the parameters was received. Note that the recording unit 20F may record the input parameters in association with a position in the voice data corresponding to the playback timing at which the input of the parameters was received.

[0063] The playback unit 20E generates synthetic voice data based on voice dictionary data corresponding to emotions according to parameters associated with each playback timing for the voice data to be edited.

[0064] Specifically, the playback unit 20E inputs linguistic features (linguistic information) at the playback timing of the audio data to be edited into audio dictionary data corresponding to each of a plurality of emotion types included in parameters set for the playback timing, thereby obtaining audio features corresponding to each emotion. The playback unit 20E then obtains a first mixed audio feature by mixing the obtained audio features corresponding to each emotion type according to the mixing ratio of the emotions included in the parameters set for the playback timing. Furthermore, the playback unit 20E inputs the linguistic features at the playback timing of the audio data to be edited into audio dictionary data corresponding to neutral emotion to obtain a second audio feature corresponding to neutral emotion. The playback unit 20E then mixes the second audio feature corresponding to neutral emotion and the first mixed audio feature in a ratio according to the intensity of the emotion included in the parameters set for the playback timing to obtain a second mixed audio feature. In particular, the playback unit 20E obtains a second mixed audio feature by increasing the ratio of the second audio feature corresponding to neutral emotion as the intensity of the emotion decreases, and increasing the ratio of the first mixed audio feature as the intensity of the emotion increases. Then, the reproduction unit 20E generates synthetic speech, which is a speech waveform represented by the second mixed acoustic feature, as synthetic speech data at the reproduction timing.

[0065] The playback unit 20E generates synthetic voice data by synthesizing the voice data according to the parameters by performing the above-mentioned processing using the parameters set at each of the multiple playback timings along the time axis contained in the voice data.

[0066] Then, the recording unit 20F records the parameters used to generate the synthetic voice at each playback timing in association with each playback timing in the generated synthetic voice data.

[0067] The playback unit 20E plays the synthetic voice data when the synthetic voice play button 40K on the display screen 30B is operated by a user's operation instruction via the input unit 14B. The playback unit 20E plays the generated synthetic voice data by outputting the generated synthetic voice data to the voice output unit 14C. Note that the playback unit 20E may generate and play the synthetic voice data when the synthetic voice play button 40K is operated by a user's operation instruction via the input unit 14B.

[0068] 4 is a schematic diagram of an example of display screen 30C. Display screen 30C is an example of display screen 30. Display screen 30C is the display screen 30 that is displayed on display unit 14A when synthetic voice data is played back. When the user operates input unit 14B to instruct synthetic voice playback button 40K, display control unit 20A displays display screen 30C on display unit 14A.

[0069] In addition to the components of display screen 30B, display screen 30C includes a playback timing image 40I. Playback timing image 40I is an image that represents the current playback timing of the waveform representing the synthetic voice data displayed in voice waveform display field 40E. Therefore, display control unit 20A moves the display position of playback timing image 40I as time passes during playback of the synthetic voice data so that it is displayed at a position corresponding to the current playback timing of the waveform representing the synthetic voice data.

[0070] In addition, it is preferable that the display control unit 20A adjusts the positions of the pointer 40F, speech speed adjustment button 40G, and gain adjustment button 40H on the display screen 30C so that the display positions correspond to the parameters set for each playback timing of the synthetic voice data.

[0071] As described above, the recording unit 20F records the parameters used to generate the synthetic speech at each playback timing in the synthetic speech data in association with each playback timing. During playback of the synthetic speech data, the display control unit 20A displays a pointer 40F at a position on the emotion map M that represents the type of emotion, the mixing ratio, and the intensity of the emotion represented by the parameters recorded in association with the current playback timing. During playback of the synthetic speech data, the display control unit 20A also displays a speech rate adjustment button 40G and a gain adjustment button 40H at positions that represent the speech rate and gain represented by the parameters recorded in association with the current playback timing.

[0072] A user may wish to edit parameters. In this case, the user operates the input unit 14B to operate the edit button 40J on the display screen 30B. When the edit button 40J is operated, the input receiving unit 20B begins receiving parameter editing requests. As described above, the input unit 14B is an input device such as a pointing device such as a digital pen, mouse, or trackball, or a keyboard. The input unit 14B may also include a first operation unit 14B1 such as a joystick, and a second operation unit 14B2 such as a pedal. Therefore, the parameter operation input by the user is not limited to the first operation unit 14B1 such as a joystick, and the second operation unit 14B2 such as a pedal, but may be input by simultaneously operating one or more of a pointing device such as a mouse, digital pen, or dragball, and a keyboard.

[0073] In detail, the user operates the input unit 14B to select an editing point to be edited in the waveform representing the voice data displayed in the voice waveform display field 40E included in the display screen 30B. Then, the user operates the input unit 14B while gazing at at least one of the pointer 40F on the emotion map M displayed on the display screen 30B, the speech rate adjustment button 40G, and the gain adjustment button 40H, thereby editing the parameters associated with the editing point. The operations for editing these parameters are the same as those for inputting parameters for voice data.

[0074] That is, the user operates at least one of the first operation unit 14B1 and the second operation unit 14B2 to edit at least one parameter of the selected editing point, including the type of desired emotion, the mixing ratio of multiple emotions, the intensity of the emotion, the speech rate, and the sound pressure level. The input receiving unit 20B also receives input for editing the parameters at the selected editing point.

[0075] The recording unit 20F records the parameters for which editing has been accepted, attaching them to the selected editing point in the synthetic voice data. In particular, for example, the recording unit 20F stores the parameters for which editing input has been accepted, in association with a timestamp representing the selected editing point in the synthetic voice data. Note that the recording unit 20F may record the parameters for which editing input has been accepted, in association with a position in the synthetic voice data corresponding to the selected editing point.

[0076] The playback unit 20E then regenerates the synthetic voice data in accordance with the edited parameters. The playback unit 20E generates synthetic voice data in accordance with the edited parameters in the same manner as generating synthetic voice data in accordance with voice data in which parameters have been set. The recording unit 20F records the regenerated synthetic voice data in association with the parameters set for each playback timing.

[0077] When the user has finished inputting the parameters, he or she operates the save button 40L. The save button 40L is a button that is operated by the user to instruct storage of the synthesized voice data generated from the voice data for which the parameters have been set in the storage unit 16. When the save button 40L is operated, the input receiving unit 20B receives the save instruction.

[0078] When input receiving unit 20B receives the save instruction, display control unit 20A displays display screen 30 on display unit 14A for receiving input of text information for the synthetic voice data.

[0079] 5 is a schematic diagram of an example of a display screen 30D. The display screen 30D is an example of the display screen 30. The display screen 30D is the display screen 30 that is displayed when the save button 40L is operated by the user.

[0080] When the user operates the save button 40L in response to an instruction from the input unit 14B, the display control unit 20A displays a display screen 30D on the display unit 14A, in which a text information input field 40M is superimposed on the display screen 30C. The text information input field 40M is an input field for text information to be added to the synthetic voice data. For example, the user operates the input unit 14B to input text information such as an explanation for the synthetic voice data into the text information input field 40M.

[0081] Input receiving unit 20B receives character information for synthetic voice data via character information input field 40M. Recording unit 20F stores the received input character information in storage unit 16 in association with the synthetic voice data.

[0082] Through these processes, text information that explains each piece of synthetic speech data is stored in association with the synthetic speech data. Therefore, users of the synthetic speech data can effectively reuse the synthetic speech data by checking the text information. Furthermore, by using the synthetic speech data and the text information assigned to the synthetic speech data as training data, a training model can be generated that outputs the text information from the synthetic speech data as a correct label.

[0083] Next, an example of the flow of information processing executed by the voice processing support device 10 of this embodiment will be described.

[0084] FIG. 6 is a flowchart showing an example of the flow of information processing executed by the voice processing support device 10 of this embodiment.

[0085] The display control unit 20A displays the display screen 30A on the display unit 14A (step S100). For example, when a signal indicating the start of editing of voice data is input by a user's operation instruction on the input unit 14B, the display control unit 20A displays the display screen 30A for accepting settings of emotion types and voice dictionary data on the display unit 14A.

[0086] The user operates the input unit 14B while viewing the display screen 30A to set the types of emotions to be used for setting the voice data and the voice dictionary data to be used for the types of emotions. The input accepting unit 20B accepts the settings of the types of emotions to be used for setting the voice data and the voice dictionary data corresponding to the types of emotions (step S102).

[0087] The setting unit 20C sets the voice dictionary data corresponding to each of the types of emotions received via the display screen 30A as voice dictionary data to be used for editing the voice data (step S104).

[0088] The display control unit 20A displays the display screen 30B for accepting parameter settings for the audio data on the display unit 14A (step S106). By the processing of step S106, the display screen 30B shown in FIG.

[0089] The user operates the input unit 14B while viewing the display screen 30B to input the file name of the audio data to be edited into the audio data file name input / display field 40C. The acquisition unit 20D acquires the audio data with the input file name as the audio data to be edited (step S108).

[0090] The user operates the playback button 40D to instruct playback of the audio data. When the input receiving unit 20B receives a playback instruction signal input by operating the playback button 40D, the playback unit 20E starts playing the audio data acquired in step S108 (step S110).

[0091] When playback of the audio data begins, the user listens to the played audio and, while gazing at at least one of the pointer 40F, speech rate adjustment button 40G, and gain adjustment button 40H on the emotion map M displayed on the display screen 30B, simultaneously operates the first operation unit 14B1 such as a joystick, the second operation unit 14B2 such as a pedal, and the input unit 14B such as a mouse to input parameters for the desired playback timing of the audio data being played. That is, while listening to the played audio, the user inputs parameters such as the desired type of emotion, the mixture ratio of multiple emotions, the intensity of the emotion, the speech rate, and the sound pressure level for one playback timing all at once. Then, by the user performing the above operations in response to the playback of the audio data, parameters including the desired type of emotion, the mixture ratio of multiple emotions, the intensity of the emotion, the speech rate, and the sound pressure level are input all at once for each playback timing of the audio data.

[0092] During playback of the audio data, the input receiving unit 20B determines whether or not parameter input has been received from the input unit 14B (step S112). If the determination in step S112 is negative (step S112: No), the process proceeds to step S116 (described later). If the determination in step S112 is positive (step S112: Yes), the process proceeds to step S114.

[0093] In step S114, the recording unit 20F records the parameters received in step S112 in association with the playback timing of the audio data at which the parameters were received (step S114).

[0094] Next, the playback unit 20E determines whether playback of the audio data has finished (step S116). For example, the playback unit 20E makes the determination in step S116 by determining whether playback has finished up to the final timing on the time axis of the audio data that started to be played in step S110. If the determination in step S116 is negative, the process returns to step S112. If the determination in step S116 is positive (step S116: Yes), the process proceeds to step S118.

[0095] The reproduction unit 20E generates synthetic voice data by synthesizing the voice data acquired in step S108 with voice dictionary data corresponding to emotions according to parameters associated with each of the reproduction timings (step S118).

[0096] The recording unit 20F associates the parameters used to generate the synthetic voice at each playback timing with each playback timing in the generated synthetic voice data, and records them (step S120).

[0097] The user can also set or edit the parameters for each playback timing again while listening to the synthetic voice data. As described above, the processing unit 20 may accept the selection of an editing point in the waveform of the synthetic voice data displayed on the display screen 30B, and may accept editing of the parameters of the editing point, thereby recording the edited parameters in association with the position in the synthetic voice data that corresponds to the selected editing point.

[0098] The display control unit 20A displays the display screen 30D on which the character information input field 40M is superimposed. The input accepting unit 20B accepts character information input into the character information input field 40M in response to an instruction from the user operating the input unit 14B (step S122).

[0099] Recording unit 20F records the character information received in step S138 in association with the synthetic voice data generated in step S132 (step S124), and then ends this routine.

[0100] As described above, the voice processing support device 10 of this embodiment includes an input receiving unit 20B and a recording unit 20F. The input receiving unit 20B receives input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of voice data to be edited. The recording unit 20F records the received input parameters in association with the playback timing of the voice data at which the input of the parameters was received.

[0101] Here, with conventional technology, it is possible to set parameters such as morphing ratios for the entire audio data, but it is difficult to set detailed parameters related to the transition of emotions that change on the time axis.

[0102] On the other hand, the voice processing support device 10 of this embodiment accepts input of parameters including at least a plurality of different emotion types and a mixing ratio of the plurality of emotion types during playback of voice data to be edited. Then, the voice processing support device 10 records the accepted input parameters in association with the playback timing of the voice data at which the input of the parameters was accepted.

[0103] Therefore, in the voice processing support device 10 of this embodiment, it is possible to set parameters including at least the types of emotions and the mixing ratio of emotions for each playback timing along the time axis of the voice data.

[0104] Therefore, the voice processing support device 10 of this embodiment can provide detailed settings for parameters related to the transition of emotions that change on the time axis.

[0105] Furthermore, in the voice processing support device 10 of this embodiment, parameters that enable dynamic voice expression in which emotions and the intensity of emotions change on the time axis can be set for voice data.

[0106] Furthermore, the voice processing support device 10 of this embodiment accepts input of parameters at each playback timing during playback of voice data, making it possible to set parameters for each playback timing in real time simultaneously with playback of voice data. Furthermore, in the voice processing support device 10 of this embodiment, the user can set parameters for each playback timing by operating the input unit 14B. Therefore, the voice processing support device 10 of this embodiment can reduce the burden on the user of input and editing and enable detailed setting of parameters that change on the time axis.

[0107] Next, the hardware configuration of the voice processing support device 10 of this embodiment will be described.

[0108] FIG. 7 is a diagram showing an example of the hardware configuration of the voice processing support device 10 according to the present embodiment.

[0109] The voice processing support device 10 of this embodiment includes a control device such as a CPU 10A, a storage device such as a ROM (Read Only Memory) 10B and a RAM (Random Access Memory) 10C, an HDD (Hard Disk Drive) 10D, an I / F 10E that connects to a network and communicates, and a bus 10F that connects each part.

[0110] The programs executed by the voice processing support device 10 of this embodiment are provided in advance in the ROM 10B or the like.

[0111] The program executed by the voice processing support device 10 of this embodiment may be configured to be provided as a computer program product by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disc).

[0112] Furthermore, the program executed by the voice processing assistance device 10 according to the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the voice processing assistance device 10 according to the present embodiment may be provided or distributed via a network such as the Internet.

[0113] The program executed by the voice processing support device 10 of this embodiment can cause a computer to function as each unit of the above-mentioned voice processing support device 10. In this computer, the CPU 10A can read the program from a computer-readable storage medium onto a main storage device and execute the program.

[0114] In the above embodiment, the voice processing assistance device 10 has been described as being configured as a single device. However, the voice processing assistance device 10 may be configured as a plurality of devices that are physically separated and connected to each other so as to be able to communicate with each other via a network or the like.

[0115] The voice processing support device 10 of the above embodiment may be realized as a virtual machine running on a cloud system.

[0116] Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0117] 10 Voice processing support device 20A Display control unit 20B Input reception unit 20C Setting unit 20D Acquisition unit 20E Playback unit 20F Recording unit

Claims

1. A voice processing support device comprising: an input receiving unit that receives input of parameters including at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions during playback of voice data to be edited; and a recording unit that records the received input parameters in association with the playback timing of the voice data at which the input of the parameters was received.

2. The voice processing support device according to claim 1, wherein the parameters further include at least one of emotional intensity, speech rate, and sound pressure level.

3. A voice processing support device as described in claim 1, further comprising a display control unit that displays a display screen including an emotion map showing the correlation between multiple types of emotions, and the input receiving unit receives as the parameter at least one of the type of emotion, the mixing ratio, and the intensity of emotion corresponding to a point specified on the emotion map by a user.

4. The voice processing support device according to claim 1, wherein the input receiving unit receives settings of voice dictionary data corresponding to types of emotions used for setting voice data.

5. The voice processing support device according to claim 1, further comprising: a playback unit that plays back synthetic voice data based on voice dictionary data corresponding to the type of emotion according to the parameters associated with each of the playback timings for the voice data.

6. The voice processing support device according to claim 5, wherein the input accepting unit accepts editing of the parameters, and the recording unit stores the edited parameters in association with a selected editing point in the synthetic voice data.

7. The voice processing support device according to claim 5, wherein the input receiving unit receives input of character information for the synthetic voice data, and the recording unit records the character information in association with the synthetic voice data.

8. A voice processing support method executed by a voice processing support device, comprising: an input receiving step of receiving, during playback of voice data to be edited, input of parameters including at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions; and a recording step of recording the received input parameters in association with the playback timing in the voice data at which the input of the parameters was received.

9. A voice processing support program for causing a computer to execute the following steps: an input receiving step of receiving, during playback of voice data to be edited, input of parameters including at least a plurality of different types of emotions and a mixing ratio of the plurality of types of emotions; and a recording step of recording the received input parameters in association with the playback timing in the voice data at which the input of the parameters was received.

Images