Voice processing method, program, and voice processing system

Abstract

In this voice processing method, event information relating to an event is acquired (S1) from an information source apparatus or an information source service, a text string that a speaking apparatus will be made to speak is determined (S4) on the basis of the acquired event information, the determined text string is segmented (S5) into one or more sub-text strings, a voice signal is generated (S7) from the text string, the generated voice signal is corrected (S6, S8) by subjecting the voice signal to filter processing in which a first filter that corresponds to a characteristic of a consonant is applied to each sub-text string, and the corrected voice signal is outputted (S11).

Description

Audio processing method, program, and audio processing system

[0001] The present disclosure relates to a voice processing method for processing human voice emitted by a speaker.

[0002] For example, Patent Document 1 discloses an automatic audio response adjustment method for improving the intelligibility of human speech received by a radio receiver by automatically adjusting the audio response according to the environmental noise level. In this automatic audio response adjustment method, when there is a large environmental noise, the relative gain of high audio frequencies is increased at the expense of low frequency response.

[0003] Special Publication No. 2000-508487

[0004] The present disclosure provides a voice processing method and the like that allows a user to easily hear human voices, regardless of the performance of a speaker provided in a speech device.

[0005] In a voice processing method according to one aspect of the present disclosure, event information relating to an event is acquired from an information source device or an information source service, a character string to be spoken by a speech device is determined based on the acquired event information, the determined character string is divided into one or more sub-character strings, a voice signal is generated from the character strings, a filter process is performed on the generated voice signal in which a first filter according to the characteristics of a consonant is applied to each of the sub-character strings, thereby correcting the voice signal, and the corrected voice signal is output.

[0006] A program according to one aspect of the present disclosure causes one or more processors to execute the above-described audio processing method.

[0007] According to one aspect of the present disclosure, there is provided a speech processing system including an input interface for acquiring event information relating to an event from an information source device or an information source service, a signal processing circuit for correcting an audio signal, and an output interface for outputting the corrected audio signal, wherein the signal processing circuit determines a character string to be spoken by a speech device based on the acquired event information, divides the determined character string into one or more sub-character strings, generates an audio signal from the character string, and corrects the generated audio signal by performing a filter process in which a first filter according to characteristics of consonants is applied to each of the sub-character strings.

[0008] The voice processing method and the like according to the present disclosure has the advantage that it becomes easier for a user to hear human voices, regardless of the performance of the speaker provided in the speech device.

[0009] FIG. 1 is a diagram showing frequency characteristics of a human voice when a speech device is caused to output the human voice. FIG. 2 is a diagram showing frequency characteristics of a human voice when a speech device is caused to output the human voice after filtering the audio signal. FIG. 3 is a diagram showing frequency characteristics of a drive audio output by an electrical appliance. FIG. 4 is a diagram showing results of DRT performed by a robot vacuum cleaner outputting evaluation audio in a noisy environment. FIG. 5 is a diagram showing results for each consonant feature when DRT was performed by a robot vacuum cleaner outputting evaluation audio in a noisy environment. FIG. 6 is a diagram showing results for each consonant feature when DRT was performed by a pet camera outputting evaluation audio in a noisy environment. FIG. 7 is a diagram showing an example of a spectrogram obtained from the audio waveform of the evaluation audio for each consonant feature. FIG. 8 is a diagram showing an example of a spectrogram obtained from the audio waveform of the evaluation audio for each consonant feature. FIG. 9 is a block diagram showing an example of an overall configuration including a speech processing system according to an embodiment. FIG. 10 is a diagram showing the correspondence between consonants and consonant features. FIG. 11 is a diagram showing the correspondence between consonant features and frequency domains. FIG. 12 is a flowchart illustrating an example of the operation of the voice processing system according to the embodiment.

[0010] [1. Findings that Form the Basis of the Present Disclosure] First, the inventor's point of view will be explained below.

[0011] Conventionally, there is a technology that allows a home appliance (speech device) equipped with a voice input / output function to speak by instructing the device on the content and timing of the speech. Here, "voice" refers to vibrations of air or the like that can be perceived by humans at least by hearing. This technology is used to notify a user who is not near a home appliance, for example, a washing machine, of the content of an event that has occurred by having the speech device equipped with a voice input / output function speak the content of the event. The event may include, for example, the occurrence of an error in the home appliance or the completion of an operation that the home appliance was executing.

[0012] Here, if the speech device is a device that is intended to mainly output human voices, such as a television receiver, the performance of the speaker provided in the speech device may be relatively high, making it easy for the user to hear the human voice output by the speech device, i.e., the intelligibility of the human voice may be relatively high. On the other hand, if the speech device is a device that is intended to mainly output system sounds other than human voices, such as beeps, the performance of the speaker provided in the speech device may be relatively low, making it difficult for the user to hear the human voice output by the speech device, i.e., the intelligibility of the human voice may be relatively low.

[0013] In view of the above problems, the inventor has investigated a technology that makes it easier for users to hear human voices, that is, that makes the intelligibility of human voices relatively high, regardless of the performance of the speaker equipped in the speaking device.

[0014] [1-1. Frequency characteristics of human voice output by a speech device] First, the inventors investigated improving the intelligibility of human voice output by a speech device by performing filter processing on the voice signal converted into sound waves and output by the speech device. The filter processing here refers to processing that amplifies the power (sound pressure level) of a specific frequency band in the voice signal.

[0015] Fig. 1 is a diagram showing the frequency characteristics of a human voice when the human voice is output from a speech device. Fig. 2 is a diagram showing the frequency characteristics of a human voice when the human voice is output from a speech device after filtering the audio signal. In both Fig. 1 and Fig. 2, the vertical axis represents the power of the human voice, and the horizontal axis represents the frequency.

[0016] Fig. 1(a) is a diagram showing the frequency characteristics of human voice output by a robot vacuum cleaner as a speech device, and Fig. 2(a) is a diagram showing the frequency characteristics of human voice output by a robot vacuum cleaner when filter processing is performed. Fig. 1(b) is a diagram showing the frequency characteristics of human voice output by a pet camera as a speech device, and Fig. 2(b) is a diagram showing the frequency characteristics of human voice output by a pet camera when filter processing is performed. Note that both the robot vacuum cleaner and the pet camera are devices intended to mainly output system voices other than human voices.

[0017] As shown in Figure 1(a), the power of human voice output by a robot vacuum cleaner is lower in the low-frequency range of 0 to 1 kHz and the high-frequency range of 3 kHz or higher than in other frequency ranges (see the circles in the figure). Therefore, a filter process was performed on the voice signal to amplify the power in the low-frequency range and the high-frequency range. As a result, as shown in Figure 2(a), the power of human voice output by a robot vacuum cleaner was increased in both the low-frequency range and the high-frequency range, and it was discovered that filter process can contribute to improving the intelligibility of human voice.

[0018] As shown in (b) of Figure 1, the power of the human voice output by the pet camera is lower in the low-frequency range of 0 to 1 kHz and the high-frequency range of 4 kHz or higher than in other frequency ranges (see the circles in the figure). Therefore, as described above, a filter process was performed on the voice signal to amplify the power in the low-frequency range and the high-frequency range. However, as shown in (b) of Figure 2, the power of the human voice output by the pet camera is not high in either the low-frequency range or the high-frequency range, and it was found that the filter process cannot contribute to improving the intelligibility of the human voice.

[0019] As described above, the inventor has found that, when filtering is performed in accordance with the frequency characteristics of the human voice output by a speech device, the filtering may contribute to improving the intelligibility of the human voice depending on the type of speech device, but may not contribute to improving the intelligibility of the human voice. Hereinafter, this filtering will also be referred to as "filtering according to a speech device."

[0020] [1-2. DRT Test with Japanese Speech] Next, the inventor conducted a DRT (Diagnostic Rhyme Test) with Japanese speech, i.e., a multiple-choice speech intelligibility test, by having the speech device output human speech in a noisy environment. Here, the noisy environment refers to an environment in which driving speech (noise) is output by driving electrical devices around the speech device.

[0021] Fig. 3 is a diagram showing the frequency characteristics of the drive sound output by an electrical appliance. In Fig. 3, the vertical axis represents the power of the drive sound, and the horizontal axis represents the frequency. Fig. 3(a) shows the frequency characteristics of the drive sound output by a vacuum cleaner, and Fig. 3(b) shows the frequency characteristics of the drive sound output by a robot vacuum cleaner. Fig. 3(c) shows the frequency characteristics of the drive sound output by a dishwasher, and Fig. 3(d) shows the frequency characteristics of the drive sound output by a washing machine.

[0022] DRT is a method of testing intelligibility in which subjects listen to one of two candidate word pairs that differ only by one initial phoneme and select one of the word pairs. In DRT, consonants are divided into six characteristics, 10 word pairs are prepared for each characteristic, and a test is conducted on a total of 120 evaluation speech words. In DRT, the intelligibility of human speech is expressed as (number of correct answers - number of incorrect answers) / total number of evaluation speech words.

[0023] Here, consonants are divided into six characteristics: voicing, nasality, sustention, sibilation, graveness, and compactness.

[0024] Voicing corresponds to "vocalic-nonvocalic" in the Jacobson, Fant, and Halle (JFH) classification of English phonemes (hereinafter referred to as "JFH classification"), which is a classification of voiced and unvoiced sounds. Voiced sounds are sounds that involve vocal cord vibration, such as "zai," while unvoiced sounds are sounds that do not involve vocal cord vibration, such as "sai."

[0025] Nasal sounds correspond to "nasal-oral" in the JFH classification and are a classification of nasal sounds. Nasal sounds are sounds in which sound energy is emitted through the nose without being emitted from the oral cavity, such as "man," or sounds in which sound energy is emitted through the nose with an emission from the oral cavity, such as "ban."

[0026] Continuity corresponds to "continuant-interrupted" in the JFH classification, and is a classification of sustained speech sounds and other speech sounds (plosives or affricates). A sustained speech sound is a speech sound in which the constriction of the vocal tract is not restricted until the / h / sound is produced, such as in "hashi." An unsustained speech sound is a plosive sound, such as in "kashi."

[0027] Irregularity corresponds to "strident-mellow" in the JFH classification, and is a classification related to the irregularity of the waveform. An example of an irregular sound is a sound like "chaku," while an example of a sound without irregularity is a sound like "kaku."

[0028] Grave-acute corresponds to "grave-acute" in the JFH classification, and corresponds to grave and acute sounds. Grave sounds are, for example, sounds like "pai" (pie), and acute sounds are, for example, sounds like "tai" (tai).

[0029] Concentration corresponds to "compact-diffuse" in the JFH classification, and is a classification of whether the spectral energy is concentrated or dispersed at one formant frequency. The former is, for example, a sound like "yaku" (to yaku), and the latter is, for example, a sound like "waku" (to wa).

[0030] Fig. 4 shows the results of a DRT performed by having a robot vacuum cleaner output evaluation voices in a noisy environment. In Fig. 4, the vertical axis represents human speech intelligibility (Speech Intelligibility), and the horizontal axis represents the type of noise source. In Fig. 4, solid-line hatched bars represent results when filter processing according to the speech device was not performed, while dotted-line bars represent results when the filter processing was performed. Fig. 4(a) shows the results when the robot vacuum cleaner output evaluation voices in a female voice, and Fig. 4(b) shows the results when the robot vacuum cleaner output evaluation voices in a male voice.

[0031] As shown in Figure 4, it was confirmed that the intelligibility of human voices improved when the noise source was any of the electrical appliances: a vacuum cleaner, a dishwasher, a robot vacuum cleaner, and a washing machine. It was also confirmed that the intelligibility of human voices was slightly higher when the robot vacuum cleaner output the evaluation voice in a female voice than when the robot vacuum cleaner output the evaluation voice in a male voice. However, when any of the electrical appliances was the noise source, the intelligibility of human voices was relatively low at 0.4 or less.

[0032] As described above, the inventor has found that simply executing a filter process according to the speaking device cannot contribute to a sufficient improvement in the intelligibility of human speech in a noisy environment.

[0033] Here, the inventors have conducted further detailed studies on the above-mentioned DRT. Specifically, they have studied the intelligibility of human speech for each consonant feature in the DRT. Fig. 5 shows the results for each consonant feature when the DRT was performed by having a robot vacuum cleaner output evaluation sounds in a noisy environment. Fig. 6 shows the results for each consonant feature when the DRT was performed by having a pet camera output evaluation sounds in a noisy environment.

[0034] In each of Figures 5 and 6, the vertical axis represents the intelligibility of human speech, and the horizontal axis represents the type of consonant feature. Also, in each of Figures 5 and 6, the bar graphs with solid hatching represent the results when filter processing according to the speech device was not performed, and the bar graphs with dotted hatching represent the results when the filter processing was performed. Also, (a) of Figure 5 and (a) of Figure 6 represent the results when the noise source was a robot vacuum cleaner, and (b) of Figure 5 and (b) of Figure 6 represent the results when the noise source was a washing machine.

[0035] As shown in Figures 5 and 6, it was found that, depending on the characteristics of the consonants, the test subjects may not be able to hear the evaluation speech even after the above-mentioned filtering process was performed. For example, as shown in Figure 5, when the evaluation speech was output from a robot vacuum cleaner, the intelligibility of the evaluation speech corresponding to each of voicedness and irregularity was relatively high, but the intelligibility of the evaluation speech corresponding to other consonant characteristics was relatively low. In particular, the intelligibility of the evaluation speech corresponding to each of nasality and continuity was very low, and the test subjects could hardly hear the evaluation speech. Furthermore, as shown in Figure 6, when the evaluation speech was output from a pet camera, when the noise source was a washing machine, the intelligibility of the evaluation speech corresponding to voicedness was relatively high, but the intelligibility of the evaluation speech corresponding to other consonant characteristics was very low, and the test subjects could hardly hear the evaluation speech.

[0036] Therefore, the inventors focused on the frequency characteristics of each consonant feature. Figures 7 and 8 are both diagrams showing examples of spectrograms obtained from the speech waveforms of evaluation speech for each consonant feature. In each of Figures 7 and 8, the upper area represents the speech waveform, and the lower area represents the spectrogram. The spectrogram here represents the frequency spectrum of the evaluation speech over time.

[0037] 7A shows a spectrogram obtained from the speech waveform of the evaluation speech "zai" corresponding to voicedness, and FIG. 7B shows a spectrogram obtained from the speech waveform of the evaluation speech "sai" corresponding to voicedness. FIG. 7C shows a spectrogram obtained from the speech waveform of the evaluation speech "man" corresponding to nasalness, and FIG. 7D shows a spectrogram obtained from the speech waveform of the evaluation speech "ban" corresponding to nasalness. FIG. 7E shows a spectrogram obtained from the speech waveform of the evaluation speech "hashi" corresponding to continuity, and FIG. 7F shows a spectrogram obtained from the speech waveform of the evaluation speech "kashi" corresponding to continuity.

[0038] Figure 8(a) shows a spectrogram obtained from the speech waveform of the evaluation speech "chaku" corresponding to irregularity, and Figure 8(b) shows a spectrogram obtained from the speech waveform of the evaluation speech "kaku" corresponding to irregularity. Figure 8(c) shows a spectrogram obtained from the speech waveform of the evaluation speech "pai" corresponding to muteness, and Figure 8(d) shows a spectrogram obtained from the speech waveform of the evaluation speech "tai" corresponding to muteness. Figure 8(e) shows a spectrogram obtained from the speech waveform of the evaluation speech "yaku" corresponding to concentration, and Figure 8(f) shows a spectrogram obtained from the speech waveform of the evaluation speech "waku" corresponding to concentration.

[0039] As shown in Figures 7 and 8, the frequency spectrum of the initial phoneme differs for each consonant feature. For example, as shown by the rectangular frames in Figures 7(a) and 7(b), when focusing on the spectrogram of the evaluation speech corresponding to voicedness, the first phoneme "za" and "sa" contain frequency components from 0 to 1 kHz, while the latter does not. Also, as shown by the rectangular frames in Figures 8(a) and 8(b), when focusing on the spectrogram of the evaluation speech corresponding to irregularity, the first phoneme "cha" and "ka" contain many frequency components from 2 to 6 kHz, while the latter contains almost none. Other consonant features also have different frequency spectra for the initial phoneme, as shown by the arrows or rectangular frames in Figures 7 and 8.

[0040] 5(a) and 5(b), when the results for the evaluation speech corresponding to voicedness are examined, the intelligibility of human speech is significantly improved when filter processing according to the speech device is performed compared to when the filter processing is not performed. This is thought to be because the effective frequency range for distinguishing the initial phoneme of the evaluation speech corresponding to voicedness is 0 to 1 kHz, and the power in the low frequency range of 0 to 1 kHz is enhanced by the filter processing, which amplifies the power in the low frequency range.

[0041] As described above, the inventor has discovered that by emphasizing a frequency region in a speech signal that corresponds to the characteristics of a consonant, it becomes easier for people to distinguish the initial phoneme of a word, resulting in improved intelligibility of human speech.

[0042] In view of the above, the inventors have come up with the present disclosure.

[0043] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, step order, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0044] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0045] (Embodiment) [2. Configuration] [2-1. Overall Configuration] First, the overall configuration including a speech processing system according to an embodiment will be described with reference to FIG. 9 . FIG. 9 is a block diagram showing the overall configuration including a speech processing system according to an embodiment. The speech processing system 10 is a system for outputting human speech indicating the content of an event from a speech device 2 when event information related to the event is acquired from an information source device 3 or an information source service 4. In the embodiment, the human speech is human speech in Japanese. In the embodiment, the speech processing system 10 is realized by a server 1. The server 1 is capable of communicating with each of the speech device 2, the information source device 3, and the information source service 4 via a network N1 such as the Internet. Note that the server 1 may communicate with some or all of the speech device 2, the information source device 3, and the information source service 4 via a LAN (Local Area Network).

[0046] In the embodiment, the server 1 (voice processing system 10) causes one speech device 2 to output a human voice indicating the content of the event, but it may also cause each of multiple speech devices 2 to output a human voice indicating the content of the event. Furthermore, the server 1 may cause one or more of the multiple speech devices 2 to output a human voice indicating the content of the event. Furthermore, the server 1 may cause different event content to be spoken by each speech device 2. For example, the server 1 may cause one of two speech devices 2 to output a human voice indicating the content of an event related to an information source device 3, and the other speech device 2 to output a human voice indicating the content of an event related to the other information source device 3.

[0047] The speech device 2 is a device that can notify the user of the content of an event that has occurred in the information source device 3 or the information source service 4 by outputting a human voice indicating the content of the event. The notification by the speech device 2 may also be performed by displaying a character string or an image on a built-in display, for example.

[0048] The speech device 2 is, for example, a device installed in a facility where a user resides, and is a device having the above-described audio output function. In the embodiment, the speech device 2 is a home appliance. Specifically, the speech device 2 may include, for example, a smart speaker, a television receiver, a lighting fixture, a pet camera, an intercom master unit, an intercom slave unit, an air conditioner, or a robot vacuum cleaner. Note that the speech device 2 may also be a portable information device carried by a user, such as a portable television receiver, a smartphone, a tablet terminal, or a laptop personal computer.

[0049] The information source device 3 is a device that is the source of information spoken by the speech device 2. In the embodiment, the information source device 3 is a home appliance. Specifically, the information source device 3 is, for example, an air conditioner, a washing machine, a vacuum cleaner, a robot vacuum cleaner, a dishwasher, a refrigerator, a rice cooker, or a microwave oven. Events that occur in the information source device 3 may include, for example, the start or end of operation of the information source device 3, the occurrence of an error in the information source device 3, or maintenance of the information source device 3. Although FIG. 9 shows one information source device 3, there may be multiple information source devices 3.

[0050] The information source service 4 is a service that is the source of information spoken from the speech device 2, and is, for example, a service provided to the user from a server operated by a service provider. The information source service 4 is, for example, a transportation service, a weather forecast service, a schedule management service, or a traffic information service. An event that occurs in the information source service 4 may include, for example, the start or end of a service provided by the information source service 4, or the occurrence of an error in the information source service 4. Although FIG. 9 shows one information source service 4, there may be multiple information source services 4.

[0051] [2-2. Server Configuration] Next, we will specifically describe the configuration of the server 1. As shown in Fig. 9, the server 1 includes a communication interface (hereinafter referred to as "communication I / F (Interface)") 11, a processor 12, a memory 13, and a storage device 14.

[0052] The communication I / F 11 is, for example, a wireless communication interface, and receives signals transmitted from the information source device 3 and the information source service 4 by communicating with the information source device 3 or the information source service 4 via the network N1 based on a wireless communication standard such as Wi-Fi (registered trademark). The communication I / F 11 also transmits signals to the speech device 2 and receives signals transmitted from the speech device 2 by communicating with the speech device 2 via the network N1 based on a wireless communication standard such as Wi-Fi (registered trademark).

[0053] The communication I / F 11 has the functions of both an input interface (hereinafter referred to as "input I / F") 11A and an output interface (hereinafter referred to as "output I / F") 11B. The input I / F 11A receives a signal transmitted from the information source device 3 or the information source service 4, thereby acquiring event information related to an event from the information source device 3 or the information source service 4.

[0054] In this embodiment, the input I / F 11A further acquires sound collection information obtained by collecting sound around the speech device 2. The sound collection information is, for example, information about audio data generated by collecting sound using a microphone 25 (described later) provided in the speech device 2. When the speech device 2 outputs a person's voice indicating the contents of an event, the sound around the speech device 2 becomes noise that makes it difficult for the user to hear the person's voice. The input I / F 11A acquires the sound collection information by receiving audio data transmitted as sound collection information from the speech device 2.

[0055] The output I / F 11B outputs the audio signal corrected by the processor 12 by transmitting a signal to the speech device 2. The output I / F 11B also outputs an instruction signal to the speech device 2 to instruct the speech device 2 to pick up surrounding sounds by transmitting a signal to the speech device 2.

[0056] The processor 12 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and performs information processing related to the transmission and reception of signals using the communication I / F 11, as well as information processing for generating and correcting audio signals based on event information acquired by the communication I / F 11. The above-mentioned processing related to the transmission and reception of signals and the information processing for generating and correcting audio signals are all realized by the processor 12 executing a computer program stored in the memory 13. The processor 12 is an example of a signal processing circuit of the audio processing system 10.

[0057] The memory 13 is a storage device that stores various types of information required for the processor 12 to perform information processing, computer programs executed by the processor 12, etc. The memory 13 is realized by, for example, a semiconductor memory.

[0058] The storage device 14 is a device that stores a database that is referenced when the processor 12 executes information processing for generating and correcting an audio signal. The storage device 14 is realized by, for example, a hard disk or a semiconductor memory such as an SSD (Solid State Drive).

[0059] [2-3. Configuration of the speech device] Next, the configuration of the speech device 2 will be specifically described.

[0060] 9, the speech device 2 includes a communication I / F 21, a processor 22, a memory 23, a speaker 24, and a microphone 25. Depending on the type of speech device 2, the speech device 2 may not include the microphone 25. In the following description, unless otherwise specified, the speech device 2 will be described as including the microphone 25.

[0061] The communication I / F 21 is, for example, a wireless communication interface, and communicates with the server 1 via the network N1 based on a wireless communication standard such as Wi-Fi (registered trademark), thereby receiving signals transmitted from the server 1 and transmitting signals to the server 1.

[0062] The processor 22 is, for example, a CPU or a DSP, and performs information processing related to the transmission and reception of signals using the communication I / F 21, information processing for causing the microphone 25 to pick up sounds around the speech device 2 based on instruction signals received by the communication I / F 21, and information processing for causing the speaker 24 to output human voices based on audio signals received by the communication I / F 21. The information processing related to the transmission and reception of signals, information processing for outputting human voices, and information processing for picking up sounds around the speech device 2 are all realized by the processor 22 executing computer programs stored in the memory 23.

[0063] The memory 23 is a storage device that stores various types of information required for the processor 22 to perform information processing, computer programs executed by the processor 22, etc. The memory 23 is realized by, for example, a semiconductor memory.

[0064] The speaker 24 reproduces human voice based on the audio signal received by the communication I / F 21. In the embodiment, the speaker 24 converts the audio signal into human voice and outputs the converted human voice.

[0065] The microphone 25 generates audio data by collecting audio around the speech device 2. In this embodiment, the microphone 25 does not always collect audio around the speech device 2, but collects audio around the speech device 2 only when it receives an instruction to collect audio from the server 1 (audio processing system 10). The audio data generated by the microphone 25 is transmitted to the server 1 via the communication I / F 21 as collected audio information.

[0066] 2-4. Generation and Correction of Audio Signals Next, the information processing for generating and correcting audio signals by the processor 12 of the server 1 (audio processing system 10) will be specifically described.

[0067] First, when the communication I / F 11 (input I / F 11A) acquires event information, the processor 12 determines which speech device 2 will output human voice indicating the content of the event. If only one speech device 2 exists, the processor 12 determines that speech device 2 will output human voice. If multiple speech devices 2 exist, the processor 12 determines that a predetermined speech device 2 among the multiple speech devices 2 will output human voice. In this case, the speech device 2 that outputs human voice is not limited to one, and may be multiple.

[0068] Next, the processor 12 outputs an instruction signal to the determined speech device 2 via the communication I / F 11 (output I / F 11B), instructing the speech device 2 to pick up surrounding sounds. As a result, the processor 12 acquires picked-up sound information from the determined speech device 2 via the communication I / F 11 (input I / F 11A). Note that if the determined speech device 2 does not have a microphone 25, the processor 12 does not acquire picked-up sound information from the speech device 2.

[0069] Next, processor 12 determines a character string indicating the content of the event based on the acquired event information. For example, when processor 12 acquires event information indicating that a washing operation has finished from a washing machine, which is information source device 3, processor 12 determines a character string such as "Washing in washing machine has finished." In the embodiment, processor 12 automatically generates a character string based on the event information using an appropriate automatic generation algorithm.

[0070] The processor 12 may determine the character string by reading out the character string corresponding to the acquired event information, for example, by referring to a database stored in the storage device 14. In this case, the database stores in advance data associating the content of the event with the character string corresponding to the event.

[0071] Next, the processor 12 segments the determined character string into one or more sub-character strings using an appropriate algorithm. In an embodiment, the processor 12 segments the determined character string into one or more sub-character strings based on syllables. Here, a syllable is a type of segmental unit that separates consecutive speech sounds and is a type of unit of human speech sounds. For example, a syllable can be a consonant, a vowel, a consonant + vowel, a vowel + consonant, or a consonant + vowel + consonant.

[0072] In the embodiment, the processor 12 divides the determined character string into one or more substrings according to the following rules, for example. First, the processor 12 basically divides the character string into one or more substrings for each consonant and each vowel, and regards a long vowel, a double consonant, and a glottal stop together with the immediately preceding consonant as one substring. The processor 12 may also regard a vowel combined with a straight consonant as one substring. For example, if the character string contains "washing machine (sentakuki)," the processor 12 divides the character string into four substrings: "sen," "ta," "ku," and "ki."

[0073] Next, the processor 12 determines a first filter to be applied to each substring. Here, the first filter is a filter according to the characteristics of the consonant, and is a filter for amplifying and emphasizing the power in the frequency domain according to the characteristics of the consonant. Note that the processor 12 does not apply the first filter to substrings consisting only of vowels.

[0074] FIG. 10 is a diagram showing the correspondence between consonants and their characteristics. (a) of FIG. 10 is a table listing consonants corresponding to each consonant characteristic (voicing, nasalness, continuity, irregularity, suppression, and compactness). For example, consonants having the consonant characteristic of being voiced are the ka, sa, ta, ga, za, and da rows. (b) of FIG. 10 is a table listing the consonant characteristics corresponding to each consonant. For example, consonants in the ka row have four consonant characteristics: voicing, sustention, irregularity, and compactness.

[0075] 11 is a diagram showing the correspondence between consonant characteristics and frequency domains. For example, for a consonant that has the characteristic of being voiced, the effective frequency domain for distinguishing the initial phoneme is 0 to 1 kHz. Also, for a consonant that has the characteristic of being nasal, the effective frequency domain for distinguishing the initial phoneme is 1 to 4 kHz.

[0076] 10 and 11 are stored in a database stored in the storage device 14. The processor 12 then refers to the data stored in the database to determine the first filter to be applied for each substring.

[0077] For example, in the case of four subcharacter strings "sen," "taku," "ku," and "ki," the processor 12 determines, for the subcharacter string "sen," that the "se" included in "sen" is a character in the "sa" row, as characteristics of the consonants corresponding to the "sa" row, based on the correspondence relationship shown in Fig. 10. Then, the processor 12 combines a filter that amplifies the power in the frequency domain corresponding to voicing, a filter that amplifies the power in the frequency domain corresponding to continuity, and a filter that amplifies the power in the frequency domain corresponding to irregularity, and determines the result as the first filter.

[0078] Furthermore, for the subcharacter string "ta", because "ta" is a character in the ta row, the processor 12 identifies voicedness, continuity, irregularity, and muteness as characteristics of the consonant corresponding to the ta row based on the correspondence relationship shown in Fig. 10. The processor 12 then combines a filter that amplifies power in a frequency domain corresponding to voicing, a filter that amplifies power in a frequency domain corresponding to continuity, a filter that amplifies power in a frequency domain corresponding to irregularity, and a filter that amplifies power in a frequency domain corresponding to muteness, and determines the result as a first filter.

[0079] Furthermore, for the substrings "ku" and "ki," because "ku" and "ki" are both in the ka row, the processor 12 identifies voicedness, continuity, irregularity, and convergence as the characteristics of the consonants corresponding to the ka row based on the correspondence relationship shown in Fig. 10. The processor 12 then combines a filter that amplifies the power in the frequency domain corresponding to voicing, a filter that amplifies the power in the frequency domain corresponding to continuity, a filter that amplifies the power in the frequency domain corresponding to irregularity, and a filter that amplifies the power in the frequency domain corresponding to convergence, to determine the first filter as a result.

[0080] In addition, when multiple consonant characteristics exist for each sub-character string, the processor 12 may determine each filter corresponding to each of the multiple consonant characteristics as the first filter without combining filters.

[0081] Next, the processor 12 generates a voice signal from the determined character string using an algorithm for generating an appropriate mechanical voice. In this embodiment, the processor 12 generates a voice signal in a female voice, taking into account the finding in [1. Findings forming the basis of the present disclosure] that female voices are slightly more intelligible than male voices. Note that the processor 12 may also generate a voice signal in a male voice.

[0082] Next, the processor 12 executes a process of correcting the generated audio signal. In this embodiment, the processor 12 executes a first filtering process of applying a first filter to the generated audio signal, a second filtering process of applying a second filter to the generated audio signal, and a third filtering process of applying a third filter to the generated audio signal. Note that these three filtering processes may be executed in the above order or in another order.

[0083] In the embodiment, prior to executing the first filtering process, the processor 12 stores in the memory 13 the position (time) corresponding to each subcharacter string in the generated audio signal. For example, when an audio signal is generated from the character string "sentaku ki," the processor 12 stores in the memory 13 that the subcharacter string "sen" corresponds to a time period from 0 to 0.7 seconds in the audio signal, the subcharacter string "ta" corresponds to a time period from 0.7 seconds to 1 second in the audio signal, the subcharacter string "ku" corresponds to a time period from 1 to 1.3 seconds in the audio signal, and the subcharacter string "ki" corresponds to a time period from 1.3 seconds to 1.6 seconds in the audio signal.

[0084] Then, in the first filter processing, the processor 12 applies the first filter determined for each subcharacter string to the position (time) corresponding to each subcharacter string stored in the memory 13. For example, when applying the first filter to the subcharacter string "sen", the processor 12 applies the first filter to the time period from 0 to 0.7 seconds of the audio signal.

[0085] The processor 12 is not limited to the above process, and may instead generate an audio signal corresponding to the determined character string by, for example, generating an audio signal for each substring and applying a first filter, and then concatenating the audio signals corresponding to all of the substrings. In other words, the processor 12 may generate an audio signal for the determined character string by generating and concatenating an audio signal for each substring. In this case, the processor 12 may apply a corresponding first filter to each audio signal generated for each substring to perform correction. However, generating and concatenating an audio signal for each substring may sound strange to a human listener, so the former method is preferable.

[0086] Here, the second filter is a filter that corresponds to the type of speech device 2. Specifically, the second filter is a filter that amplifies and emphasizes the power in a frequency range with relatively low power, based on the frequency characteristics of the human voice output by the speech device 2. For example, if the speech device 2 is a robot vacuum cleaner, the human voice output by the robot vacuum cleaner has lower power in the low frequency range of 0 to 1 kHz and the high frequency range of 3 kHz or higher than in other frequency ranges (see (a) in FIG. 1). In this case, the processor 12 determines, as the second filter, a filter that amplifies the power in the frequency range of 0 to 1 kHz and the frequency range of 3 kHz or higher.

[0087] The third filter is a filter that corresponds to the sound collection information. Specifically, the third filter amplifies and emphasizes the power in a frequency range with relatively high power based on the frequency characteristics of the sound around the speech device 2 obtained from the sound collection information. For example, assume that a dishwasher is running near the speech device 2. In this case, the processor 12 analyzes the frequency characteristics of the sound data included in the sound collection information and determines that the power is relatively high in the frequency range of 0 to 500 Hz (see (c) in FIG. 3). Therefore, in this case, the processor 12 determines that the third filter is a filter that amplifies the power in the frequency range of 0 to 500 Hz.

[0088] The frequency characteristics of the sound around the speech device 2 may be calculated by the processor 22 of the speech device 2 when the speech device 2 picks up the sound around it, or may be calculated by the processor 12 of the server 1 that has acquired the picked-up sound information.

[0089] Then, the processor 12 transmits (outputs) the audio signal corrected by executing the first filtering process, the second filtering process, and the third filtering process to the determined speech device 2 via the communication I / F 11 (output I / F 11B). As a result, the target speech device 2 acquires the corrected audio signal via the communication I / F 21, and reproduces the human voice based on the corrected audio signal from the speaker 24.

[0090] [3. Operation] Below, an example of the operation of the server 1 (voice processing system 10) according to the embodiment, i.e., an example of a voice processing method, will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the operation of the voice processing system 10 according to the embodiment. Below, it is assumed that an event occurs in the information source device 3 or the information source service 4, and that event information is transmitted from the event source to the server 1 via the network N1. Also, below, it is assumed that the speech device 2 is equipped with a microphone 25 and can provide collected voice information to the server 1.

[0091] First, the processor 12 acquires event information via the communication I / F 11 (input I / F 11A) (S1). Then, the processor 12 determines the speech device 2 that will output a human voice indicating the content of the event (S2).

[0092] Next, the processor 12 outputs an instruction signal to the determined speech device 2 via the communication I / F 11 (output I / F 11B), instructing the speech device 2 to pick up surrounding sounds. As a result, the processor 12 acquires picked-up sound information from the determined speech device 2 via the communication I / F 11 (input I / F 11A) (S3).

[0093] Next, the processor 12 determines a character string indicating the content of the event using an appropriate automatic generation algorithm based on the acquired event information (S4).Then, the processor 12 divides the determined character string into one or more sub-character strings using an appropriate algorithm (S5).Here, the processor 12 divides the determined character string into one or more sub-character strings based on syllables.

[0094] Next, the processor 12 determines a first filter to be applied for each substring (S6). Here, the processor 12 determines a first filter to be applied for each substring by referring to the data shown in Figures 10 and 11 stored in the database stored in the storage device 14.

[0095] Next, the processor 12 generates a voice signal from the determined character string using an algorithm for generating an appropriate mechanical voice (S7). In this example, the processor 12 generates a voice signal in a female voice.

[0096] Next, the processor 12 performs a first filtering process to apply a first filter to each substring in the generated audio signal (S8). As already described, in the embodiment, the processor 12 stores in the memory 13 the position (time) corresponding to each substring in the generated audio signal. Then, in the first filtering process, the processor 12 applies the first filter determined for each substring to the position (time) corresponding to each substring stored in the memory 13. The processor 12 also performs a second filtering process to apply a second filter to the generated audio signal (S9). The processor 12 also performs a third filtering process to apply a third filter to the generated audio signal (S10). Note that the order in which steps S8, S9, and S10 are performed is not limited to this order and may be other orders.

[0097] The processor 12 then transmits (outputs) the corrected audio signal to the selected speech device 2 via the communication I / F 11 (output I / F 11B) (S11). As a result, the selected speech device 2 acquires the corrected audio signal via the communication I / F 21 and reproduces the human voice based on the corrected audio signal from the speaker 24.

[0098] [4. Effects, etc.] As described above, in the voice processing method executed by a computer such as the processor 12, a first filter corresponding to the characteristics of the consonants is applied to each substring, that is, the voice signal is corrected by amplifying and emphasizing the frequency region corresponding to the characteristics of the consonants, and the corrected voice signal is transmitted (output) to the speech device 2. As a result, a user who listens to a person's voice output by the speech device 2 based on the corrected voice signal can easily distinguish one initial phoneme for each substring, which tends to improve the intelligibility of the person's voice. Therefore, such a voice processing method has the advantage of making it easier for the user to hear the person's voice, regardless of the performance of the speaker 24 provided in the speech device 2.

[0099] Furthermore, in the voice processing method of the embodiment, a second filter according to the type of speech device 2 is applied, that is, the voice signal is further corrected by amplifying and emphasizing the power in a frequency range with relatively low power based on the frequency characteristics of the human voice output by the speech device 2. This has the advantage that the human voice output by the speech device 2 is corrected according to the characteristics of the speech device 2, making it even easier for the user to hear the human voice output by the speech device 2.

[0100] Furthermore, in the embodiment, the audio processing method applies a third filter according to the sound collection information, that is, the audio signal is further corrected by amplifying and emphasizing the power in a frequency range with relatively large power based on the frequency characteristics of the audio around the speech device 2. This has the advantage that the human audio output by the speech device 2 is corrected so that it is not buried in the audio around the speech device 2, making it easier for the user to hear the human audio output by the speech device 2.

[0101] 5. Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0102] In the above embodiment, the processor 12 divides the generated character string into one or more sub-character strings based on syllables, but this is not limited to this. For example, the processor 12 may divide the generated character string into one or more sub-character strings based on words. As an example, if the generated character string includes the character string "sentakuki," the processor 12 may divide "sentakuki" into one sub-character string.

[0103] For example, the processor 12 may divide the generated character string into one or more sub-character strings based on mora (beats). For example, if the generated character string includes the character string "sentaku ki," the processor 12 may divide the character string into five sub-character strings: "se," "n," "ta," "ku," and "ki."

[0104] Furthermore, for example, the processor 12 may divide the generated character string into one or more sub-character strings based on the kanji characters that make up the word. For example, if the generated character string includes the character string "sentakuki (washing machine)," the processor 12 may divide the character string into three sub-character strings: "sen," "taku," and "ki."

[0105] In the above embodiment, when a substring includes multiple consonant characteristics, the processor 12 determines a filter corresponding to all of the consonant characteristics as the first filter, but this is not limited to this. For example, the processor 12 may determine a priority for each of the multiple consonant characteristics for each substring in the audio signal.

[0106] As an example, the processor 12 may determine, as the first filter, only a filter corresponding to a consonant feature with a high priority among the multiple consonant features. Here, high priority means that the user can understand the meaning of the character string if the user can hear the consonant; in other words, the user is unlikely to erroneously recognize the meaning of the character string. Note that the processor 12 may determine, as the first filter, only a filter corresponding to a consonant feature with the highest priority among the multiple consonant features, or may determine, as the first filter, filters corresponding to consonant features with a predetermined priority ranking.

[0107] In the above embodiment, the processor 12 performs the first filtering process, the second filtering process, and the third filtering process in the filtering process for correcting the audio signal, but this is not limited to this. For example, the processor 12 may not perform either the second filtering process or the third filtering process, or may not perform both processes.

[0108] In the above embodiment, the speech processing system 10 causes the speech device 2 to output human speech in Japanese, but this is not limited to this. For example, the speech processing system 10 may cause the speech device 2 to output human speech in another language, such as English or Chinese. In this case, the processor 12 may determine, for each sub-character string, a first filter that corresponds to the language of the human speech that is to be output by the speech device 2.

[0109] For example, when outputting English human speech to the speech device 2, the processor 12 divides the determined character string into one or more sub-character strings on a word-by-word basis. If a word contains multiple syllables, the processor 12 may further divide the determined character string into one or more sub-character strings on a syllable-by-syllable basis. In this case, the processor 12 may determine a first filter for each sub-character string based on consonant characteristics (voicedness, nasalness, continuity, irregularity, muteness, and convergence), as in the case of Japanese. For example, words with the consonant characteristic of voicedness include "veal" and "feel." For example, words with the consonant characteristic of nasalness include "moot" and "boot." For example, words with the consonant characteristic of continuity include "sheet" and "cheat." For example, words with the consonant characteristic of irregularity include "sing" and "thing." For example, words with a consonant characteristic of muteness include "weed" and "reed." For example, words with a consonant characteristic of concentration include "key" and "tea."

[0110] In the above embodiment, when there are multiple speech devices 2, the processor 12 determines that a predetermined speech device 2 among the multiple speech devices 2 is to output the human voice, but this is not limited to this. For example, when a detection device capable of acquiring the user's position is provided in an environment where multiple speech devices 2 are placed, the processor 12 may determine the speech device 2 that is to output the human voice based on the user's position.

[0111] For example, assume that one or more human presence sensors are installed in an environment where multiple speech devices 2 are placed, and information indicating speech devices 2 present around the human presence sensors is stored for each human presence sensor in the memory of the processor 12. In this case, the processor 12 may acquire detection results from the one or more human presence sensors via the communication I / F 11 (input I / F 11A), and determine to cause the speech device 2 corresponding to the human presence sensor that indicates the presence of a user to output human voice.

[0112] In the above embodiment, the communication I / F 11 of the server 1 serves as both the input I / F 11A and the output I / F 11B, but this is not limitative. For example, the input I / F 11A and the output I / F 11B may be different interfaces.

[0113] Furthermore, in the above embodiment, the speech processing system 10 is realized by a single device, but it may be realized by multiple devices. When the speech processing system 10 is realized by multiple devices, the functional components of the speech processing system 10 may be distributed among the multiple devices in any manner. For example, the speech processing system 10 may be realized by being distributed across multiple servers. Also, for example, the speech processing system 10 may be realized by being distributed across a server and a speech device. Also, for example, the speech processing system 10 may be realized by only a speech device.

[0114] Furthermore, the communication method between the devices in the above-described embodiments is not particularly limited. When two devices communicate with each other in the above-described embodiments, a relay device (not shown) may be interposed between the two devices.

[0115] The order of the processes described in the above embodiments is merely an example. The order of multiple processes may be changed, or multiple processes may be executed in parallel. Furthermore, a process executed by a specific processing unit may be executed by another processing unit. Furthermore, part of the digital signal processing described in the above embodiments may be realized by analog signal processing.

[0116] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0117] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0118] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. Also, they may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. For example, the present disclosure may be implemented as a speech processing method executed by a computer, or may be realized as a program for causing a computer to execute such a speech processing method. Also, the present disclosure may be realized as a computer-readable non-transitory recording medium on which such a program is recorded. Note that the program here includes an application program for causing a general-purpose information terminal to function as the speech processing system of the above-described embodiment.

[0119] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0120] (Summary) As described above, in the voice processing method according to the first aspect, event information relating to an event is acquired from the information source device 3 or the information source service 4 (S1), a character string to be spoken by the speech device 2 is determined based on the acquired event information (S4), the determined character string is divided into one or more sub-character strings (S5), a voice signal is generated from the character string (S7), the generated voice signal is corrected by performing a filter process in which a first filter according to the characteristics of the consonants is applied to each sub-character string (S6, S8), and the corrected voice signal is output (S11).

[0121] This has the advantage that the user can easily hear human voices regardless of the performance of the speaker 24 provided in the speech device 2.

[0122] In addition, in the speech processing method according to the second aspect, in the first aspect, the character string is divided into one or more sub-character strings based on syllables.

[0123] This has the advantage that the intelligibility of human speech can be ensured while the processing load for correcting the audio signal can be reduced compared to when a character string is separated character by character.

[0124] In the voice processing method according to the third aspect, in the first or second aspect, a second filter according to the type of the speech device 2 is further applied to the voice signal in the filter processing (S9).

[0125] This has the advantage that the human voice output by the speech device 2 is corrected in accordance with the characteristics of the speech device 2, making it easier for the user to hear the human voice output by the speech device 2.

[0126] In addition, in the audio processing method according to the fourth aspect, in any one of the first to third aspects, audio pickup information obtained by picking up audio around the speaking device 2 is acquired (S3), and in the filter processing, a third filter corresponding to the audio pickup information is further applied to the audio signal (S10).

[0127] This has the advantage that the human voice output by the speech device 2 is corrected so that it is not buried in the sounds around the speech device 2, making it easier for the user to hear the human voice output by the speech device 2.

[0128] A program according to a fifth aspect causes one or more processors to execute the audio processing method according to any one of the first to fourth aspects.

[0129] This has the advantage that the user can easily hear human voices regardless of the performance of the speaker 24 provided in the speech device 2.

[0130] Furthermore, a voice processing system 10 according to a sixth aspect includes a processor 12 that corrects a voice signal, and an output I / F 11B that outputs the corrected voice signal. The processor 12 is an example of a signal processing circuit. The processor 12 determines a character string to be spoken by the speech device 2 based on acquired event information, divides the determined character string into one or more sub-character strings, generates a voice signal from the character string, and corrects the voice signal by performing filtering on the generated voice signal in which a first filter corresponding to the characteristics of a consonant is applied to each sub-character string.

[0131] This has the advantage that the user can easily hear human voices regardless of the performance of the speaker 24 provided in the speech device 2.

[0132] The voice processing method of the present disclosure is applicable to systems that process human voices to be reproduced by speakers.

[0133] REFERENCE SIGNS LIST 1 Server 10 Voice processing system 11 Communication I / F 11A Input I / F 11B Output I / F 12 Processor 13 Memory 14 Storage device 2 Speaking device 21 Communication I / F 22 Processor 23 Memory 24 Speaker 25 Microphone 3 Information source device 4 Information source service N1 Network

Claims

1. A voice processing method comprising: acquiring event information relating to an event from an information source device or an information source service; determining a character string to be spoken by a speech device based on the acquired event information; dividing the determined character string into one or more sub-character strings; generating a voice signal from the character strings; correcting the generated voice signal by performing a filter process that applies a first filter according to the characteristics of consonants to each of the sub-character strings; and outputting the corrected voice signal.

2. The speech processing method according to claim 1, wherein the character string is divided into the one or more sub-character strings based on syllables.

3. The audio processing method according to claim 1 or 2, wherein the filtering further applies a second filter to the audio signal according to the type of the speaking device.

4. The audio processing method according to claim 1 or 2, further comprising: acquiring audio pickup information obtained by picking up audio around the speaking device; and further applying a third filter corresponding to the audio pickup information to the audio signal in the filtering process.

5. A program causing one or more processors to execute the speech processing method according to claim 1 or 2.

6. A voice processing system comprising: an input interface that acquires event information relating to an event from an information source device or an information source service; a signal processing circuit that corrects a voice signal; and an output interface that outputs the corrected voice signal, wherein the signal processing circuit determines a character string to be spoken by a speech device based on the acquired event information; separates the determined character string into one or more sub-character strings; generates a voice signal from the character string; and corrects the generated voice signal by performing filter processing that applies a first filter according to the characteristics of consonants to each of the sub-character strings.