Hearing assistance method, hearing aid speech dataset and processing method set amendment method

Abstract

A hearing aid synthetic speech data set correction method according to an aspect of the present invention includes: outputting a first detection synthetic speech based on first detection synthetic speech data associated with a first phoneme label in a detection synthetic speech data set; receiving a first response selected by a user; outputting a second detection synthetic speech based on second detection synthetic speech data associated with a second phoneme label in the detection synthetic speech data set; receiving a second response selected by the user; and in a case where the first response is consistent with the second phoneme label and the second response is inconsistent with the second phoneme label, replacing second hearing aid synthetic speech data with the first hearing aid synthetic speech data in association with the second phoneme label in a hearing aid synthetic speech data set.

Description

Technical Field

[0001] This invention relates to a method for correcting a synthetic speech dataset for hearing aids. Background Technology

[0002] In existing speech synthesis technology, after parsing the input human speech into text, specific speech data is synthesized to produce artificial speech. Speech synthesis technology is used, for example, in the field of hearing aids, which help hearing-impaired individuals hear sounds.

[0003] For example, Patent Document 1 describes a speech processing apparatus comprising a language processing unit, a segment database, and a speech synthesis unit. The language processing unit parses text, the segment database stores speech in advance in the form of required synthesis units such as vowels / consonants / vowel liaison, and the speech synthesis unit synthesizes the speech stored in the segment database based on the reading information input to the language processing unit. Furthermore, this speech processing apparatus also includes a phonological enhancement processing unit, an auditory measurement unit, and a sound quality control unit. The phonological enhancement processing unit enhances the synthesis units stored in the segment database, the auditory measurement unit measures the user's auditory characteristics, and the sound quality control unit determines the enhancement processing settings based on the measured auditory characteristics. Thus, this speech processing apparatus can output synthesized speech according to the user's auditory characteristics.

[0004] [Patent Documents]

[0005] Patent Document 1: Japanese Patent Application Publication No. 2004-004952 Summary of the Invention

[0006] However, while enhanced processing increases the likelihood of correct recognition by users, excessive manipulation of the amplitude and frequency of the original speech waveform can be uncomfortable for users. Speech synthesis methods have evolved to include (1) waveform methods, (2) HMM methods, and (3) DNN methods. However, due to the lack of a unified standard for the characteristics of consonants and vowels in synthesized speech, even people with normal hearing may hear different sounds when comparing synthesized speech from different companies. It is particularly noteworthy that significant differences exist in how hearing-impaired individuals perceive sound.

[0007] Therefore, the purpose of this invention is to provide a hearing aid method that enables users to correctly identify hearing and reduces user discomfort.

[0008] The present invention relates to a method for correcting a synthetic speech dataset for hearing aids. This method corrects a synthetic speech dataset for hearing aids based on a synthetic speech dataset for detection. In the synthetic speech dataset for detection, for each of a plurality of phoneme labels, there is associated synthetic speech data for outputting synthetic speech for detection. The method further includes: outputting first synthetic speech for detection based on first synthetic speech data associated with a first phoneme label in the synthetic speech dataset for detection; the output first synthetic speech for detection... After the phoneme label represented by the sound is perceived by the user, the first answer is selected by the user based on the reason; based on the second detection synthesized speech data associated with the second detection phoneme label in the detection synthesized speech dataset, the second detection synthesized speech is output; after the phoneme label represented by the output second detection synthesized speech is perceived by the user, the second answer is selected by the user based on the reason; if the first answer is consistent with the second phoneme label and the second answer is inconsistent with the second phoneme label, in the hearing aid synthesized speech dataset, the second hearing aid synthesized speech data associated with the second phoneme label is replaced, and the first hearing aid synthesized speech data associated with the first phoneme label is associated with the second phoneme label.

[0009] According to this method, even if the second synthesized speech output based on the second synthesized speech data associated with the second phoneme label contained in the synthesized speech dataset for hearing aids cannot be correctly heard by the user, if the first synthesized speech output based on the first synthesized speech data associated with other first phoneme labels in the synthesized speech dataset for detection can be recognized as the phoneme represented by the second phoneme label, a new association can be made between the second synthesized speech data associated with the second phoneme label and the first phoneme label in the synthesized speech dataset for hearing aids. Therefore, without processing the synthesized speech data in the synthesized speech dataset for hearing aids, the user can hear it correctly by correcting the synthesized speech data for hearing aids. Thus, synthesized speech that can be correctly recognized by the user and reduces user discomfort can be output.

[0010] [Invention Effects]

[0011] According to the present invention, a synthetic speech dataset for hearing aids can be provided, which can be used to output synthetic speech for hearing aids that reduces user discomfort. Attached Figure Description

[0012] Figure 1 This is an exemplary structural diagram of the voice processing device 1 according to an embodiment of the present invention.

[0013] Figure 2A This is a schematic diagram of the data structure of the synthetic speech dataset used for detection.

[0014] Figure 2B This is a schematic diagram of the data structure of a synthetic speech dataset for hearing aids.

[0015] Figure 2C This is a schematic diagram of the data structure of the processing method set.

[0016] Figure 2D This is a schematic diagram of the data structure of the processing method set.

[0017] Figure 3 This is an example diagram illustrating the action flow for explaining the speech detection and processing of the speech processing device 1.

[0018] Figure 4 This is an example image of the test screen.

[0019] Figure 5A This is an example flowchart of the process for correcting synthetic speech datasets used in hearing aids.

[0020] Figure 5B This is an example diagram of the action flow for the correction process of the processing method set.

[0021] Figure 6A This is an example diagram illustrating the workflow of hearing aid processing using speech synthesis.

[0022] Figure 6B This is an example diagram illustrating the workflow of hearing aid processing using input speech processing methods.

[0023] Figure 6C This is an example diagram illustrating other action flows in hearing aid processing using input speech processing methods.

[0024] Figure 7 This is a structural example diagram of the speech detection device 1A according to other embodiments of the present invention.

[0025] Figure 8 This is a structural example diagram of the hearing aid 1B according to other embodiments of the present invention. Detailed Implementation

[0026] Preferred embodiments of the present invention will be described with reference to the accompanying drawings. (Furthermore, components marked with the same reference numerals in the drawings have the same or similar structures.)

[0027] (1) Structure

[0028] Figure 1This is an exemplary structural diagram of a speech processing apparatus 1 according to an embodiment of the present invention. The speech processing apparatus 1 is an apparatus that outputs speech based on input speech (e.g., a speaker's voice) using either speech synthesis or input speech processing methods. Speech synthesis refers to a method of synthesizing speech based on specific synthesized speech data. Input speech processing refers to a method of performing specific processing on the input speech itself and outputting speech generated by that processing.

[0029] As will be explained later, the speech processing device 1 can correct the hearing aid synthesized speech dataset used in the speech synthesis method, which will be described later, based on user speech test results based on the synthesized speech dataset for detection. Similarly, the speech processing device 1 can correct the set of processing methods used in the input speech processing method, which will be described later, based on user speech test results based on the synthesized speech dataset for detection.

[0030] Furthermore, in the following description, the voice processing device 1 is described as being composed of a single information processing device; however, the voice processing device 1 can also be a system composed of multiple information processing devices. The information processing device can be any information processing device capable of implementing the functions and methods described in this application, and can be various types of information processing devices, such as smartphones, mobile phones (feature phones), computers, server devices, handheld computer devices, wearable terminals (e.g., glasses-type devices, watch-type devices, etc.), and any other computer. The information processing device includes, for example, a processor, memory, storage, input / output interfaces (input devices and output devices), and a communication interface interconnected via a bus.

[0031] like Figure 1 As shown, the voice processing device 1 includes, for example, an operation unit 2, a voice input unit 3, a display unit 5, a voice output unit 4, a storage unit 6, and a processing unit 7.

[0032] The operation unit 2 is composed of input devices such as touch panels or buttons, and accepts input operations such as text, numbers, and symbols performed by the user, and supplies the signal corresponding to the operation to the processing unit 7.

[0033] The voice input unit 3, for example, is composed of a microphone, which converts the user's voice into an electrical signal voice waveform and supplies it to the processing unit 7.

[0034] The voice output unit 4, for example, is composed of a speaker, and outputs voice based on synthesized voice data (synthetic voice data for detection and synthesized voice data for hearing aids, etc.). In particular, the voice output unit 4 outputs synthesized voice for detection based on synthesized voice data for detection, and outputs synthesized voice for processing detection based on synthesized voice data for processing detection. Furthermore, the voice output unit 4 outputs synthesized voice for hearing aids based on synthesized voice data for hearing aids.

[0035] The display unit 5 is composed of an output device such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays images and the like based on the display data supplied by the processing unit 7.

[0036] Storage unit 6 includes, for example, a memory such as a magnetic tape device, a disk device, or an optical disk device, storing computer programs, data, etc., used in the processing of processing unit 7. Furthermore, storage unit 6 stores the synthesized speech dataset for detection, the synthesized speech dataset for hearing aids, and the processing method set, which will be described later. Additionally, the programs of each embodiment of this disclosure can also be provided stored in a computer-readable storage medium. The storage medium refers to a "non-temporary tangible medium" capable of storing programs. The program is not particularly limited, and for example includes software programs and computer programs.

[0037] Figure 2A This is a schematic diagram of the data structure of the synthetic speech dataset for detection. In the synthetic speech dataset for detection, each phoneme label is associated with synthetic speech data for detection, which serves as the basis for speech synthesis for detection. Figure 2A In the example shown, the left column displays phoneme labels, and the right column displays the synthesized speech data for detection associated with those phoneme labels. Phoneme labels are tags used to identify phonemes. Furthermore, a phoneme refers to a sound that constitutes a unit of speech (not necessarily the smallest unit), and can be a consonant, vowel, or a combination of consonants and vowels, etc. Phoneme labels are not limited to these; for example, they can be phonetic characters (e.g., Japanese kana characters) or phonetic symbols. In this example, phoneme labels are represented by "[ka]", and the synthesized speech data for detection is represented by "S[ka]". Specifically, in the initial state of the synthesized speech dataset for detection, the synthesized speech data for detection is associated with each phoneme label, and the synthesized speech data for detection is used to output synthesized speech for detection, which allows users with normal hearing (healthy individuals) to fully recognize the sounds represented by the phoneme labels.

[0038] Figure 2BThis is a schematic diagram of the data structure of the synthetic speech dataset for hearing aids. In the synthetic speech dataset for hearing aids, each phoneme label is associated with synthetic speech data for hearing aids, which serves as the basis for hearing aid speech synthesis. Figure 2B In the example shown, the left column displays phoneme labels, and the right column displays the synthesized speech data for hearing aids associated with those phoneme labels. At least a portion of the phoneme labels in the synthesized speech dataset for hearing aids are identical to at least a portion of the phoneme labels in the synthesized speech dataset for detection. Furthermore, the phoneme labels in the synthesized speech dataset for hearing aids may also be completely identical to those in the synthesized speech dataset for detection. In this example, phoneme labels are represented by "[ka]", etc., and synthesized speech data for hearing aids is represented by "H[ka]", etc. Specifically, in the initial state of the synthesized speech dataset for hearing aids, the synthesized speech data for hearing aids is associated with each phoneme label, and the synthesized speech data for hearing aids is used to output synthesized speech for hearing aids that allows users with normal hearing (healthy individuals) to fully recognize the sounds represented by the phoneme labels.

[0039] Figure 2C This is a schematic diagram of the data structure of the processing method set for each phoneme tag. Figure 2C In the example shown, for each phoneme tag, a method for processing the input sound (e.g., the speaker's voice) is associated. Figure 2C In the example shown, the left column displays phoneme labels, and the right column displays the processing methods associated with those phoneme labels. In the initial settings, the processing methods in the processing method set may be minimal (including processing such as outputting the input sound as is) without enhancement. At least a portion of the phoneme labels in the processing method set are identical to at least a portion of the phoneme labels in the synthesized speech dataset for detection. Furthermore, the phoneme labels in the processing method set may be completely identical to those in the synthesized speech dataset for detection. In this example, phoneme labels are represented by "[ka]", etc., and processing methods are represented by "F[ka]", etc. In particular, in the initial state of the processing method set, the minimal processing method for outputting the input sound as is (faithfully) is associated with each phoneme label. Also, in the initial state of the processing method set, the processing methods for all phoneme labels may be identical to each other.

[0040] Figure 2D This is a schematic diagram of the data structure of the processing methods set for each consonant group. Figure 2D In the example shown, for each consonant group, a method for processing the input sound (e.g., the speaker's speech) is associated. Figure 2DIn the example shown, the left column displays consonant groups, and the right column displays the processing methods associated with those consonant groups. For example, as... Figure 2D As shown, consonant groups can include "(1) voiceless consonants", "(2) voiced consonants", and "(3) voiced initial consonants" as major categories. In the processing method set, these consonant groups are each associated with processing methods F(1), F(2), and F(3). Furthermore, for example, using this processing method set, if the input speech contains "(1) voiceless consonants", the speech corresponding to the "(1) voiceless consonant" part of the input speech can be processed by processing method F(1). Furthermore, "(1) voiceless consonants" can also be further subdivided into "(1-1) voiceless initial consonants in a narrow sense", "(1-2) voiceless plosives / k,ky,t,p,py / ", "(1-3) voiceless affricates / ch,ts / ", and "(1-4) voiceless fricatives / s,sh,h,hy / " as minor categories. Furthermore, “(2) voiced consonants” can be further subdivided into “(2-1) nasal consonants / s,sh,h,hy / ”, “(2-2) semi-vowels / y,w / ”, and “(2-3) voiced plosives / r,ry / ”. Also, “(3) voiced initial consonants” can be further subdivided into “(3-1) voiced plosives and voiced fricatives / d,b,by / ”, “(3-2) voiced plosives and voiced fricatives / g,gy / ”, and “(3-3) voiced affricates and voiced fricatives / z,j / ”. With the processing methods centralized, processing methods such as F(1-1) can be set for each of these subdivided consonant groups. Then, for example, if the input speech contains “(1-1) voiceless initial consonants in a narrow sense”, the speech corresponding to the “(1-1) voiceless initial consonants in a narrow sense” part of the input speech can be processed using processing method F(1-1). Furthermore, the above-mentioned consonant group classification method and level of refinement in the processing method set are just examples; other methods and levels of refinement can also be used to set the processing method set.

[0041] The processing unit 7 includes one or more processors and their peripheral circuitry. The processing unit 7, for example, is a CPU, which provides comprehensive control over the overall operation of the voice processing device 1. The processing unit 7 controls the operation of the operation unit 2, etc., based on programs stored in the storage unit 6, causing the voice processing device 1 to perform various processes in an appropriate sequence. The processing unit 7 executes these processes based on the computer programs stored in the storage unit 6.

[0042] The processing unit 7 includes, for example, a detection unit 71, a correction unit 72, a hearing aid processing unit 73, and a display control unit 74. The detection unit 71 includes a phoneme tag selection unit 71a, a synthesized speech output unit 71b, a response acceptance unit 71c, a judgment unit 71d, and a processing unit 71e. The correction unit 72 includes a hearing aid synthesized speech dataset correction unit 72a and a processing method set correction unit 72b. The hearing aid processing unit 73 includes an input speech processing unit 73a and an output speech processing unit 73b. The input speech processing unit 73a includes a text recognition processing unit 73a1 and an acoustic recognition processing unit 73a2. The output speech processing unit 73b includes a speech synthesis unit 73b1 and an input speech processing unit 73b2.

[0043] In speech detection processing, the phoneme label selection unit 71a selects, for example, phoneme labels from the synthesized speech dataset for detection that have not yet been tested. The synthesized speech output unit 71b controls the speech output unit 4 to output synthesized speech (synthetic speech for detection and synthesized speech for hearing aids, etc.) based on synthesized speech data or synthesized speech data for processing the synthesized speech data for detection. The response acceptance unit 71c accepts responses input by the user through operations performed on the operation unit 2. Responses may include, for example, selections of phoneme labels associated with the output speech or responses that are uncomfortable to hear. The judgment unit 71d performs various judgments in the speech detection processing. The processing unit 71e, under specified conditions, processes the synthesized speech data for detection to generate processed synthesized speech data for detection. In the synthesized speech dataset for hearing aids, the synthesized speech dataset correction unit 72a, under specified conditions, performs replacement processing and / or processing of the synthesized speech data for hearing aids associated with phoneme labels. The processing method set correction unit 72b corrects the processing method set based on the speech detection results. The text recognition processing unit 73a1 analyzes the input speech and generates text corresponding to the speech, while the acoustic recognition processing unit 73a2 performs acoustic recognition processing on the input speech. The acoustic recognition processing, for example, determines the acoustic features of the input speech by referring to pre-defined acoustic features (which can be stored in components such as the storage unit 6) according to each consonant group or vowel, and identifies the consonant group or vowel corresponding to the speech. The consonant group can be, for example, referred to... Figure 2DThe consonant groups are described. The speech synthesis unit 73b1 uses a synthesized speech dataset for hearing aids and outputs synthesized speech corresponding to the generated text via the speech output unit 4. The input speech processing unit 73b2 processes the input speech using a set of processing methods and outputs the speech via the speech output unit 4. The display control unit 74 displays various images on the display unit 5 based on prescribed display data. The speech analysis unit 7i analyzes the speech input to the speech input unit 3 and generates text data representing the speech. The speech synthesis unit 7j synthesizes the speech representing the text data generated by the speech analysis unit 7i, etc., based on the synthesized speech dataset for hearing aids, and then outputs the speech via the speech output unit 4.

[0044] (2) Action

[0045] (2-1) Speech detection

[0046] Figure 3 This is an example flowchart illustrating the speech detection processing of the speech processing device 1. In this process, after outputting either synthesized speech for detection based on a synthesized speech dataset for detection or processed synthesized speech for detection, the speech processing device 1 receives responses from users.

[0047] (S101) First, the phoneme tag selection unit 71a selects any phoneme tag whose test has not yet been completed (as the phoneme tag in the test) from the phoneme tags contained in the detection synthetic speech dataset stored in the storage unit 6.

[0048] (S102) Next, the synthesized speech output unit 7b outputs the synthesized speech for detection through the speech output unit 4 based on the synthesized speech data for detection that is associated with the phoneme labels in the test in the synthesized speech data set for detection.

[0049] (S103) Next, the user operates the operation unit 2 and selects the perceived phoneme tag from the phoneme tag selection unit displayed on the test screen shown in FIG5, based on the speech (detection synthesized speech) output in S2. The response receiving unit 7c accepts this selection as a response to the detection synthesized speech. The accepted response is stored, for example, in the storage unit 6.

[0050] (S104) Next, the judgment unit 7d judges whether the answer received in S3 is correct. Specifically, the judgment unit 7d judges the answer received in S3 as correct if the phoneme label is the same as the phoneme label in the test, and judges it as incorrect if they are different.

[0051] (S105) If the judgment unit 7d determines that the answer received in S103 is correct (S4, Yes), the judgment unit 7d then determines whether the test for all phoneme labels contained in the synthetic speech dataset for detection has been completed.

[0052] (S106) If the judgment unit 7d determines that the answer received in S103 is incorrect (S4, No), the processing unit 7e processes the detection synthesized speech data (wherein, regarding the detection synthesized speech data associated with the phoneme tags in the test, if the processing in S6 has already been performed, it refers to the latest processed detection synthesized speech data) to generate processed detection synthesized speech data.

[0053] The processing can be performed by pre-selecting one of several processing methods. These processing methods may include, for example, consonant phoneme sound pressure amplification, consonant duration prolongation, silence time insertion between consonants and vowels, and speech rate conversion, but are not limited to these.

[0054] Furthermore, for example, for synthesized speech generated from synthesized speech data for processing detection after at least one processing step in S106, as explained later, if the user selects an answer that indicates discomfort, the processing method used for processing performed again in S106 can be changed. Also, the order in which various processing methods are changed can be arbitrarily set; for example, it is preferable to change the processing methods according to the order of consonant phoneme sound pressure amplification, consonant duration extension, silence time insertion between consonants and vowels, and speech rate conversion, as explained later. Furthermore, the order in which various processing methods are changed can also be set to vary depending on the type of consonant (plosive consonants (voiced / voiceless), fricative consonants (voiced / voiceless), nasal consonants, etc.).

[0055] (S107) Next, the synthesized speech output unit 7b outputs the synthesized speech for processing detection through the speech output unit 4 based on the synthesized speech data for processing detection generated in S106.

[0056] (S108) Next, the response receiving unit 7c receives the response to the synthesized speech for processing detection output in S107. Specifically, after the user selects a perceived phoneme tag based on the speech output in S7, the response receiving unit 7c receives that selection as the response. Alternatively, if the user selects that the synthesized speech for processing detection output in S7 is unpleasant, the response receiving unit 7c receives that selection as the response. The received response is stored, for example, in the storage unit 6.

[0057] (S109) Next, the judgment unit 7d judges whether the response to the synthesized speech for processing detection received in S108 is correct. Specifically, the judgment unit 7d judges the response received in S8 as correct if it selects a phoneme tag and the phoneme tag involved in the response is the same as the phoneme tag in the test; otherwise, it judges it as incorrect. If the judgment unit 7d judges the response received in S108 as correct (S109, Yes), the process proceeds to step S105.

[0058] (S110) If the judgment unit 7d determines that the answer received in S108 is incorrect (S109, No), the judgment unit 7d determines whether the answer indicating that the synthesized speech for processing detection output in S108 is uncomfortable was accepted. If it is determined that no answer indicating discomfort was accepted (S110, No), the process proceeds to S106.

[0059] (S111) If the determination unit 7d determines that the synthesized speech output in S108 is unpleasant (S110, Yes), the determination unit 7d determines whether there are any other processing methods that have not been executed among the preset multiple processing methods. Then, if the result of the determination is affirmative (S111, Yes), the process proceeds to S106; if the result of the determination is negative (S111, No), the process proceeds to S105. Thus, the speech detection ends.

[0060] Figure 4 This is an example diagram of a test screen displayed by display unit 5 in the aforementioned speech detection processing. This test screen is displayed on display unit 5 by display control unit 74, for example, based on display data stored in storage unit 6. Figure 4 As shown, the test screen displays the selection sections for each phoneme label (e.g., [a], [ka], [sa], etc.) contained in the synthesized speech dataset. After the user listens to the speech output from the speech output unit 4, they can select from these selection sections when answering with the phoneme label perceived based on that speech. Furthermore, when the user answers that the output speech is uncomfortable, they can select the selection section indicating "uncomfortable" instead of choosing a phoneme label, or they can select both the phoneme label and the selection section indicating "uncomfortable".

[0061] (2-2-1) Correction of synthetic speech dataset for hearing aids

[0062] Figure 5A This is an example flowchart of the process for correcting synthetic speech datasets used in hearing aids.

[0063] (S201) First, the hearing aid synthesized speech dataset correction unit 72a refers to the storage unit 6 and selects the arbitrary phoneme label that the response to the detection synthesized speech received in S103 above is incorrect.

[0064] (S202) Next, regarding other phoneme labels that are different from the phoneme labels selected in S201, the hearing aid synthesized speech data set correction unit 72a determines whether there is an answer that is consistent with the phoneme label selected in S201 in the response of the synthesized speech for detection corresponding to the other phoneme labels.

[0065] (S203) If the determination result of S202 is positive (S202, Yes), the hearing aid synthesized speech dataset correction unit 72a performs a new association between the hearing aid synthesized speech data associated with the other phoneme tags and the phoneme tags selected in S201 in the hearing aid synthesized speech dataset. Then, the process proceeds to step S206.

[0066] (S204) If the judgment result of S202 is negative (S202, No), the hearing aid synthesized speech data set correction unit 72a determines whether the answer to the synthesized speech for processing detection received in S108 is correct.

[0067] (S205) If the hearing aid synthesized speech dataset correction unit 72a determines that the answer to the processed synthesized speech received in S108 is correct (S204, Yes), it performs the same processing as in S106 (the last processing) on ​​the hearing aid synthesized speech data associated with the phoneme tags selected in S201 in the hearing aid synthesized speech dataset. Then, in the hearing aid synthesized speech dataset, the hearing aid synthesized speech dataset correction unit 72a re-associates the processed hearing aid synthesized speech data with the phoneme tags selected in S201.

[0068] If the response to the synthesized speech for processing detection received in S108 above is determined to be incorrect (S204, No), the process proceeds to S206.

[0069] (S206) Next, regarding all phoneme labels for the incorrect responses to the synthesized speech received in S103 above, the hearing aid synthesized speech dataset correction unit 72a determines whether the processing of S201 to S205 has been completed. If the determination result is affirmative (S206, Yes), the processing ends; if the determination result is negative (S206, No), the processing proceeds to S201.

[0070] (2-2-2-1) Correction of the processing method set for each phoneme tag

[0071] Figure 5B yes Figure 2C Example diagram of the correction process flow for the processing method set of each phoneme tag in the image.

[0072] (S301) First, the processing method set correction unit 72b refers to the storage unit 6 and selects the arbitrary phoneme tag that was accepted in S103 above for the incorrect answer to the synthesized speech for detection.

[0073] (S302) Regarding the phoneme tag selected in S301, the processing method set correction unit 72b determines whether the response to the synthesized speech for processing detection received in S108 is correct.

[0074] (S303) If the processing method set correction unit 72b determines that the answer to the synthesized speech for processing detection received in S108 is correct (S302, Yes), then in the processing method set of each phoneme tag, for the processing method associated with the phoneme tag selected in S301, it synthesizes the same processing method as in S106 (wherein, the processing method involved in the last executed processing). Then, in the processing method set, the processing method set correction unit 72b associates the synthesized new processing method with the phoneme tag selected in S301.

[0075] (S304) Next, regarding all phoneme tags for the incorrect response to the synthesized speech used for detection received in S103 above, the processing method set correction unit 72b determines whether the processing of S301 to S303 has been completed. If the determination result is affirmative (S304, Yes), the processing ends; if the determination result is negative (S304, No), the processing proceeds to S201.

[0076] (2-2-2-2) Generation of processing method sets for each consonant group

[0077] For example, the processing unit 7 can generate a response based on the answer to "(2-1) speech detection" mentioned above. Figure 2D The processing unit 7 can generate a processing method F(1) corresponding to the voiceless consonant "(1)" based on multiple responses made by synthesized speech in speech detection for the detection of the voiceless consonant "(1)". In this case, the processing can be strengthened as the number of incorrect responses increases. The processing can include amplification of the sound pressure of the consonant phoneme, prolongation of the consonant duration, insertion of silence time between consonants and vowels, and speech rate conversion, etc.

[0078] (2-3-1) Hearing aid processing / speech synthesis methods

[0079] Figure 6A This is an example diagram illustrating the workflow of hearing aid processing using speech synthesis.

[0080] (S401) First, when a speaker emits a voice into the voice input unit 3, the voice input unit 3 supplies a signal based on the voice to the processing unit 7, and the text recognition processing unit 73a1 of the voice processing unit 73a performs text recognition processing on the voice. As a result, text corresponding to each part of the input voice is generated.

[0081] (S402) Next, the speech synthesis unit 73b1, for example, uses the hearing aid synthesized speech dataset that has been corrected by the aforementioned "(2-2-1) Correction of the synthesized speech dataset for hearing aids" process performed based on a listener as a test subject, to cause the speech output unit 4 to output synthesized speech corresponding to the generated text. Thus, the listener is able to hear synthesized speech generated based on the synthesized speech data, which has undergone appropriate replacement processing and / or processing according to the listener's hearing ability.

[0082] (2-3-2-1) Hearing aid processing / Input speech processing methods for each phoneme tag

[0083] Figure 6B This is an example diagram illustrating the workflow of hearing aid processing using input speech processing methods.

[0084] (S501) First, when a speaker emits a voice into the voice input unit 3, the voice input unit 3 supplies a signal based on the voice to the processing unit 7, and the text recognition processing unit 73a1 of the input voice processing unit 73a performs text recognition processing on the voice. As a result, text corresponding to each part of the input voice is generated. Furthermore, the input voice processing unit 73a separates words and particles (e.g., Japanese "wa", "ga", "wo", etc.) from the input voice, and detects consonant parts from these words and particles.

[0085] (S502) Next, the input speech processing unit 73b2 of the output speech processing unit 73b uses a set of processing methods to perform processing on each part of the input speech, and controls the speech output unit 4 to output the generated speech. At this time, for example, the input speech processing unit 73b2 may also use... Figure 2C The set of processing methods for each phoneme tag shown performs different processing on each phoneme tag contained in the input speech. Alternatively, for example, the input speech processing unit 73b2 can also use... Figure 2DThe processing method set for each consonant group shown determines the consonant group corresponding to the phoneme label of the input speech, and then performs processing according to the processing method corresponding to the determined consonant group. Alternatively, for example, the input speech processing unit 73b2 can also perform processing only on the initial consonant of each word using the processing method corresponding to the consonant group to which the consonant belongs. Specifically, for example, in the Japanese word "ka mi hi ko ki", the Japanese words "ka", "mi", "hi", "ko", and "ki" are consonants. In this case, processing can be performed only on the initial consonant "ka" according to the processing method corresponding to the consonant group to which "ka" belongs. This prevents the overall word length from becoming too long, thus often improving the listener's listening experience.

[0086] (2-3-2-2) Hearing aid processing / Input speech processing methods for each consonant group

[0087] Figure 6C This is an example diagram illustrating other actions in hearing aid processing using input speech processing methods.

[0088] (S601) First, when a speaker emits speech into the speech input unit 3, the speech input unit 3 supplies a signal based on the speech to the processing unit 7. The acoustic recognition processing unit 73a2 of the input speech processing unit 73a then performs acoustic processing on the speech, identifying the consonant groups contained in the input speech based on the acoustic features of the speech. This acoustic processing utilizes the principle that consonants belonging to the same consonant group have similar acoustic features. For example, for each consonant group, it can be determined whether the speech belongs to that consonant group based on whether the difference between a predefined spectrum and the spectrum of the input speech is within a predetermined threshold. Alternatively, after transforming the speech waveform to a frequency range, Bayesian estimation can be used to detect phonologies that sound like fricatives, plosives, nasals, and vowels. Thus, consonant groups corresponding to each part of the speech are generated.

[0089] (S602) Next, the input speech processing unit 73b2 of the output speech processing unit 73b uses Figure 2D The processing method set for each consonant group shown performs processing on the portion of the input speech corresponding to a specified consonant group within the processing method set, and controls the speech output unit 4 to output the generated speech. Thus, the speaker's input speech is processed by the processing methods for each consonant group, and the processed input speech is output.

[0090] The embodiments described above are for the purpose of understanding the present invention and are not intended to limit or explain the present invention. The elements, their arrangement, materials, conditions, shapes, and dimensions in the embodiments are not limited to the examples shown and can be appropriately modified. Furthermore, structures in different embodiments can be partially substituted or combined.

[0091] (3) Other

[0092] In other embodiments of the present invention, the speech detection device 1A and the hearing aid 1B may also be configured as separate information processing devices. Therefore, by configuring the speech detection device 1A and the hearing aid 1B as separate information processing devices, it is advantageous to miniaturize the hearing aid 1B. In particular, in various embodiments of the present invention, acoustic analysis of the input speech enables a reduction in the size of the hearing aid 1B.

[0093] Figure 7 This is a structural example diagram of a voice detection device 1A according to other embodiments of the present invention. The voice detection device 1A can be configured as any information processing device such as a personal computer (PC), smartphone, tablet computer, or dedicated terminal. The voice detection device 1A includes, for example, an operation unit 2, a voice input unit 3, a voice output unit 4, a display unit 5, a storage unit 6, and a processing unit 7. The processing unit 7 includes, for example, a detection unit 71, a correction unit 72, and a display control unit 74.

[0094] The voice detection device 1A also includes, for example, a communication unit 8A. This communication unit 8A has a communication interface circuit, allowing the voice detection device 1A to be connected to other information processing devices such as the hearing aid 1B via short-range wireless communication (such as Bluetooth) or mobile communication. The communication unit 8A supplies data received from the information processing device, such as the hearing aid 1B, to the processing unit 7. Furthermore, the communication unit 8A sends data supplied by the processing unit 7 to the hearing aid 1B, etc.

[0095] Figure 8 This is a structural example diagram of a hearing aid 1B according to other embodiments of the present invention. The hearing aid 1B can be constructed from any information processing device such as a personal computer (PC), smartphone, tablet computer, or dedicated terminal. The hearing aid 1B includes, for example, an operation unit 2, a voice input unit 3, a voice output unit 4, a display unit 5, a storage unit 6, and a processing unit 7. The processing unit 7 includes, for example, a hearing aid processing unit 73 and a display control unit 74. The storage unit 6 stores a dataset of synthesized speech for hearing aids or a set of processing methods.

[0096] The hearing aid 1B also includes, for example, a communication unit 8B. This communication unit 8B has a communication interface circuit, allowing the hearing aid 1B to be connected to other information processing devices such as the speech detection device 1A via near-field wireless communication (Bluetooth, a Japanese registered trademark) or mobile communication. The communication unit 8A supplies data received from the information processing device such as the speech detection device 1A to the processing unit 7. Furthermore, the communication unit 8A sends data supplied by the processing unit 7 to the speech detection device 1A, etc.

[0097] In other embodiments of the present invention, when the speech detection device 1A detects the speech of the test subject, it determines the overall degree of micro-correction of speech synthesis or enhancement processing not for each individual phoneme tag or each consonant group, but after comparing it with each of a plurality of predefined model speech.

[0098] [Explanation of reference numerals in the attached drawings]

[0099] 1…speech processing device, 2…operation unit, 3…speech input unit, 4…speech output unit, 4…display unit, 6…storage unit, 7…processing unit, 7a…phoneme tag selection unit, 7b…synthesized speech output unit, 7c…response acceptance unit, 7d…judgment unit, 7e…processing unit, 7f…synthesized speech dataset correction unit for hearing aids, 7g…replacement processing unit, 74…display control unit, 7i…speech analysis unit, 7j…speech synthesis unit, 1A…speech detection device, 1B…hearing aid.

Claims

1. A method for correcting a synthetic speech dataset for hearing aids, comprising correcting the synthetic speech dataset for hearing aids based on detection results using a synthetic speech dataset for detection, wherein for each of a plurality of phoneme labels, each is associated with synthetic speech data for outputting synthetic speech for detection, and for each of the plurality of phoneme labels, each is associated with synthetic speech data for outputting synthetic speech for hearing aids, characterized in that, The method includes: Based on the first synthetic speech data associated with the first phoneme label in the synthetic speech dataset for detection, the first synthetic speech for detection is output. The phoneme label represented by the first detected synthesized speech is perceived by the user and then subject to the first answer selected by the user as described above. Based on the second synthetic speech data associated with the second phoneme label in the synthetic speech dataset for detection, the second synthetic speech for detection is output; The phoneme label represented by the second synthesized speech output is perceived by the user and then subject to the second answer selected by the user. If the first answer is consistent with the second phoneme label and the second answer is inconsistent with the second phoneme label, in the synthetic speech dataset for hearing aids, the second synthetic speech data for hearing aids associated with the second phoneme label is replaced, and the first synthetic speech data for hearing aids associated with the first phoneme label is associated with the second phoneme label.

2. The method for correcting a synthetic speech dataset for hearing aids according to claim 1, characterized in that... Also includes: Based on the processed synthetic speech data for detection obtained after processing the second synthetic speech data for detection, the processed synthetic speech for detection is output. The phoneme labels represented by the synthesized speech used for processing and detection are perceived by the user and then subject to a third response selected by the user. If the third answer matches the second phoneme tag, the same processing is performed on the second hearing aid synthesized speech data associated with the second phoneme tag as described above.

3. The method for correcting a synthetic speech dataset for hearing aids according to claim 2, characterized in that, The processing includes at least one of the following: amplification of consonant phonemes, extension of consonant duration, insertion of silence time between consonants and vowels, and speech rate conversion.

4. The method for correcting a synthetic speech dataset for hearing aids according to claim 2 or 3, characterized in that... Also includes: The fourth response indicated that the synthesized speech output for the processing and testing was unpleasant to hear; If the first answer matches the second phoneme label, the second answer does not match the second phoneme label, and the fourth answer is accepted, then in the synthetic speech dataset for hearing aids, the second synthetic speech data for hearing aids is replaced, and the first synthetic speech data for hearing aids is associated with the second phoneme label.

5. The method for correcting a synthetic speech dataset for hearing aids according to any one of claims 1 to 3, characterized in that, The phoneme label represents any one of the following: consonant phonemes, vowel phonemes, or combinations of consonants and vowels.

6. A hearing aid method, comprising: Accepts voice input; Analyze speech and generate text; The method for correcting a synthetic speech dataset for hearing aids according to any one of claims 1 to 5, after correcting the synthetic speech dataset corresponding to the generated text, outputs speech based on the synthetic speech data associated with the phoneme tags corresponding to the generated text in the synthetic speech dataset.

7. A method for modifying a processing method set, modifying the processing method set based on detection results using a detection synthetic speech dataset, wherein for each of a plurality of phoneme labels, detection synthetic speech data for outputting detection synthetic speech is associated, and the processing method set specifies processing of the portion of the input speech corresponding to a specific phoneme label, the method being characterized in that it includes: Based on the synthetic speech data associated with the first phoneme label in the synthetic speech dataset for detection, output synthetic speech for detection; The phoneme labels represented by the synthesized speech used for detection are perceived by the user and are subject to the first answer selected by the user. The synthetic speech data associated with the first phoneme label in the synthetic speech dataset for detection is processed by a specified processing method to generate processed synthetic speech data for detection. Based on the processed synthetic speech data for detection, the processed synthetic speech for detection is output. The phoneme labels represented by the synthesized speech used for processing and detection are perceived by the user and then subject to the second answer selected by the user. If the first answer is inconsistent with the first phoneme tag and the second answer is consistent with the first phoneme tag, in the processing method set, for the processing associated with the first phoneme tag, a processing method identical to the specified processing method is synthesized, and the synthesized new processing method is associated with the first phoneme tag.

8. A hearing aid method, comprising: It accepts and processes voice input from the speaker. The speech produced by the speaker is analyzed; Based on the analysis results of the speech emitted by the speaker, the speech emitted by the speaker is processed by the prescribed processing method after modification using the method for modifying the processing method set as described in claim 7. The processed speech is then output.

9. The hearing aid method according to claim 8, characterized in that, During the parsing of the speaker's speech, instead of performing parsing to reduce ambient noise mixed in with the speaker's speech, parsing is performed to enhance the features of the speaker's speech.

10. The hearing aid method according to claim 8 or 9, characterized in that, In the process of parsing the speech produced by the speaker, acoustic analysis is performed on the speech produced by the speaker to identify consonant groups, which represent the consonant features of the speech produced by the speaker. During the processing of the speech produced by the speaker, the speech produced by the speaker is processed by the prescribed processing method corresponding to the consonant group.

11. The hearing aid method according to claim 8 or 9, characterized in that, During the process of parsing the speech produced by the speaker, text recognition processing of the speech is performed to identify the text corresponding to the speech produced by the speaker. During the processing of the speech produced by the speaker, the speech produced by the speaker is processed by the prescribed processing method corresponding to the text.

Images