Recognition or synthesis of human-uttered harmonic sounds

IN598008BActive Publication Date: 2026-08-05FRIDMAN MINTZ BORIS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
IN202317070989
Authority / Receiving Office
IN · IN
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-05-18
Filing Date
2023-10-18
Publication Date
2026-08-05
Estimated Expiration
2042-05-13

AI Technical Summary

Technical Problem

Current speech processing technologies fail to effectively utilize harmonic acoustic components for recognizing and synthesizing human-uttered speech, relying primarily on fundamental frequencies and disregarding harmonic information, which limits their ability to accurately identify and generate phonetic chords.

Method used

A computer-implemented method that analyzes temporal waveforms to identify phonetic chords by detecting harmonic components with intensities exceeding a threshold, using primary cap frequencies greater than 410 Hz to select corresponding phonetic notes, and synthesizes sound by incorporating these harmonic components into the waveform, thereby exploiting harmonic spectral information for improved recognition and synthesis.

Benefits of technology

This approach enhances the accuracy of speech recognition and synthesis by leveraging harmonic components, allowing for more precise identification and generation of phonetic chords, demonstrating that vowel identity can be based on categorical perception of harmonic features.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

Within each harmonic spectrum of a sequence of spectra derived from analysis of a waveform representing human speech are identified two or more fundamental or harmonic components that have frequencies that are separated by integer multiples of a fundamental acoustic frequency. The highest harmonic frequency that is also greater than 410 Hz is a primary cap frequency, which is used to select a primary phonetic note that corresponds to a subset of phonetic chords from a set of phonetic chords for which acoustic spectral is available. The spectral data can also include frequencies for primary band, secondary band (or secondary note), basal band, or reduced basal band acoustic components, which can be used to select a phonetic chord from the subset of phonetic chords corresponding to the selected primary note.
Need to check novelty before this filing date? Find Prior Art

Description

FIELD OF THE INVENTION

[0001] The field of the present invention relates to recognition or synthesis of human-uttered speech.In particular, computer-implemented methods are disclosed for recognizing or synthesizing humanuttered harmonic sounds.BACKGROUND

[0002] Some examples of speech processing or synthesizing apparatus or methods are disclosed in:- U. S. Pat. 5,406,635 entitled "Noise attenuation system" issued 04 / 11 / 1995 to Jarvinen;- U. S. Pub. 2004 / 0181411 entitled "Voicing index controls for CELP speech encoding"published 09 / 16 / 2004 in the name of Gao;- U. S. Pat. 6,925,435 entitled "Method and apparatus for improved noise reduction in a speechencoder" issued 08 / 02 / 2005 to Gao;- U. S. Pat. 7,516,067 entitled "Method and apparatus using harmonic-model-based front endfor robust speech recognition" issued 04 / 07 / 2009 to Seltzer et al;- U. S. Pat. 7,634,401 entitled "Speech recognition method for determining missing speech"issued 12 / 15 / 2009 to Fukuda;- U. S. Pat. 8,566,088 entitled "System and method for automatic speech to text conversion"issued 10 / 22 / 2013 to Pinson et al;- U. S. Pat. 8,606,566 entitled "Speech enhancement through partial speech reconstruction"issued 12 / 10 / 2013 to Li et al;- U. S. Pat. 8,812,312 entitled "System, method and program for speech processing" issued08 / 19 / 2014 to Fukuda et al;- U. S. Pat. 9,087,513 entitled "Noise reduction method, program product, and apparatus"issued 07 / 21 / 2015 to Ichikawa et al;- U. S. Pat. 9,190,072 entitled "Local peak weighted-minimum mean square error (LPW-MMSE) estimation for robust speech" issued 11 / 17 / 2015 to Ichikawa;- U. S. Pat. 9,570,072 entitled "System and method for noise reduction in processing speechsignals by targeting speech and disregarding noise" issued 02 / 14 / 2017 to Pinson;- Dieter Maurer, Acoustic of the Vowel: Preliminaries, Peter Lang AG, Bern 2016;- Bruno H. Repp, "Categorical Perception: Issues, Methods, Findings," SPEECH ANDLANGUAGE: Advances in Basic Research and Practice, Vol. 10 p. 243 (Academic Press1984), https: / / doi.org / 10.1016 / B978-0-12-608610-2.50012-1;- U. S. Pat. 9,147,393 entitled "Syllable based speech processing method" issued 09 / 29 / 2015to Fridman-Mintz (the inventor here);- U. S. Pat. 9,460,707 entitled "Method and apparatus for electronically recognizing a series ofwords based on syllable-defining beats" issued 10 / 04 / 2016 to Fridman-Mintz (the inventorhere); and- U. S. Pat. 9,747,892 entitled "Method and apparatus for electronically synthesizing acousticwaveforms representing a series of words based on syllable-defining beats" issued08 / 29 / 2017 to Fridman-Mintz (the inventor here).

[0003] Each of the last three patents listed above (each issued to Fridman-Mintz) is incorporated byreference as if set forth herein in its entirety.SUMMARY

[0004] A computer-implemented method is employed for identifying one or more phonetic chords(e.g., harmonic phones) represented within an electronic temporal waveform derived from an utteranceof human speech. In some instances, from a temporal sequence of acoustic spectra derived from thewaveform, each of a plurality of harmonic acoustic spectra among the temporal sequence are analyzedto identify within that harmonic acoustic spectrum two or more fundamental or harmonic componentsthat each have an intensity exceeding a detection threshold. The identified components havefrequencies that are separated by at least one integer multiple of a fundamental acoustic frequencyassociated with that acoustic spectrum. For at least some of the plurality of acoustic spectra, a primarycap frequency is identified, the primary cap frequency being greater than 410 Hz and also being thehighest harmonic frequency among the identified harmonic components. For each acoustic spectrumfor which a primary cap frequency is identified, that identified primary cap frequency is used to select asa primary phonetic note at least one phonetic note from among a set of phonetic notes. The selectedprimary phonetic note corresponds to a subset of phonetic chords from among a set of phonetic chords.

[0005] In some instances the acoustic spectra of the temporal sequence can correspond to one of asequence of temporal sample intervals of the waveform; in other instances the acoustic spectracorrespond to one of a sequence of distinct temporal segments during which a time-dependent acousticspectrum of the waveform remains consistent with a single phonetic chord. In some instances, aphonetic chord can be selected on the basis of harmonic components present in the harmonic acousticspectrum that include one or more of a primary band, a secondary band, a basal band, or a reducedbasal band (each discussed below).

[0006] A computer-implemented method is employed for analyzing human-uttered speech andgenerating spectral data that can be used for the identification of harmonic phonetic chords in the method described above. For each phonetic chord, waveforms derived from respective utterances ofthat phonetic chord by one or more human test subjects are spectrally analyzed. The spectral analysisincludes, for each electronic waveform, estimation of a fundamental acoustic frequency andidentification of two or more fundamental or harmonic components, each having an intensity thatexceeds a detection threshold and an acoustic frequency that is the fundamental acoustic frequency ora harmonic thereof. The primary cap frequency is identified and stored for each phonetic chord, alongwith acoustic frequencies for each identified fundamental or harmonic component. A focal frequencycan be estimated for a primary phonetic note common to a subset of phonetic chords, using theobserved primary cap frequencies (e.g., mean, or median). In some instances, stored spectral data caninclude data for one or more of a primary band, a secondary band, a basal band, or a reduced basalband.

[0007] A computer-implemented method is employed for synthesizing a temporal segment of anelectronic waveform. Applying the waveform segment to an electroacoustic transducer produces soundof a phonetic chord. Data indicative of a primary phonetic note corresponding to the selected phoneticchord, and a focal frequency of that primary phonetic note, are used to determine a primary capfrequency. The primary cap frequency is (i) an integer multiple of a selected fundamental frequency,(ii) greater than 410 Hz, and (iii) closer to the focal frequency of the corresponding primary note than tofocal frequencies of other phonetic notes. A harmonic component at the primary cap frequency isincluded in the waveform segment. The primary cap frequency is the largest frequency amongharmonic components included in the waveform segment. The waveform segment can further includecomponents at one or more harmonic frequencies of a primary band, a secondary band, a basal band,or a reduced basal band. The method can be repeated for each phonetic chord among a temporalsequence of multiple different harmonic or hybrid segments, along with inharmonic or silencedsegments, and transitional segments between them, that together comprise a human utterance.

[0008] Objects and advantages pertaining to recognition or synthesis of human speech may becomeapparent upon referring to the example embodiments illustrated in the drawings and disclosed in thefollowing written description or appended claims.

[0009] This Summary is provided to introduce a selection of concepts in a simplified form that arefurther described below in the Detailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Fig. 1 is an example acoustic spectrum of a recorded natural [ɛ|a] phonetic chord. Intensitydetection thresholds, basal cap frequency, secondary cap frequency, and primary cap frequency areindicated with their corresponding frequencies.

[0011] Fig. 2 is an example of an acoustic spectral data table of basic phonetic notes, each followedby its defining focal frequency (e.g., mean, median, or otherwise characteristic cap frequency ofcorresponding phonetic note categories). It is proposed as an acoustically based reference for thecategorical identification of harmonic phones as phonetic chords across all spoken languages. The thirdcolumn indicates how the proposed set of phonetic notes is correlated with specific phoneticarticulations, as represented by the canonical alphabet of the International Phonetic Association (IPA).

[0012] Figs. 3A and 3B together constitute a single example acoustic spectral data table showing, formultiple recorded phonetic chords, their primary cap frequencies, and (if applicable) their secondarycap frequencies. The acoustic data were extracted from recordings of John Wells, Jill House, and PeterLadefoged (available at phonetics.ucla.edu / course / chapter1 / wells / wells.html).

[0013] Figs. 4A and 4B show example temporal waveforms and corresponding acoustic spectra for aone-note [u] phonetic chord. The waveform of Fig. 4A is synthesized using a fundamental frequency of192 Hz; the waveform of Fig. 4B is recorded and exhibits an estimated fundamental frequency of211.4 Hz.

[0014] Fig. 5 shows example temporal waveform and corresponding acoustic spectrum for arecorded two-note [ʌ|u] phonetic chord exhibiting an estimated fundamental frequency of 103.4 Hz.

[0015] Figs. 6A and 6B show example temporal waveforms and corresponding acoustic spectra for athree-note [ã|o] phonetic chord. The waveform of Fig. 6A is recorded and exhibits an estimatedfundamental frequency of 127 Hz; the waveform of Fig. 6B is synthesized using a fundamentalfrequency of 127 Hz.

[0016] Fig. 7 shows example temporal waveform and corresponding acoustic spectrum for arecorded two-note [i|y] phonetic chord exhibiting an estimated fundamental frequency of 138.4 Hz.

[0017] Fig. 8 shows example temporal waveform and corresponding acoustic spectrum for a two note [i|y] phonetic chord synthesized using an estimated fundamental frequency of 138.4 Hz.

[0018] The examples or embodiments depicted are shown only schematically; all features may notbe shown in full detail or in proper proportion; for clarity certain features or structures may beexaggerated or diminished relative to others or omitted entirely; the drawings should not be regarded asbeing to scale unless explicitly indicated as being to scale. The embodiments shown are only examples and should not be construed as limiting the scope of the present disclosure or inventive subject matter.Identical reference numbers refer to like elements throughout the different figures.DETAILED DESCRIPTION

[0019] The following detailed description should be read with reference to the drawings. The detaileddescription illustrates principles of the subject matter by way of example, not by way of limitation.

[0020] Methods disclosed herein rely in part on the observation that, for phonetic chords (e.g.,harmonic phones such as vowels, nasalized vowels, nasals, and approximants), harmonic acousticspectral components (i.e., components having frequencies that are integral multiples of a fundamentalacoustic frequency) carry the core of linguistically relevant information signaling those phonetic chords.The disclosed methods therefore include recognition or synthesis of phonetic chords that employdetection, identification, or generation of such harmonic components. Each phonetic note correspondsto a basic harmonic element (e.g., as in the example list of Fig. 2); each phonetic chord can includemultiple harmonic frequencies in the context of a given phonetic note (e.g., as in the examples listed inFigs. 3A and 3B).

[0021] As part of operating a speech recognition system, a computer-implemented method isemployed to identify one or more phonetic chords that are represented within an electronic temporalwaveform derived from an utterance of human speech. A temporal sequence of acoustic spectra isderived from the waveform in a suitable way (discussed further below), and some of those acousticspectra are identified as being harmonic acoustic spectra, i.e., spectra that contain frequencycomponents at one or more integer multiples of a fundamental acoustic frequency. For some or all ofthe harmonic acoustic spectra in the temporal sequence, two or more fundamental or harmoniccomponents are identified that have respective intensities that exceed a detection threshold and thatare separated by an integer multiple of a corresponding fundamental acoustic frequency for thatharmonic acoustic spectrum. In some instances the fundamental frequency component might bemissing or hidden for a number of reasons, e.g., interference with other nearby intense frequencycomponents, attenuation by obstructions in an acoustic pathway, or weak production of low acousticfrequencies by speakers of small stature. Even if no fundamental component is present in a harmonicspectrum, that harmonic spectrum can nevertheless be characterized by the correspondingfundamental acoustic frequency, integer multiples of which separate harmonic components of theharmonic spectrum.

[0022] The fundamental and harmonic components typically manifest themselves as peaked spectralfeatures rising above a background level (e.g., as in Fig. 1). The corresponding frequency of acomponent can be defined in a suitable way, e.g., the frequency of the maximum intensity or the centroid of the component. Non-zero width of the spectral features can result in some degree ofuncertainty in assigning corresponding frequencies to those features, and typically there is someleeway in determining whether a spectral feature is an integer multiple of a fundamental frequency. Theintensity of a component can be characterized in a suitable way, e.g., as a maximum or peak intensityor as an integrated intensity; intensities are typically expressed in relative terms (e.g., decibels), relativeto a background noise level, a hearing threshold level, or another reference level. However the relativeintensities are defined, a suitable detection threshold can be selected for deciding whether a givenspectral feature has been "detected" and "identified" as a fundamental or harmonic component of agiven harmonic spectrum. In some examples, a certain decibel level might be specified, above aselected reference level or above a background noise level. In some examples, the detection thresholdcan be frequency dependent, and so differ among the fundamental and harmonic components of aspectrum (e.g., as in Fig. 1).

[0023] For at least some of the harmonic acoustic spectra, a primary cap frequency is identified thatis the highest harmonic frequency among the identified harmonic components, provided that suchhighest harmonic frequency is also greater than 410 Hz. For each harmonic acoustic spectrum forwhich such a primary cap frequency is identified, that primary cap frequency is used to select at leastone phonetic note from among a set of basic phonetic notes (e.g., as in the table of Fig. 2). Thatselection is enabled because it has been observed, through recording and analysis of numeroushuman-uttered phonetic chords, that it is the primary cap frequency that most closely correlates with theparticular phonetic chord being uttered. Data gathered for establishing the table of basic phonetic notes(e.g., as in Fig. 2) can include for each note a characteristic primary cap frequency (i.e., a focalfrequency) and the primary note label with which it is correlated. Other phonetic note datasets can beemployed; generation of such phonetic note datasets is discussed below. Other speech recognitionsystems frequently rely on the fundamental frequency, and in some cases two or three formants(temporally aligned peaks of intensities whose frequency values are defined as independent of thefundamental frequency), while substantially disregarding any harmonic components present in theacoustic spectrum, under the assumption that vowel perception and recognition are not based oncategorical perception or feature analysis. The methods disclosed herein exploit those previouslyunutilized harmonic spectral components for identification of the corresponding phonetic chord and itsone or more constituent phonetic notes, demonstrating that vowel identity can in some instances bebased on prototype categorical perception.

[0024] For a given phonetic chord uttered at different pitches (i.e., with different fundamentalfrequencies), the primary cap frequency can vary, so a definitive identification (e.g., an unambiguousselection from the data table of Fig. 2) cannot always be realized. An example set of instantiated phonetic chord data is shown in the table of Figs. 3A and 3B. Each instantiated primary cap frequencyfalls between focal frequencies of a pair of adjacent phonetic notes (e.g., in Table 2), or else is higherthan the highest focal frequency. If between a pair of phonetic notes (labelled as "identified" and"proximal" chords in the table of Figs. 3A and 3B), the instantiated phonetic chord can be identified ascorresponding to one of those two notes (the "identified" chord in this example). If above the highestfocal frequency (e.g., above 4160 Hz in the table of Fig. 2), the instantiated chord can be identified ascorresponding to an [i] phonetic note. In some examples, an instantiated phonetic chord can beidentified as corresponding to the phonetic note, of a pair of adjacent notes, that has a focal frequencycloser to the instantiated primary cap frequency. The confidence columns in the table of Figs. 3A and3B indicate the relative closeness of the instantiated primary cap frequency to the corresponding pair ofadjacent focal frequencies. Other suitable tests or algorithms can be employed for deciding amongdifferent possible phonetic chords that might be consistent with a particular instantiated harmonicspectrum. In some examples, an artificial intelligence (AI) system or a neural network can be trained tomake the selection; in some examples so-called "fuzzy logic" algorithms can be employed. In someexamples, identification of additional fundamental or harmonic spectral components (e.g., primary band,secondary band, basal band, as discussed below) of an observed harmonic spectrum can be employedto distinguish among different phonetic notes that have focal frequencies consistent with the observedprimary cap frequency, secondary cap frequency, or basal cap frequency (discussed further below).

[0025] Some example methods can include deriving from the electronic temporal waveform thetemporal sequence of acoustic spectra; in other examples the temporal sequence of acoustic spectraare already derived from the electronic waveform before performing the method. In either case thetemporal sequence of acoustic spectra can be derived from the waveform in a suitable way, e.g., usingan electronic spectrum analyzer to process the electronic waveform itself, or using Fourier transformtechniques to process a numerical representation of the waveform.

[0026] In some examples, each of the acoustic spectra corresponds to one of a sequence oftemporal sample intervals of the waveform; such sample intervals can be of equal durations, but thatneed not always be so. At least some of the acoustic spectra among such a temporal sequence can beclassified as only one of harmonic, inharmonic, hybrid, or silenced. For at least some of the temporalsample intervals classified as harmonic, the methods described above or below can be employed toidentify a phonetic chord among a set of phonetic chords (each including its characteristic combinationof notes) as corresponding to that temporal sample interval of the electronic waveform.

[0027] In some examples, each of the acoustic spectra corresponds to one of a sequence of distincttemporal segments during which that time-dependent acoustic spectrum remains consistent with a single phonetic chord. A determination that the acoustic spectrum "remains consistent with a singlephonetic chord" can be made without having already identified the chord; it need only be observed thatno transition has occurred to an acoustic spectrum indicative of a different chord. At least some of thetemporal segments can be classified as only one of harmonic, inharmonic, hybrid, or silenced based onthe acoustic spectrum thereof. For at least some of the temporal segments classified as harmonic, themethods described above or below can be employed to identify a phonetic chord among a set ofphonetic chords as corresponding to that temporal segment of the electronic waveform.

[0028] Identification of additional fundamental or harmonic frequencies, in addition to the primary capfrequency, can be employed to further facilitate identification of phonetic chords that correspond toharmonic spectra in the temporal sequence. In some instances, identification of those additionalfundamental or harmonic components can be helpful in distinguishing among phonetic chords havingsimilar primary cap frequencies. Additional fundamental or harmonic components can in someexamples form one or more of a primary band, a secondary band, or a basal band.

[0029] In some examples, a primary band of harmonic components can be identified in at least someharmonic spectra of the temporal sequence. The primary band can include harmonic components at theprimary cap frequency and at the one, two, three, or more largest consecutive multiples of thefundamental acoustic frequency that are (i) less than the primary cap frequency, (ii) greater than410 Hz, and (iii) greater than the smallest integer multiple of the fundamental acoustic frequency that isabove 410 Hz. Stored data for the set of phonetic chords can include, in addition to the primary capfrequency, frequencies of those other harmonic components of the primary band. Based on acomparison of that primary band data with observed frequencies of a particular harmonic spectrumderived from the electronic waveform, a phonetic chord of the set can be selected as corresponding tothat harmonic spectrum and the corresponding temporal portion of the waveform. Figures 4A / 4B, 5,6A / 6B, 7, and 8 show a few examples of recorded and synthesized waveforms and harmonic spectrathat could potentially be differentiated based on a primary band of harmonic components.

[0030] In some examples, a secondary band of harmonic components can be identified in at leastsome harmonic spectra of the temporal sequence. The secondary band can include harmoniccomponents at one or more harmonic acoustic frequencies that are greater than the smallest integermultiple of the fundamental acoustic frequency that is above 410 Hz and separated from the primarycap frequency by at least one intervening multiple of the fundamental acoustic frequency for which theacoustic spectrum lacks a harmonic component. In other words, a secondary band falls below a"harmonic gap" or one or more "missing harmonics" between the lowest-frequency component of theprimary band and the highest-frequency component of the secondary band. The frequency of the highest-frequency component of the secondary band can be referred to as the secondary capfrequency.

[0031] Stored data for the set of phonetic chords can include, in addition to the primary capfrequency (or a corresponding primary note), the secondary cap frequency (or a correspondingsecondary note) and frequencies of one or more other harmonic components of the secondary band (ifany). Based on a comparison of that secondary band data with observed frequencies of a particularharmonic spectrum derived from the electronic waveform, a phonetic chord of the set can be selectedas corresponding to that harmonic spectrum and the corresponding temporal portion of the waveform.For example, observation of a secondary band can enable distinction between (i) a first phonetic chordhaving a secondary band separated from its primary band by one missing harmonic, and (ii) a secondphonetic chord having a secondary band separated from its primary band by two or more missingharmonics. In some examples, comparison of secondary band data and observed components can beused in conjunction with comparison of primary band data and observed components; in otherexamples, secondary data and components can be used without also using primary band data andcomponents. Figures 4A / 4B, 5, 6A / 6B, 7, and 8 show several examples of recorded and synthesizedwaveforms and harmonic spectra that could potentially be differentiated based on a secondary band ofharmonic components.

[0032] In some examples, a basal band of harmonic components can be identified in at least someharmonic spectra of the temporal sequence. The basal band can include harmonic components at oneor more fundamental or harmonic acoustic frequencies that are less than 410 Hz, and can also includea harmonic component at the smallest harmonic acoustic frequency above 410 Hz (unless thatharmonic frequency is the primary cap frequency). The frequency of the highest-frequency componentof the basal band can be referred to as the basal cap frequency. In examples wherein the primary capfrequency is also the only harmonic frequency above 410 Hz, the harmonic spectrum includes only theprimary cap component and basal band components; there are no other primary band components andno secondary band components. Stored data for the set of phonetic chords can include, in addition tothe primary cap frequency (or corresponding primary note), the basal cap frequency and frequencies ofone or more other harmonic components of the basal band. Based on a comparison of that basal banddata with observed frequencies of a particular harmonic spectrum derived from the electronicwaveform, a phonetic chord of the set can be selected as corresponding to that harmonic spectrum andthe corresponding temporal portion of the waveform.

[0033] In some examples, comparison of basal band data and observed components can be used inconjunction with comparison of primary band data and observed components; in other examples, basal band data and components can be used in conjunction with comparison of secondary band data andobserved components; in other examples, basal band data and components can be used in conjunctionwith comparison of both primary and secondary band data and observed components; in otherexamples, basal data and components can be used without also using either primary or secondaryband data and components. Figures 4A / 4B, 5, 6A / 6B, 7, and 8 show several examples of recorded andsynthesized waveforms and harmonic spectra that could potentially be differentiated based on a basalband of harmonic components.

[0034] In some examples, a harmonic acoustic spectrum of the temporal sequence might includeonly basal band components with a cap frequency below 410 Hz. Such a harmonic acoustic spectrumis referred to as a reduced basal band, and it can correspond to certain harmonic acoustic schemata(e.g., nasals, nasalized vowels, or approximants) or hybrid acoustic schemata (e.g., voiced fricatives).Stored data for those acoustic schemata can include frequencies of harmonic components of thereduced basal band. Based on a comparison of that reduced basal band data with observedfrequencies of a particular harmonic spectrum derived from the electronic waveform, a harmonic orhybrid acoustic schema of the set can be selected as corresponding to that harmonic spectrum and thecorresponding temporal portion of the waveform. Presence or absence of higher-frequency inharmonicfrequency components also can be employed to distinguish among (i) reduced-basal-band harmonicschemata (e.g., corresponding to the first row in Fig. 2) and (ii) hybrid schemata (e.g., voiced fricativessuch as β, v, ð, z, ʒ, ʐ, ʝ, ɣ, ʁ, ʕ, ɦ).

[0035] Various methods disclosed herein for recognizing phonetic notes and chords within utteredhuman speech rely upon stored data indicative of harmonic spectra for each of multiple phonetic notesor chords, including harmonic frequencies expected for each phonetic note or chord (e.g., the tables ofFigs. 2, 3A, and 3B). Methods for generating that data for a set of phonetic notes and cords can includeperforming an acoustic spectral analysis on multiple utterances of phonetic chords by one or morehuman test subjects who are instructed to articulate certain specific sounds or words, with informationbeing kept labeling the instructed utterances. Such a process is analogous to "teaching" the system torecognize each phonetic chord when it occurs within the waveform representing an utterance of humanspeech. In some examples, the harmonic spectral data can be generated based on utterances of asingle human test subject, to train the system for recognition of speech from that particular subject. Inother examples, the phone spectral data can be generated based on averages or other analyses ofutterances of multiple human test subjects, to train the system for more general speech recognition.

[0036] For each phonetic note or chord, the spectral analysis includes estimation of a fundamentalacoustic frequency and identification of two or more fundamental or harmonic components detected or identified in the spectrum. As described above, a component is "detected" or "identified" is one havingan intensity that exceeds one or more suitably defined detection thresholds. Each component has anacoustic frequency that is the fundamental acoustic frequency or a harmonic acoustic frequency (i.e.,an integer multiple of the fundamental acoustic frequency). The spectrum can be characterized by afundamental acoustic frequency, although the spectrum may or may not include a fundamentalcomponent at that fundamental acoustic frequency. The highest harmonic acoustic frequency amongthe identified harmonic components that is also greater than 410 Hz is identified as the primary capfrequency, and that frequency is stored as part of the data for the phonetic chord. Also stored areacoustic frequencies for each identified fundamental or harmonic component, as described above. Forsome of the phonetic chords, the data can also include acoustic frequencies for identified harmoniccomponents of one or more of a primary band, a secondary band, or a basal band, as described above.

[0037] To estimate a focal frequency for a phonetic note, multiple utterances of a subset of phoneticchords, at multiple different fundamental frequencies by one or more human test subjects, can bespectrally analyzed. The phonetic chords of the subset share the primary note. The focal frequency ofthe common primary note can be estimated from the primary cap frequencies of the uttered phoneticchords. The focal frequency can be estimated from the observed primary cap frequencies in a suitableway, e.g., mean or median of the observed primary cap frequencies.

[0038] For some harmonic acoustic schemata (e.g., nasals, nasalized vowels, or approximants) orhybrid acoustic schemata (e.g., voiced fricatives), the spectral analysis may identify fundamental orharmonic components only at frequencies below 410 Hz, described above as reduced basal bandfrequencies. Acoustic frequencies of those reduced basal band components can be included in the dataset describing the corresponding harmonic or hybrid acoustic schemata, along with an indicator ofabsence or presence, respectively, of higher-frequency inharmonic components.

[0039] Recognition of the importance of harmonic components in the recognition of phonetic notesand chords in human-uttered speech also allows for improved speech synthesis. The spectral datadescribed above used for recognition of phonetic chords can also be employed to produce or generatethe sound of those chords. To synthesize a selected phonetic chord, a primary cap frequency can bedetermined based on a corresponding primary note and a selected fundamental frequency (i.e., pitch).For a harmonic phonetic chord, the primary cap frequency is (i) an integer multiple of the selectedfundamental frequency, (ii) greater than 410 Hz, and (iii) closer to the focal frequency of thecorresponding primary note than to focal frequencies of other phonetic notes. A frequency componentat the primary cap frequency is included in a synthesized waveform segment. In some examples, if theselected phonetic chord includes a secondary note, a secondary cap frequency can be determined as described above for the primary note and cap frequency, and a frequency component at the secondarycap frequency can be included in the synthesized waveform segment.

[0040] The method can be repeated for each set of chords and fundamental frequencies among atemporal sequence of multiple different interspersed harmonic segments (e.g., vowels, nasalizedvowels, nasals, and approximants) and hybrid segments (e.g., voiced fricatives). The synthesizedharmonic segments, together with synthesized inharmonic segments (e.g., voiceless fricatives),synthesized silenced segments (e.g., occlusives, within trills, and within flaps), and the transitionalsegments between them, comprise a synthesized human utterance. The electronic waveform thusproduced using the spectral data is applied to an electroacoustic transducer (e.g., a speaker) toproduce the sound of the synthesized phonetic chord. A string of such chords can be produced toconstruct a synthesized speech sequence.

[0041] In some examples, to produce one of the phonetic chords for which spectral data areavailable, an electronic waveform corresponding to that chord is created using the correspondingspectral data (the primary cap frequency along with acoustic frequencies of harmonic components). Forsome phonetic notes and chords, the data can include acoustic frequencies for identified harmoniccomponents of one or more of a primary band, a secondary band, or a basal band, as described above.For some harmonic or hybrid acoustic schemata (e.g., phonetic chords that lack primary and secondarycomponent, such as nasals or voiced fricatives), the data can include reduced basal band componentsand an indicator of presence or absence of higher-frequency inharmonic components.

[0042] The systems and methods disclosed herein can be implemented as or with general or specialpurpose computers or servers or other programmable hardware devices programmed through software,or as hardware or equipment "programmed" through hard wiring, or a combination of the two. A"computer" or "server" can comprise a single machine or can comprise multiple interacting machines(located at a single location or at multiple remote locations). Computer programs or other softwarecode, if used, can be implemented in tangible, non-transient, temporary or permanent storage orreplaceable media, such as by including programming in microcode, machine code, network-based orweb-based or distributed software modules that operate together, RAM, ROM, CD-ROM, CD-R,CD-R / W, DVD-ROM, DVD±R, DVD±R / W, hard drives, thumb drives, flash memory, optical media,magnetic media, semiconductor media, or any future computer-readable storage alternatives.Electronic indicia of a dataset can be read from, received from, or stored on any of the tangible, non-transitory computer-readable media mentioned herein.

[0043] In addition to the preceding, the following example embodiments fall within the scope of thepresent disclosure or appended claims:

[0044] Example 1. A computer-implemented method for identifying one or more phonetic chordsrepresented within an electronic temporal waveform derived from an utterance of human speech, themethod comprising: using one or more electronic processors of a programmed computer system: (a) foreach of a plurality of harmonic acoustic spectra among a temporal sequence of acoustic spectraderived from the waveform, identifying within that harmonic acoustic spectrum two or more fundamentalor harmonic components, each identified component having an intensity exceeding a detectionthreshold, and the identified components having frequencies that are separated by at least one integermultiple of a fundamental acoustic frequency associated with that acoustic spectrum; (b) for at leastsome of the plurality of acoustic spectra, identifying as a primary cap frequency a highest harmonicfrequency among the identified harmonic components, which highest harmonic frequency is alsogreater than 410 Hz; and (c) for each of the plurality of acoustic spectra for which a primary capfrequency is identified in part (b), using the identified primary cap frequency to select as a primaryphonetic note at least one phonetic note from among a set of phonetic notes, the selected primaryphonetic note corresponding to a subset of phonetic chords from among a set of phonetic chords.

[0045] Example 2. The method of Example 1 wherein each of the acoustic spectra corresponds toone of a sequence of temporal sample intervals of the waveform, the method further comprising, usingone or more of the electronic processors of the programmed computer system: (A) classifying at leastsome of the acoustic spectra among the temporal sequence as only one of harmonic, inharmonic,hybrid, or silenced; and (B) for at least some of the temporal sample intervals classified as harmonic inpart (A), performing parts (a) through (c) and identifying the selected primary phonetic note ascorresponding to that temporal sample interval of the electronic waveform.

[0046] Example 3. The method of Example 1 wherein each of the acoustic spectra corresponds toone of a sequence of distinct temporal segments during which a time-dependent acoustic spectrum ofthe waveform remains consistent with a single phonetic chord, the method further comprising, usingone or more of the electronic processors of the programmed computer system: (A) classifying at leastsome of the temporal segments as only one of harmonic, inharmonic, or hybrid based on the acousticspectrum thereof; and (B) for at least some of the temporal segments classified as harmonic in part (A),performing parts (a) through (c) and identifying the selected primary phonetic note as corresponding tothat temporal segment of the electronic waveform.

[0047] Example 4. The method of any of Examples 1 through 3 further comprising, using one ormore of the electronic processors of the programmed computer system, deriving from the electronictemporal waveform the temporal sequence of acoustic spectra.

[0048] Example 5. The method of any of Examples 1 through 4 wherein, for two or more differentfundamental or harmonic components, the respective detection thresholds differ from one anotheraccording to acoustic frequency.

[0049] Example 6. The method of any of Examples 1 through 5 further comprising, using one ormore of the electronic processors of the programmed computer system, for at least one of the harmonicacoustic spectra among a temporal sequence of acoustic spectra derived from the waveform,identifying within the harmonic acoustic spectrum a fundamental component having the fundamentalacoustic frequency and an intensity that exceeds a detection threshold.

[0050] Example 7. The method of any of Examples 1 through 6 further comprising, using one ormore of the electronic processors of the programmed computer system, selecting, for a particularharmonic acoustic spectrum of the plurality of acoustic spectra, a phonetic chord from among thesubset of phonetic chords of part (c), based at least in part on a comparison of (i) stored data, indicativeof harmonic frequencies expected for the phonetic chords of the subset, with (ii) harmonic frequenciesof harmonic components of a primary band of the harmonic acoustic spectrum, the primary bandincluding harmonic components at the primary cap frequency and at one, two, or three largestconsecutive multiples of the fundamental acoustic frequency that are less than the primary capfrequency, greater than 410 Hz, and greater than the smallest integer multiple of the fundamentalacoustic frequency that is above 410 Hz and less than the primary cap frequency.

[0051] Example 8. The method of any of Examples 1 through 7 further comprising, using one ormore of the electronic processors of the programmed computer system, selecting, for a particularharmonic acoustic spectrum of the plurality of acoustic spectra, a phonetic chord from among thesubset of part (c), based at least in part on a comparison of (i) stored data, indicative of harmonicfrequencies expected for the phonetic chords of the set, with (ii) harmonic frequencies of harmoniccomponents of a secondary band of the harmonic acoustic spectrum, the secondary band includingharmonic components at one or more harmonic acoustic frequencies that are greater than the smallestinteger multiple of the fundamental acoustic frequency that is above 410 Hz and separated from theprimary cap frequency by at least one intervening multiple of the fundamental acoustic frequency forwhich the acoustic spectrum lacks a harmonic component.

[0052] Example 9. The method of any of Examples 1 through 8 further comprising, using one ormore of the electronic processors of the programmed computer system, making the selection of part (c)for a particular harmonic acoustic spectrum of the plurality of acoustic spectra based at least in part ona comparison of (i) stored data, indicative of harmonic frequencies expected for the phonetic chords ofthe set, with (ii) harmonic frequencies of harmonic components of a basal band of the harmonic acoustic spectrum, the basal band including harmonic components at one or more fundamental orharmonic acoustic frequencies that are less than 410 Hz or equal to the smallest integer multiple of thefundamental acoustic frequency that is between 410 Hz and the primary cap frequency.

[0053] Example 10. The method of any of Examples 1 through 9 further comprising, for a selectedharmonic acoustic spectrum for which the highest harmonic acoustic frequency of an identifiedharmonic component is less than 410 Hz, using one or more of the electronic processors of theprogrammed computer system, selecting from among a set of harmonic or hybrid acoustic schemataone of those schemata based at least in part on (i) a comparison of (A) stored data, indicative ofharmonic frequencies expected for the acoustic schemata of the set, with (B) harmonic frequencies ofeach identified fundamental or harmonic component, and (ii) presence or absence of higher-frequencyinharmonic frequency components.

[0054] Example 11. A computer-implemented method for generating stored data indicative ofharmonic frequencies expected for respective harmonic acoustic spectra of phonetic chords of a set ofphonetic chords, the method comprising: using one or more of the electronic processors of aprogrammed computer system: (a) for each phonetic chord of the set, spectrally analyzing multipleelectronic waveforms derived from respective utterances of that phonetic chord by one or more humantest subjects, the spectral analysis including, for each electronic waveform, estimation of a fundamentalacoustic frequency and identification of two or more fundamental or harmonic components each havingan intensity that exceeds a detection threshold and an acoustic frequency that is the fundamentalacoustic frequency or a harmonic acoustic frequency that is equal to an integer multiple of thefundamental acoustic frequency; (b) identifying as a primary cap frequency the highest harmonicacoustic frequency among the identified harmonic components that is also greater than 410 Hz, and foreach phonetic chord for which a primary cap frequency is thus identified, storing, on a tangible, non-transitory computer-readable storage medium that is not a transitory propagating signal, electronicindicia of the primary cap frequency, and acoustic frequencies for each identified fundamental orharmonic component.

[0055] Example 12. The method of Example 11 further comprising, for a subset of the phoneticchords having a common primary phonetic note, and for multiple utterances at multiple differentfundamental frequencies by one or more human test subjects, (i) estimating from the primary capfrequencies a focal frequency for the common primary phonetic note that corresponds to the subset ofphonetic chords, and (ii) storing, on a tangible, non-transitory computer-readable storage medium thatis not a transitory propagating signal, electronic indicia of the focal frequency of the primary phoneticnote.

[0056] Example 13. The method of any of Examples 11 or 12 wherein the detection threshold variesas a function of acoustic frequency.

[0057] Example 14. The method of any of Examples 11 through 13 wherein the stored data include,for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of aprimary band of a corresponding harmonic acoustic spectrum, the primary band including harmoniccomponents at the primary cap frequency and at one, two, three, or more largest consecutive multiplesof the fundamental acoustic frequency that are (i) less than the primary cap frequency, (ii) greater than410 Hz, and (iii) greater than the smallest integer multiple of the fundamental acoustic frequency that isabove 410 Hz and less than the primary cap frequency.

[0058] Example 15. The method of any of Examples 11 through 14 wherein the stored data include,for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of asecondary band of a corresponding harmonic acoustic spectrum, the secondary band including one ormore harmonic components at harmonic acoustic frequencies that are greater than the smallest integermultiple of the fundamental acoustic frequency that is above 410 Hz and separated from the primarycap frequency by at least one intervening multiple of the fundamental acoustic frequency for which theacoustic spectrum lacks a harmonic component.

[0059] Example 16. The method of any of Examples 11 through 15 wherein the stored data include,for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of abasal band of a corresponding harmonic acoustic spectrum, the basal band including one or moreharmonic components at one or more fundamental or harmonic acoustic frequencies that are less than410 Hz or equal to the smallest integer multiple of the fundamental acoustic frequency that is between410 Hz and the primary cap frequency.

[0060] Example 17. The method of any of Examples 11 through 16 further comprising: (a) for eachone of an additional set of harmonic, or hybrid acoustic schemata, spectrally analyzing multipleelectronic waveforms derived from respective utterances of that schemata by one or more human testsubjects, the spectral analysis including, for each electronic waveform, estimation of a fundamentalacoustic frequency and identification of two or more fundamental or harmonic components each havingan intensity that exceeds a detection threshold and an acoustic frequency that is the fundamentalacoustic frequency or a harmonic acoustic frequency that is equal to an integer multiple of thefundamental acoustic frequency, each of the fundamental and harmonic acoustic frequencies beingless than 410 Hz; and (b) for each acoustic schema for which multiple waveforms are analyzed in part(a), storing, on a tangible, non-transitory computer-readable storage medium that is not a transitorypropagating signal, electronic indicia of the one or more fundamental and harmonic acoustic frequencies for each identified fundamental or harmonic component, and presence or absence ofhigher-frequency inharmonic components.

[0061] Example 18. A computer-implemented method for synthesizing a temporal segment of anelectronic waveform that, when the waveform segment is applied to an electroacoustic transducer,produces sound of a phonetic chord selected from among a set of phonetic chords, the methodcomprising: using one or more of the electronic processors of a programmed computer system:(a) using received, retrieved, or calculated data indicative of a primary phonetic note corresponding tothe selected phonetic chord and a focal frequency of that primary phonetic note, determining a primarycap frequency using the focal frequency and a selected fundamental acoustic frequency, the primarycap frequency being (i) an integer multiple of the selected fundamental frequency, (ii) greater than410 Hz, and (iii) closer to the focal frequency of the corresponding primary note than to focalfrequencies of other phonetic notes; and (b) including in the waveform segment a harmonic componentat the primary cap frequency, the primary cap frequency being greater than acoustic frequencies of allother harmonic components included in the waveform segment.

[0062] Example 19. The method of Example 18 further comprising repeating the method of Example18 for each phonetic chord among a temporal sequence of multiple different harmonic or hybridsegments, along with inharmonic or silenced segments, and transitional segments therebetween, thattogether comprise a human utterance.

[0063] Example 20. The method of any of Examples 18 or 19 further comprising applying to anelectroacoustic transducer the waveform segment to produce sound of the selected phonetic chord.

[0064] Example 21. The method of any of Examples 18 through 20 further comprising, for one ormore of the phonetic chords of the set, including in the waveform segment harmonic components of aprimary band, the primary band including harmonic components at the primary cap frequency and atone, two, three, or more largest consecutive multiples of the fundamental acoustic frequency that are(i) less than the primary cap frequency, (ii) greater than 410 Hz, and (iii) greater than the smallestinteger multiple of the fundamental acoustic frequency that is above 410 Hz and less than the primarycap frequency.

[0065] Example 22. The method of any of Examples 18 through 20 further comprising, for one ormore of the phonetic chords of the set, including in the waveform segment harmonic components of aprimary band, the primary band including harmonic components at the primary cap frequency and atleast three largest consecutive multiples of the fundamental acoustic frequency that are (i) less than theprimary cap frequency, (ii) greater than 410 Hz, and (iii) greater than the smallest integer multiple of thefundamental acoustic frequency that is above 410 Hz and less than the primary cap frequency.

[0066] Example 23. The method of any of Examples 18 through 22 further comprising, for one ormore of the phonetic chords of the set: (a) using received, retrieved, or calculated data indicative of asecondary phonetic note corresponding to the selected phonetic chord and a focal frequency of thatsecondary phonetic note, determining a secondary cap frequency using the focal frequency and theselected fundamental acoustic frequency, the secondary cap frequency being (i) an integer multiple ofthe selected fundamental frequency, (ii) greater than 410 Hz, and (iii) closer to the focal frequency ofthe corresponding secondary note than to focal frequencies of other phonetic notes; and (b) including inthe waveform segment one or more harmonic components of a secondary band, the secondary bandincluding harmonic components at harmonic acoustic frequencies that are (i) greater than the smallestinteger multiple of the fundamental acoustic frequency that is above 410 Hz and (ii) less than or equalto the secondary cap frequency.

[0067] Example 24. The method of any of Examples 18 through 23 further comprising, for one ormore of the phonetic chords of the set, including in the waveform segment one or more harmoniccomponents of a basal band, the basal band including harmonic components at one or morefundamental or harmonic acoustic frequencies that are (i) less than 410 Hz or (ii) equal to the smallestinteger multiple of the fundamental acoustic frequency that is between 410 Hz and the primary capfrequency.

[0068] Example 25. The method of any of Examples 18 through 24 further comprising, for at leastone additional temporal segment of the electronic waveform corresponding to a selected harmonic orhybrid acoustic schema of a set of such acoustic schemata, including in the additional waveformsegment one or more harmonic components of only a reduced basal band, the reduced basal bandincluding only harmonic components at one or more fundamental or harmonic acoustic frequencies thatare less than 410 Hz.

[0069] Example 26. The method of any of Examples 18 through 25 further comprising, for at leastone additional temporal segment of the electronic waveform corresponding to a selected harmonic orhybrid acoustic schema of a set of such acoustic schemata, including in the additional waveformsegment (i) one or more harmonic components of only a reduced basal band, the reduced basal bandincluding only harmonic components at one or more fundamental or harmonic acoustic frequencies thatare less than 410 Hz, and (ii) one or more higher-frequency inharmonic components.

[0070] Example 27. A programmed computerized machine comprising one or more electronicprocessors and one or more tangible computer-readable storage media operationally coupled to theone or more processors, the machine being structured and programmed to perform the method of anyof Examples 1 through 26.

[0071] Example 28. An article comprising a tangible medium that is not a transitory propagatingsignal, the medium encoding computer-readable instructions that, when applied to a computer system,instruct the computer system to perform the method of any of Examples 1 through 26.

[0072] This disclosure is illustrative and not limiting. Further modifications will be apparent to oneskilled in the art in light of this disclosure and are intended to fall within the scope of the presentdisclosure. It is intended that equivalents of the disclosed example embodiments and methods, ormodifications thereof, shall fall within the scope of the present disclosure.

[0073] In the foregoing Detailed Description, various features may be grouped together in severalexample embodiments for the purpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that any identified embodiment requires more features thanare expressly recited therein. Rather, inventive subject matter may lie in less than all features of asingle disclosed example embodiment. Therefore, the present disclosure shall be construed asimplicitly disclosing any embodiment having any suitable subset of one or more features - whichfeatures are shown, described, or specified in the present application - including those subsets thatmay not be explicitly disclosed herein. A "suitable" subset of features includes only features that areneither incompatible nor mutually exclusive with respect to any other feature of that subset. It should befurther noted that the cumulative scope of the examples listed above can, but does not necessarily,encompass the whole of the subject matter disclosed in the present application.

[0074] The following interpretations shall apply for purposes of the present disclosure. The words"comprising," "including," "having," and variants thereof, wherever they appear, shall be construed asopen-ended terminology, with the same meaning as if a phrase such as "at least" were appended aftereach instance thereof, unless explicitly stated otherwise. The article "a" shall be interpreted as "one ormore," unless "only one," "a single," or other similar limitation is stated explicitly or is implicit in theparticular context; similarly, the article "the" shall be interpreted as "one or more of the," unless "onlyone of the," "a single one of the," or other similar limitation is stated explicitly or is implicit in theparticular context. The conjunction "or" is to be construed inclusively unless: (i) it is explicitly statedotherwise, e.g., by use of "either…or," "only one of," or similar language; or (ii) two or more of the listedalternatives are understood or disclosed (implicitly or explicitly) to be incompatible or mutually exclusivewithin the particular context. In that latter case, "or" would be understood to encompass only thosecombinations involving non-mutually-exclusive alternatives. In one example, each of "a dog or a cat,""one or more of a dog or a cat," and "one or more dogs or cats" would be interpreted as one or moredogs without any cats, or one or more cats without any dogs, or one or more of each. In anotherexample, each of "a dog, a cat, or a mouse," "one or more of a dog, a cat, or a mouse," and "one ormore dogs, cats, or mice" would be interpreted as (i) one or more dogs without any cats or mice, (ii) oneor more cats without and dogs or mice, (iii) one or more mice without any dogs or cats, (iv) one or moredogs and one or more cats without any mice, (v) one or more dogs and one or more mice without anycats, (vi) one or more cats and one or more mice without any dogs, or (vii) one or more dogs, one ormore cats, and one or more mice. In another example, each of "two or more of a dog, a cat, or amouse" or "two or more dogs, cats, or mice" would be interpreted as (i) one or more dogs and one ormore cats without any mice, (ii) one or more dogs and one or more mice without any cats, (iii) one ormore cats and one or more mice without and dogs, or (iv) one or more dogs, one or more cats, and oneor more mice; "three or more," "four or more," and so on would be analogously interpreted.

[0075] For purposes of the present disclosure or appended claims, when terms are employed suchas "about equal to," "substantially equal to," "greater than about," "less than about," and so forth, inrelation to a numerical quantity, standard conventions pertaining to measurement precision andsignificant digits shall apply, unless a differing interpretation is explicitly set forth. For null quantitiesdescribed by phrases such as "substantially prevented," "substantially absent," "substantiallyeliminated," "about equal to zero," "negligible," and so forth, each such phrase shall denote the casewherein the quantity in question has been reduced or diminished to such an extent that, for practicalpurposes in the context of the intended operation or use of the disclosed or claimed apparatus ormethod, the overall behavior or performance of the apparatus or method does not differ from that whichwould have occurred had the null quantity in fact been completely removed, exactly equal to zero, orotherwise exactly nulled.

[0076] For purposes of the present disclosure, any labelling of elements, steps, limitations, or otherportions of an embodiment or example (e.g., first, second, third, etc., (a), (b), (c), etc., or (i), (ii), (iii),etc.) is only for purposes of clarity, and shall not be construed as implying any sort of ordering orprecedence of the portions so labelled. If any such ordering or precedence is intended, it will beexplicitly recited in the embodiment or example, or, in some instances, it will be implicit or inherentbased on the specific content of the embodiment or example. If the provisions of 35 USC § 112(f) orany corresponding law related to "means plus function" or "step plus function" claim format are desiredto be invoked in a description of an apparatus, then the word "means" will appear in that description. Ifthose provisions are desired to be invoked in a description of a method, the words "a step for" willappear in that description. Conversely, if the words "means" or "a step for" do not appear, then suchprovisions are not intended to be invoked.

[0077] If any one or more disclosures are incorporated herein by reference and such incorporateddisclosures conflict in part or whole with, or differ in scope from, the present disclosure, then to the extent of conflict, broader disclosure, or broader definition of terms, the present disclosure controls. Ifsuch incorporated disclosures conflict in part or whole with one another, then to the extent of conflict,the later-dated disclosure controls.

[0078] The Abstract is provided as an aid to those searching for specific subject matter within thepatent literature. However, the Abstract is not intended to imply that any elements, features, orlimitations recited therein are necessarily encompassed by any particular description or are in any wayrequired.

Claims

1. A computer-implemented method for recognizing audible human speech represented by an electronic temporal waveform derived from an utterance of that human speech, the method comprising: using one or more electronic processors of a programmed computer system: (a) for each of a plurality of harmonic acoustic spectra among a temporal sequence of acoustic spectra derived from the waveform, identifying within that harmonic acoustic spectrum two or more fundamental or harmonic components, each identified component having an intensity exceeding a detection threshold, and the identified components having frequencies that are separated by at least one integer multiple of a fundamental acoustic frequency associated with that acoustic spectrum; (b) for at least some of the plurality of acoustic spectra, identifying as a primary cap frequency a highest harmonic frequency among the identified harmonic components, which highest harmonic frequency is also greater than 410 Hz; (c) for each of the plurality of acoustic spectra for which a primary cap frequency is identified in part (b), using the identified primary cap frequency to select as a primary phonetic note at least one phonetic note from among a set of phonetic notes, the selected primary phonetic note corresponding to a subset of phonetic chords from among a set of phonetic chords; and (d) based in part on selections of part (c), generating electronic indicia of text representing the audible human speech.

2. The method of Claim 1 wherein each of the acoustic spectra corresponds to one of a sequence of temporal sample intervals of the waveform, the method further comprising, using one or more of the electronic processors of the programmed computer system: (A) classifying at least some of the acoustic spectra among the temporal sequence as only one of harmonic, inharmonic, hybrid, or silenced; and (B) for at least some of the temporal sample intervals classified as harmonic in part (A), performing parts (a) through (c) and identifying the selected primary phonetic note as corresponding to that temporal sample interval of the electronic waveform.

3. The method of Claim 1 wherein each of the acoustic spectra corresponds to one of a sequence of distinct temporal segments during which a time-dependent acoustic spectrum of the waveform remains consistent with a single phonetic chord, the method further comprising, using one or more of the electronic processors of the programmed computer system: (A) classifying at least some of the temporal segments as only one of harmonic, inharmonic, or hybrid based on the acoustic spectrum thereof; and (B) for at least some of the temporal segments classified as harmonic in part (A), performing parts (a) through (c) and identifying the selected primary phonetic note as corresponding to that temporal segment of the electronic waveform.

4. The method of Claim 1 further comprising, using one or more of the electronic processors of the programmed computer system, deriving from the electronic temporal waveform the temporal sequence of acoustic spectra.

5. The method of Claim 1 wherein, for two or more different fundamental or harmonic components, the respective detection thresholds differ from one another according to acoustic frequency.

6. The method of Claim 1 further comprising, using one or more of the electronic processors of the programmed computer system, for at least one of the harmonic acoustic spectra among a temporal sequence of acoustic spectra derived from the waveform, identifying within the harmonic acoustic spectrum a fundamental component having the fundamental acoustic frequency and an intensity that exceeds a detection threshold.

7. The method of Claim 1 further comprising, using one or more of the electronic processors of the programmed computer system, selecting, for a particular harmonic acoustic spectrum of the plurality of acoustic spectra, a phonetic chord from among the subset of phonetic chords of part (c), based at least in part on a comparison of (i) stored data, indicative of harmonic frequencies expected for the phonetic chords of the subset, with (ii) harmonic frequencies of harmonic components of a primary band of the harmonic acoustic spectrum, the primary band including harmonic components at the primary cap frequency and at one, two, or three largest consecutive multiples of the fundamental acoustic frequency that are less than the primary cap frequency, greater than 410 Hz, and greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and less than the primary cap frequency.

8. The method of Claim 1 further comprising, using one or more of the electronic processors of the programmed computer system, selecting, for a particular harmonic acoustic spectrum of the plurality of acoustic spectra, a phonetic chord from among the subset of part (c), based at least in part on a comparison of (i) stored data, indicative of harmonic frequencies expected for the phonetic chords of the set, with (ii) harmonic frequencies of harmonic components of a secondary band of the harmonic acoustic spectrum, the secondary band including harmonic components at one or more harmonic acoustic frequencies that are greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and separated from the primary cap frequency by at least one intervening multiple of the fundamental acoustic frequency for which the acoustic spectrum lacks a harmonic component.

9. The method of Claim 1 further comprising, using one or more of the electronic processors of the programmed computer system, making the selection of part (c) for a particular harmonic acoustic spectrum of the plurality of acoustic spectra based at least in part on a comparison of (i) stored data, indicative of harmonic frequencies expected for the phonetic chords of the set, with (ii) harmonic frequencies of harmonic components of a basal band of the harmonic acoustic spectrum, the basal band including harmonic components at one or more fundamental or harmonic acoustic frequencies that are less than 410 Hz or equal to the smallest integer multiple of the fundamental acoustic frequency that is between 410 Hz and the primary cap frequency.

10. The method of Claim 1 further comprising, for a selected harmonic acoustic spectrum for which the highest harmonic acoustic frequency of an identified harmonic component is less than 410 Hz, using one or more of the electronic processors of the programmed computer system, selecting from among a set of harmonic or hybrid acoustic schemata one of those schemata based at least in part on (i) a comparison of (A) stored data, indicative of harmonic frequencies expected for the acoustic schemata of the set, with (B) harmonic frequencies of each identified fundamental or harmonic component, and (ii) presence or absence of higher frequency inharmonic frequency components.

11. The method of Claim 1 wherein the primary phonetic note is selected based at least in part on stored data indicative of harmonic frequencies expected for respective harmonic acoustic spectra of the phonetic chords of the set of phonetic chords, the stored data being generated by a method comprising: using one or more of the electronic processors of a programmed computer system: (a) for each phonetic chord of the set, spectrally analyzing multiple electronic waveforms derived from respective utterances of that phonetic chord by one or more human test subjects, the spectral analysis including, for each electronic waveform, estimation of a fundamental acoustic frequency and identification of two or more fundamental or harmonic components each having an intensity that exceeds a detection threshold and an acoustic frequency that is the fundamental acoustic frequency or a harmonic acoustic frequency that is equal to an integer multiple of the fundamental acoustic frequency; (b) identifying as a primary cap frequency the highest harmonic acoustic frequency among the identified harmonic components that is also greater than 410 Hz, and for each phonetic chord for which a primary cap frequency is thus identified, storing, on a tangible, non-transitory computer-readable storage medium that is not a transitory propagating signal, electronic indicia of the primary cap frequency, and acoustic frequencies for each identified fundamental or harmonic component.

12. The method of Claim 11 further comprising, for a subset of the phonetic chords having a common primary phonetic note, and for multiple utterances at multiple different fundamental frequencies by one or more human test subjects, (i) estimating from the primary cap frequencies a focal frequency for the common primary phonetic note that corresponds to the subset of phonetic chords, and (ii) storing, on a tangible, non-transitory computer-readable storage medium that is not a transitory propagating signal, electronic indicia of the focal frequency of the primary phonetic note.

13. The method of Claim 11 wherein the detection threshold varies as a function of acoustic frequency.

14. The method of Claim 11 wherein the stored data include, for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of a primary band of a corresponding harmonic acoustic spectrum, the primary band including harmonic components at the primary cap frequency and at one, two, three, or more largest consecutive multiples of the fundamental acoustic frequency that are (i) less than the primary cap frequency, (ii) greater than 410 Hz, and (iii) greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and less than the primary cap frequency.

15. The method of Claim 11 wherein the stored data include, for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of a secondary band of a corresponding harmonic acoustic spectrum, the secondary band including one or more harmonic components at harmonic acoustic frequencies that are greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and separated from the primary cap frequency by at least one intervening multiple of the fundamental acoustic frequency for which the acoustic spectrum lacks a harmonic component.

16. The method of Claim 11 wherein the stored data include, for at least some of the phonetic chords of the set, harmonic frequencies of harmonic components of a basal band of a corresponding harmonic acoustic spectrum, the basal band including one or more harmonic components at one or more fundamental or harmonic acoustic frequencies that are less than 410 Hz or equal to the smallest integer multiple of the fundamental acoustic frequency that is between 410 Hz and the primary cap frequency.

17. The method of Claim 11 further comprising: (a) for each one of an additional set of harmonic, or hybrid acoustic schemata, spectrally analyzing multiple electronic waveforms derived from respective utterances of that schemata by one or more human test subjects, the spectral analysis including, for each electronic waveform, estimation of a fundamental acoustic frequency and identification of two or more fundamental or harmonic components each having an intensity that exceeds a detection threshold and an acoustic frequency that is the fundamental acoustic frequency or a harmonic acoustic frequency that is equal to an integer multiple of the fundamental acoustic frequency, each of the fundamental and harmonic acoustic frequencies being less than 410 Hz; and (b) for each acoustic schema for which multiple waveforms are analyzed in part (a), storing, on a tangible, non-transitory computer-readable storage medium that is not a transitory propagating signal, electronic indicia of the one or more fundamental and harmonic acoustic frequencies for each identified fundamental or harmonic component, and presence or absence of higher-frequency inharmonic components.

18. A computer-implemented method for synthesizing audible human speech from text representing that speech, the method comprising: using one or more of the electronic processors of a programmed computer system: (a) for each of a plurality of harmonic segments of an electronic waveform that contains a temporal sequence of waveform segments, using received, retrieved, or calculated data indicative of a primary phonetic note corresponding to a selected phonetic chord and a focal frequency of that primary phonetic note to determine a primary cap frequency using the focal frequency and a selected fundamental acoustic frequency, the primary cap frequency being (i) an integer multiple of the selected fundamental frequency, (ii) greater than 410 Hz, and (iii) closer to the focal frequency of the corresponding primary note than to focal frequencies of other phonetic notes; (b) wherein, for each of the plurality of harmonic segments of the temporal sequence, the waveform segment includes a harmonic component at the primary cap frequency, the primary cap frequency being greater than acoustic frequencies of all other harmonic components included in that waveform segment; and (c) wherein the electronic waveform, when applied to an electroacoustic transducer, produces sound of the human speech corresponding to the text.

19. The method of Claim 18 further comprising applying the electronic waveform to an electroacoustic transducer to produce the sound of the human speech.

20. The method of Claim 18 further comprising, for one or more of the phonetic chords of the set, including in the waveform segment harmonic components of a primary band, the primary band including harmonic components at the primary cap frequency and at one, two, three, or more largest consecutive multiples of the fundamental acoustic frequency that are (i) less than the primary cap frequency, (ii) greater than 410 Hz, and (iii) greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and less than the primary cap frequency.

21. The method of Claim 18 further comprising, for one or more of the phonetic chords of the set, including in the waveform segment harmonic components of a primary band, the primary band including harmonic components at the primary cap frequency and at least three largest consecutive multiples of the fundamental acoustic frequency that are (i) less than the primary cap frequency, (ii) greater than 410 Hz, and (iii) greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and less than the primary cap frequency.

22. The method of Claim 18 further comprising, for one or more of the phonetic chords of the set: (a) using received, retrieved, or calculated data indicative of a secondary phonetic note corresponding to the selected phonetic chord and a focal frequency of that secondary phonetic note, determining a secondary cap frequency using the focal frequency and the selected fundamental acoustic frequency, the secondary cap frequency being (i) an integer multiple of the selected fundamental frequency, (ii) greater than 410 Hz, and (iii) closer to the focal frequency of the corresponding secondary note than to focal frequencies of other phonetic notes; and (b) including in the waveform segment one or more harmonic components of a secondary band, the secondary band including harmonic components at harmonic acoustic frequencies that are (i) greater than the smallest integer multiple of the fundamental acoustic frequency that is above 410 Hz and (ii) less than or equal to the secondary cap frequency.

23. The method of Claim 18 further comprising, for one or more of the phonetic chords of the set, including in the waveform segment one or more harmonic components of a basal band, the basal band including harmonic components at one or more fundamental or harmonic acoustic frequencies that are (i) less than 410 Hz or (ii) equal to the smallest integer multiple of the fundamental acoustic frequency that is between 410 Hz and the primary cap frequency.

24. The method of Claim 18 further comprising, for at least one additional temporal segment of the electronic waveform corresponding to a selected harmonic or hybrid acoustic schema of a set of such acoustic schemata, including in the additional waveform segment one or more harmonic components of only a reduced basal band, the reduced basal band including only harmonic components at one or more fundamental or harmonic acoustic frequencies that are less than 410 Hz.

25. The method of Claim 18 further comprising, for at least one additional temporal segment of the electronic waveform corresponding to a selected harmonic or hybrid acoustic schema of a set of such acoustic schemata, including in the additional waveform segment (i) one or more harmonic components of only a reduced basal band, the reduced basal band including only harmonic components at one or more fundamental or harmonic acoustic frequencies that are less than 410 Hz, and (ii) one or more higher-frequency inharmonic components.