Education and assessment of spoken language skills by detailed evaluation of human speech

Abstract

To provide a system, a method, and an apparatus for teaching and / or evaluating one or more spoken language skills.SOLUTION: An apparatus includes: an educational character string providing circuit 252 for providing an educational character string corresponding to a user's utterance related to the educational character string; an educational character string voice sample data receiving circuit 254; an individual language component analyzing circuit 256; and an evaluation presentation providing circuit 258. The apparatus analyzes one or more pronunciation characteristics of one or more individual language components.SELECTED DRAWING: Figure 2A

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] If an Application Data Sheet (ADS) was filed on the filing date of this application, it is incorporated herein by reference. All applications claimed in the ADS for priority under 35 U.S.C. 35 U.S.C. 365(c), and any and all parent, grandparent, great-grandparent, etc. applications of such applications, are also incorporated by reference, including all priority claims made in such applications, and all material incorporated by reference, to the extent such subject matter is not inconsistent with this specification.

[0002] This application is related to and / or claims the benefit of the earliest available effective filing dates, if any, of the applications listed below ("priority applications"), as set forth below (e.g., for non-provisional patent applications, claims the earliest available priority date, or for provisional patent applications, claims the benefit under 35 U.S.C. § 119(e) of any and all applications that are the parent, grandparent, great-grandparent, etc., of the priority applications). Additionally, this application is related to the "related applications" in some cases, as set forth below.

[0003] [Priority application] For purposes of the USPTO's non-statutory requirements, this application is a continuation-in-part of International Patent Application No. PCT / US17 / 34065, entitled "Teaching and Assessing Spoken Language Skills Through Fine-Grained Assessment," naming Chun Ho Cheung as the inventor, filed on May 23, 2017, attorney docket number 3003-003-001PCT, and is an application that is currently co-pending or entitled to the benefit of the filing date of such currently co-pending application.

[0004] For purposes of the USPTO's non-statutory requirements, this application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 476,733, entitled TEACHING AND ASSESSING Spoken Language Skills Through Fine-Grained Assessment of Human Speech, which names Chun Ho Cheung as the inventor, and which was filed on March 25, 2017, which is an application filed within twelve months of the filing date of this application or which is currently pending and is entitled to the benefit of the filing date of such application.

[0005] [Related Applications] None as of the filing date.

[0006] If the list of applications provided above does not match the list provided via the ADS, it is Applicant's intent to claim priority to each of the applications that appear in the priority application section of the ADS and each of the applications that appear in the priority application section of this application.

[0007] The entire subject matter of the priority application and related applications, and the parent, grandparent, great-grandparent, etc. applications of the priority application and related applications, including all priority claims, is incorporated herein by reference to the extent such subject matter is not inconsistent herewith. [Background technology]

[0008] The present invention is in the technical field of language teaching and learning, more particularly language learning through computational linguistic techniques. Summary of the Invention

[0009] The advent of computational power to store and process large amounts of data has brought about many changes throughout human existence. One area that is rapidly progressing is speech analysis. Using large training sets of data, computers can now process and analyze speech at the same level as humans. Many modern devices, such as smartphones, can receive commands in the form of speech and recognize the speech through complex algorithms and sets of training data. Apple's Siri, Microsoft's Cortana, and Amazon's Echo are just a few of these products currently on the market.

[0010] However, there has not been much progress in the area of ​​computer-implemented language skills teaching, i.e., the use of computer training to teach someone how to speak without recognizing their existing speech. Existing speech recognition programs, such as those mentioned above, are trained to recognize a wide range of forms of speech. For example, existing speech recognition programs are trained to recognize speakers who speak incorrectly, speakers who speak with accents, speakers who have disabilities or are under duress, and many other variations. When attempting to perform speaker pattern matching, the nuances of speech are lost, and thus voice commands can be recognized even if the speaker slurs their words because they are drunk, speaks quickly because of stress or illness, speaks differently because the speaker is under duress, or speaks in a noisy room with a lot of ambient noise.

[0011] However, the above problem and solutions deal with speech detection, not speech teaching. The following solutions include training to accurately analyze and speak those words, rather than discarding a lot of speech data and hoping to hit a pattern match so that meaning can be gleaned, and accurately analyzing how the speaker's speech differs from the baseline accurate neutral pronunciation of words and phrases that a native speaker would exhibit. Some implementations disclosed herein hope to perform, among other things, a fine-grained analysis of where the speaker's speech deviates from the native speaker's, e.g., "perfect" pitch, sound, intonation, frequency, stress, and accent. This accurate and fine-grained analysis helps inform the speaker where exactly they are making mistakes and what they can do to correct them. Thus, some of the systems and methods described herein detect and bring to the user's attention the user's poor speech, rather than compensating for the user's poor speech, so that the speech can be corrected.

[0012] Additionally, accurate and fine-grained analysis of discrepancies with the user's speech can provide further training benefits, particularly by enabling adaptive training over time. For example, each time a user speaks a sentence, that sentence can be analyzed with fine-grained analysis, as described in more detail herein. This analysis of the sentence can be combined with analysis of other words, sentences, and phrases to develop lessons targeted to the user. The analysis can be used to identify the language user's weaknesses and target them for improvement and / or determine the language user's strengths to reduce time. Because this process is iterative, in some implementations the overall time required to master a new language at whatever level of fluency the user desires is significantly reduced compared to traditional language teaching methods that rely on extensive stroke analysis of stroke lessons and the user's performance in those lessons.

[0013] The apparatus, methods, and systems herein thus constitute computer improvements to the way computers operate, for example, by enabling the computer to more carefully and finely analyze a user's speech, provide better feedback to the user, and enable the user to learn a language more quickly.

[0014] In one or more various aspects described herein, a method of teaching and / or evaluating one or more spoken language skills, as described in the claims, includes, but is not limited to, providing at least one educational string configured to be read by a user, receiving educational string audio sample data corresponding to an utterance of the user regarding the educational string, analyzing one or more individual language components of the educational string audio sample data, having a measurement of one or more pronunciation characteristics of the at least one individual language component against a corresponding one or more pronunciation characteristics of a baseline individual language component, providing an evaluation presentation to the user, the evaluation presentation including a presentation to the user of an evaluation of the educational string audio sample data based on the one or more pronunciation characteristics of the analyzed one or more individual language components, and collecting an evaluation of the educational string audio sample data, the evaluated educational string audio sample data including an evaluation of each of the one or more pronunciation characteristics of the one or more individual language components. In addition to the above, other method aspects are described in the claims, drawings, and text forming part of the disclosure described herein.

[0015] Similarly, in one or more various aspects described herein, a device configured to teach and / or assess one or more spoken language skills includes, but is not limited to, one or more of device input and device output components, a memory, and a processor operably coupled to the memory and configured to communicate data to and from device interface components, the processor including one or more circuits. In various aspects, the processor may comprise one or more of: an educational string providing circuit configured to implement one or more educational strings read by a user; an educational string audio sample data receiving circuit configured to receive educational string audio sample data corresponding to the user's speech regarding the educational string from a user via a device input component; an individual language component analysis circuit configured to analyze the educational string audio sample data through a measurement of one or more pronunciation characteristics of at least one individual language component against a corresponding one or more pronunciation characteristics of a baseline individual language component; an evaluation presentation providing circuit configured to provide an evaluation presentation to the user on a device output component, the evaluation presentation including, for each of the one or more individual language components, an evaluation of the educational string audio sample data based on the one or more individual language components of the analyzed educational string audio sample data; and an educational string audio sample data evaluation collecting circuit configured to collect an evaluation of the educational string audio sample data including an evaluation of each of the one or more pronunciation characteristics of the one or more individual language components.

[0016] In one or more various embodiments, the one or more associated systems may be implemented in the manufacture of a machine, composition of matter, or system, limited to patentable subject matter under U.S. patent law. 101. The one or more associated systems may include, but are not limited to, circuitry and / or programming for implementing aspects of the methods referenced herein.

[0017] Because the above is a summary, it may contain simplifications, generalizations, inclusions, and / or omissions of details, so that those skilled in the art will appreciate that the summary is illustrative only and is not intended to be limiting in any way. Other aspects, features, and advantages of the apparatus and / or processes and / or other subject matter described herein will become apparent by reference to the detailed description, the corresponding drawings, and / or the teachings set forth herein. [Brief description of the drawings]

[0018] For a more complete understanding of the embodiments, reference is made to the following description taken in conjunction with the accompanying drawings. The use of the same symbols in different drawings generally indicates similar or identical items, unless otherwise indicated in context, other than arrows, which may be used to indicate exemplary and non-limiting process flows when displayed in process / method flowcharts, and may be used to indicate exemplary and non-limiting data flows when displayed in system and apparatus diagrams, as the context indicates. The exemplary embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

[0019] [Figure 1] 1 illustrates an environmental diagram of a device 50 operating in an environment 100, according to various embodiments of the present invention.

[0020] [Figure 2A] One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention. [Figure 2B] One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention. [Figure 2C] One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention. [Figure 2D]One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention. [Figure 2E] One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention. [Figure 2F] One or more exemplary implementations of device 50 operating in environment 200 are shown, in accordance with an embodiment of the present invention.

[0021] [Figure 3A] 3 illustrates an exemplary implementation of a speech evaluation engine 310 according to an embodiment of the present invention. [Figure 3B] 3 illustrates an exemplary implementation of a speech evaluation engine 310 according to an embodiment of the present invention.

[0022] [Figure 4] 4 illustrates an exemplary implementation of rating presentation 402 according to various embodiments of the present invention.

[0023] [Figure 5A] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention. [Figure 5B] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention. [Figure 5C] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention. [Figure 5D] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention. [Figure 5E] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention. [Figure 5F] 5 is a high level logical flowchart of a process, eg, operational flow 500, in accordance with an embodiment of the present invention.

[0024] [Figure 6]5 is a high level logical flow diagram for providing educational string manipulation 502 in accordance with an embodiment of the present invention.

[0025] [Figure 7A] 5 is a high level logical flow chart for providing a rating submission operation 508 in accordance with an embodiment of the present invention. [Figure 7B] 5 is a high level logical flow chart for providing a rating submission operation 508 in accordance with an embodiment of the present invention. [Figure 7C] 5 is a high level logical flow chart for providing a rating submission operation 508 in accordance with an embodiment of the present invention. [Figure 7D] 5 is a high level logical flow chart for providing a rating submission operation 508 in accordance with an embodiment of the present invention.

[0026] [Figure 8A] 5 is a high level logical flow diagram of an analysis operation 506 according to an embodiment of the present invention. [Figure 8B] 5 is a high level logical flow diagram of an analysis operation 506 according to an embodiment of the present invention.

[0027] [Figure 9] 1 is an exemplary screenshot / implementation of a training string presentation according to an embodiment of the present invention.

[0028] [Figure 10] 1 is an exemplary screenshot / implementation of the presentation of an evaluation of a user's voice sample of a training string according to an embodiment of the present invention.

[0029] [Figure 11] 11 is a high level logical flowchart of a process, eg, operational flow 1100, in accordance with various embodiments of the present invention.

[0030] [Figure 12] 12 is a high level logical flowchart of a process, eg, operational flow 1200, in accordance with various embodiments of the present invention.

[0031] [Figure 13] 13 is a high level logical flowchart of a process, eg, operational flow 1300, in accordance with various embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] table of contents

[0033] I. Introduction:

[0034] II. Advantages of Various Embodiments:

[0035] III. The claims are directed to statutory subject matter:

[0036] IV. Description of Various Embodiments:

[0037] V. Various implementations and open-ended language:

[0038] VI. Preamble to the Claimed Subject Matter:

[0039] Beginning of detailed description

[0040] I. Introduction

[0041] The following presents a description of the apparatus, methods, systems, articles of manufacture, and / or compositions of matter that make up the present invention. Those portions of the detailed description that are not claims are to be understood as examples only, with the scope of the invention being defined entirely by the claims themselves, read in light of the foregoing and following descriptions.

[0042] II. Advantages of Various Embodiments

[0043] It should be understood that the following are only some of the advantages of one or more various implementations of the present invention. The following advantages are not intended to describe all implementations of the present invention or to outline what is necessary for the present invention. They do not limit the present invention in any way. The present invention is defined only by the claims, and any statements or advantages presented herein that are not included in the claims should not be read into the claims in any way. This section is provided solely to allow the subsequent reader to understand and appreciate some of the advantages the present invention offers over existing technology.

[0044] Existing algorithms for speech determination generally attempt to compensate for so-called "flaws" or speech inconsistencies, e.g. due to accent, grammar, local differences, changes in the user's state, and / or environmental conditions such as ambient noise. These existing algorithms do not focus on or capture the exact nature of the speech inconsistencies related to generalized neutrality, in a process disclosed in more detail herein. Thus, a user may only get a rough indication that an utterance is "wrong" even if the utterance is assigned a numerical value (e.g., "60% correct"), which does not help the user to specifically learn why the utterance is wrong. Thus, in various implementations, a fine-grained evaluation of the user's speech is performed and the fine-grained evaluation is provided to the user so that the user can know exactly why he or she is pronouncing a particular word incorrectly, which phoneme stress is too low, which is too high, etc.

[0045] Furthermore, in various implementations, the user may be given instant or near-instant visual feedback regarding a fine-grained assessment of the user's speech. This feedback may use colorful visual presentations, which may be combined with auditory feedback to trigger the user's brain functions to efficiently correct mistakes. The feedback may be as fine-grained as necessary or useful to the user, depending on the user's skill level, the type of exercise, and the specific implementation below. For example, feedback regarding the user's performance of a particular pronunciation characteristic may be looped back to further training strings.

[0046] III. The claimed invention relates to statutory subject matter

[0047] The claims, descriptions, and drawings of this application may describe one or more of the present techniques in an operational / functional language, e.g., as a series of operations performed by a computer. In most cases, such operational / functional descriptions will be understood by those skilled in the art as specially configured hardware (e.g., as a general-purpose computer, in effect, becomes a special-purpose computer when programmed to perform particular functions pursuant to instructions from program software).

[0048] Importantly, while the descriptions of operations / functions described herein are understandable to the human mind, they are not abstract ideas of operations / functions separated from the computational implementation of those operations / functions. Rather, the operations / functions represent specifications for highly complex computing machines or other means. As explained in more detail below, the operation / function language should be read in the appropriate technical context, i.e., as a concrete specification of a physical implementation.

[0049] The logical operations / functions described herein are a distillation of machine specifications or other physical mechanisms specified by the operations / functions, such that otherwise incomprehensible machine specifications become understandable to the human mind. Distillation allows one of ordinary skill in the art to adapt the description of the operations / functions of a technology across many different specific vendor hardware configurations or platforms, without being limited to a specific vendor hardware configuration or platform.

[0050] Some of the present technical descriptions (e.g., detailed description, drawings, claims, etc.) may be described in terms of logical operations / functions. As will be explained in more detail in the following paragraphs, these logical operations / functions are not expressions of abstract ideas, but rather represent static or ordered specifications of various hardware elements. Stated differently, unless the context dictates otherwise, logical operations / functions will be understood by those skilled in the art as representing static or ordered specifications of various hardware elements. This is true because the tools available to those skilled in the art for implementing the technical disclosures described in the form of operations / functions - either in the form of high-level programming languages ​​(e.g., C, Java, Visual Basic, etc.) or in the form of very fast hardware description languages ​​("VHDL", a language that uses text to describe logic circuits) - are generators of static or ordered specifications of various hardware configurations. This fact may be obscured by the broad term "software", but as will be shown in the following description, those skilled in the art will understand that what is called "software" is shorthand for very complex chains / specifications of ordered elements. The term "ordered element" may refer to the physical components of a computation, such as an assembly of electronic logic gates, molecular computing logic components, or quantum computing mechanisms.

[0051] For example, a high-level programming language is one with strong abstractions, e.g., multiple levels of abstraction from the details of the sequential organization, state, input, output, etc. of the machine that the high-level programming language actually specifies. To facilitate human understanding, high-level programming languages ​​often resemble or even share symbols with natural languages.

[0052] Because high-level programming languages ​​use strong abstractions (e.g. because they may resemble or share symbols with natural languages), they are not considered "pure mental constructs" (e.g. "software" is a computer program or computer programming - somehow an ineffable mental construct, because at a high level of abstraction it is conceived and understood in the human mind). This description has been used to characterize technical descriptions in the form of functions / operations as somehow "abstract ideas". Indeed, in technical fields (e.g. information and communication technology), this is not true.

[0053] The fact that high-level programming languages ​​use powerful abstractions to facilitate human understanding should not be taken as an indication that what is being expressed is an abstract idea. In fact, those skilled in the art understand that the exact opposite is true. If a high-level programming language is a tool used to implement a technical disclosure in the form of functions / operations, those skilled in the art will understand that such a tool is not abstract, imprecise, "fuzzy," or "mental" in any meaningful sense; such a tool is instead an almost incomprehensibly precise sequential specification of a particular computing machine, parts of which are constructed by activating / selecting such parts from more general computing machines typical over time (e.g., timekeeping). This fact can sometimes be obscured by superficial similarities between high-level programming languages ​​and natural languages. These superficial similarities can reveal the fact that implementations of high-level programming languages ​​ultimately perform valuable work by creating / controlling many different computing machines.

[0054] The many different computers that high-level programming languages ​​specify are almost unimaginably complex. Essentially, the hardware used in a computer is typically composed of some sort of ordered material (e.g., conventional electronic devices (e.g., transistors), deoxyribonucleic acid (DNA), quantum devices, mechanical switches, optics, fluidics, pneumatics, optical devices (e.g., optical interference devices), molecules, etc.) arranged to form logic gates. Logic gates are physical devices that can usually be electrically, mechanically, chemically, or otherwise actuated to change a physical state, creating the physical reality of Boolean logic.

[0055] Logic gates can be arranged to form logic circuits, which are physical devices that can be driven, typically electrically, mechanically, chemically, or both, to create the physical reality of a particular logic function. Types of logic circuits include devices such as multiplexers, registers, arithmetic logic units (ALUs), and computer memory, and each type can be combined to form yet other types of physical devices, such as central processing units (CPUs) - the best known of which is the microprocessor. Modern microprocessors often contain over 100 million logic gates (and often over a billion transistors) in many logic circuits. See, for example, Wikipedia, Logic gates, http: / / en.wikipedia.org / wiki / Logic_gates (5 June 2012, 21:03 GMT).

[0056] The logic circuits forming a microprocessor are arranged to provide a microarchitecture for the microprocessor to execute instructions defined by a defined instruction set architecture. The instruction set architecture is the part of the microprocessor architecture that is relevant to programming, including native data types, instructions, registers, addressing modes, memory architecture, interrupt and exception handling, external input / output, etc.

[0057] An instruction set architecture contains the specification of the machine language that a programmer can use to use / control the microprocessor. Machine instructions are such that they can be directly executed by the microprocessor, and so usually consist of strings of binary digits or bits. For example, a typical machine instruction may be many bits long (e.g., 32, 64, or 128-bit strings are common today). A typical machine instruction may take the form "11110000101011110000111100111111" (a 32-bit instruction).

[0058] The key here is that even though machine instructions are written as sequences of binary digits, those binary digits actually specify physical reality. For example, in the physical realization of Boolean logic operations using certain semiconductors, the seemingly mathematical bits "1" and "0" in a machine instruction actually constitute shorthands that specify the application of certain voltages to certain wires. For example, in some semiconductor technologies, the binary digit "1" (e.g., logic "1") in a machine instruction specifies the application of approximately +5 volts to a certain "wire" (e.g., a metal trace on a printed circuit board), while the binary digit "0" (e.g., logic "0") in a machine instruction specifies approximately -5 volts to be applied to a certain "wire". In addition to specifying the voltages of the machine's configuration, such machine instructions select and activate a particular group of logic gates from the millions of logic gates of a more general machine. Thus, far from being abstract mathematical formulas, a program of machine instructions, even written as a string of zeros and ones, specifies a multitude of constructed physical machines or physical machine states.

[0059] Machine code is typically incomprehensible to most humans (e.g., the above example is just one instruction; some personal computers execute over 2 billion instructions per second). Thus, programs written in machine code - which can be tens of millions of machine instructions long - are incomprehensible. With this in mind, early assembly languages ​​were developed that referenced machine instructions using mnemonic codes rather than using the numeric values ​​of the machine instructions directly (e.g., to perform a multiplication operation, a programmer would code the abbreviation "multi", which represents the binary digits "011000" in the MIPS machine code). Assembly languages ​​were initially a great aid to humans controlling microprocessors to perform tasks, but over time the complexity of the tasks that humans needed to perform outstripped the ability of humans to simply control microprocessors using assembly languages.

[0060] At this point, it was noted that the same tasks needed to be performed repeatedly, and the machine code required to perform those repeated tasks was the same. With this in mind, compilers were created. A compiler is a device that takes a statement that is easier for humans to understand than machine or assembly language, such as "add 2+2 and print the result," and translates that human understandable statement into complex, tedious, and sprawling machine code (e.g., millions of 32, 64, or 128-bit long strings). Thus, a compiler translates a high-level programming language into machine code.

[0061] This compiled machine code is then used as a technical specification, as described above, to sequentially structure and cause the interoperation of various computing machines to perform tasks that are tangible, concrete, and useful to humans. For example, as described above, such machine code (compiled versions of high-level languages) serves as a technical specification that selects hardware logic gates, specifies voltage levels, voltage transition timings, etc., so that tasks useful to humans are accomplished by the hardware.

[0062] Thus, the functional / operational technical description, when viewed by one of ordinary skill in the art, is far from an abstract idea. Rather, such functional / operational technical description, when understood through tools available in the art, such as those just described, is instead understood to be a human-understandable representation of hardware specifications, the complexity and specificity of which is far beyond almost any human understanding. With this in mind, one of ordinary skill in the art, in view of the disclosure herein and the knowledge of one of ordinary skill in the art, will understand that such operational / functional technical description can be understood as operations made into physical reality by (a) one or more inter-chain physical machines, (b) inter-chain logic gates configured to create one or more physical machines that represent sequential / combinational logic, (c) inter-chain ordered matter that constitutes the logic gates (e.g., inter-chain electronic devices (e.g., transistors), DNA, quantum devices, mechanical switches, optics, fluidics, pneumatics, molecules, etc.), or (d) virtually any combination of the above. Indeed, machines can be constructed based on the above technical description using physical objects in stable, measurable, and modifiable states. For example, Charles Babbage built the first computer out of wood and powered it by turning a handle.

[0063] Thus, far from being understood as an abstract idea, a person skilled in the art would recognize a functional / operational technical description as a human-understandable representation of one or more almost unimaginably complex and time-series hardware instantiations. The fact that a functional / operational technical description may easily lend itself to a high-level computing language (or high-level block diagrams for that matter) that shares some words, structures, phrases, and the like with natural language cannot be taken as merely an indication that such a functional / operational technical description is an abstract idea, or merely a representation of an abstract idea. In fact, as outlined herein, this is simply not true in the art. Viewed through the tools available to a person skilled in the art, such a functional / operational technical description is deemed to specify a hardware configuration of almost unimaginably complex.

[0064] As outlined above, there are at least two reasons for using functional / operational technical descriptions. First, functional / operational technical descriptions allow nearly infinitely complex machines and the machine operations resulting from inter-chain hardware elements to be described in ways that the human mind can process (e.g., by mimicking natural language and logical narrative flow). Second, the use of functional / operational technical descriptions aids those skilled in the art in understanding the described subject matter by providing a description that is somewhat independent of specific vendor hardware components.

[0065] The use of functional / operational technical descriptions aids those skilled in the art in understanding the described subject matter, as is evident from the above description, and the technical descriptions described in this document can be easily transcribed as trillions of ones and zeros, billions of single lines of assembly-level machine code, millions of logic gates, thousands of gate arrays, or any number of intermediate levels of abstraction. However, if such low-level technical descriptions were to replace the current technical descriptions, those skilled in the art may encounter undue difficulty in implementing the present disclosure, since such low-level technical descriptions would likely add complexity without corresponding benefits (e.g., by describing the subject matter utilizing one or more vendor-specific pieces of hardware). Thus, the use of functional / operational technical descriptions aids those skilled in the art by separating the technical descriptions from the conventions of vendor-specific pieces of hardware.

[0066] In view of the above, the logical operations / functions described in this technical description represent static or ordered specifications of various ordered elements, so that such specifications are intelligible to the human mind and allow for the creation of many different hardware configurations. Simply because the specifications they represent are presented in a manner that can be readily understood and applied by those skilled in the art in a manner that is independent of a particular vendor's hardware implementation, the logical operations / functions disclosed herein should be treated as such and not disparagingly characterized as abstract ideas.

[0067] Those skilled in the art will recognize that the state of the art has advanced to a point where little distinction remains between hardware, software, and / or firmware implementations of aspects of a system, and the use of hardware, software, and / or firmware is generally (but not always, as the choice between hardware and software can be significant in certain circumstances) a design choice that represents a cost vs. efficiency trade-off. Those skilled in the art will appreciate that there are a variety of vehicles (e.g., hardware, software, and / or firmware) in which the processes and / or systems and / or other techniques described herein may be implemented, and that the preferred vehicle will vary depending on the context in which the processes and / or systems and / or other techniques are deployed. For example, if the implementer determines that speed and accuracy are paramount, the implementer may select a primarily hardware and / or firmware vehicle, or if flexibility is paramount, the implementer may select a primarily software implementation, or again alternatively, the implementer may select one or more combinations of hardware, software, firmware, compositions of matter, and articles of manufacture, limited to patentable subject matter under 35 U.S.C. 101. Thus, there are several possible vehicles in which the processes and / or apparatus and / or other techniques described herein may be implemented, none of which is inherently better than the others, and all of which may vary, in that the choice will depend on the context in which the utilized vehicle is deployed and the particular concerns of the implementer (e.g., speed, flexibility, predictability, etc.) One of skill in the art will recognize that the optical aspects of the implementation will typically use optically oriented hardware, software, or firmware.

[0068] In some implementations described herein, logic and similar implementations may include software or other control structures. For example, an electronic circuit may have one or more paths of electrical current constructed and arranged to implement various functions as described herein. In some implementations, one or more media may be configured to carry a device-detectable implementation when such media holds or transmits device-detectable instructions operable to perform as described herein. In some variations, for example, the implementation includes updating or modifying existing software or firmware, or gate arrays or programmable hardware, such as by performing the reception or transmission of one or more instructions associated with one or more operations described herein. Alternatively or additionally, in some variations, the implementation may include dedicated hardware, software, firmware components, and / or general-purpose components that execute or otherwise invoke the dedicated components. The specification or other implementation may be transmitted by one or more instances of a tangible transmission medium described herein, optionally by packet transmission, or by passing through a distributed medium at various times.

[0069] Alternatively or additionally, implementations may include executing dedicated instruction sequences or invoking circuits to enable, trigger, coordinate, request, or otherwise cause the occurrence of one or more of substantially any functional operations described herein. In some variations, the operational or other logical descriptions herein may be expressed as source code and compiled or invoked as executable instruction sequences. For example, depending on the context, they may be implemented in whole or in part by source code or other code sequences such as C++. In other implementations, source or other code implementations may be compiled / implemented / substituted / converted into a high-level descriptor language using techniques that are commercially available and / or in the art (e.g., first implementing the described techniques in C or C++ programming language and then converting the programming language implementation into a logic synthesizable language implementation, a hardware description language implementation, a hardware design simulation implementation, and / or other similar modes of expression). For example, some or all of the logical expressions (e.g., a computer programming language implementation) may be manifested as a Verilog-type hardware description (e.g., via a Hardware Description Language (HDL) and / or a Very High Speed ​​Integrated Circuit Hardware Descriptor Language (VHDL)) or other circuit model that can be used to create a physical implementation having hardware (e.g., an application specific integrated circuit).Those skilled in the art will recognize how to obtain, configure, and optimize suitable transmission or computational elements, material supplies, actuators, or other structures in light of these teachings.

[0070] IV. Description of Various Embodiments

[0071] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar or identical components or items, unless the context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0072] Thus, according to various embodiments, computationally implemented methods, systems, circuits, articles of manufacture, ordered chains of matter, and computer program products are designed to, among other things, provide an interface with the environment depicted in FIG.

[0073] Referring now to FIG. 1, FIG. 1 illustrates a system diagram according to one or more embodiments. Specifically, FIG. 1 illustrates a system environment 100 including a user 150, a device 50, and a server 10 in various embodiments. In various implementations, the server 10 may communicate with the device 50 via a communication network 105. It is noted that in various implementations, all of the various circuits described herein may be executed in the device 50. The server 10 may be entirely optional and may be omitted in various implementations. In various implementations, the device 50 may perform all of the functions described herein without connecting to a network, such as the communication network 105. In various other implementations, some of the operations described herein and some of the circuits described herein may be executed in part in or on the device 50 and in part in or on the server 10.

[0074] In various implementations, device 50 may be a server-type device or may be a user-level device including, but not limited to, for example, a desktop computer, a laptop computer, a mobile phone, a network phone, a smartphone, a tablet, a music player, a walkie-talkie, a radio, an augmented reality device (e.g., augmented reality glasses and / or headphones), wearable electronics such as a watch, a belt, earphones, or “smart” clothing, earphones, headphones, audio / visual equipment, a media player, a television, a projection screen, a flat screen, a monitor, a clock, an appliance (e.g., a microwave oven, a convection oven, a stove, a refrigerator, a freezer), a navigation system (e.g., a Global Positioning System (“GPS”) system), a medical alert device, a remote control, a peripheral device, a video camera, a personal video recorder, a personal audio recorder, and the like.

[0075] In various implementations, the communication network 105 may be any form of one or more networks, wired or wireless or some combination thereof, that serve to transmit data and / or information between two endpoints. In various embodiments, the communication network 105 may include one or more of a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a wireless local area network (WLAN), a personal area network (PAN), a Worldwide Interoperability for Microwave Access (WiMAX), a public switched telephone network (PTSN), a general packet radio communication service (GPRS) network, a cellular network (GSM, CDMA, etc.), a microwave network, etc. The communication network 105 may be a wired network, a wireless network, or a combination of wired and wireless networks. Note that a "communication network" as used herein refers to one or more communication networks, which may or may not interact with each other.

[0076] In various implementations, server 10 may be any piece or set of computing equipment capable of being implemented in a request-response model. For example, server 10 may provide various services, functions, or computational tasks to one or more clients, including but not limited to device 50. The term "server" does not imply a particular set or class of hardware, although in some applications hardware designed for servers may be desirable. Furthermore, the term "server" does not imply multiple clients, or even one client, if the server can process clients. Furthermore, server 10 may be part of a distributed server and / or set of other computing equipment, where the locations of different parts are variable (e.g., memory may be in one physical location and the CPU may be in a completely different location), and may have various redundant parts that may also be in various entirely different locations. In other implementations, server 10 may be local to device 50 or internal to device 50 (e.g., in some implementations, a part of device 50 may act as server 10 for one or more processes active on device 50). Additionally, a server may refer to a particular computer program or process that results in a server, a device that functions as a server by executing a program, or some combination thereof. Use of the term "server" does not exclude or preclude peer-to-peer or other network arrangements. Although the device 50, server 10, and communication network 105 are described as shown in FIG. 1 for ease of understanding, other arrangements and setups are contemplated and are within the scope of the following disclosure.

[0077] Referring now to FIG. 2A, FIG. 2A illustrates an apparatus 50 according to one or more implementations. As shown in FIG. 2A, the apparatus 50 may include a processor 222, an apparatus memory 245, an apparatus interface component 220, and other apparatus components 224. These components are described in more detail herein. Note that this is not an exhaustive list of the components of the apparatus 50, and some components described may be omitted or performed elsewhere. For example, in a distributed environment, the apparatus memory 245 may be physically separate from the apparatus 50, e.g., a shared memory. Similarly, the processor 222 may be a distributed processor.

[0078] As mentioned above, in one or more various implementations, the device 50 may include device memory 245. In an embodiment, the device memory 245 may include one or more of random access memory ("RAM"), read only memory ("ROM"), flash memory, hard drive, disk-based media, magnetic storage, optical storage, volatile memory, non-volatile memory, and any combination thereof. In an embodiment, the device memory 245 may be separate from the device and may be available to different devices on a network or wirelessly, for example. For example, in a networked system, there may be one or more devices 50 where the device memory 245 is located on a central server that may be located a few feet away or across an ocean. In an embodiment, the device memory 245 may include one or more mass storage devices, read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), cache memory such as random access memory (RAM), flash memory, synchronous random access memory (SRAM), dynamic random access memory (DRAM), and / or other types of memory devices. In some embodiments, device memory 245 may be located at a single network site. In some embodiments, device memory 245 may be located at multiple network sites, including sites that are distant from one another.

[0079] As mentioned above, in one or more various implementations, the device 50 may include a device interface component 220, as shown in FIG. 2A. As described herein, the device interface component 220 is intended to represent all components that handle the input and output of the device 50, e.g., interactions with the user 150. For ease of explanation, in various implementations, the device interface component 220 may include a device input component 220A and a device output component 220B. Although shown separately in FIG. 2A, it is noted that there may be substantial overlap between the device input component 220A and the device output component 220B, both in hardware and software / programming, as described throughout this application. For example, a touch screen monitor may function as part of both the device input component 220A and the device output component 220B. Each of the device input component 220A and the device output component 220B may include many different components and associated drivers, programs, and software that enable those components to function. In various implementations, a non-exhaustive, non-limiting list of components that may be included in device input components 220A includes a keyboard, a keypad, a gamepad, a touch screen, a microphone, a headset, a gesture tracker, an optical tracker, a camera, and a webcam, or any combination thereof. In various implementations, a non-exhaustive, non-limiting list of components that may be included in device output components 220B includes a display, a touch screen, a projector, an augmented reality projection, a virtual reality projection, a speaker, a set of headphones, and haptic feedback.

[0080] Additionally, the device interface component 220 may include circuitry formed by a software program that enables interaction with the device 50, e.g., a computer. For example, in an implementation where the device 50 is a computer, the device interface component 220 may include circuitry formed by a software program that configures a web browser. The web browser circuitry, in some implementations, allows user interaction to reach the circuitry claimed in the present invention. Alternatively, this circuitry may be part of the processor 222 and may be integrated into the circuitry claimed in the present invention, but is not expressly claimed as part of the present invention, allowing for various interface contemplations. As another example, in an implementation where the device 50 is a smartphone, e.g., an Apple iPhone or Samsung phone running Android, and connected to various "app stores" or "play stores," applications that configure the circuitry of the processor 222 into the circuitry claimed in the present invention may interface with the circuitry of the processor 222 that enables the user 150 to interact with the smartphone device, e.g., via the device interface component 220.

[0081] As mentioned above, in one or more various implementations, device 50 can include other device components 224. Other device components 224 are shown primarily for purposes of illustration, and the list of components described herein for device 50, for example, is not intended to be an exhaustive or complete list. For example, other device components 224 can include, for example, an operating system (OS) or kernel 223. In other implementations, other device components 224 can include a graphics card, a GPU, a processor other than processor 222, persistent storage, and other components known in the art as optionally part of device 50.

[0082] Referring again to FIG. 2A, the device 50 may include the processor 222, as previously described. In some implementations, the processor 222 may be configured by instructions that form the processing circuitry 250. In other implementations, the processing circuitry 250 may be a permanent part of the processor 222, for example in the form of an application specific integrated circuit (ASIC) or field programmable gate array (FPGA). In some implementations, only a portion of the processing circuitry 250 may exist at a given snapshot in time, as the processor 222 executes instructions and then reuses its logic gates and transistors according to received instructions. In implementations, for example, where the processor 222 performs pipeline processing of instructions, a portion of the processing circuitry 250 may not exist at the exact same moment, but is fully formed when the processor 222 forms itself based on received instructions.

[0083] Referring again to FIG. 2A, the processing circuit 250 may include an educational string providing circuit 252, an educational string audio sample data receiving circuit 254, an individual language component analysis circuit 256, and an evaluation presentation providing circuit 258, which may be implemented one or more times, and not necessarily simultaneously or in the order shown in FIG. 2A.

[0084] As shown in FIG. 2A, in one or more implementations, the educational string providing circuit 252 may be configured to provide an educational string to the user 252A via a device output component of the device interface component for the user to read. For example, the educational string providing circuit 252 may first provide a visual or audio representation of the educational string to the user. For example, the educational string may be "I cleaned 30 rooms yesterday. I'm so tired." The visual representation may first show the user the words to be pronounced. For example, in FIG. 9, FIG. 9 illustrates visual representations of educational strings according to various implementations. As shown in FIG. 9, an educational string, for example educational string 930, is "I cleaned 30 rooms yesterday. I'm so tired."

[0085] As shown in FIG. 9, the visual representation can have various components according to a number of different implementations. For example, in one implementation, FIG. 9 represents the first educational string of a sequence of 16 educational strings. This is indicated by an educational string identification number 902 that identifies the educational string as the string "1 of 16". In other implementations, this visual representation can have other formats such as "1 / 16" or 15 red dots and 1 green dot. In one implementation, a play button 910 allows the user to use an interactive button that presents an audio representation of the entire educational string. As shown in FIG. 9, the educational string 930 is prominently displayed and each word, e.g., word 932, is separated and clearly indicated. In some implementations, interacting with an individual word (e.g., clicking on an individual word) can cause a visual representation of the word's phonemes to be displayed, while in other implementations the pronunciation of the word can be presented audibly. In other implementations, interacting with an individual word can result in a visual representation of one or more phonetic characteristics of the individual word. Additionally, as shown in FIG. 9, there is an information icon, e.g., an information button 920. The information icon can display the educational string 930 in the user's native language or a default native language (e.g., Spanish) to orient the user or remind the user of the meaning of the educational string 930. When the user is ready to speak the educational string, the user can interact with a speak button 950, which triggers the device interface component 220 to begin accepting the user's speech regarding the educational string 930. Finally, the visual representation 900 can include an exit button 940 for ending the lesson or program and returning to another screen.

[0086] 2A, in one implementation, the instructional string providing circuit 252 may perform an operation 502, for example, providing at least one instructional string configured to be read by a user, which will be further described in more detail herein in connection with FIGs. 5A-5F. In one implementation, the instructional string providing circuit 252 interacts with the device interface component 220 to provide visual and audio representations of the instructional string.

[0087] Referring again to FIG. 2A, in some implementations, the processing circuit 250 may include an educational character string voice sample data receiving circuit 254 in various implementations. As shown in FIG. 2A, for example, in sub-operation 254A, the educational character string voice sample data receiving circuit 254 may be configured to receive educational character string voice sample data from a user via the device input component 220A, corresponding to the user's speech regarding the educational character string. For example, when a user speaks an exemplary educational character string, "I cleaned 30 rooms yesterday. I'm so tired," into the device input component 220A, for example, a microphone, the voice data captured by the device input component 220A may be received by the processing circuit 250, for example, the educational character string voice sample data receiving circuit 254. In some implementations, the educational character string voice sample data receiving circuit 254 may handle some of the data processing of the voice data, such as compression, conversion to a specific format, signal pre-processing (for example, to reduce background noise, but not normalization, as in a typical speech recognition engine), etc. In other implementations, these functions are handled by multiple different circuits of the processor 222. As described in more detail in connection with Figures 5A-5F, in one implementation, the training string voice sample data receiving circuit 254 can perform an operation 504 to receive training string voice sample data corresponding to a user's speech regarding the training string, for example.

[0088] Referring again to FIG. 2A, in some implementations, the processing circuit 250 may include an individual language component analysis circuit 256. For example, in some implementations, the individual language component analysis circuit 256 may be configured to analyze the training string speech sample data by measuring one or more pronunciation characteristics of at least one individual language component against one or more corresponding pronunciation characteristics of the baseline individual language component data, for example, as shown in suboperation 256A. That is, the analysis of the training string is primarily performed by the individual language component analysis circuit 256, while in other implementations, other parts of the processing circuit 250 may perform parts of the analysis. In some implementations, some or all of the individual language component analysis circuit 256 may be incorporated in a device external to the device 50, while in other implementations, the individual language component analysis circuit 256 is part of the processor 222. Note that the circuit is referred to as an "individual language component analysis circuit" because the primary analysis is performed on the individual language components. However, the names of the circuits should not imply that the individual language component analysis circuit 256 cannot or does not perform an analysis of the entire training string, e.g., a global analysis of the training string audio sample data received by the training string audio sample data receiving circuit 254. For example, in addition to analyzing the individual language components, the training string audio sample data can be analyzed for overall speed, pitch changes, and smoothness of transitions.

[0089] In one implementation, as shown in FIGS. 3A-3B, the individual language component analysis circuit 256 may include a speech evaluation engine 310. Details of the speech evaluation engine 310 are described in more detail in connection with FIGS. 3A-3B. However, in one implementation, the individual language component analysis circuit 256 decomposes the educational string audio sample data into one or more individual language components. In one implementation, the individual language component may be a unit of speech, e.g., a phoneme, e.g., a phonological system of a language, which corresponds to a speech sound and is perceived by the human ear as a single distinctive sound of the language. In other embodiments, the individual language component may be a syllable, a diphthong, a digraph, any other phonetic unit, a word, a phrase, a sentence, a paragraph, or a set of paragraphs or a group of text. The individual language component analysis circuit 256 may separate the educational string audio sample data into individual language components by one or more known techniques for separating speech audio data into its component parts. In one implementation, the individual language component analysis circuit 256 can iteratively search the training string speech sample data to extract individual language components known to be part of the training string speech sample data. In one implementation, the individual language component analysis circuit 256 can perform this search using audio pattern matching, among other techniques. The individual language components are then measured against baseline individual language components drawn from a database, for example, as described in more detail herein.

[0090] In one implementation, the individual language components are measured against the baseline individual language components using one or more pronunciation characteristics. The pronunciation characteristics are any measurable components of the speech of the individual language components. Some exemplary pronunciation characteristics include pitch, intonation, frequency, stress, and accent. Each of the pronunciation characteristics of each of the individual language components is compared to each of the pronunciation characteristics of its corresponding baseline individual language components to generate various scores for each of the individual language components. Various scoring mechanisms can be used, for example, a binary scoring mechanism can be used, where the pronunciation characteristics of the individual language components match (1) or do not match (0) the corresponding pronunciation characteristics of the individual language components of the corresponding baseline. In another implementation, a numeric scoring mechanism can be used, for example, 1 to 100, where 1 is the minimum match between the pronunciation characteristics of the individual language components and the corresponding pronunciation characteristics of the individual language components of the corresponding baseline, and 100 is a perfect match between the pronunciation characteristics of the individual language components and the corresponding pronunciation characteristics of the individual language components of the corresponding baseline. In yet another implementation, a threshold scoring mechanism may be used, for example, 1 to 5, or phrases such as "very bad", "bad", "average", "good", "very good", etc., depending on the closeness of the match and whether the match exceeds a required threshold line. In this implementation, the score of the match may be accounted for by color coding the outputted rating, as described in more detail herein, although any known method of describing a scoring system may be used. In another implementation, the individual language component analysis circuit 256 may provide feedback to the user on how the spoken individual language components actually sounded, using the fine-grained scores outlined above, which may be derived from scores given to one or more pronunciation characteristics of the individual language components, or from pattern matching of the individual language components performed against the training string speech sample data.

[0091] After completing the analysis of the educational string speech sample data, the individual language component analysis circuit 256 completes its operation and the evaluation presentation providing circuit 258 can start its operation. Referring again to FIG. 2A, in one implementation, the processing circuit 250 of the processor 222 may include an evaluation presentation providing circuit 258. In one implementation, the evaluation presentation providing circuit 258 may be configured to provide an evaluation presentation to the user on the device output component 220B. In one implementation, as shown in sub-operation 258A, the evaluation presentation providing circuit is configured to provide an evaluation presentation to the user on the device output component, the evaluation presentation including an evaluation of the educational string speech sample data based on one or more individual language components of the analyzed educational string speech sample data for each of the one or more individual language components. The evaluation presentation includes an evaluation of the educational string speech sample data based on one or more individual language components of the analyzed educational string speech sample data for each of the one or more individual language components. In one implementation, a score, e.g., a quality score, is associated with each pronunciation characteristic of each individual language component measured against the baseline, and these quality scores are part of the evaluation presentation. The quality scores may be displayed in their individual capacity or may be summed using various techniques to arrive at an overall score. For example, FIG. 4 shows a rating presentation 402 that includes an overall score 412. The overall score 412 of the user's speech regarding the educational string may include the statistical calculations 414 described above and / or statistical calculations 416 that also take into account the scores assigned to the educational string audio sample data as a whole.

[0092] 2A , in one implementation, the rating suggestion providing circuit 258 may present a rating based on the analysis performed by the individual language component analysis circuit 256, as described above. In one implementation, the rating suggestion providing circuit 258 may perform, for example, operation 508 of providing a rating suggestion to a user. An exemplary rating suggestion is illustrated in conjunction with the screenshot shown in FIG.

[0093] Referring to FIG. 10, FIG. 10 illustrates an assessment presentation according to an exemplary embodiment. As shown in FIG. 10, FIG. 10 includes an overall score 1010, which in the illustrated example is 95%, which causes the system to add the sentence 1012, "That's right! Are you a native speaker?". FIG. 10 also illustrates an entire educational string 1030 similar to educational string 930, except that here, the use of color shows each particular word in the educational string that did not meet the overall score in red, for example, word 1032 in FIG. 10, e.g., the word "tired", in red to indicate that the overall score for that particular word was insufficient to indicate correct pronunciation. (Note that the scores here are merely examples, e.g., missing words would normally result in a score below 95%, but FIG. 10 is intended to illustrate merely examples and not precise numerical calculations).

[0094] Referring again to FIG. 10, in one implementation, when a user interacts with a particular word, e.g., the word "cleaned" in FIG. 10, a popup, e.g., popup box 1050, appears. The popup box 1050 includes a large amount of additional details regarding the particular "cleaned" word and its pronunciation. The popup box 1050 includes a "syllable" or "whole word" column, in this case shown as column 1020, indicating that the word is "cleaned". A second column, phoneme column 1022, indicates that the word "cleaned" has been broken down into its individual linguistic components, e.g., in the example shown, phonemes or sounds, e.g., "K", "L", "IY", "N", and "D". A third column, score column 1024, indicates the score of that particular individual linguistic component. In the example shown in FIG. 10, row 1025 indicates the individual linguistic component "IY" that received a score of "good", as indicated by the text word and color in this example. In the example of row 1027, the phoneme "L" did not receive a good score, as indicated by the color and the textual indication "sounded like AY." This is because in one implementation, as described above, the individual language component analysis circuit 256 scores the pronunciation characteristics of the individual language components to determine which other language components the actual utterance sounded like, in this case the "L" phoneme / sound sounded like the "AY" phoneme / sound, and the red color indicates that the score for that individual language component was not high enough to receive a good score. Note, however, that because four of the five individual language components of the word were close enough to their baseline individual language components to be determined to be "good" scores, the pronunciation score for the entire word was good enough to receive the color green for the entire word.

[0095] Referring again to Figure 10, in one implementation the pop-up box also includes a play baseline button 1062 and a play user sample button 1064. Pressing the play baseline button 1062 causes the device output component 220B to deliver an audio presentation of the individual linguistic components of the baseline that make up the word, e.g., "cleaned" in Figure 10. Pressing the play user sample button 1064 causes the device output component 220B to deliver an audio presentation of the training string audio sample data that corresponds to that set of individual linguistic components.

[0096] Referring now to FIG. 2B, FIG. 2B illustrates some additional circuits that may be part of the processing circuit 250 in various implementations. For example, in one implementation, the processing circuit 250 may include an educational string voice sample data evaluation collection circuit 262. In one implementation, for example, as shown in sub-operation 262A, the educational string voice sample data evaluation collection circuit 262 may be configured to collect an evaluation of the educational string voice sample data, including an evaluation of each of one or more pronunciation characteristics of one or more individual language components. For example, in one implementation, the evaluation is collected and stored, for example, in a memory, for example, the device memory 245, or another memory that may be remote from the device 50. For example, in one implementation, a user may complete multiple lessons, each lesson including its own educational string voice sample data. Evaluations of the educational string voice sample data generated by and presented by the individual language component analysis circuit 256 may be collected by the educational string voice sample data evaluation collection circuit 262 for further analysis and use in providing user feedback. This serves several purposes. First, the user can go back and review those exact mistakes, so that the user can see without undue difficulty where they are having problems. Second, a language instructor, tutor, teacher, or other person helping the user learn or improve their language skills can see exactly where the user's language skills are weak and strong. Third, machine learning / intelligence amplification can be applied to the various collected scores to determine, through automation, the user's strong language skills and the user's weak language skills. The end result is that the user's weaknesses and strengths can be more quickly identified and targeted for further iterations of teaching strings, reducing the overall time required for the user to achieve a desired level of fluency in the target language.

[0097] 2B, the educational character string audio sample data evaluation collection circuit 262 can include an educational character string audio sample data evaluation collection and storage circuit 268, which can be configured to store the collected educational character string audio sample data evaluations and store them in a memory, such as the device memory 245, or another memory that can be remote from the device 50. The storage in the memory can be temporary, such as long enough for processing, or permanent, such as stored in a physical medium, such as a hard disk drive, a flash memory, a memory card, or other similar.

[0098] Referring again to FIG. 2B, FIG. 2B shows some additional circuits that may be part of the processing circuit 250 in various implementations. For example, as shown in FIG. 2B, the processing circuit 250 may include an additional educational string determination circuit 264. In one implementation, for example, as shown in sub-operation 264A, the additional educational string determination circuit 264 may be configured to determine one or more additional educational strings based at least in part on the evaluation of the collected educational string voice sample data. The one or more additional educational strings are calculated to improve the user's spoken language skills when captured from the user for analysis and evaluation by the device. For example, if the user has repeatedly low scores for the pronunciation characteristic of "pitch" across multiple individual language components, future lessons (e.g., sets of educational strings presented to the user) may include educational strings that focus on situations where accurate "pitch" is emphasized. Similarly, if the user scores low on multiple pronunciation characteristics of the individual language component that is the "IY" phoneme, future lessons may include educational strings that include the "IY" phoneme in various letter forms and / or various words and / or various word positions, allowing the user to train his / her weak points more quickly. Referring again to FIG. 2B, this determination may be made at a remote server, for example, server 10. An example of this implementation is described in FIG. 2B in conjunction with the further educational string receiving circuit 266, which in various implementations may be configured to receive the determined one or more further educational strings at the remote server. The one or more further educational strings are calculated based at least in part on the evaluation of the collected educational string voice sample data and to improve the user's spoken language skills when captured from the user for analysis and evaluation.

[0099] In another implementation, an individual or organization that is helping users learn or improve their language skills can review the collected educational character string audio sample data evaluations. The individual may be helping a group of people, and in various implementations, the individual may have access to see the collected educational character string audio sample data evaluations for the entire group of people at once, so that the individual can determine which of his or her users need more help in certain areas, which users are falling behind, and which users have not yet grasped core concepts of language. These and other implementations are described in more detail herein.

[0100] Now referring to FIG. 2C, FIG. 2C shows one or more additional circuits that may be part of the processing circuit 250 in various implementations. For example, in one implementation, the training string providing circuit 252 of the processing circuit 250 may include a training string providing circuit 294 (e.g., as shown in FIG. 2F). The training string providing circuit 294 may include a training string visual providing circuit 272. In one implementation, for example, as shown in sub-operation 272A, the training string visual providing circuit 272 may be configured to provide a visual representation of the training string that the user is to read to the user via the device output component 220B. For example, in one implementation, the device output component 220B may include a touch screen of a mobile phone. On the touch screen, the training string visual providing circuit 272 may cause a training string, for example the training string 930 of FIG. 9, to be displayed. In some implementations, the educational string providing circuit 252 may further include an educational string audio providing circuit 274, which may be configured to provide an audio representation of the educational string to be read by the user, e.g., a sound delivered via a speaker or headphones, to the user via the device output component 220B, as shown in sub-operation 274A. In the example shown in FIG. 9, the user presses a button to trigger the audio representation, but in other implementations, the audio representation may be played automatically, or at a predetermined time, or based on other interactions with the user via the device input component 220A.

[0101] Referring again to FIG. 2C, in one implementation, the educational string providing circuit 294 (shown in more detail in FIG. 2F) of the educational string providing circuit 252, which is part of the processing circuit 250, can include an educational string interactive visual providing circuit 273, which can be configured to provide an interactive visual representation of the educational string that the user is to read to the user via the device output component 220B, for example, as shown in sub-operation 273A. The interactive visual representation can be clicked, for example, in the example shown in FIG. 9, the user presses a button to trigger an audio representation. In one implementation, the audio representation that is triggered can be the entire educational string, and is executed by the educational string response audio providing circuit 275, which is configured to provide an audio representation of the entire educational string that the user is to read to the user via the device output component 220B, for example, as shown in sub-operation 275A, in response to the user's interaction with at least a part of the interactive visual representation of the educational string. The "interactive visual representation of an educational string" does not have to be the educational string itself; rather, in various implementations, the interactive visual representation can include one or more buttons, and a user can click on one or more of these buttons to trigger an audio presentation of the entire educational string.

[0102] In another implementation, the audio presentation triggered may be a portion of the educational string to be read by the user, for example, the portion of the educational string to be read by the educational string response audio providing circuit sub-string 277, which may be configured to provide the audio presentation of the portion of the educational string to be read by the user (e.g., the sub-string) to the user via the device output component 220B in response to the user's interaction with at least a portion of the interactive visual presentation of the educational string, for example, as shown in sub-operation 277A. For example, in one implementation, clicking on a particular word of the educational string may trigger the audio presentation of only that word to be played. In another implementation, clicking on a particular word of the educational string may display a menu of the individual linguistic components of the word, and clicking on one of those components may trigger the audio presentation of only that individual linguistic component.

[0103] 2D, in one implementation, the evaluation presentation providing circuit 258 of the processing circuit 250 may include a visual presentation providing circuit 282 configured to provide visual feedback to the user on a display of the user's utterance regarding each of one or more individual language components analyzed against the corresponding one or more baseline individual language components. For example, the visual feedback may be a breakdown of how the user's pronunciation of the individual language components actually sounded. In another implementation, the visual feedback may be a color indicating a binary or threshold score for the pronunciation of the individual language components, which may be phonemes, syllables, words, or sentences. Similarly, in one implementation, the evaluation presentation providing circuit 258 of the processing circuit 250 may include an audible presentation providing circuit 284 configured to provide audible feedback of one or more of the user's utterance regarding a particular individual language component via a speaker in response to an interaction with the particular individual language component. For example, as shown in FIG. 10 and previously described, when a user clicks on a particular word or syllable button 1025, the audible presentation providing circuit 284 plays audible feedback as to how that individual language component is supposed to sound, and when a user clicks on a particular word or syllable button 1027, the audible presentation providing circuit 284 presents audible feedback of how the user's pronunciation of the individual language component sounded.

[0104] 2E, in one implementation, the training string voice sample data receiving circuit 254 of the processing circuit 250 can include a training string microphone data receiving circuit 286, which is configured to receive training string voice sample data, i.e., voice data of a user speaking a training string. In one implementation, the training string microphone data receiving circuit 286 may automatically trigger recording from the microphone. In another implementation, the user can interact with some part of the presentation to trigger the start of recording of the training string voice sample data by the microphone, for example as shown in FIG. 9.

[0105] As mentioned above, the individual language component analysis circuits 256 may include a speech evaluation engine 310, previously described in the context of the individual language component analysis circuits 256 and the processor 250. Figures 3A-3B, described in more detail immediately below, illustrate the details of a speech evaluation engine according to one or more implementations. It is noted that a specific component referred to as a speech evaluation engine is not required, i.e., in various implementations, the individual language component analysis circuits 256 may handle all of the functions of the speech evaluation engine 310 without being referred to or functioning as a speech evaluation engine 310.

[0106] 2F, as described above, the training strings configured for the user to read may or may not be shown to the user through the device output. In one implementation, the training string providing circuit 252 of the processing circuit 250 may include a training string providing circuit 290 configured to provide the user with one or more training strings to read. For example, in one implementation, the user may read the training strings from a display of the device, may read the training strings from a display of a different device (e.g., a separate screen that may or may not be connected to the device 50 and / or that may or may not be in communication with the device 50), or may read the training strings from a pre-populated sheet of training strings. In another embodiment, the user may repeat the training strings after they are pronounced or otherwise signaled by a teacher, tutor, assessor, or other third party (e.g., a parent, another student, or other party, whether or not trained to assist with speech assessment).

[0107] 2F, the training string providing circuit 290 may include a training string providing circuit 294 configured to provide one or more training strings to the user for reading via the device output component 220B of the device interface component 220. It is noted that in some implementations, one training string may be shown or displayed at a time, however, in other implementations, multiple training strings may be shown or displayed at a single point in time or at various times in succession.

[0108] 2F, the training string providing circuit 290 can include a training string providing circuit 298 configured to provide the user with one or more training strings to be read from an external source, such as a display of a different device, a piece of paper (e.g., a worksheet), or from an initial utterance by an instructor or other third party. It is noted that in some implementations, one training string may be shown or displayed at a time. However, in other implementations, multiple training strings may be shown or displayed at a single point in time or at various times in succession.

[0109] [Speech evaluation engine]

[0110] 3A-3B, e.g., FIG. 3A and FIG. 3B, which show various implementations of a speech evaluation engine, e.g., speech evaluation engine 310, and some of the components associated with the operation of speech evaluation engine 310. As mentioned above, there are many systems for mere recognition of speech. These systems are primarily designed for recognition across a wide range of use cases, e.g., many accents, speech patterns, vocabulary, and levels of speech impairment of users. Many of these systems use a speech recognition engine that recognizes the user's speech and corrects the user's poor or unusual speech. In contrast, the systems, methods, and apparatus described throughout this application use a speech evaluation engine rather than a speech recognition engine. In particular, the speech evaluation engine does not correct poor or unusual speech, but points out it. Thus, this engine has much more nuance than a speech recognition engine. In contrast to speech recognition engines that normalize the user's speech to be able to perform pattern matching, discarding much of the data on accent, intonation, pitch, frequency, and stress, the speech evaluation engine takes all of these factors into account when evaluating the quality of the speech, but does not consider the actual content of the speech. In various implementations, the speech evaluation engine is not interested in recognizing the words the user is speaking, since the system may already know the words the user is speaking in some part of the database. For example, the speech evaluation engine receives the user's speech data, e.g., educational string voice sample data, and has already cataloged the speech the user is about to speak, e.g., educational string. In this way, the speech evaluation engine can focus on finely evaluating the subtle nuances in the way the user speaks that are discarded or intentionally smoothed and minimized by almost all speech recognition engines that focus on determining which words the user speaks.

[0111] Thus, in various implementations, the speech evaluation engine is language and definition independent. The speech evaluation engine may be less interested in the meaning of the words and more interested in how close the pronunciation of the words is to the "ideal". The specific features of the speech evaluation engine then raise the question of how to determine the "ideal" pronunciation of words and sentences in a given language, e.g., English. In various implementations, the speech evaluation engine is "trained" with a large number, e.g., thousands or hundreds of thousands, of known training data sets that generate pronunciations. Note that, as a common convention, "trained" or "training" refers to machine learning, e.g., intelligence amplification (IA), or, in some implementations, artificial intelligence (AI). In contrast, "education" is generally used to indicate the effect of various implementations of the present invention on a user, e.g., "teaching" a new language or "teaching" how to speak properly. These rules apply generally throughout this application unless the context dictates otherwise. Thus, in various implementations, the training set is analyzed and modified using various analysis techniques, including machine learning techniques and intelligence amplification techniques. These techniques are described in more detail herein.

[0112] Referring now to FIG. 3A, FIG. 3A illustrates one exemplary implementation of a speech evaluation engine, e.g., a speech evaluation engine 310 operating in the environment 300. It is important to note that the layout of the speech evaluation engine 310 is merely for illustrative purposes and should not be limited to that particular layout, which was chosen merely for convenience and ease of illustration. In various implementations, parts of the speech evaluation engine 310 may be in different locations or may be omitted entirely. In other implementations, the speech evaluation engine 310 itself may be omitted or scattered across various parts of one or more of the server 10 and / or device 50. Since it is nearly impossible to sketch all permutations of the speech evaluation engine 310 contemplated by the invention herein, the illustrated implementation is merely exemplary and is left as a reminder that the full scope of the claimed invention is limited only by the claims.

[0113] Additionally, as generally stated above, the arrows in Figure 3A depict "typical data flows" rather than process flows, as these are exemplary components of the speech evaluation engine 310, rather than being part of a method. However, as with all components in Figure 3A, these arrows should not be considered the only way data may flow in the speech evaluation engine 310. Data flows that are not helpful to understanding are not shown, and in various other possible implementations, data may flow in various directions or in various ways or manners.

[0114] With particular reference to FIG. 3A, the speech evaluation engine 310 may be part of the server 10 as shown in FIG. 3A, but in other implementations, the speech evaluation engine 310 may be integrated into the device 50 as part of the device's circuitry (e.g., whether it is a multi-purpose device coded to deploy the circuitry as described herein or a specific disposable device). In various implementations, the speech evaluation engine 310 may be split across the server 10 and the device 50, with the server 10 having some of the circuitry and components and performing some of the functions of the speech evaluation engine 310, and the device 50 having other circuitry, components, and performing the functions of the speech evaluation engine 310. In various implementations, the roles of the server 10 and the device 50 are fixed with respect to the implementation of the speech evaluation engine 310, while in other implementations, the roles of the server 10 and the device 50 in the implementation of the speech evaluation engine 310 may change dynamically based on one or more factors, such as network conditions of the communication network 105, or the type of the device 50. For example, in some implementations, the device 50 may not have the computational resources to run all of the components and circuitry of the speech evaluation engine 310 because the computational resources of the device 50 are tied up in the performance of other tasks or because the device 50 does not have the physical attributes to have sufficient computational resources. In such scenarios, some of the load of the speech evaluation engine 310 may be shifted to the server 10.

[0115] Referring again to FIG. 3A, FIG. 3A illustrates the speech evaluation engine 310 according to one or more implementations. In one implementation, the speech evaluation engine 310 may receive the educational character string audio sample data 340 from the device 50. If the speech evaluation engine 310 is internal to the device 50, this reception may occur internal to the device 50, for example, via a bus or other internal path. If the speech evaluation engine 310 is external to the device 50, for example, in the server 10, the reception of the educational character string audio sample data 340 may occur via the communication network 105 or the like. When the speech evaluation engine 310 receives the educational character string audio sample data 340, the speech evaluation engine 310, particularly the individual speech component separator 312, may break down the educational character string audio sample data 340 into one or more individual language components 333. As discussed in more detail herein, the individual linguistic components 333 may be phonemes, e.g., units of the phonological system of a language that correspond to speech sounds and are perceived by the human ear as a single, distinctive sound of the language. In other embodiments, the individual linguistic components 333 may be syllables, diphthongs, digraphs, any other phonetic units, words, phrases, sentences, paragraphs, or sets of paragraphs or groups of text.

[0116] Referring again to FIG. 3A, in some implementations, after the individual speech component separator 312 of the speech evaluation engine 310 breaks down the training string speech sample data 340 into one or more individual language components 333, the one or more individual language components 333 are then compared to corresponding baseline individual language components 335. As briefly described above, the baseline individual language components 335 can be obtained from an internal or external source, for example, a baseline speech data database 350 as shown in FIG. 3A. In various implementations, the baseline speech data database 350 can be part of the device 50, the server 10, or can be a separate component, for example, a different server in a different location. In various implementations, the baseline speech data database 350 can include training data that determines characteristics of the baseline individual language components 335. The baseline speech data database 350 may be a standard training set, such as the British English speech corpus "WSJCAMO" derived from The Wall Street Journal spoken text corpus and the "Santa Barbara Corpus of Spoken American English" as two examples, although other corpora or custom corpora or sets of corpora may be used in the implementations described herein. Additionally, the baseline speech data database may be a modified corpus or set of corpora through the use of computer recognition, profiling, and other machine learning / intelligence amplification techniques to create baseline level values ​​for various phonemes and speech features within the corpus.

[0117] 3A-3B, for example, FIG. 3A, in various implementations, the baseline speech data database 350 may deliver one or more baseline individual language components 335 to the individual speech component analyzer 314 of the speech evaluation engine 310. As shown in FIG. 3A, the individual speech component analyzer 314 may receive the individual language components 333 and the baseline individual language components 335. Upon receiving this data, the individual speech component analyzer 314 may compare the received individual language components to the baseline individual language components and check various pronunciation characteristics. In various implementations, the pronunciation characteristics may include features such as pitch, intonation, frequency, stress, and accent (represented in FIG. 3A as boxes containing the letters P, I, F, S, A).

[0118] In one implementation, each feature is given a numerical score indicating how closely its individual linguistic component matches the corresponding baseline individual linguistic component. For example, for the sound "th," the comparison may suggest an 80% match in pitch, 60% match in intonation, 75% match in frequency, 20% match in stress, and 44% match in accent. This information is captured and used to evaluate the utterance as a whole, and can be tracked for use in determining more accurate feedback to help the user correct problems (e.g., if stress problems continue to occur on the same sound, that sound in words with various levels of stress can be emphasized in future lessons).

[0119] In another implementation, each feature is given a binary score to determine whether a particular individual linguistic component matches a corresponding individual linguistic component of the baseline, e.g., "yes" if the match is close enough and "no" if the match is not sufficient. In yet another implementation, each feature is evaluated as one or more of a set of discrete values, e.g., 1 to 5, with 5 being the closest match and 1 being the furthest match. In some implementations, each discrete value of the set of discrete values ​​may represent a range of closeness between a pronunciation characteristic of one of the individual linguistic components 333 and a pronunciation characteristic of one of the individual linguistic components 335 of the baseline. In some implementations, these thresholds may be displayed as color-coded values, e.g., 1 being red, 2 being bright red, 3 being yellow, 4 being bright green, and 5 being bright green. Other implementations may use other known methods of scoring and determining closeness between a pronunciation characteristic of one of the individual linguistic components 333 and a pronunciation characteristic of one of the individual linguistic components 335 of the baseline.

[0120] Further, in another implementation, the various individual linguistic components 333 may be assigned an overall score, e.g., a numerical value, a binary value, a threshold value, etc., that may combine one or more pronunciation characteristics for each of the individual linguistic components 333. The combination may be simply additive or may include a weighted average, e.g., in various implementations, pitch and intonation may be weighted more than stress and timing. In some implementations, this may result in an overall score being assigned to each of the individual linguistic components 333. Similarly, in some implementations, the various pronunciation characteristics may be assigned an overall score, e.g., a numerical value, a binary value, a threshold value, etc., and certain pronunciation characteristics may be combined across each individual linguistic component 333. As above, the individual linguistic components 333 may be averaged or added, or some of the individual linguistic components 333 may be weighted differently.

[0121] In some implementations, various combinations of the scores of the individual linguistic components 333 and the pronunciation characteristics may be combined to obtain an overall sample performance score. In such implementations, the overall sample performance component analyzer 322 (described in the next paragraph herein) may be removed or not used.

[0122] Referring again to FIG. 3A, the educational string speech sample data 340 may be delivered to the full sample performance component analyzer 322, which is shown in FIG. 3A as separate from the other components of the speech evaluation engine 310, for illustrative purposes only and for ease of illustration. In various implementations, the full sample performance component analyzer 322 may be combined with other parts of the speech evaluation engine 310, may appear in a different location than shown in FIG. 3A, or may be eliminated entirely, for example as described above. It is important to note that the layout of the speech evaluation engine 310 is for illustrative purposes only and should not be limited to a particular layout selected merely for convenience and ease of illustration.

[0123] Referring again to FIG. 3A, various analyses performed on the educational character string audio sample data 340 can be combined and returned as an analysis 345 of the educational character string audio sample data. In some implementations, the analysis 345 of the educational character string audio sample data can include the above analyses, as well as other analyses known in the art but not mentioned here. In implementations where the speech evaluation engine 310 is part of the device 50, there may be no "movement" of data except in the memory 245 of the device 50, or data may move along internal connections, such as through a bus or other connection of the device 50. In implementations where the speech evaluation engine 310 is implemented partially or entirely in the server 10 and / or additional servers, the analysis 345 of the educational character string audio sample data can be transmitted over the communication network 105.

[0124] 3B, in one implementation, the individual speech component analyzer 314 may include a pitch evaluation circuit 314A that compares, for example, the "pitch" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. The pitch pronunciation characteristics include, but are not limited to, characteristics that represent perceptual characteristics of sound characteristics (such as frequency) that are applied to phonemes for the purpose of differentiation or perception. In one implementation, the individual speech component analyzer 314 may include an intonation evaluation circuit 314B that compares, for example, the "intonation" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. The intonation pronunciation characteristics include, but are not limited to, variations in spoken pitch that can be used to change the meaning, attitude, and emotion of the speaker, for example, to differentiate between questions and statements. In one implementation, the individual speech component analyzer 314 may include a frequency evaluation circuit 314C that compares, for example, the "frequency" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. Frequency pronunciation characteristics include, but are not limited to, measurements of the frequency of sound made when speaking an individual language component, such as Hertz. In some implementations, the individual speech component analyzer 314 may include a stress evaluation circuit 314D that compares, for example, the "stress" pronunciation characteristics of the individual language component 333 and the baseline individual language component 335. In some implementations, stress pronunciation characteristics include, but are not limited to, the relative emphasis given to a particular phoneme, syllable, word or phrase, or other lexical unit that is part of a larger set of lexical units, which may be caused by characteristics such as increased loudness, phoneme length, changes in pitch, and other factors. In some implementations, the individual speech component analyzer 314 may include an accent evaluation circuit 314E that compares, for example, the "accent" pronunciation characteristics of the individual language component 333 and the baseline individual language component 335. In some implementations, accent pronunciation characteristics include, but are not limited to, modified or adjusted ways of speaking, speaking, or dialect specific to an individual, group, location, or other set of single or multiple individuals or entities.

[0125] In some implementations, the individual speech component analyzer 314 may include a speech rate evaluation circuit 314F, for example, to compare the "speech rate" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. In some implementations, the speech rate pronunciation characteristics include, but are not limited to, the speed at which phonemes are spoken individually or as a group, and may optionally include intervals and pauses between phonemes and / or other lexical units. In some implementations, the individual speech component analyzer 314 may include a hesitation evaluation circuit 314G, for example, to compare the "hesitation" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. In some implementations, the hesitation pronunciation characteristics include, but are not limited to, an analysis of the amount of hesitation before, during, or after the pronunciation of a particular individual language component. In some implementations, the individual speech component analyzer 314 may include a filler sound evaluation circuit 314H, for example, to compare the "filler sound" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. In some implementations, the filler sound pronunciation characteristics include, but are not limited to, evaluation of placeholders, filler sounds, and / or filler words, i.e., sounds or words spoken in a conversation as a signal that a pause has occurred but the speaker has not finished speaking, such as, for example, "um," "like," "ah," "you know," "er," etc. In some implementations, the individual speech component analyzer 314 may include a chunk evaluation circuit 314J that compares, for example, "chunk" pronunciation characteristics of the individual language components 333 and the baseline individual language components 335. In some implementations, the chunk pronunciation characteristics include, but are not limited to, pauses between words or sentences to add emphasis. It is important to note here that other pronunciation characteristics are considered and the nine illustrated in FIG. 3B are for illustrative purposes only. Additionally, various implementations may omit some or all of the illustrated pronunciation characteristics or weight or emphasize them differently.

[0126] 5A-5F onward, a series of flowcharts illustrating implementations follow. For ease of understanding, the flowcharts are organized such that an initial flowchart presents implementations via implementation examples, and then subsequent flowcharts present alternative implementations and / or extensions of the initial flowchart as subcomponent operations or additional component operations based on one or more flowcharts already presented. Those skilled in the art will appreciate that the style of presentation utilized in this specification (e.g., beginning with the presentation of flowcharts illustrating implementation examples, and providing additional and / or further details to subsequent flowcharts) generally allows for quick and easy understanding of various process implementations. In addition, those skilled in the art will further appreciate that the style of presentation used in this specification is also suitable for modular and / or object-oriented program design paradigms.

[0127] Additionally, in Figures 5A-5F and subsequent figures, various operations may be depicted in a boxed manner. Such depictions may indicate that the operations within the inner box may include optional exemplary embodiments of operational steps depicted in one or more outer boxes. However, it should be understood that the inner box operations are considered to be independent operations separate from any associated outer box and may be performed in any order or simultaneously with respect to all other depicted operations. Additionally, these operations depicted in Figures 5A-5F through 8A-8B, as well as other operations described herein, may be performed by at least one of a machine, an article of manufacture, or a composition of matter.

[0128] 5A-5F, which illustrate operations 500 that may include an operation 502 illustrating providing at least one educational string configured to be read by a user. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an educational string providing circuit 252, which may be configured in various implementations to provide at least one educational string configured to be read by a user. The operations 500 may further include an operation 504 illustrating receiving educational string voice sample data corresponding to a user's speech regarding the educational string. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an educational string voice sample data receiving circuit 254, which may be configured in various implementations to provide educational string voice sample data corresponding to a user's speech regarding the educational string. The operations 500 may further include an operation 506 illustrating analyzing one or more individual language components of the educational string voice sample data, (506a) which includes measuring one or more pronunciation characteristics of at least one individual language component against a corresponding one or more pronunciation characteristics of the baseline individual language component voice data. For example, Figures 2A-2F, e.g., Figure 2A, show an individual language component analysis circuit 256 that may be configured to analyze one or more individual language components of the educational string audio sample data in various implementations. The operation 500 may further include an operation 508 that illustrates providing an evaluation presentation to the user, (508a) the evaluation presentation including a presentation to the user of an evaluation of the educational string audio sample data based on one or more pronunciation characteristics of the analyzed one or more individual language components. For example, Figures 2A-2F, e.g., Figure 2A, show an evaluation presentation providing circuit 258 that may be configured to provide to the user an evaluation of the educational string audio sample data based on one or more pronunciation characteristics of the analyzed one or more individual language components in various implementations. These operations have already been described in more detail in connection with Figures 2A-2F, but further implementations are described in more detail herein with respect to additional flow charts and figures.

[0129] 5B, it is noted that the at least one educational string may include multiple educational strings that are part of the same processing set to which, for example, the steps and circuits already described may be applied. For example, in one implementation, all educational strings related to a particular lesson may be displayed on the screen at once and processed at once. In another implementation, multiple educational strings that are not necessarily related may be grouped together and processed together in serial, parallel, or other formats. Thus, throughout this document, unless otherwise indicated directly or by context, "educational string" should be understood to include the possibility of multiple educational strings. Referring to FIG. 5B, operation 502 may include operation 560, which illustrates a step of providing a set of multiple different educational strings configured for a user to read.

[0130] Referring again to FIG. 5B, FIG. 5B illustrates one or more additional operations that may be part of the operations 500 according to one or more implementations. For example, in an implementation, the operations 500 may include an operation 510 illustrating receiving a determination of one or more additional educational strings. For example, referring to FIG. 2B, in an implementation, the additional educational string receiving circuit 266 may be configured to receive a determination of one or more additional educational strings. In an implementation, the one or more additional educational strings are determined by an external entity, which in some implementations is another human being, and in some implementations is a machine, circuit, software, or combination thereof. The one or more additional educational strings may be based at least in part on an evaluation of the collected educational string speech sample data (510A). In an implementation, the determination is performed by an examiner who has access to an evaluation of each of one or more pronunciation characteristics of one or more individual language components.

[0131] Although not explicitly shown, the examiner has extensive access to the data generated by operation 500. For purposes of this application, the examiner is defined as an entity that has access to the user's evaluation and educational string speech sample data, but is not necessarily itself a user, and may be human or computationally based. The examiner may be a teacher, a tutor, a therapist (e.g., a speech therapist), a parent, a friend of the user, a computer monitoring the user, part of the device 50, part of the server 10, a separate computer from the computer shown in FIG. 1, or any other human or computational entity that can process the evaluation of the user's pronunciation of the educational strings. In various embodiments, the examiner may have multiple users under their guidance. In such embodiments, the examiner may have access to various identifying information of the users, including names, photos, email addresses, medical history, previous experience levels with languages, etc. In various implementations, the examiner can see, for each of the users, an audio or visual representation of some lessons the user has completed (e.g., lessons in this context means "a set of training strings with a common theme"), the amount of curriculum the user has completed (e.g., the total number of training strings the user has completed), each user's absolute performance, each user's high and low performance, each user's weighted average performance, each user's unweighted performance, each user's curve performance relative to other users under the examiner's guidance, each user's curve performance relative to larger demographic groups, etc.

[0132] Now, referring to FIG. 5C, FIG. 5C illustrates one or more additional operations that may be part of the operation 500 according to one or more implementations. For example, as shown in FIG. 5C, the operation 500 may illustrate an operation 512 of collecting an evaluation of educational character string voice sample data (512A), the evaluated educational character string voice sample data includes an operation 514 of determining one or more further educational character strings based at least in part on the evaluation of each of one or more pronunciation characteristics of one or more individual language components and / or the evaluation of the collected educational character string voice sample data, and as shown in sub-operation 514A, the one or more further educational character strings are calculated to improve the user's spoken language skills when captured from the user for analysis and evaluation. For example, in one implementation, referring to FIG. 2B, the educational character string voice sample data evaluation collection circuit 262 may be configured to perform an operation 512 of collecting an evaluation of the educational character string voice sample data, for example, and an operation 514 of determining one or more further educational character strings based at least in part on the evaluation of the collected educational character string voice sample data. As mentioned above, operation 512 collects an evaluation, which allows the user to go back and review the exact mistakes, allowing the user to see where they are having problems without undue difficulty. Secondly, the examiner, who is helping the user learn or improve their language skills, can see exactly the weaknesses and strengths of the user's language skills. This can reduce the overall time required for the user to achieve a desired level of fluency in the target language, as more targeted lessons can be brought to the user so that the user's weaknesses and strengths can be more quickly identified and targeted for further iterations of teaching strings.

[0133] Referring again to FIG. 5C, operation 514 includes operation 520 illustrating selecting one or more further instructional strings that emphasize a particular pronunciation characteristic of one or more individual language components that has been determined to have a low comparison score when measured against a particular pronunciation characteristic of the individual language components of the corresponding baseline or baselines. For example, one or more further instructional strings that emphasize a particular pronunciation characteristic (e.g., "accent") can be selected if the accent pronunciation characteristic is determined to have a low comparison score when measured against an accent pronunciation characteristic of the corresponding individual language components. In another implementation, operation 520 may include operation 522 illustrating selecting a set of further instructional strings that emphasize an intonation characteristic of the individual language components because the intonation characteristic has been determined to have a low comparison score of at least one individual language component when measured against an intonation characteristic of the individual language components of the corresponding baselines, as shown in FIG. 5C.

[0134] 5D, in one implementation, operation 514 may include operation 530 illustrating selecting one or more further instructional strings that highlight one or more individual language components that are determined to have a low comparison score when measured against one or more corresponding baseline individual language components. For example, a user may have difficulty with a particular individual language component, such as a "k" sound, which may be detected across multiple different pronunciation characteristics. In such an example, the further instructional strings are selected to provide the user with a variety of situations involving the individual language components, preventing the user from wasting time going back through concepts that the user has already understood or mastered, and allowing the user to spend more time on weaker areas.

[0135] In an implementation, if one or more of the measured pronunciation characteristics of the diphthong "YI" are determined to have a lower comparison score when measured against the corresponding pronunciation characteristics of the baseline of the diphthong "YI", the operation 530 may further include an operation 532 indicating selecting a set of further instructional strings that emphasize the diphthong "YI". In a different implementation, the operation 530 may further include an operation 534 indicating selecting one or more further instructional strings, the one or more further instructional strings emphasizing teaching the user regarding one or more errors of intonation, frequency, stress, accent, and timing detected in at least one evaluation of the collected instructional string audio sample data. In an implementation, the operation 514 may include an operation 536 indicating removing one or more further instructional strings intended to emphasize teaching the particular pronunciation characteristic from the set of further instructional strings, the instructional string audio sample data demonstrating little or no difference between the particular pronunciation characteristics of the one or more individual language components and the particular pronunciation characteristics of the one or more baseline individual language components. For example, in one implementation, if the pronunciation characteristics of the hesitation ratings of one or more individual language components of the training string speech sample data are determined to be within 95% of the pronunciation characteristics of the hesitation ratings of the corresponding individual language components of the baseline, then the particular training string that emphasizes training the pronunciation characteristics of the hesitation ratings can be removed from the set of further training strings displayed to the user.

[0136] Now referring to FIG. 5E, in one implementation, the operations 500 may further include an operation 542, which is an operation of repeating the operations 502 (providing educational strings configured to be read by the user), 504 (receiving educational string audio sample data corresponding to the user's speech regarding the educational strings), 506 (analyzing one or more individual language components of the educational string audio sample data), and 508 (providing an evaluation presentation to the user) for a plurality of different educational strings a certain number of times. For example, in one implementation, these operations are repeated 16 times, e.g., using a plurality of different but related educational strings to construct a lesson. However, the number 16 is merely exemplary, as any reasonable number of lessons greater than one are possible. Furthermore, the educational strings may be related, but this is not required. In one embodiment, the educational strings are related as described above with respect to the operations 530, 532, and 534, and detection of weaknesses in a particular individual language component or pronunciation characteristic is detected in the user, and a set of educational strings is constructed around the particular individual language component or pronunciation characteristic.

[0137] 5E, in one implementation, operations 500, which may include operation 542 already described, may also include operation 544 illustrating collecting an evaluation of the training string voice sample data at each particular time (544A), and the evaluated training string voice sample data is used, at least in part, to determine one or more additional training strings that, if captured from the user, are calculated to improve the user's spoken language skills. The captured additional training strings are used for analysis, e.g., at operation 506, and for providing an evaluation, e.g., at operation 508.

[0138] Referring again to FIG. 5E, in some implementations, operations 500, which may include operations 542 and 544 already described, may also include operation 546, which indicates using an evaluation of the collected educational string speech sample data to determine one or more additional educational strings. As noted above, the process may be effective as an iterative process, where a set of educational strings is analyzed, the user's weaknesses and strengths are selected by a human examiner or an analysis algorithm, and further sets of educational strings are presented to the user, and the process continues in an iterative manner. This iterative process, coupled with the fine-tuning available using the pronunciation characteristics of individual language components, allows the user to progress more quickly with less time wasted and slippage.

[0139] 5F, in one implementation, the operation 500, which may include operations 542, 544, and 546 as previously described, may also include an operation 552 of grouping one or more sets of users into a class. In this context, a class refers to a group of students of any size, whether under the supervision of an examiner or under the supervision of the same examiner. In one implementation, the operation 500 also includes an operation 554 that illustrates performing 542 (repeat the process for a certain number of strings) and 544 (collect each result of the process for each string) for each user in the class. In one implementation, this operation is a way that the examiner can receive detailed data regarding the performance of the students. In one implementation, the operation 500 also includes an operation 556 that illustrates visually displaying the list of classes to an instruction entity, e.g., an examiner, along with one or more performance characteristics of the classes. For example, as already described, the instruction entity may display for each of the users an audio or visual representation of a number of lessons the user has completed (e.g., lessons in this context means "a set of training strings with a common theme"), the amount of curriculum the user has completed (e.g., the total number of instructional strings the user has completed), each user's absolute performance, each user's high and low performance, each user's weighted average performance user, each user's unweighted performance, each user's curve performance relative to other users under the instruction entity's guidance, and each user's curve performance relative to larger demographic groups.

[0140] 5F, in one implementation, operations 500, which may include operations 542, 544, 546, 552, 554, and 556, may also include operation 557, in which a selection is received from the instruction entity of one or more users in the class, the instruction entity would like to see more characteristics, such as, for example, pronunciation characteristics, or performance of a particular individual language component. The instruction entity may make this selection for the human or other eye-based instruction entity based on the visual display described in step 556. In one implementation, after the selection of one or more users from the class, operations 500 may include operation 558, which illustrates presenting a component score to the instruction entity for each of the one or more pronunciation characteristics of the one or more individual language components, which represents how closely the particular individual language component matches a particular baseline individual language component.

[0141] Now referring to FIG. 6, FIG. 6 illustrates various alternative implementations of operation 502. For example, as shown in FIG. 6, operation 502 may include operation 602 illustrating providing a visual representation of an educational string configured to be read by a user. In one implementation, referring to FIG. 2C, the educational string visual providing circuit 272 may be configured to provide a visual representation of an educational string configured to be read by a user. The visual representation may be displayed, for example, on a monitor or a touch screen, for example, as shown in FIG. 9.

[0142] Referring again to FIG. 6, in one implementation, operation 502 may include operation 604, which indicates providing a visual representation of the educational string configured to be read by the user, and operation 606, which indicates providing an audio representation of the educational string configured to be read by the user in response to a request from the user. In one implementation, referring to FIG. 2C, the educational string visual providing circuit 272 may be configured to provide a visual representation of the educational string configured to be read by the user, and the educational string audio providing circuit 274 may be configured to provide an audio representation of the educational string configured to be read by the user in response to a request from the user. As shown in FIG. 9, the request from the user is made in the form of an interaction with a part of the visual providing of the educational string, for example, by pressing a button below the visual representation of the educational string.

[0143] Referring again to FIG. 6 , in one implementation, operation 502 may include one of operation 608 illustrating providing an interactive visual representation of the educational string configured to be read by the user, operation 610 illustrating providing an audio representation of the educational string configured to be read by the user in response to the user interacting with the interactive visual representation of the educational string, and operation 612 illustrating providing an audio representation of a portion of the educational string configured to be read by the user in response to the user interacting with a corresponding portion of the interactive visual representation of the educational string. In one implementation, referring to FIG. 2C , the educational string interactive visual providing circuit 273 may be configured to provide an interactive visual representation of an educational string configured to be read by a user, the educational string responsive audio providing circuit 275 may be configured to provide an audio representation of an educational string configured to be read by a user in response to a user interacting with the interactive visual representation of the educational string, and the educational string responsive audio providing circuit 277 may be configured to provide an audio representation of a portion of an educational string configured to be read by a user in response to a user interacting with a corresponding portion of the interactive visual representation of the educational string as already described in connection with FIG. 2C .

[0144] In some implementations, user interaction with a particular portion of the visual representation of the educational string may trigger the provision of an audio representation of that portion of the educational string, or another sub-portion of that portion of the educational string, such as one or more individual linguistic components that make up that portion of the educational string. Referring again to FIG. 6, in some implementations, operation 612 may further include operation 614 illustrating providing an audio and / or visual representation of one or more pronunciation characteristics of the portion of the educational string configured to be read by the user in response to the user interacting with a corresponding portion of the interactive visual representation of the educational string. For example, in some implementations, clicking on a portion of the educational string that is part of the visual representation may present a box that includes, for example, a visual or audio representation of how the individual linguistic components of that portion of the educational string are pronounced.

[0145] 7A, in one implementation, operation 508 may include operation 702 illustrating providing, as part of the evaluation presentation, for each of one or more pronunciation characteristics of one or more individual language components, (702A) a component score, which represents how closely the particular individual language component matches the particular baseline individual language component. For example, the component score may take into account various pronunciation characteristics and each of the individual deltas between the individual language component and the particular baseline individual language component. In various implementations, each pronunciation characteristic may have an independent weighting, which may be modified or optimized based on the user's previous results (e.g., some users may need more work on an accented pronunciation characteristic, so that the pronunciation characteristic is weighted more so that the user cannot "cheat" by getting other pronunciation characteristics closer to accurate, but lacking an accent). In one implementation, the component score 702A may include a binary indicator 704, which represents whether the particular individual language component matches the particular baseline individual language component. In another implementation, the component score 702A can include a numeric score indicator 706, which represents a discrete, fine-grained amount that a particular individual linguistic component matches an individual linguistic component of a particular baseline. In yet another implementation, the component score 702A can include a color-coded score indicator 708 that visually represents, on a graduated color scale, how closely a particular individual linguistic component matches an individual linguistic component of a particular baseline. For example, as shown in FIG. 10, "red" indicates an error and "green" indicates a closer match, and in various implementations the shades of red and green can be darker or lighter depending on how far or close the match was.

[0146] Referring again to FIG. 7A, in some implementations, operation 508 may include operation 710, which indicates providing the user with a numerical assessment of the user's fluency in the language of the educational string based on the analyzed educational string speech sample data. As can be seen in FIG. 10, after the educational string is analyzed with a score assigned to each individual language component, a cumulative score is tallied, which may serve as an assessment of the user's fluency in the language of the educational string. In some implementations, the numerical assessment of the user's fluency in the language of the educational string is based only on the particular educational string, while in other implementations, the numerical assessment of the user's fluency in the language of the educational string is based on the particular educational string and one or more previous educational strings that were analyzed.

[0147] Referring again to FIG. 7A, in one implementation, operation 508 may include operation 712 illustrating providing, as part of the evaluation presentation, for each of one or more individual language components, a visual representation of the user's speech with respect to the individual language component based on the evaluation of one or more pronunciation characteristics of the individual language component, to assist in improving the language skills of a speech or hearing impaired person. For example, in one implementation, the system may be used to assist in improving the language skills of a mentally, speech, hearing, or otherwise impaired person. For example, a hearing impaired person may not be able to hear or process language differences between the way of speaking and the sound intent of a language. By providing a visual representation of the pronunciation characteristics of one or more individual language components, a different path for learning the pronunciation characteristics may be provided to the hearing impaired person. This may allow the hearing impaired person to improve their language skills without relying on learning paths that are unavailable to the hearing impaired person.

[0148] 7B, in one implementation, operation 508 may include operation 720 illustrating displaying an assessment presentation to the user on a screen of a device associated with the user, (720A) the assessment presentation including one or more interactive objects configured to receive interaction from the user. For example, in one implementation, the assessment presentation may include an interactive object for viewing each word of the educational string for each of its individual language components. In another implementation, it may include an interactive object for hearing the user's pronunciation of a particular individual language component next to the baseline pronunciation of the particular individual language component. As previously mentioned, some of these examples are illustrated in FIG. 10, which includes a box 1050 that appears as a pop-up when a particular word of the educational string is interacted with.

[0149] Referring again to FIG. 7B, in one implementation, the evaluation presentation 720A can include an evaluation presentation 722 that displays at least one of the elements of the individual language constructs as an interactive object in response to an interaction from a user, and displays an individual score comparison between the individual language construct and a corresponding baseline individual language construct of one or more baseline individual language constructs. For example, as shown in FIG. 10, when at least one word (e.g., the word "cleaned") is interacted with, an individual score comparison of the individual language construct (e.g., the phonemes of the word "cleaned", e.g., "K", "L", "IY", "N", "D") is displayed, and an individual score comparison (e.g., a color-coded binary system in FIG. 10, but other scores are possible and are noted throughout) is displayed between the individual language construct and the baseline individual language construct. In another implementation, the evaluation presentation 722 may include individual comparisons 724 of pitch, individual comparisons of intonation, individual comparisons of stress, individual comparisons of accent, individual comparisons of speaking rate, individual comparisons of hesitation, individual comparisons of filler sounds, and individual comparisons of chunks between the individual language components and the corresponding baseline individual language components. In yet another implementation, the individual comparisons 724 may include individual comparisons 726 showing individual comparisons of pitch, individual comparisons of intonation, individual comparisons of stress, individual comparisons of accent, individual comparisons of speaking rate, individual comparisons of hesitation, individual comparisons of filler sounds, and individual comparisons of chunks between the individual language components and the corresponding baseline individual language components, each with a numerical score representing the delta between the individual language components and the corresponding baseline individual language components.

[0150] Referring again to FIG. 7B, in some implementations, operation 722 may include operation 728 indicating that the individual score comparisons between the individual language components and the corresponding baseline individual language components are displayed for all individual score comparisons below a certain threshold level to encourage the user to learn faster. In some implementations, the certain threshold level may be dynamically adjusted, e.g., the threshold level for indicating the individual score comparisons may increase over time as the user improves, and the user approaches more and more perfection. In another implementation, the certain threshold level is based on the user's demonstrated level of fluency, as calculated by the overall score of one or more training strings, as previously described. In some implementations, this dynamic targeting of the user's "weak spots," e.g., where the user consistently scores low, may reduce the overall amount of time it takes for the user to achieve a certain level of fluency in a language, e.g., a level that is quite fluent, compared to existing techniques for teaching a language to humans.

[0151] 7C, in one implementation, operation 720A showing an evaluation presentation including one or more interactive objects configured to receive interaction from a user may include various interactive objects, such as interactive object 740A and interactive object 740B. Interactive objects 740A and 740B may be the same type of interactive object or different types of interactive objects on the same presentation. For example, the interactive objects may be any object presented in a visual, auditory, tactile, or VR / AR environment, any device, such as a clickable / touchable interface, or an interface that responds to spoken commands, or an interface that responds to augmented reality interactions. In one implementation, interactive object 740A may include operation 742 showing an interactive object configured to present an evaluation audio presentation to a user, the evaluation audio presentation including one or more of an audio presentation of a particular portion of an educational string, and an audio presentation of a user's pronunciation of a particular portion of an educational string. For example, in some implementations, interaction with an interactive object may trigger an evaluative audio presentation, e.g., via a speaker, headphones, or a display of a device such as device 50. This audio presentation may highlight one or more of the proper pronunciation of the baseline individual language components associated with the interactive object and the user pronunciation of the individual language components associated with the interactive object. The audio presentation may also include visual components, e.g., a chart overlay of one or more characteristics of each of the proper pronunciation of the baseline individual language components associated with the interactive object, the user pronunciation of the individual language components associated with the interactive object. In some implementations, the user may repeat the audio presentation as many times as the user desires, and with this or other individual language components in any order.

[0152] Referring again to FIG. 7C, in some implementations, operation 742 may include operation 744 of presenting an interactive object configured to present an evaluation audio presentation to the user, the evaluation audio presentation being amplified or slowed down based on an analysis of the user's performance in pronouncing the particular portion of the educational string and other related portions of other educational strings to enhance the user's ability to recognize and correct mispronunciations of the particular portion of the educational string. For example, in some implementations, the audio presentation may be adjusted based on one or more factors, such as slowing down, speeding up, partially amplified, emphasized or de-emphasizing the audio presentation depending on one or more different factors. The one or more factors may include the user's performance on a set of educational strings, which may be similar to the educational string being displayed and evaluated, which may be a set of all educational strings spoken by the user, or which may be a set of educational strings in a particular time frame. In another implementation, the system can detect patterns of user errors and adjust the audio presentation based on the user's patterns of errors, e.g., if the user responds affirmatively to being shown slowed-down versions of individual language components that are mispronounced, the sounds of future mispronounced individual language components can be slowed down, and vice versa.

[0153] 7D, in one implementation, operation 720A may include operation 752 illustrating displaying an assessment presentation to the user on a screen of a device associated with the user, the assessment presentation including an interactive object for each particular individual language component identified as problematic for the user. For example, in one implementation, the assessment presentation includes interactive objects such as clickable / touchable interfaces, or interfaces that respond to verbal commands, or interfaces that respond to augmented reality interactions. In one implementation, interaction with one of the interactive objects displays an interactive demonstration for each particular individual language component identified as problematic for the user. In another implementation, interaction with the interactive object opens a new set of one or more lessons that focus on the particular individual language components and / or pronunciation characteristics determined to be problematic through analysis of the user's speech. In one implementation, interaction with the interactive object may allow the user to receive feedback in different forms, for example, verbal, audible, color-coded, visual, or via one or more visual or other representations of the manner in which the user is mispronouncing the individual language component. In some implementations, interaction with an interactive object allows a user to optionally practice more exercises related to that individual language component. In some implementations, interaction with an interactive object causes information about the individual language component to be displayed in one or more media.

[0154] Referring again to FIG. 7D, in an implementation in which operation 720A includes operation 752, operation 720A may further include operation 754, which indicates a step of allowing the user to directly interact with each interactive object, through isolation and through targeted interaction with at least one particular individual language component identified as problematic for the user, to reduce the time for the user to achieve a particular level of fluency in the language associated with the educational string. For example, interaction with the interactive object may isolate and target specific individual language components, such as those language components with which the user struggles, as described above. In this way, the user focuses training on the particular individual language components that the user has not yet fully mastered. This reduces the time wasted in training individual language components that the user does not need to train, in order to also train the particular individual language component. In various implementations, this may have the effect of reducing the time for the user to develop a particular level of fluency in the language associated with the educational string.

[0155] Referring now to FIG. 8A, in some implementations, operation 506 may include operation 802, which illustrates grouping one or more individual language components into one or more individual language component groups. For example, as shown in FIG. 10, in some implementations, the phonemes of the individual language components are grouped into words. In alternative implementations, referring again to FIG. 8A, one or more individual language components 808 may be grouped into groups of syllables, words, phrases, and / or sentences. Referring again to FIG. 8A, in some implementations, operation 506, which may include operation 802, may also include operation 804, which illustrates analyzing one or more individual language component groups, for example, through measurements of pitch, intonation, frequency, stress, accent, timing, speech rate, hesitation, filler sounds, and chunks on the baseline individual language component speech data, and may include operation 806, which illustrates including the results of the one or more individual language component groups analyzed as part of the assessment presentation.

[0156] Now referring to FIG. 8B, in some implementations, operation 506 may include operation 820, which indicates separating the educational string audio sample data corresponding to the user's speech regarding the educational string into one or more individual linguistic components. In some implementations, this is done by performing pattern matching of the educational string audio sample data to find known individual linguistic components present in the educational string audio sample data. However, in other implementations, other techniques for separating the educational string audio sample data into individual linguistic components may be used. In some implementations, operation 506, which may include operation 820, may also include operation 822, which indicates obtaining one or more baseline individual linguistic components corresponding to at least a portion of the one or more individual linguistic components. As already described, the baseline individual linguistic components may be obtained from any source, corpus, or set of corpora, but may be used as is, modified with machine learning techniques, or modified by averaging or weighted averaging one or more pronunciation characteristics based on one or more features of the source to generate "baseline" individual linguistic components. In one implementation, operation 506, which may include operations 820 and 822, may also include operation 824, which indicates a step of comparing at least one individual language component of the one or more individual language components with a corresponding baseline individual language component of the one or more baseline individual language components, which may be achieved by fine-grained detection of differences in one or more pronunciation characteristics of the individual language component relative to the corresponding baseline individual language component.

[0157] Figures 11, 12, and 13 illustrate various implementations of the system previously described herein. For the sake of brevity, some parts of the operation of the system are not repeated, and unless otherwise specified, details not mentioned in relation to Figures 11, 12, and 13 function in a similar manner to that described in relation to Figures 5A-5F and other figures, except where specifically delineated or internally contradictory. Now referring to Figure 11, Figure 11 illustrates an operation 1100 that may include an operation 1102 illustrating a stage of providing at least one partially edited educational string. For example, Figures 2A-2F, e.g., Figure 2A, illustrates an educational string providing circuit 252 that may be configured to provide at least one partially edited educational string in various implementations. In one implementation, as shown in sub-operation 1102A, the partially edited educational string includes one or more edited portions edited from the partially edited educational string when the partially edited educational string is presented to a user. For example, this method can be used to have the user fill in blanks to edit words to improve language fluency, such as "We all rode ____ to the mall." In other implementations, there can be multiple edit parts, such as "We all do _ from _ to _." In other implementations, clues about the number of words or letters may be given in the edit part as well, for example, if the edit part is "Thank you," the edited sentence might have two underlined blanks, such as "_ _ for the nice gift." In another example, the number of blanks may indicate the number of missing letters, such as "Alana rode _ _ _ chestnuts," where the missing word is "horse." In some implementations, one purpose of these exercises is to teach fluency along with accurate language pronunciation, to give the user the flexibility to speak additional words, and to break up lessons to make the overall educational program more interesting.

[0158] Referring again to FIG. 11, in one implementation, operation 1100 may include operation 1104 illustrating receiving educational string voice sample data corresponding to a user's speech regarding the educational string. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an educational string voice sample data receiving circuit 254 that may be configured in various implementations to receive educational string voice sample data corresponding to a user's speech regarding the educational string. In one implementation, as shown in sub-operation 1104A, the educational string may include a user's speech regarding one or more target strings corresponding to the user's speech regarding the evaluation of the words that make up one or more edited portions. For example, as described above, if the edited educational string is "we all went to the mall in a car," the user may speak a string such as "we all went to the mall in a car," "we went to the mall in a pushcart," or other similar phrases. The target string in the above example is the text that fills the edited portion (e.g., "car" or "pushcart," etc.) and can be recognized by the speech evaluation engine as part of the user-supplied string.

[0159] Referring again to FIG. 11, in some implementations, operation 1100 may include operation 1106, which indicates analyzing one or more individual linguistic components of the educational string audio sample data, including one or more edited portions. For example, FIGS. 2A-2F, e.g., FIG. 2A, show an individual linguistic component analysis circuit 256, which in various implementations may be configured to analyze one or more individual linguistic components of the educational string audio sample data, including one or more edited portions. In some implementations, as shown in sub-operation 1106A, the analyzing step may include measuring one or more pronunciation characteristics of at least one individual linguistic component of the educational string audio sample data against the corresponding one or more pronunciation characteristics of the individual linguistic components of the baseline of the educational string, as previously described. In other implementations, when sub-operation 1106A is present, operation 1106 may also include sub-operation 1106B, which indicates determining changes between the one or more target strings and the one or more edited portions by further measuring one or more pronunciation characteristics of the individual linguistic components of at least one or more target strings. For example, in one implementation, for the word "car" in the previous example "we went to the mall in a car", the edited string is "car" and the user's pronunciation of this word can be analyzed as already described. As the user selects different words in various implementations, the words are recognized and compared to the corresponding baseline words (e.g., if the user said "car" or "horse" or "phone"). In one implementation, points can be deducted from the user's score based on the words the user selects. For example, in one implementation, in the previous example, words such as "car", "van", "truck", and "subway" may still receive full points because the sentence makes sense with those words. Words such as "horse" and "wheelbarrow" may be deducted slightly in points because they may logically fit but not fit the context, and words such as "phone" may be deducted significantly in points because they do not logically fit into a partially edited educational string, such as "we all went to the mall in a phone".

[0160] Referring again to FIG. 11, in some implementations, the operation 1100 may include an operation 1108 illustrating providing a quality measure of the educational character string audio sample data. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an evaluation suggestion providing circuit 258, which in various implementations may be configured to provide a quality measure of the educational character string audio sample data. In some implementations, as shown in sub-operation 1108A, the quality measure of the educational character string audio sample data may be based at least in part on the determined changes between the one or more target strings and the one or more edited portions, as previously described. In various embodiments, the balance between the selection of the correct word or logic word may be weighted against the correct pronunciation of the word with various weightings to arrive at a user score useful for user feedback. The weighting of the various factors may change in response to factors such as the user's experience, language fluency, the user's performance with previous educational strings, and the emphasis of the particular lesson being given to the user. In some implementations, the instructor may manually assign the weighting of the various factors according to the instructor's lesson goals for the user.

[0161] Referring now to FIG. 12, FIG. 12 illustrates operations 1200 that may include an operation 1202 illustrating providing at least one option selection instructional set 1202. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an instructional string providing circuit 252, which in various implementations may be configured to provide at least an option selection instructional set, which itself may include various instructional strings, as further described herein. In one implementation, as shown in sub-operation 1202A, the option selection instructional set includes a set of two or more related option selection instructional strings, and a preferred option selection instructional string of the set of two or more related option selection instructional strings is a more accurate instructional string than other option selection instructional strings of the set of two or more related option selection instructional strings. For example, this method may be used to teach pronunciation and language skills while improving a user's language fluency. In various implementations, the option selection instructional set has a varying number of option selection instructional strings, and the strings may be closely related or distantly related based on the level of difficulty at which a particular user is operating. For example, in one implementation, if the user is operating at a more difficult level, the option selection instructional set may have more option selection instructional strings from which the user can select. In another implementation, if the user is operating at a more difficult level, the option selection instructional set may have option selection instructional strings that are closely related, e.g., only a few words apart, or have similar meanings, or have subtle grammatical or logical errors. In other implementations, all option selection instructional strings may be "correct" and the user may need to select the "most correct" option. An option selection instructional set may be displayed in a variety of ways, but not necessarily on the device. In one implementation, all of the option selection instructional strings of an option selection instructional set are displayed at once. In another implementation, one option selection instructional string is displayed to the user at a time, while in other implementations a varying number of option selection instructional strings may be displayed at a time, and the user may be able to switch back and forth between members of the option selection instructional set.

[0162] Referring again to FIG. 12, in some implementations, operation 1200 may include operation 1204 illustrating receiving training string voice sample data corresponding to a user's speech regarding a training string selected by the user from a set of two or more related option selection training strings. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate a training string voice sample data receiving circuit 254, which in various implementations may be configured to receive training string voice sample data corresponding to a user's speech regarding a training string selected by the user from a set of two or more related option selection training strings. In some implementations, the user speaks only the selected option selection training string, while in other implementations, the user may speak more option selection training strings.

[0163] Referring again to FIG. 12, in some implementations, the operation 1200 may include an operation 1206 illustrating analyzing one or more individual linguistic components of the educational string audio sample data. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an individual linguistic component analysis circuit 256 that may be configured to analyze one or more individual linguistic components of the educational string audio sample data in various implementations. In some implementations, as shown in sub-operation 1206A, the analyzing may include measuring one or more pronunciation characteristics of at least one individual linguistic component against the corresponding one or more pronunciation characteristics of the individual linguistic components of the baseline of the educational string, as previously described. In other implementations, if sub-operation 1206A is present, the operation 1206 may also include sub-operation 1206B, illustrating determining whether the educational string selected by the user matches the preferred option selected educational string, and further measuring one or more pronunciation characteristics of the individual linguistic components of the educational string selected by the user. For example, in some implementations, it is determined whether the educational string selected by the user is the correct string or the "most accurate" string. Various implementations may assign a point score based on whether the instructional string selected by the user is the correct one, and if not, whether it is the 2nd best answer, 3rd best answer, etc. In some implementations, only the correct answer is worth points under the scoring system used. In other implementations, points are awarded for some of the other answers in the optional instructional set, depending on the complexity of the question, the difficulty of selecting the correct answer, and the user's skill level / goals.

[0164] Referring again to FIG. 12, in some implementations, the operation 1200 may include an operation 1208 illustrating providing a quality measurement of the educational string audio sample data. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an evaluation suggestion providing circuit 258, which in various implementations may be configured to provide a quality measurement of the educational string audio sample data. In some implementations, as shown in sub-operation 1208A, the quality measurement of the educational string audio sample data may be based at least in part on the measured one or more pronunciation characteristics and the determined match of the educational string selected by the user with the preferred option selected educational string. In various implementations, the balance of selecting the correct option or one of the more accurate options may be weighted with various weights against the correct pronunciation of that option to arrive at a user score that is useful for user feedback. The weighting of the various factors may change in response to factors such as the user's experience, language fluency, the user's performance with previous educational strings, and the emphasis of the particular lesson being given to the user. In some implementations, the instructor may manually assign the weighting of the various factors according to the instructor's lesson goals for the user. In some implementations, if the user selects an incorrect option, the user repeats the exercise using the same option selection instructional set; in various implementations, all options may be the same or the selected incorrect option may be replaced with a different instructional string.

[0165] Referring now to FIG. 13, FIG. 13 illustrates operation 1300, which may include operation 1302 illustrating providing a prompt string audibly, visually, or a combination thereof. As noted above, in all instances throughout this application, the term "provide" is intended to include any kind of sensory presentation, e.g., audible, visual, tactile, or other sensory engagement, with audio and visual presentation specifically illustrated in FIG. 13. This is for illustrative purposes only and should not be considered as limiting other instantiations of the term provide, which do not specifically call out a particular sensory stimulus that serves as a medium for providing. Rather, the term is explicitly added here for ease of understanding and convenience of the reader. Returning to operation 1302, for example, FIG. 2A-2F, e.g., FIG. 2A, in various implementations, the prompt string may be configured to provide audibly, visually, or a combination thereof. In some implementations, as illustrated in sub-operation 1302A, the prompt string may be associated with a corresponding baseline response string responsive to the prompt string. In various implementations, there may be multiple corresponding baseline response strings. For example, in one implementation, if the prompt string is "Where would you take someone who just fell and broke their leg?", the only corresponding baseline response string may be "hospital" or two response strings, e.g., "hospital" and "emergency room". In another implementation, if the prompt string is "How many sunflower seeds can you eat at one time?", there may be many corresponding baseline response strings containing various numbers. In one implementation, the prompt string may be repeated to the user as many times as necessary. In another implementation, the prompt string may be presented first audibly and then visually, or vice versa, or simultaneously, or any combination thereof.

[0166] Referring again to FIG. 13, in some implementations, operations 1300 may include operation 1304 illustrating receiving educational string audio sample data corresponding to a user's response string to the prompt string. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an educational string audio sample data receiving circuit 254 that may be configured in various implementations to receive educational string audio sample data corresponding to a user's response string to the prompt string. In some implementations, the user may first repeat the prompt string and then respond with a response string or speak the response string alone. In some implementations, the user may ask a follow-up question about the prompt string via speech or other input entry (e.g., keyboard or touch screen). In other implementations, the prompt string may be an open-ended question without a specific answer, e.g., "What is the nature of free will and is it possible to determine whether such exists," and it is the speech rather than a specific response that is scored in the implementation.

[0167] Referring again to FIG. 13, in some implementations, the operation 1300 may include an operation 1306 which indicates analyzing one or more individual linguistic components of the training string audio sample data. For example, FIGS. 2A-2F, e.g., FIG. 2A, show an individual linguistic component analysis circuit 256 which may be configured to analyze one or more individual linguistic components of the training string audio sample data in various implementations. In some implementations, as shown in sub-operation 1306A, the analyzing may include measuring one or more pronunciation characteristics of at least one individual linguistic component of the response string against the corresponding one or more pronunciation characteristics of the corresponding individual linguistic component of the baseline response string, as previously described. In other implementations, when sub-operation 1306A is present, operation 1306 may also include sub-operation 1306B which indicates determining whether the response string is responsive to the prompt string. For example, an implementation may determine whether the user's selected response string responds to the prompt string, which in various implementations may be a two-part test, e.g., a response string that responds to the prompt string, and the response string is the correct answer; e.g., if the prompt string is "the color of the sky", then "green" is a syntactically correct answer but not a logically correct answer and may be scored accordingly. In other implementations, this distinction may not be made and a logically incorrect answer may be scored the same as a syntactically incorrect answer. In other implementations, there may be no "correct" answer, or there may be a set of correct answers, e.g., "name the prime numbers less than 100".

[0168] Referring again to FIG. 13, in some implementations, the operations 1300 may include an operation 1308 illustrating providing a quality measure of the educational string audio sample data. For example, FIGS. 2A-2F, e.g., FIG. 2A, illustrate an evaluation suggestion providing circuit 258, which in various implementations may be configured to provide a quality measure of the educational string audio sample data. In some implementations, as shown in sub-operation 1308A, the quality measure of the educational string audio sample data may be based at least in part on one or more pronunciation characteristics of the measured response strings and on the determined responsiveness of the response strings to the prompt strings. In various implementations, the balance of providing logically and / or syntactically correct response strings may be weighted against the correct pronunciation of the response strings to arrive at a user score useful for user feedback. The weighting of the various factors may vary in response to factors such as the user's experience, language fluency, the user's performance with previous educational strings, the emphasis of the particular lesson being given to the user, etc. In some implementations, the instructor may manually assign the weighting of the various factors according to the instructor's lesson goals for the user.

[0169] The foregoing examples are for illustrative purposes only, and the omission of an example herein should not be construed as an intentional or unintentional disclaimer of subject matter. The scope of the invention described herein is defined solely by the following claims at the end of this application.

[0170] V. Various Alternative Implementations and Open-Ended Statements

[0171] Those skilled in the art will recognize that it is common in the art to implement devices and / or processes and / or systems and then use engineering and / or other techniques to integrate such implemented devices and / or processes and / or systems into more comprehensive devices and / or processes and / or systems, i.e., at least portions of the devices and / or processes and / or systems described herein can be integrated into other devices and / or processes and / or systems with a reasonable amount of experimentation. Those skilled in the art will recognize that examples of such other devices and / or processes and / or systems, depending on the context and application, may include all or a portion of devices and / or processes and / or systems such as: (a) air transportation (e.g., airplanes, rockets, helicopters, etc.); (b) ground transportation (e.g., cars, trucks, locomotives, tanks, armored personnel carriers, etc.); (c) buildings (e.g., homes, warehouses, offices, etc.); (d) appliances (e.g., refrigerators, washers, dryers, etc.); (e) communications systems (e.g., network systems, telephone systems, Voice over IP systems, etc.); (f) business entities (e.g., Internet Service Provider (ISP) entities such as Comcast Cable, Qwest, Southwestern Bell, etc.); or (g) wired / wireless service entities (e.g., Sprint, Cingular, Nextel, etc.).

[0172] In certain cases, use of a system or method may occur in a domain even if components are outside the domain. For example, in the context of distributed computing, use of a distributed computing system may occur within a domain even if parts of the system are outside the domain (e.g., relays, servers, processors, signaling media, sending computers, receiving computers, etc. that are outside the domain).

[0173] A sale of a system or method may occur in a domain even if components of the system or method are located and / or used outside the domain. Furthermore, implementation of at least a portion of a system to perform a method in one domain does not preclude use of the system in another domain.

[0174] In a general sense, those skilled in the art will recognize that the various embodiments described herein may be implemented, individually and / or collectively, by various types of electromechanical systems having a wide variety of electrical components, such as hardware, software, firmware, and / or virtually any combination thereof, limited to subject matter patentable under U.S. patent law 101; and a wide variety of components capable of imparting mechanical force or motion, such as rigid bodies, spring or torsion bodies, hydraulics, electromagnetic actuators, and / or virtually any combination thereof. As a result, an "electromechanical system" as used herein includes, but is not limited to, an electrical circuit operably coupled with a transducer (e.g., an actuator, a motor, a piezoelectric crystal, a microelectromechanical system (MEMS), etc.), an electrical circuit having at least one discrete electrical circuit, an electrical circuit having at least one integrated circuit, an electrical circuit having at least one application specific integrated circuit, an electrical circuit forming a general purpose computing device configured by a computer program (e.g., a general purpose computer configured by a computer program that at least partially executes the processes and / or devices described herein, or a microprocessor configured by a computer program that at least partially executes the processes and / or devices described herein), an electrical circuit forming a memory device (e.g., a form of memory (e.g., random access, flash, read only, etc.)), an electrical circuit forming a communication device (e.g., a modem, a communication switch, an optical electrical device, etc.), and / or any non-electrical analog, such as optical or other analog (e.g., graphene-based circuitry). Those skilled in the art will also appreciate that examples of electromechanical systems include, but are not limited to, various consumer electronics systems, medical devices, and other systems such as motorized transportation systems, factory automation systems, security systems, communication / computing systems, etc. Those skilled in the art will recognize that electric machine, as used herein, is not necessarily limited to systems having both electrical and mechanical actuation, unless the context dictates otherwise.

[0175] In a general sense, those skilled in the art will recognize that the various aspects described herein, which may be individually and / or collectively implemented by a wide range of hardware, software, firmware, and / or any combination thereof, may be considered to be comprised of various types of "electrical circuitry." Thus, as used herein, "electrical circuitry" includes, but is not limited to, electrical circuitry having at least one discrete electrical circuit, electrical circuitry having at least one integrated circuit, electrical circuitry having at least one application specific integrated circuit, electrical circuitry forming a general purpose computing device configured by a computer program (e.g., a general purpose computer configured by a computer program that at least partially executes the processes and / or devices described herein, or a microprocessor configured by a computer program that at least partially executes the processes and / or devices described herein), electrical circuitry forming a memory device (e.g., a form of memory (e.g., random access, flash, read only, etc.)), and / or electrical circuitry forming a communication device (e.g., a modem, a communication switch, an optoelectronic device, etc.). Those skilled in the art will recognize that the subject matter described herein may be implemented in an analog or digital manner or any combination thereof.

[0176] Those skilled in the art will recognize that at least a portion of the devices and / or processes described herein can be integrated into a data processing system. Those skilled in the art will recognize that a data processing system typically includes one or more of a system unit housing, a video display device, a memory such as a volatile or non-volatile memory, a processor such as a microprocessor or digital signal processor, a computational entity such as an operating system, drivers, a graphical user interface, and application programs, one or more interaction devices (e.g., touch pad, touch screen, antenna, etc.), and / or a control system including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, control motors for moving and / or adjusting components and / or quantities). The data processing system can be implemented utilizing suitable commercially available components, such as those commonly found in data computing / communication systems and / or network computing / communication systems.

[0177] For purposes of this application, "cloud" computing may be understood as described in the cloud computing literature. For example, cloud computing may be a method and / or system for delivering computing power and / or storage capacity as a service. "Cloud" refers to one or more hardware and / or software components that provide or assist in the delivery of computing and / or storage capacity, including, but not limited to, one or more of clients, applications, platforms, infrastructure, and / or servers. Cloud may refer to any of the hardware and / or software associated with clients, applications, platforms, infrastructure, and / or servers. For example, cloud and cloud computing may refer to one or more of computers, processors, storage media, routers, switches, modems, virtual machines (e.g., virtual servers), data centers, operating systems, middleware, firmware, hardware backends, software backends, and / or software applications. Cloud may refer to private clouds, public clouds, hybrid clouds, and / or community clouds. A cloud is a shared pool of configurable computing resources and may be public, private, semi-private, distributable, scalable, flexible, ephemeral, virtual, and / or physical. A cloud or cloud services may be delivered over one or more types of networks, for example, a mobile communications network, the Internet, etc.

[0178] As used herein, cloud or cloud services may include one or more of infrastructure as a service ("IaaS"), platform as a service ("PaaS"), software as a service ("SaaS"), and / or desktop as a service ("DaaS"). As non-exclusive examples, IaaS may include, for example, the instantiation of one or more virtual servers, which can be started, stopped, accessed, and / or configured (e.g., providing one or more processor, storage space, and / or network resources on demand, e.g., EMC and Rackspace). PaaS may include, for example, one or more software and / or development tools hosted on the infrastructure (e.g., a computing platform and / or solution stack where a client can create software interfaces and applications, e.g., Microsoft Azure). SaaS may include, for example, software hosted by a service provider and accessible over a network (e.g., software for an application and / or data associated with that software application may be maintained on a network, e.g., Google Apps, SalesForce, etc.). DaaS may include, for example, providing desktops, applications, data, and / or services to users over a network (e.g., providing a multi-application framework, applications within the framework, data associated with the applications, and / or services related to the applications and data over a network, e.g., Citrix). The above are intended to be illustrative of the types of systems and / or methods referred to herein as "cloud" or "cloud computing" and should not be considered complete or exhaustive.

[0179] Those skilled in the art will recognize that the components (e.g., operations), devices, objects, and accompanying descriptions described herein are used as examples to clarify concepts, and that various configuration changes are contemplated. As a result, as used herein, the specific examples described and the accompanying descriptions are intended to represent more general classes. In general, the use of a specific example is intended to represent the class, and non-inclusion of a specific component (e.g., operation, etc.), device, or object should not be construed as limiting.

[0180] The subject matter described herein sometimes shows a number of different components contained within or connected to a number of different other components. It should be understood that such illustrated architectures are merely examples, and that in fact many other architectures that achieve the same functionality can be implemented. In a conceptual sense, an arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Thus, any two components combined herein to achieve a particular function can be considered to be "associated" with one another such that a desired "function" is achieved, regardless of the architecture or intermediate components. Similarly, two components so associated can be considered to be "operably connected" or "operably coupled" to one another to achieve a desired functionality, and two associated components can be considered to be "operably coupled" to one another to achieve a desired functionality. Specific examples of operably coupleable include, but are not limited to, physically matable and / or physically interacting components, and / or wirelessly interacting and / or wirelessly interacting components, and / or logically interacting and / or logically interacting components.

[0181] To the extent that formal overview headings are present in this application, it is understood that the overview headings are for presentation purposes and that various types of subject matter may be described throughout the application (e.g., an apparatus / structure may be described under a process / operation heading, and / or a process / structure may be described under a structure / process heading, and / or the description of a single topic may span two or more topic headings). Thus, use of formal overview headings in this application is for presentation purposes and is not intended to be limiting in any way.

[0182] Throughout this application, examples and lists are presented using parentheses, the abbreviation "Examples," or both. Unless otherwise specified, these examples and lists are merely illustrative and not exhaustive. In most cases, listing every example and every combination is prohibited. Therefore, smaller, illustrative lists and examples are used with a focus on understanding the claim terms, rather than limiting the scope of such terms.

[0183] With respect to the use of virtually any plural and / or singular terminology herein, those of skill in the art can convert from plural to singular and / or from singular to plural as appropriate to the context and / or application. For purposes of clarity, the various singular / plural permutations have not been explicitly set forth herein.

[0184] Those skilled in the art will recognize that the components (e.g., operations), devices, objects, and accompanying descriptions described herein are used as examples to clarify concepts, and that various configuration changes are contemplated. As a result, as used herein, the specific examples described and the accompanying descriptions are intended to represent more general classes. In general, the use of a specific example is intended to represent the class, and non-inclusion of a specific component (e.g., operation, etc.), device, or object should not be construed as limiting.

[0185] While one or more users may be illustrated herein as one illustrated diagram, for example in FIG. 1 and elsewhere, those skilled in the art will understand that one or more users may represent one or more human users, robotic users (e.g., computational entities), and / or substantially any combination thereof (e.g., a user may be assisted by one or more robotic agents), unless the context dictates otherwise. Those skilled in the art will understand that the same can generally be said about "sender" and / or such other entity-oriented terms are used herein, unless the context dictates otherwise.

[0186] In some cases, one or more components may be referred to herein as being "configured to," "configured by," "configurable to," "operable / operate to," "adapted / compatible to," "capable of," "compatible / compliant to," etc. Those skilled in the art will recognize that such terms (e.g., "configured to") generally encompass active state components and / or inactive state components and / or standby state components unless the context requires otherwise.

[0187] VI. Preamble to the Claimed Subject Matter

[0188] While particular embodiments of the subject matter described herein have been shown and described, it will be apparent to one skilled in the art that, based on the teachings herein, changes and modifications can be made without departing from the subject matter described herein and its broader aspects, and therefore, the appended claims are intended to encompass within their scope all such changes and modifications that are within the true spirit and scope of the subject matter described herein. In general, it will be understood by those skilled in the art that the terms used in this specification and in the appended claims in particular (e.g., the body of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "including" should be interpreted as "including but not limited to," etc.).

[0189] Moreover, those skilled in the art will further understand that where a particular number of introduced claim recitations is intended, such intent will be expressly recited in the claim, and in the absence of such recitation, no such intent exists. For example, to aid in understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce the claim recitation. However, the use of such phrases should not be construed as meaning that the introduction of a claim recitation with the indefinite article "a" or "an" limits a particular claim that includes such introduced claim recitation to a claim that includes only one such recitation, even if the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should generally be construed to mean "at least one" or "one or more"), and the same applies to the use of definite articles used to introduce claim recitations. Further, even if a particular number of introduced claim recitations is explicitly recited, one of ordinary skill in the art will recognize that such recitation should generally be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations" without other qualifiers generally means at least two recitations, or more than two recitations).

[0190] Furthermore, when a rule similar to "at least one of A, B, and C, etc." is used, such a structure is generally intended in the sense that one of ordinary skill in the art would understand the rule (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B and C together, etc.). When a rule similar to "at least one of A, B, or C, etc." is used, such a structure is generally intended in the sense that one of ordinary skill in the art would understand the rule (e.g., a system having at least one of A, B, or C includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will further be appreciated by those skilled in the art that disjunctions and / or phrases expressing two or more alternative terms, whether in the description, claims, or drawings, should normally be understood to contemplate the possibility of including one of the terms, either of the terms, or both terms unless the context dictates otherwise. For example, the phrase "A or B" is normally understood to include the possibilities of "A" or "B" or "A and B."

[0191] With respect to the appended claims, those skilled in the art will appreciate that the operations described therein may generally be performed in any order. Also, while various operational flows are presented in a sequence, it should be understood that various operations may be performed in orders other than those illustrated, or may be performed simultaneously. Examples of such alternative orders may include overlapping, alternating, interrupting, reordering, incremental, preparatory, supplementary, simultaneous, reverse, or other variant orders, unless the context dictates otherwise. Moreover, terms such as "responsive to," "related to," or other past tense adjectives are generally not intended to exclude such variants, unless the context dictates otherwise.

[0192] This application may refer to one or more trademarks, e.g., words, letters, symbols, or devices, adopted by one manufacturer or merchant and used to identify and / or distinguish its products from others. Trademark names as used herein are described in such language as to clarify their identity and distinguish them from common descriptive nouns, have a fixed and definite meaning, or are accompanied by other specific identification using terms not covered by the trademark in many, if not all, cases. Furthermore, the trademark names used herein do not refer to products or compounds that have a well-known and defined meaning in the literature or that require knowledge of one or more trade secrets to infer their meaning. All trademarks referred to in this application are the property of their respective owners, and the appearance of one or more trademarks in this application does not impair the validity of the one or more trademarks. All trademarks, registered or unregistered, that appear in this application are assumed to include the appropriate trademark symbol (e.g., a circled R or capital letters in brackets (e.g., [trademark name])) even if such trademark symbol is not explicitly displayed next to the trademark. To the extent a trademark is used in a descriptive manner to refer to a product or process, the trademark should be construed to represent the corresponding product or process as of the filing date of this patent application.

[0193] Throughout this application, terms such as "in one embodiment," "in one embodiment," "in some embodiments," "in multiple embodiments," "in at least one embodiment," "in various embodiments," and the like may be used. Each of these terms, and all such similar terms, should be interpreted as "in at least one embodiment, but not necessarily in all embodiments," unless otherwise specified. Specifically, unless otherwise specified, the intent of phrases such as these is to provide non-exclusive and non-limiting examples of implementations of the invention. The mere description that one, some, or many embodiments include one or more things or have one or more features does not mean that all embodiments include one or more things or have one or more features, nor does it mean that such embodiments must exist. It is merely an indication of an example and should not be interpreted as otherwise unless expressly stated to be so.

[0194] Throughout this application, terms such as "in an implementation," "in one implementation," "in several implementations," "in multiple implementations," "in at least one implementation," "in various implementations," and the like may be used. Each of these terms, and all such similar terms, should be interpreted as "in at least one implementation, but possibly, but not necessarily, in all implementations," unless otherwise specified. Specifically, unless otherwise specified, the intent of phrases such as these is to provide non-exclusive and non-limiting examples of implementations of the invention. The mere description that one, some, or many implementations include one or more things or have one or more features does not imply that all implementations include one or more things or have one or more features, nor does it imply that such implementations must exist. It is merely an indication of an example and should not be interpreted as otherwise unless expressly stated to be so.

[0195] Those skilled in the art will appreciate that the specific example processes and / or apparatus and / or techniques described above represent more general processes and / or apparatus and / or techniques that are taught elsewhere herein, such as in the claims filed herein and / or elsewhere in this application. [Item 1] 1. An apparatus configured to teach and / or assess one or more spoken language skills, the apparatus comprising: a device interface component having one or more of a device input component and a device output component; Memory, a processor operably coupled to the memory and configured to transmit and receive data to and from the device interface component; wherein the processor: an instructional string providing circuit configured to implement one or more instructional strings for a user to read; a training string voice sample data receiving circuit configured to receive training string voice sample data corresponding to the user's speech regarding the one or more training strings from the user via the device input component; an individual language component analysis circuit configured to analyze the training string speech sample data through a measurement of one or more pronunciation characteristics of at least one individual language component against a corresponding one or more pronunciation characteristics of a baseline individual language component; a rating representation providing circuit configured to provide a rating representation to the user on the device output component; and wherein the evaluation presentation includes, for each of the one or more individual linguistic components, an evaluation of the educational string audio sample data based on the one or more individual linguistic components of the analyzed educational string audio sample data. [Item 2] The educational character string providing circuit includes: a training string providing circuit configured to provide one or more training strings to a user for reading by the user; 2. The device according to item 1, comprising: [Item 3] The educational character string providing circuit includes: an instructional string providing circuit configured to provide one or more instructional strings to the user via the device output component of the device interface component for reading by the user; 3. The device according to item 2, comprising: [Item 4] an educational string speech sample data evaluation collection circuit configured to collect the evaluation of the educational string speech sample data including an evaluation of each of the one or more pronunciation characteristics of the one or more individual language components; 4. The apparatus of any one of claims 1 to 3, further comprising: [Item 5] 5. The apparatus of claim 4, further comprising a further educational string determination circuit configured to determine one or more further educational strings based at least in part on the evaluation of the collected educational string audio sample data, the one or more further educational strings being calculated to improve the user's spoken language skills when captured from the user for analysis and evaluation by the apparatus. [Item 6] The educational character string providing circuit includes: an instructional string visual providing circuit configured to provide a visual representation of the one or more instructional strings to be read by the user to the user via the device output component; an educational string audio providing circuit configured to provide to the user via the device output component an audio representation of the one or more educational strings that the user is to read; 6. The device according to any one of items 3 to 5, comprising: [Item 7] The educational character string providing circuit includes: an educational string interactive visual providing circuit configured to interact with the user via the device input component and configured to provide to the user via the device output component an interactive visual representation of the one or more educational strings that the user is to read; 7. The device according to any one of items 3 to 6, comprising: [Item 8] an educational string responsive audio providing circuit configured to provide to the user via the device output component an audio representation of the one or more educational strings to be read by the user in response to the user's interaction with the interactive visual representation of the one or more educational strings; 8. The apparatus of item 7, further comprising: [Item 9] 9. The apparatus of any one of claims 1 to 8, wherein the evaluation presentation includes an overall score of the user's speech with respect to the one or more training strings. [Item 10] The device output components include a display and a speaker, and the rating presentation providing circuitry further comprises: a circuit for providing a visual presentation to the user on the display of the user's utterance for each of the one or more individual language components analyzed against a corresponding one or more baseline individual language components; an audible presentation providing circuit for presenting, via the speaker, audible feedback of one or more of the user's utterances relating to a particular individual linguistic component in response to the interaction with the particular individual linguistic component; 10. The device according to any one of items 1 to 9, comprising: [Item 11] 1. A method of teaching and / or assessing one or more speech language skills, comprising: providing at least one instructional string configured to be read by a user; receiving training string voice sample data corresponding to the user's speech regarding the at least one training string; Analyzing one or more individual language components of the training string speech sample data, comprising measuring one or more pronunciation characteristics of at least one individual language component against one or more corresponding pronunciation characteristics of a baseline individual language component; providing an evaluation presentation to the user, the evaluation presentation including a presentation to the user of an evaluation of the training string audio sample data based on the one or more pronunciation characteristics of the one or more individual linguistic components analyzed; A method for providing the above. [Item 12] 12. The method of claim 11, further comprising receiving a decision of the one or more further educational strings based at least in part on the evaluation of the collected educational string audio sample data, the decision being performed by an examiner who has access to the evaluation of each of the one or more pronunciation characteristics of the one or more individual language components. [Item 13] 13. The method of claim 11 or 12, further comprising a step of collecting the evaluation of the educational character string audio sample data, the evaluated educational character string audio sample data including an evaluation of each of the one or more pronunciation characteristics of the one or more individual language components, the one or more pronunciation characteristics including one or more of a pitch pronunciation characteristic, an intonation pronunciation characteristic, a frequency pronunciation characteristic, a stress pronunciation characteristic, an accent pronunciation characteristic, a speaking rate pronunciation characteristic, a hesitation pronunciation characteristic, a filler sound pronunciation characteristic, and a chunk pronunciation characteristic. [Item 14] Item 14. The method of item 13, further comprising determining one or more further educational strings based at least in part on the evaluation of the collected educational string audio sample data, wherein the one or more further educational strings are calculated to improve the user's spoken language skills when captured from the user for analysis and evaluation by targeting one or more pronunciation characteristics including one or more of the pitch pronunciation characteristic, the intonation pronunciation characteristic, the frequency pronunciation characteristic, the stress pronunciation characteristic, the accent pronunciation characteristic, the speaking rate pronunciation characteristic, the hesitation pronunciation characteristic, the filler sound pronunciation characteristic, and the chunk pronunciation characteristic. [Item 15] providing at least one educational string configured to be read by a user, receiving educational string audio sample data corresponding to an utterance of the user relating to the at least one educational string, analyzing one or more individual linguistic components of the educational string audio sample data, and providing an evaluation suggestion to the user; repeating the steps a specific number of times, and for different educational strings for at least a portion of the specific number of times; collecting the evaluation of the training string voice sample data at each of the particular times, the evaluated training string voice sample data being used at least in part to determine one or more further training strings that, when captured from the user for analysis and evaluation, are calculated to improve the user's spoken language skills; determining the one or more additional training strings using the evaluation of the collected training string audio sample data; 15. The method of any one of items 11 to 14, further comprising: [Item 16] grouping a set of one or more users into a class; performing the steps of repeating and collecting the evaluations of the training string speech sample data for each of the users in the class; Item 16. The method of item 15, further comprising: [Item 17] visually displaying the list of classes to an instruction entity along with one or more performance characteristics of the classes; receiving from the command entity a selection of one or more users within the class for additional characteristic viewing; presenting to the instruction entity, for each of the one or more pronunciation characteristics of the one or more individual language components, a component score representing how closely a particular individual language component matches a particular baseline individual language component; Item 17. The method of item 16, further comprising: [Item 18] The step of providing a rating submission to the user further comprises: providing, as part of said evaluation presentation, for each of said one or more pronunciation characteristics of said one or more individual language components, a component score representing how closely a particular individual language component matches a particular baseline individual language component. 18. The method according to any one of items 11 to 17, further comprising: [Item 19] The component score, which represents how closely a particular individual language component matches a particular baseline individual language component, is A binary indicator representing whether the particular individual linguistic component matches the particular baseline individual linguistic component. 20. The method of claim 18, further comprising: [Item 20] The component score, which represents how closely a particular individual language component matches a particular baseline individual language component, is A numerical score indicator representing the discrete amount that the particular individual linguistic component matches the particular baseline individual linguistic component. 20. The method according to item 18 or 19, further comprising: [Item 21] The component score, which represents how closely a particular individual language component matches a particular baseline individual language component, is a color-coded score indicator that visually represents, on a graduated color scale, how closely the particular individual language component matches the particular baseline individual language component; 21. The method according to any one of items 18 to 20, further comprising: [Item 22] The step of providing a rating submission to the user further comprises: providing the user with a numerical assessment of the user's fluency in the language of the at least one training string based on the analyzed training string speech sample data. 22. The method according to any one of items 11 to 21, comprising the steps of: [Item 23] The step of providing a rating submission to the user further comprises: 23. The method of any one of items 11 to 22, comprising displaying the assessment presentation to the user on a screen of a device associated with the user, the assessment presentation including one or more interactive objects configured to receive interaction from the user. [Item 24] The evaluation submission includes one or more interactive objects configured to receive an interaction from the user, and the method further comprises: displaying the assessment presentation to the user on a screen of a device associated with the user, the assessment presentation including an interactive object for each particular individual language component identified as problematic for the user; enabling said user to directly interact with each interactive object to enable said user to reduce the time it takes to achieve a particular level of fluency in the language associated with said at least one training string through isolation and targeted interaction with said at least one particular individual linguistic component identified as problematic for said user; 24. The method of claim 23, comprising: [Item 25] the evaluation submission includes one or more interactive objects configured to receive an interaction from the user; and displaying at least one of the individual language components as an interactive object that displays an individual score comparison between the individual language component and a corresponding individual language component of the one or more baselines in response to an interaction from the user. 25. The method according to item 23 or 24, comprising: [Item 26] 26. The method of claim 25, wherein the individual score comparisons between the individual language components and corresponding baseline individual language components are shown for all individual score comparisons below a certain threshold level, facilitating a reduction in the user's learning time. [Item 27] The individual score comparison between the individual language components and corresponding baseline individual language components is individual comparisons of pitch, individual comparisons of intonation, individual comparisons of stress, individual comparisons of accent, individual comparisons of speech rate, individual comparisons of hesitations, individual comparisons of filler sounds, and individual comparisons of chunks between the individual speech components and the corresponding individual speech components of the baseline. 27. The method according to item 25 or 26, comprising: [Item 28] 28. The method of claim 27, wherein the pitch individual comparison, the intonation individual comparison, the stress individual comparison, the accent individual comparison, the speaking rate individual comparison, the hesitation individual comparison, the filler sound individual comparison, and the chunk individual comparison each comprise a numerical score representing the delta between the individual language component and the corresponding baseline individual language component. [Item 29] The evaluation submission comprises one or more interactive objects configured to receive an interaction from the user; an interactive object configured to present an evaluation audio presentation to the user; 29. The method of any one of claims 23 to 28, wherein the evaluation audio presentation includes one or more of an audio presentation of a particular portion of the at least one training string and an audio presentation of the user's pronunciation of the particular portion of the at least one training string. [Item 30] 30. The method of claim 29, wherein at least a portion of the evaluation audio presentations are amplified or slowed down based on an analysis of the user's performance in pronouncing the particular portion of the at least one educational string and other related portions of other educational strings to enhance the user's ability to recognize and correct the user's mispronunciations of the particular portion of the at least one educational string. [Item 31] The step of analyzing one or more individual linguistic components of the training speech sample data includes: grouping the one or more individual linguistic components into one or more individual linguistic component groups; analyzing said one or more individual language component groups through one or more measures of pitch, intonation, frequency, stress, accent, timing, speech rate, hesitation, filler sounds and chunks for said baseline individual language component groups; including as part of the assessment presentation the results of the one or more individual language component groups that have been analyzed; 31. The method according to any one of items 11 to 30, comprising the steps of: [Item 32] said step of grouping said one or more individual linguistic components into one or more individual linguistic component groups further comprising: grouping said one or more individual linguistic components into groups of syllables, words, phrases, and / or sentences. 32. The method according to claim 31, comprising: [Item 33] Item 33. The method of any one of items 11 to 32, wherein the at least one training string is one or more of a syllable, a word, a sentence, a paragraph, or a set of paragraphs. [Item 34] 34. The method of any one of claims 11 to 33, wherein the individual linguistic components are phonemes or other perceptually distinct units of sound. [Item 35] 1. A method for improving a user's language fluency, comprising: providing at least one partially edited training string, the at least one partially edited training string including one or more edited portions edited out of the at least one partially edited training string when the at least one partially edited training string is presented to the user; receiving training string voice sample data corresponding to the user's speech regarding the at least one partially edited training string including the user's speech regarding one or more target strings corresponding to the user's speech regarding evaluation of words constituting the one or more edited portions; Analyzing one or more individual linguistic components of the training text speech sample data including the one or more edited portions, measuring one or more pronunciation characteristics of at least one individual linguistic component of the training string speech sample data against one or more corresponding pronunciation characteristics of individual linguistic components of a baseline of the at least one partially edited training string; determining changes between the one or more subject strings and the one or more edited portions by at least further measuring the one or more pronunciation characteristics of the individual linguistic components of the one or more subject strings; and providing a quality measure of the training string audio sample data based at least in part on the determined changes between the one or more subject strings and the one or more edited portions; A method for providing the above. [Item 36] 1. A method for improving a user's language fluency, comprising: providing at least one option selection training set, the at least one option selection training set including a set of two or more related option selection training strings, a preferred option selection training string of the set of two or more related option selection training strings being a more accurate training string than other option selection training strings of the set of two or more related option selection training strings; receiving training string voice sample data corresponding to a user's speech regarding a training string selected by the user from the set of two or more related optional selection training strings; Analyzing one or more individual linguistic components of the training text speech sample data, measuring one or more pronunciation characteristics of at least one individual language component against a corresponding one or more pronunciation characteristics of a baseline individual language component; determining whether the user selected training string matches the preferred option selected training string, and further measuring the one or more pronunciation characteristics of the individual linguistic components of the user selected training string; and providing a quality measurement of the training string audio sample data based at least in part on the measured one or more pronunciation characteristics and the determined match between the user selected training string and the preferred option selected training string; A method for providing the above. [Item 37] 1. A method for improving a user's language fluency, comprising: providing a prompt string audibly, visually, or a combination thereof, where the prompt string has a corresponding baseline response string responsive to the prompt string; receiving training voice sample data corresponding to a response string of the user to the prompt string; Analyzing one or more individual linguistic components of the training text speech sample data, measuring one or more pronunciation characteristics of at least one individual linguistic component of the response string against a corresponding one or more pronunciation characteristics of a corresponding individual linguistic component of the baseline response string; determining whether the response string responds to the prompt string; and providing a quality measure of the training string audio sample data based at least in part on the measured one or more pronunciation characteristics of the response strings and the determined responsiveness of the response strings to the prompt strings; A method for providing the above.

Claims

[Claim 1] 1. An apparatus configured to teach and / or assess one or more spoken language skills, the apparatus comprising: a device interface component having one or more of a device input component and a device output component; Memory, a processor operably coupled to the memory and configured to transmit and receive data to and from the device interface component; wherein the processor: an instructional string providing circuit configured to implement one or more instructional strings for a user to read; a training string voice sample data receiving circuit configured to receive training string voice sample data corresponding to the user's speech regarding the one or more training strings from the user via the device input component; a circuit for analyzing individual linguistic constituents, Separating the training string voice sample data into at least one individual language component without recognizing the user's speech, the at least one individual language component comprising one or more of the following phonetic units: a phoneme, a syllable, a diphthong, or a digraph; measuring one or more pronunciation characteristics of the at least one individual linguistic component, the one or more pronunciation characteristics comprising one or more of a pitch pronunciation characteristic, an intonation pronunciation characteristic, a frequency pronunciation characteristic, a stress pronunciation characteristic, an accent pronunciation characteristic, a speaking rate pronunciation characteristic, a hesitation pronunciation characteristic, a filler sound pronunciation characteristic, and a chunk pronunciation characteristic; an individual language component analysis circuit configured to analyze the training string speech sample data by comparing one or more pronunciation characteristics of one or more baseline individual language components corresponding to the training string speech sample data with one or more pronunciation characteristics of each of the measured individual language components, and generate a quality score for each of the individual language components; a rating representation providing circuit configured to provide a rating representation to the user on the device output component; and wherein the evaluation presentation includes, for each of the one or more individual language components, an evaluation of the educational string audio sample data regarding how accurately the one or more educational strings were pronounced by the user based on the one or more individual language components of the analyzed educational string audio sample data.

Images