System and methods for monitoring reading performance and providing reading assistance

Abstract

A system, implemented in part by an application running on a computing device, displays one or more written words from a book within a graphical user interface. The application captures audio of a user's spoken word using a microphone of the computing device. The application processes the audio using speech recognition to determine whether the user's spoken word matches the one or more written words. A application updates a word graph, based on the matching of the spoken word with the written word, to define accuracy and fluency of the user.

Description

SYSTEM AND METHODS FOR MONITORING READING PERFORMANCE ANDPROVIDING READING ASSISTANCERELATED APPLICATION

[0001] This application claims priority to US Provisional Patent Application Serial Number 63 / 610,311, filed December 14, 2023, titled “System and Methods for Monitoring Reading Performance and Providing Reading Assistance,” and incorporated herein by reference in its entirety.BACKGROUND

[0002] Reading provides a fundamental mode of communication and learning, however, a significant number of people struggle with learning how to read. An inability to properly read words may be frustrating, stressful, and may further reduce reading confidence leading to later learning difficulties. Instructing students to read has typically been performed in a classroom environment where, for example, an entire class would read a passage from a text. For example, a teacher may select one student in a group to read a passage and the teacher would evaluate the student during the reading. The teacher would then select another student to read a different passage, and continue the exercise for all students in the class. Recently, electronic books (e - books) have provided convenient paperless access to reading materials such as books. This electronic medium permits customization of the format and presentation of the reading material that is not possible with a paper format. The electronic medium also permits customization of the user interface in which the reading material is presented. These customizations may extend to designs that enhance teaching effectiveness for particular groups of students.SUMMARY

[0003] According to various embodiments, the present technology is directed to a user interface for administering a reader application comprising electronic speech recognition. The user interface presented on a computer display includes a user identifier tag uniquely identifying a user of the reader application. Also included is a listing of a plurality of monitored reading performance metrics connected with the user identifier tag, and a user- selectable list of at least one reading assistance process connected with the user identifier tag.Selecting an item from the list launches the respective reading assistance process that includes at least one instruction to administer the reading assistance process.

[0004] According to some embodiments, the present technology is also directed to a user interface for a reader application comprising electronic speech recognition. The user interface presented on a computer display includes a user identifier tag uniquely identifying a user of the reader application, the tag being presented in a first portion of the user interface. Also included is a listing of a plurality of monitored reading performance metrics for the user identifier tag, the metrics being presented in a second portion of the user interface. In addition, the user interface includes a user-selectable list of at least one reading assistance process, the list being presented in a third portion of the user interface, wherein selecting an item from the list launches the respective reading assistance process.

[0005] In accordance with various embodiments, the present technology is directed to a system for administering a reader application comprising electronic speech recognition. The system includes a microphone for recording an administrator's voice output, a speaker for listening to read material from at least one user, a memory for storing executable instructions, a processor configured to execute the instructions, the instructions being executed by the processor to administer the reader application, and a user interface for administering a reader application presented on a computer display. The user interface includes a user identifier tag uniquely identifying a user of the reader application, and a listing of a plurality of monitored reading performance metrics connected with the user identifier tag. Also included is a user- selectable list of at least one reading assistance process connected with the user identifier tag, wherein selecting an item from the list launches the respective reading assistance process comprising at least one instruction to administer the reading assistance process.

[0006] It should also be noted that the present embodiments may provide reading assistance to users who have learning differences. For instance, dyslexic children who may use this application to practice their reading may skip words, so the application has a feature indicating the written word that was skipped. Furthermore, for a dyslexic child, currently the only true feedback they receive is from a teacher who measures the time to read one or more written words and who notes the reading errors that the child made during that timed reading. These timed reading sessions are infrequent, which impacts their benefit to the child.

[0007] Dyslexic children need to practice on a regular basis, to improve their reading skills, and they need to be able to practice reading independently. Most dyslexic children also want to know if their reading is improving over time, and how many words they read correctly in a given reading session. The present embodiments address the needs of dyslexicchildren, by providing them a vehicle to practice reading independently on a regular basis and providing them also with their accuracy and fluency metrics.

[0008] Dyslexic child users will benefit from the present embodiments since with the reader application, the users may see their data in real time and track their progress. The data regarding the user's reading skills as obtained by the systems described herein are far more accurate than the data collected by a child’s teacher in a timed reading. A user may also see trends concerning their reading accuracy and fluency based on their usage of the application. For instance, by way of a non-limiting example, a user may see that if they only read twice a week without a warm up sequence, they may not have improved, but if they read three times a week with a warm up sequence and a cool down sequence, the user may discover that their reading skills improved. Also a user may see that users with similar profiles saw a huge improvement if they read consistently using the application. The application will also prescribe to the user a number of reading sessions per week to realize significant reading skill improvement based on users with similar profiles. Further, based on users with similar profiles, the application might recommend that the user read for a given number of minutes per day or per week, to realize significant improvement in their reading skills.

[0009] In certain embodiments, the techniques described herein relate to a method for monitoring reading performance, including: displaying, by an application running on a computing device and within a graphical user interface displayed, one or more written words from a book by the computing device; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user's spoken word matches the one or more written words; and updating a word graph based on the matching of the spoken word with the written word; wherein the word graph defines accuracy and fluency of the user.

[0010] In certain embodiments, the techniques described herein relate to a method for assisting reading of a book using a reader application, including: determining, by the reading application running on a computing device, next reading words of the book for a next reading session; generating a list of unmastered challenge words in the next reading words; selecting a most difficult challenge word from the list; determining a number of times the most difficult challenge word occurs in a remainder of the book; generating a flashcard presenting the most difficult challenge word and the number; presenting the flashcard on a display of the computing device; capturing, using a microphone of the computing device, audio of a user's spoken word when reading the most difficult challenge word; and processing the audio usingspeech recognition of the application to determine whether the user’s spoken word matches the most difficult challenge word.

[0011] In certain embodiments, the techniques described herein relate to a method for providing independent reading assistance, including: displaying, by an application running on a computing device, one or more written words selected from a book; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user’s spoken word matches the one or more written words; determining accuracy and fluency metrics based on the user's spoken word matching the one or more written words; and presenting, by the application, the metrics on a display of the computing device.BRIEF DESCRIPTION OF THE FIGURES

[0012] FIG. 1 illustrates one example system for monitoring reading performance and providing reading assistance, in embodiments.

[0013] FIG. 2 is a block diagram illustrating a reader server of the reader system of FIG. 1 in further example detail, in embodiments.

[0014] FIG. 3 is a block diagram illustrating a reader application of the reader system of FIG. 1 in further example detail, in embodiments.

[0015] FIG. 4A is a schematic diagram illustrating the user data stored in the reader server of FIG. 2 in further example detail, in embodiments.

[0016] FIGs. 4B, 4C, and 4D show example GUIs, in embodiments.

[0017] FIG. 5 shows the GUI of FIG. 3 displaying example page text from the book for the user to read in an Oral (read out loud) mode, in embodiments.

[0018] FIG. 6 is a flowchart illustrating one example method for teaching imminently encounterable new words to the user, in embodiments.

[0019] FIG. 7 is a block diagram illustrating example data used by the vocabulary virtuoso of FIG. 3, in embodiments.

[0020] FIG. 8 shows one example flashcard listing the most difficult challenge words found in a next portion of the book of FIG. 2, in embodiments.

[0021] FIG. 9A shows one example flashcard defining one of the most difficult challenge words and providing context for the challenge word from three places in the book, in embodiments.

[0022] FIG. 9B shows another example flashcard defining one challenge word and providing the user with an opportunity to practice reading the word three times, in embodiment.

[0023] FIG. 10 shows one example flashcard defining root words that correspond to the most difficult challenge words, in embodiments.

[0024] FIG. 11 shows one example flashcard providing a definition of one of root words, in embodiments.

[0025] FIG. 12 shows one example flashcard defining one root word variant and with context in the book, in embodiments.

[0026] FIG. 13 is a block diagram illustrating the reader server of FIGs. 1 and 2 further including reader artificial intelligence, in embodiments.

[0027] FIG. 14 shows the teacher dashboard of FIG. 2 in further example detail.

[0028] FIG. 15 shows one example competition scenario, in embodiments.

[0029] FIG. 16 shows a diagrammatic representation of a computing device, in embodiments.DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The user interface will support a reader application that uses speech recognition in order to detect if a user is accurately reading one or more words out loud from a given text. In various embodiments, the reader application is an electronic book (e-book) reader application.

[0031] Many people, adult and children alike, have a desire to practice reading on a regular basis. They wish to read words aloud accurately from a text. The text may be provided from any source of the written word, including but not limited to, books, articles, flashcards, pamphlets, magazines, journals, trade papers, newspapers, news, newspaper clippings, news aggregators, Internet forum messages, one or more webpages from the Internet, and any combination thereof.

[0032] The present embodiments provide a system, methods, and user interfaces for an Al-enabled reading application that uses machine learning and speech to text technology, and provides real-time feedback to the user. The application may also track the user's progress on rates of fluency and accuracy. That is, the application may indicate and also track errors that the user made while reading one or more written words that were provided to the user to read aloud. The embodiments use artificial intelligence and speech recognition such that a user may independently and privately learn how to read, and the user may receivefeedback regarding the accuracy of their pronunciation from the reader application. Further, the present embodiments provide for tracking mechanisms to determine how many words are read per minute by the user, the amount of time the user read during that given session (i.e. a contiguous block of time a user read) or for a particular day, and to track fluency. Fluency, as used in this present embodiments, not only refers to the words read per minute by a user, but also the user’s ability to articulate text with phonetically natural sound, not stilted or robotic. Furthermore, fluency addresses the ability to read with the correct voice inflection (such as in the case of reading the end of sentence versus reading the end of a question) and the ability to read with pauses to indicate the end of a sentence or a comma in the text.

[0033] The present embodiments further provides a system, methods, and user interfaces that track and improve accuracy. Accuracy is the percentage of the number of words read that was spoken accurately by the user the first time versus the total number of words read by the user. For example, if a user currently read 300 words but only 4 of those 300 words were spoken accurately the first time, then the user has an accuracy of 4 / 300 words times 100% which is equal to 1.33%. The accuracy metrics may be mapped against standards from State and Federal agencies.

[0034] Other tracked metrics reported in the user interface may include, but are not limited to: number of minutes spent reading per day, number of words read per day, words per / minute read per day, challenge words, mastered words (words that were “challenge words” and have been mastered by the user), persistence (a measure of the user’s tenacity to keep trying problematic words without skipping them), days when the user completed the warm up sequence, days within the user completed the cool down sequence, streak (the number of consecutive days the user read), suggested advice (advice given to the reader to improve abilities), location in reading material in which to continue reading, the device on which the user engaged with the application, whether a user used headphones w / microphone, and the time of day when the user used the application.

[0035] Mastered words are selected from the one-thousand most common English words. Words from this list are considered mastered by the user when they have been read correctly at least three times. The user may see a list of unmastered words in a “Word Wiz” section of the reader application that allows the user to practice these challenge words.

[0036] Persistence is determined based on whether the user skips words while reading. When the user never skips a word (no matter how many wrong tries are made before successfully reading the word), persistence would indicate one-hundred percent.

[0037] Via the user interface of the reader application, the user has access to these personal metrics and may view them on a daily basis, over time, and relative to peer groups. Users in each user type may selectively share metrics with other people depending upon the user type. For example, a user interface for a teacher may show (and may allow the teacher to share) a listing of multiple students and their respective metrics, which may be used to make personalized recommendations to users regarding ways to increase reading fluency and accuracy. A user interface for a student or parent may only show and share their personal metrics, along with other factors including, hut is not limited to visual presentation of reading material.

[0038] Additionally, the present embodiments includes a system, methods, and user interfaces that may indicate to the user the next word or punctuation that is to be read. This provides the user with support for visual tracking of the next word or punctuation to be read aloud. The present embodiments also includes a system, methods, and user interfaces to indicate to the user if they skipped a word, read a word inaccurately, and / or failed to pause for a punctuation mark (such as a comma or period). In certain embodiments, the systems and methods indicate to the user when the user failed to use the correct inflection of voice for a punctuation mark (such as a question mark or an exclamation mark). With the present embodiments, detection of reading errors made by a user may also include tracking and storing of the words that were difficult for the user to speak.

[0039] The present embodiments provide ways for the user to practice “challenge words” and “trip words.” Challenge words are extracted from the book the user is reading. For example, based on the user’s reading position in the book and the user’s lexile score, when available, the reader application analyzes subsequent passages in the book to extract challenge words, which are words expected to be difficult for the user, excluding words the user has already encountered and correctly read. Advantageously, the user may pause reading from the book and practice these challenge words in advance to become more prepared for when the challenge words are encountered later in the book.

[0040] Trip words are words that the user has had problems reading and / or pronouncing. When the number of correct readings of a particular word is less than fifty percent of the total encounters of the word (e.g., incorrect reading attempts + skipped reading attempts + correct reading attempts), then that word is marked as a trip word for the user. Advantageously, the user may pause reading of the book and practice reading these trip words in preparation for future encounters of the trip word.

[0041] The same training format is used for practicing both the challenge words and the trip words. The words (challenge words or trip words) are presented in a list on a flashcard. The user is first invited to read the word on its own, then the user is invited to read the word within a context phrase from the book, and then the use is invited to read the word on its own again. When the user reads the word correctly all three times, the word is removed from the corresponding list of challenge words or trip words.

[0042] Advantageously, the present embodiments record both (a) when the user reads each word correctly and (b) when the user reads each word incorrectly. Thus, errors that the user made while reading aloud and words that the user read correctly aloud are both recorded in the user's own voice and may be replayed later for educational purposes and / or used as an encouragement mechanism. Occurrences of reading errors of the user may be stored in a database. Both portions of audio segments in the user's own voice that included the errors, as well as portions of audio segments of the user reading words aloud correctly may also be stored.

[0043] The present embodiments also include a system and user interfaces that permit users to initiate processes that offer reading assistance. In some embodiments, a process may be initiated to suggest reading material based on user-selectable preferences. Another user- selectable process may allow a user to review details of previous reading sessions, and select challenge words to review. Various other embodiments may support a process that offers one or more questions to the user during the reading process or thereafter.

[0044] These and other features of the system, methods, and user interfaces will be described in greater detail herein.

[0045] The following embodiments and examples are mostly directed towards a school environment where a teacher supervises a class of students. However, the embodiments described herein may operate in other environments, such as an individual environment where the student learns independently from other students, and in environments that are not child or school related (e.g., adult education, self-help, and so on). As noted throughout the following description, the reader application provides the user with reading assistance, encouragement, and motivation while allowing the user to select book texts from a large library.

[0046] FIG. 1 illustrates one example system 100 for monitoring reading performance and providing reading assistance, in embodiments. System 100 includes a reader server 105 and a computing device 110 and a reader application 125 that runs on computing device 110 (e.g., a client device such as one of a personal computer (PC), hand held computing system,telephone, mobile computing system, workstation, tablet, phablet, e-book reader, wearable, mobile phone, a smart phone, server, minicomputer, mainframe computer, or any other computing device). In certain embodiments, reader application 125 controls computing device 110 to provide a reading environment that both encourages user 120 to read and improves the reading ability and vocabulary of user 120.

[0047] Reader application 125 is for example an app and computing device 110 is a smartphone, where reader application 125 is downloaded to computing device 110. Reader application 125 controls computing device 110 to communicate with reader server 105 via a network 115 (e.g., one or more of a Local Area Network (LAN), a Wide Area Network (WAN), a wireless network, a cellular network, the Internet, etc.). However, reader application 125 may operate independently (e.g., without connectivity to reader server 105). For example, reader application 125 may cache data on computing device 110, resynchronizing (e.g., uploading modified data) with reader server 105 when communication between reader application 125 and reader server 105 becomes available. Continuous communication between reader application 125 and reader server 105 is not required, whereby reader application 125 communicates with reader server 105 at intervals to synchronize data. Where connectivity is available, reader application 125 may communicate continuously with reader server 105.

[0048] Reader server 105 is a computer based service that may include a database implemented as one or more digital storage media devices. Reader application 125, running on computing device 1 10, interacts with a user 120 to monitor reading performance and to provide reading assistance.

[0049] In one example of operation, computing device 110 authenticates with reader server 105 using credentials such as a username / password combination, or any other known authentication means. Reader application 125 may download books or other reading material from reader server 105 and controls computing device 110 to monitor reading of the reaming material by user 120 on computing device 110. Reader application 125 may send data regarding progress of user 120 to reader server 105 as described in further detail below.

[0050] By way of a non-limiting example, computing device 110 is a tablet or a smart phone running a client (or other software application). Additionally or alternatively, the computing device 110 may be a PC running a web browser. In certain embodiments, computing device 110 is one of a toy (such as a computerized toy, a phone toy, a mobile toy, or a plush toy), a game, a gaming device, and the like, that is capable of running reader application 125. For example, where computing device 1 10 is a smart device, computingdevice 110 includes a web browser (or similar software application) for communicating with reader server 105, whereby functionality of reader application 125 is implemented by a web page generated by reader server 105. In one example, user 120 may download and install an applet and / or a browser plug-in to control computing device 110 to implement certain functionality of reader application 125. However, the use of reader application 125 is preferred since reader application 125 may operable independently of reader server 105 and may communicate with reader server 105 at intervals to exchange data. Reader application 125 monitors reading performance of user 120 and may provide reading assistance to the user as needed or as requested.

[0051] FIG. 2 is a block diagram illustrating reader server 105 of reader system 100 of FIG. 1 in further example detail, in embodiments. Reader server 105 includes a book library 250 that stores a plurality of books 252 (e.g., digital text, e-books, etc.) that may be used by reader application 125. Book 252 may represent any text that is in digital character form (e.g., ascii / Unicode characters), such as an e-book, where the one or more written words represents a portion of book 252, such as a word, a sentence, or a chapter. Book library 250 is for example a database that may be part of reader server 105 or may be external to reader server 105. Reader server 105 may invoke a book loader 256 to import an e-book 258 into book library 250. The term book is used in the following examples for clarity, but book 252 may refer to any digital text suitable for user 120 to read. For example, book 252 may be a page or a paragraph of text provided by a teacher for user 120 to read. Book loader 256 may process e-book 258 (e.g., before, during, or after the import) to determine book data 254 that is associated with book 252 and that defines content of book 252. Book loader 256 may generate a word graph defining one or more of whole words, root words, word parts, and sounds user 120 will encounter when reading book 252, and may also define one or both of a reading level and a difficulty level of book 252. Book loader 256 may process e-book 258 to categorize words, such as by frequency of usage. For example, a first category may include one thousand words that occur at the highest frequency. The remaining words (e.g., words not in the highest frequency category) may be categorized based on frequency within e-book 258 and difficulty (e.g., length, number of syllables, trip words from a known list, determined using an API to a third party tool, etc.). Although difficulty of a word may be represented in different forms, sometimes the complexity of a word is due to the word being multisyllabic, such as the word “phenomenal.” Sometimes the complexity of a word is because it does not follow pronunciation conventions, referred to as sight words, such as the word “gnome.”

[0052] Book loader 256 may also determine a reading level (e.g., a grade level or other type of categorization) for e-book 258. For example, book loader 256 may retrieve a reading level from an external database or determine the reading level based on the categorization of words in the book and one or more of local standards, state standards, and national standards. User 120 selects and downloads at least one book 252 from reader server 105 based on personal choice, however, one or both of reader server 105 and reader application 125 may advise user 120 when a selected book 252 is too difficult or too easy for user 120. In certain embodiments, when user 120 selects a next book 252 to read, book loader 256 determines and presents certain statistics on that particular book. For example, when user 120 selects book 252 to read next, book loader 256 determines a number of classmates of user 120 who have read part and / or all of that book, and then presents this information to user 120. Advantageously, this information motivates user 120 to read more and / or to interact more with other classmates.

[0053] Reader server 105 may also include a definition dictionary 260 that stores definitions and pronunciations of words used in books 252. In certain embodiments, definition dictionary 260 is provided by a third party service that is external to reader server 105.

[0054] Reader server 105 also includes a user database 270 that stores a user ID 272 in association with corresponding user data 274. User ID 272 uniquely identifies user 120 and user data 274 may define reading progress of user 120. Reader server 105 may also include a teacher interface 280 that generates a teacher dashboard 282 for display on a computing device 290 to a teacher 292. In certain embodiments, teacher dashboard 282 is implemented as at least one web page that is accessible by teacher 292 via a browser running on computing device 290. Teacher dashboard 282 may include one or more of user management 284, user metrics 286, and class data 288. Where teacher 292 teaches a class of several students, class data 288 may define a status and progress of each student within the class relative to other students within the class. For example, class data 288 may rank users based on an amount of time read per week, a reading level achieved, an amount of reading progress made, and so on. User metrics 286 may include individual reading statistics and achievements, thereby allowing teacher 292 to evaluate an individual’s reading performance and thereby learn when intervention is needed. In certain embodiments, user metrics 286 indicate proficiency of user 120 based on user data 274 (see also proficiency data 470 of FIG. 4A) and reading goals of user 120 (see user profile 324 of FIG. 3).

[0055] Reader server 105 may also include a competition manager 262 that implements a reading competition (e.g., a read-a-thon) between multiple entities (e.g., two or more individuals, two or more groups of users within a class, two or more classes, two or more schools, etc.). Accordingly, competition manager 262 generates a reader leader board 264 that shows an ordered table of competitors. Competition manager 262 and reader leader board 264 are described in further detail below with reference to FIG. 15.

[0056] FIG. 3 is a block diagram illustrating reader application 125 of reader system 100 of FIG. 1 in further example detail, in embodiments. Computing device 110 includes at least one digital processor 302, memory 304, a display 306 (e.g., a touch screen / user interface), a speaker 308, a microphone 310, and a camera 312 (e.g., a forward facing camera). Reader application 125 controls display 306 to present at least part of book 252 (e.g., a page text 356) to user 120 and controls microphone 310 to capture sounds of user 120 when reading aloud. Reader application 125 control camera 312 to capture an image of a user 120, processing that image to determine that user 120 is focusing on page text 356.

[0057] Reader application 125 may include additional modules, engines, or functional components without departing from the scope hereof. Reader application 125 may also be implemented as any of an application- specific integrated circuit (“ASIC”), an electronic circuit, a processor (shared, dedicated, or group) that executes one or more software or firmware programs, a combinational logic circuit, and / or other suitable components that provide the described functionality. In other embodiments, individual modules and / or components of reader application 125 may be implemented separately from computing device 110, such as on a web server or other device.

[0058] User data 274 includes user ID 272, at least part of book 252 being read by user 120, a bookmark 326 indicating a current reading position within book 252, a user word graph 320, and one session metric 322 for each reading session performed by user 120. Each time user 120 reads using reader application 125, reader application 125 creates a new session metric 322. User word graph 320 is generated by reader application 125 and lists sounds, word parts, root words, and whole words that the use has encountered (in all books 252 read by user 120). User word graph 320 may conform to one or more of local standards, state standards, and national standards for tracking reading proficiency of user 120. Particularly, user word graph 320 stores more information than just encountered words, and previously done. User word graph 320 is described in more detail below with reference to FIG. 4A.

[0059] User data 274 also includes a user profile 324 that defines certain characteristics of user 120, and may include a reading goal 325 that indicates how often and how long user 120 is expected to read. For example, user profile 324 may indicate that user 120 is expected to read for ten minutes each day. User profile 324 may also define one or more of the user's age, gender, school grade, etc.

[0060] Reader application 125 includes a reading monitor 330, a recommender 332, a tracker 334, a motivator 336, and a vocabulary virtuoso 338. Reading monitor 330 controls computing device 110 to display page text 356 (e.g., a part of book 252) on display 306 that includes a current reading position of user 120 within book 252. For example, when user 120 opens reader application 125 to start reading, reader application 125 retrieves the current reading position from bookmark 326, selects page text 356 accordingly from book 252, and controls computing device 1 10 to display page text 356 and an indication of the current reading position to user 120. Reading monitor 330 processes audio data from microphone 310 that includes user 120 reading aloud from page text 356 to detect one or more sounds, word parts, and whole words, and matches them to words of page text 356. As user 120 reads, reading monitor 330 controls computing device 110 to advance indicators on page text 356 that represents the current reading position of user 120. Reading monitor 330 may also control computing device 110 to indicate when a word is read correctly, and / or when a word is read incorrectly or skipped. In certain embodiments, reading monitor 330 controls computing device 110 to prevent further reading progress (e.g., through display of indicators and page text 356) by user 120, causing user 120 to reattempt reading of a misread or skipped word. Reading monitor 330 updates a current session metric 322 and bookmark 326 as user 120 progresses through page text 356 and updates page text 356 as needed. Reading monitor 330 uses a clock of computing device 110 to track the time user 120 spends reading for the current session metric 322.

[0061] In certain embodiments, reader application 125 sends audio (e.g., divided into audio segments) to reader server 105 where one or more algorithms of reader server 105 process the audio data to further categorize oral reading of user 120.

[0062] Recommender 332 monitors the reading by user 120 (e.g., monitors the sounds, word parts, and whole words that are read by user 120) and detects one or more patterns in the hesitations, errors and / or skipped words. Where a detected pattern indicates a particular reading deficiency or weakness, recommender 332 may suggest a treatment to help user 120 overcome the deficiency. For example, where user 120 repeatedly stumbles when reading words ending in “ion,” recommender 332 may invoke tools 344 to generate a set offlashcards that contain words ending in “ion” already encountered by user 120 but not yet mastered. In certain embodiments, recommender 332 controls computing device 110 to intervene during reading by user 120 and provide additional teaching and practice of words not yet mastered by user 120.

[0063] Recommender 332 may also be invoked when user 120 starts searching for a next book 252 to read. Recommender 332 may suggest one or more of books 252 with content that would challenge user 120, but that would not be too difficult for user 120. For example, based on one or more of session metrics 322, book 252, and user word graph 320, recommender 332 may recommend a new book that includes many of the most challenging words of current book 252, and that also includes a several new challenge words that are not yet encountered by user 120, but that are not too difficult.

[0064] Tracker 334 updates user word graph 320 (and a word graph within current session metric 322) to reflect words encountered, words mastered, words encountered but not yet mastered, sounds used correctly, sounds used incorrectly, and so on. Tracker 334 also tracks the number of words read in each session, and a duration of the session.

[0065] Motivator 336 generates motivational prompts and motivational notifications to help motivate user 120 to read more. For example, where user 120 has not read sufficiently to meet reading goal 325(e.g., 10 minutes per day) defined within user profile 324, motivator 336 controls computing device 110 to display a notification to user 120 reminding user 120 to read and listing at least one benefit of reading. In another example, as user 120 reads in a current session, motivator 336 detects that user 120 is reading slower than usual and generates a motivational notification to entice user 120 to keep reading and finish strongly. In another example, at the end of the reading session, motivator 336 notifies user 120 of the number of words read. Advantageously, user 120 wants to maintain and / or increase the number of words being read in each session, and is motivated by this feedback. This is similar to learning how many steps have been made by reviewing a step counter when exercising.

[0066] Reading monitor 330 also determines when user 120 skips reading page text 356 (e.g., when user 120 swipes left to change the page before sufficient time has elapsed for the user to read the words). For example, reading monitor 330 and / or tracker 334 may determine that user 120 is skipping the current page when insufficient time on the page has elapsed give the user’s reading rate (e.g., word / minute 474 of proficiency data 470). Reading monitor 330 and / or tracker 334 may track skipped pages within session metrics 322 for example. Further, reader application 125 may display (e.g., to user 120 via user dashboard350 and / or to teacher 292 via teacher dashboard 282) reading metrics of user 120 to indicate one or more of pages read, pages skipped, percentage of book read, and other similar metrics. In certain embodiments, reading monitor 330 and / or tracker 334 may control computing device 110 to prevent user 120 from swiping left to change the page when reading by user 120 has not been detected or when insufficient time has passed for user 120 to read the text on the page.

[0067] FIG. 4 A is a schematic diagram illustrating user data 274 stored in reader server 105 of FIG. 2 in further example detail, in embodiments. FIGs. 4B, 4C, and 4D show example GUIs 352(1), 352(2), and 352(3), respectively, for motivation and reading streaks, in embodiments. For example, GUI 352(1) graphically illustrates minutes read by user 120 as compared to a goal of user 120, thereby providing motivation for beating the goal by reading more. GUI 352(2) shows that user 120 has a reading streak of 17 days, thereby providing motivation for user 120 to read each day to prevent loss of the streak. GUI 352(3) illustrates how user 120 may use a streak restore (accumulated through learning and reading as described below) to prevent loss of a reading streak.

[0068] Session metrics 322 may include a start time 402 (e.g., set to a current time of a real-time clock of computing device 110 when user 120 starts a new reading session), a word counter 404 (e.g., a number of words read in that session), a reading duration 406 (e.g., time user 120 has spent attentively reading based on the real-time clock), and a session word graph 408. Session word graph 408 is similar to user word graph 320 but is updated only for the current reading session, whereas session word graph 408 is maintains for all reading sessions (e.g., across multiple books 252). For example, reading monitor 330 may monitor images from camera 312 to verify that user 120 is focused on display 306 and may not increment reading duration 406 while user 120 is facing away from display 306. Accordingly, reading duration 406 provides an accurate measurement of the reading time of user 120.

[0069] In certain embodiments, reading duration 406 operates as a clock that is started and stopped by reading monitor 330 based on whether the user’s attention is on book 252 (e.g., viewing display 306 of computing device 110) by processing images or video of user 120 captured by camera 312. For example, reading monitor 330 tracks one or more of eye movement, head position, hand position, and body position, to determine the user’s attention. Accordingly, reading monitor 330 controls starting and stopping of reading duration 406 (e.g., the “timer” and “progress recording” of user 120) when their attention is on book 252 and automatically pause reading duration 406 (e.g., the timer and progresstracking) when the user looks away from display 306. Advantageously, reading monitor 330 increases accuracy of proficiency data 470, particularly where user 120 reads silently and reading monitor 330 cannot track reading progress based on spoken words. For example, where user 120 reads silently, reading monitor 330 tracks reading progress through book 252 based on the time user 120 is focused on each page, and the number of words on each page. In certain embodiments, reading monitor 330 implements a swipe lock (e.g., controls computing device 110 to prevent user 120 from swiping left to turn the page of book 252) until user 120 has spent an adequate amount of time focused on the page (e.g., on display 306) to have read the word. For example, reading monitor 330 determines sufficient time based on words / minute 474 and the number of words on the page.

[0070] Reading monitor 330 may determine when user 120 is frustrated by detecting stress indicators in their aloud reading. When stress is detected, or when the user says “time out” to pause the current reading session, reading monitor 330 stops accumulating time in reading duration 406. Reading monitor 330 may then invoke one or more tools 344 that control computing device 110 to interact with user 120 to perform one or more of a guided breathing exercise, a stretching exercise, or some other mechanism to help the user relax. Reading monitor 330 may control computing device 110 to return to reading aloud after detecting that user 120 is less stressed. Such breaks are suggested by reader application 125 to improve reading results and accuracy of aloud reading by user 120. In certain embodiments, tools 344 cause the guided breathing exercise to include a physical component, such as asking user 120 to tap his / her finger to synchronize inhalation and exhalation. By way of a non-limiting example, tools 344 may indicate that user 120 should inhale while tapping his / her finger to a count of four, hold the breath while tapping the finger for a count of four, and exhale while tapping the finger to a count of six. Multi-modal or multi-sensory techniques may be particularly effective in teaching people with learning challenges like dyslexia. Reader application 125 may track whether the guided breathing or stretching exercises had any impact on the user's reading accuracy or fluency.

[0071] Reader application 125 tracks a word count, a duration, and the sounds, word parts, root words, and whole words encountered by user 120. User word graph 320 is divided into encountered 412 and not yet encountered 414 sections. Encountered 412 includes a list of encountered sounds 422 (e.g., the 44 phonemes in English) such as “a”, “ee”, “ow”, etc. For each sound 422, tracker 334 stores at least three counters: a correct use count 424 indicative of a number of times user 120 has correctly made the sound, an incorrect use count426 indicative of a number of times user 120 has incorrectly made the sound, and a skippedcount 428 indicative of a number of times user 120 has skipped making the sound. Encountered 412 also includes a list of word parts 432 (e.g., syllables) such as “un”, “bi”, “ing”, etc. For each word part 432, tracker 334 stores at least three counters: a correct use count 434 indicative of a number of times user 120 has correctly read the word part, an incorrect use count 436 indicative of a number of times user 120 has incorrectly read the word part, and a skipped count 438 indicative of a number of times user 120 has skipped the word part. Encountered 412 also includes a list of root words 442 (e.g., stem words) such as “cycle”, “ped”, “read”, etc. For each root word 442, tracker 334 stores at least three counters: : a correct use count 444 indicative of a number of times user 120 has correctly read the root word, an incorrect use count 446 indicative of a number of times user 120 has incorrectly read the root word, and a skipped count 448 indicative of a number of times user 120 has skipped the root word. Encountered 412 also includes a list of whole words 452 (e.g., the word as written in book 252) such as “bicycling”, “pedestal”, “reading”, etc. For each whole word 452, tracker 334 stores at least three counters: : a correct use count 454 indicative of a number of times user 120 has correctly read the whole word, an incorrect use count 456 indicative of a number of times user 120 has incorrectly read the whole word, and a skipped count 458 indicative of a number of times user 120 has skipped the whole word.

[0072] Not yet encountered 414 includes a list of sounds 462, word parts 464, root words 466, and whole words 468 that appear in book 252 but have not yet been encountered by user 120. Reader application 125 updates not yet encountered 414 when user 120 selects a new book 252 to read, adding sounds, word parts, root words, and whole words from the new book to not yet encountered 414 when they are not found in encountered 412. Further, tracker 334 removes sounds, word parts, root words, and whole words from not yet encountered 414 as user 120 encounter them while reading book 252.

[0073] Accordingly, user word graph 320 tracks proficiency of user 120 in mastering all encountered sounds, word parts, root words, and whole words. In certain embodiments, one or both of reader server 105 and reader application 125 categorize elements of user word graph 320 relative to local / state / national expectations by grade level.

[0074] Reader application 125 invokes tracker 334 to track certain metrics associated with the reading of the written words by user 120, including but not limited to, metrics regarding fluency, accuracy, persistence, and automaticity, as well as reading benchmarks. Tracker 334 updates user data 274 including user word graph 320, session metrics 322, progress through book 252 previously read, and other past readings, and / or reading sessions.Accordingly, user data 274 may also include proficiency data 470 that is updated by tracker334 based on session metrics 322 and user word graph 320. Tracker 334 may track total minutes read 472 by user 120 by summing duration 406 of each session metric 322, and may also determine a words / minute 474 as the reading rate of user 120 from one or more most recent session metrics 322. Tracker 334 may also determine an accuracy 476, which indicates a percentage of words read being read correctly (e.g., where 100% indicates the highest accuracy). For example, where tracker 334 determines there is a match between the user's spoken words and the one or more written words, then tracker 334 measures the accuracy of the reading, and store the accuracy measurement in user data 274. In certain embodiments, reading monitor 330 also stores a portion of the captured audio that includes the user's spoken words as matched with the one or more written words.

[0075] Where reading monitor 330 determines that a spoken word does not match the written word, then reading monitor 330 indicates the mismatch on GUI 352 at the written word within page text 356. Similarly, where reading monitor 330 determines that user 120 has mispronounced or read the written word incorrectly, reading monitor 330 controls computing device 110 to highlight the written word, underline the written word and / or change the color of the written word via G I 352. Tracker 334 records the successfully read words, the mismatched words, and the skipped words within user word graph 320 and / or session metrics 322. In certain embodiments, reading monitor 330 and / or tracker 334 records the user’s experience. For example, user 120 may configure reader application 125 to record the reading experience (e.g., audio of the user’s voice and / or video of display 306). For example, these recording may be useful for teachers wishing to watch and analyze problematic sessions of user 120.

[0076] Tracker 334 may also determine a persistence 478, which is a percentage that indicates how persistent user 120 is at reading correctly. Tracker 334 may also determine a reading proficiency 480 of user 120 from session metrics 322 and / or user word graph 320. Reading proficiency 480 is a percentage value determined by normalizing session metrics 322 and / or user word graph 320 against at least one of local standards, state standards, and national standards. Accordingly, proficiency 480 indicates a reading ability of user 120 against expected reading proficiency. For example, where user 120 is at 6thgrade, standards for 6thgrade reading are used to normalize statistics from session metrics 322 and / or user word graph 320. Accordingly, in this example, reading proficiency 480 indicates a reading ability of user 120 against an expected reading ability for a child in 6th grade.

[0077] In certain embodiments, reading monitor 330 implements speech recognition that is enhanced for use with reader application 125 to track reading accuracy of the user.Conventional speech recognition is configured to recognize any number of possible words spoken by a user since it does not know what the user's next spoken word will be. Accordingly, the processing time of conventional speech recognition incurs a delay in recognizing and matching the user's spoken word with possible word. On the other hand, reading monitor 330 is configured to recognize a limited vocabulary of words based on the expected words from page text 356. Advantageously, speech recognition by reading monitor 330 occurs more quickly than conventional speech recognition, enabling read-time (e.g., substantially immediate) responses to words spoken by user 120. Tn certain embodiments, the number of expected words is between ten and fifty and is selected based on a reading speed of user 120 and / or a processing speed of digital processor 302. In one operational example, ten words are selected from page text 356 based on the user’s current position, and reading monitor 330 operates to recognize and compare the aloud reading of user 120 to only those ten words. As user 120 progresses through page text 356, different words are selected as the expected vocabulary of reading monitor 330. This reduced vocabulary also allows reading monitor 330 and / or tracker 334 to provide a better analysis of the aloud reading, thereby improving accuracy and consistency of the analyzed performance of user 120.

[0078] In one example of operation, reading monitor 330 parses book 252 chapter by chapter (e.g., one chapter at a time) based on bookmark 326. For example, reading monitor 330 first retrieves chapter text 327 for the chapter of book 252 that bookmark 326 references. Reading monitor 330 then invokes a parser 340 that processes chapter text 327 to generate a decoding graph 329 that is used by reading monitor 330. Parser 340 generates a sentence array 328 from chapter text 327. Parser 340 analyzes each word of each sentence within sentence array 328 to generate decoding graph 329. For example, parser 340 first categorizes each word as one of an English word, a numeric, and a roman numeral. For English words (e.g., found in the speech engine’s lexicon), parser 340 generates a corresponding phonetic spelling of the word within decoding graph 329. Where the word is part of a homophones set, parser 340 generates decoding graph 329 with corresponding variations. Examples of homophones set include [acclamation, acclimation], [altar, alter], [leased, least], and so on. Where the word is a numeric, parser 340 generates decoding graph 329 with a spelled out version of the number. Where the number is possibly a year, parser 340 generates decoding graph 329 with a spelled out version of the year. Where the number is possibly an hour, parser 340 generates decoding graph 329 with a spelled out version of the hour. Where the word is a roman numeral, parser 340 generate decoding graph 329 with a spelled out corresponding numeric version of the roman numeral (e.g., V becomes 5, X becomes 10,XXII becomes 22, and so on. For each alternative pronunciation of the word, parser 340 generates a new sentence within decoding graph 329 that slows for recognition of the variation by reading monitor 330.

[0079] In certain embodiments, reading monitor 330 also recognizes similar phonemes to those required by the limited vocabulary, thereby improving accuracy measurements of aloud reading by user 120. In certain embodiments, reading monitor 330 adjusts its limited vocabulary when it detects the user is pausing. For example, when user 120 pauses while reading aloud from page text 356, reading monitor 330 may build the upcoming vocabulary based on upcoming words of page text 356 during those pauses.

[0080] Reading monitor 330 may operate based on one or both of syllables and phonemes. Reading monitor 330 may filter detected syllables and / or phonemes as spoken by user 120, pass these syllables or phonemes as sounds to the speech recognition module, and by utilizing speech recognition, reassemble the syllables to detect the word that is being said by the user. In particular, reading monitor 330 determines when the collective number of syllables matches a given word. Conventional speech recognition cannot perform this reassembly of syllables or phonemes and recognition. In one example of operation, as user 120 reads aloud the word “phenomenon,” syllable by syllable, reading monitor 330 reassembles the syllables to detect the word “phenomenon.” In certain embodiments, reading monitor 330 may synthesize phonemes and / or syllables (e.g., parts of a whole word) spoken by user 120, such that the phonemes and / or syllables are transformed into a whole word that is then compared to the one or more written words of page text 356. Accordingly, reading monitor 330 and / or tracker 334 determines accuracy of words spoken by user 120.Reading Streak

[0081] Tracker 334 may also determine a reading streak length 482 of user 120 based on recorded session metrics 322 and reading goal 325 of user profile 324. Tracker 334 increments reading streak length 482 when user 120 meets their reading goal 325 each day. When a day is skipped, or the reading goal 325 is not achieved, tracker 334 resets reading streak length 482. However, to motivate user 120 to read more, user 120 may earn streak restores 484 based on reading achievements. For example, where a current session exceeds their reading goal 325 by more than a predefined amount (e.g., 50% longer, 300 additional words, etc.), tracker 334 adds one to streak restores 484. In another example, vocabulary virtuoso 338 adds one to streak restores 484 when user 120 completes a training for challengewords. For example, the challenge words may represent words misread by user 120, or may represent upcoming challenge words that are not yest mastered by user 120.

[0082] Accumulated streak restores 484 may be used to continue a reading streak when they would otherwise lose the reading streak by not meeting their reading goal 325 on a particular day. For example, assuming streak restores 484 is not zero when user 120 skips reading, user 120 may use an accumulated streak restore 484 to continue their current reading streak. Accordingly, tracker 334 does not reset reading streak length 482, but subtracts one from streak restores 484. Advantageously, user 120 is motivated to read consistently to increase their reading streak length 482 and is further motivated to read more to accumulate streak restores 484 to prevent loss of their current reading streak when they are unable to meet their reading goal 325.

[0083] Reader application 125 provides a further motivation through use of by word counter 404, recorded for each reading session, since it allows a word count goal (e.g., per day, per week, etc.) to be set and provides motivation for user 120 to meet this goal. For example, reader application 125 may generate user dashboard 350 to show word counter 404 for each session metric 322 such that user 120 sees how many words they are reading in each session. Further, teacher interface 280 may process and / or display word counter 404 from each session metric 322 for each user on dashboard 282, such that teacher 292 sees the effort being made by each user.

[0084] Reader application 125 generates one or more of a user dashboard 350 and a graphical user interface 352 (GUI 352) that forms a user interface to allow user 120 to interact with reader application 125. For example, reading monitor 330 may generate at least one GUI 352 to display one or more written words of book 252 to for the user 120 to read aloud, where GUI 352 include visual prompts and options to help user 120. In certain embodiments, one or more GUIs 352 presents the one or more written words of book 252 with the appearance of one or more pages of a physical book as shown in FIG. 5.

[0085] Reading monitor 330 includes speech recognition algorithms that process captured audio (e.g., from microphone 310) to determine whether the user's spoken word or words match the one or more written words displayed within GUI 352.

[0086] Reader application 125 may also include tools 344, such as one or more of a warm-up tool, a ‘stop the clock’ guided breathing tool and stretching exercise tool implemented within one or more GUIs 352. Tools 344 may be invoked by one or more of reading monitor 330, tracker 334, recommender 332, motivator 336, and vocabulary virtuoso338 where needed and / or when selected by user 120 via one or both of user dashboard 350 and GUI 352.

[0087] Recommender 332 may also base recommendations on use of tools 344 by user 120, the time of day at which user 120 read, the device on which user 120 read, and use of an additional device such as headphones with a microphone by user 120. Although reading monitor 330 may implement speech recognition, alternatively, speech recognition may be performed by local servers, remote servers, or in a hybrid manner using both local and remote servers.

[0088] User profile 324 may also store configuration parameters that define the view or operation of one or more GUIs 352 and / or user dashboard 350. For example user 120 may select fonts, font sizes and contrasts for written words to be displayed within GUI 352. For example, user 120 may choose to read text using different contrasts, such as navy on white or black on white, for visual aid. For some people, reading with a particular font, font size and / or particular contrast increases the accuracy of their reading. Accordingly, where reader server 105 determines that users with similar profiles have improved accuracy in their reading that is attributable to a given font, font size and / or contrast, recommender 332 may recommend that user 120 try using the same font, font size and / or contrast. In one example, recommender 332 generates a recommendation that says “Based on your profile and profiles of other users with similar profiles, you may wish to use Courier point size fifteen font. Others with similar profiles to you had improved accuracy in their reading using this font.”

[0089] In another example, recommender 332 recommends that user 120 select a font with a heavier baseline, to help visually track the one or more written words. The heavier baseline of the font along the bottom of the letters may help users, such as dyslexic users, to track and read more accurately. A font with a heavier baseline appears to a user as if a regular font is applied on the top portion of a particular letter, while a bold font is applied on the bottom portion of the same letter.

[0090] Reader application 125 may also allow user 120 to activate a reading wave feature that, when activated, shows an emphasis on the words within GUI 352 that the user should be placing more emphasis when reading. For example, the reading wave feature magnifies or zoom in on written words that user 120 should emphasize while reading aloud.

[0091] In one example of operation, recommender 332 interacts with user 120 using one or more GUIs 352, to receive a selection of a topic (e.g., one of animals, bugs, pets, art, creativity, music, fiction, non-fiction, poetry, hobbies, sports, outdoors, science fiction, fantasy, real life, science and technology, mystery and suspense, reference, etc.).Recommender 332 then selects one or more of books 252 from book library 250 that user 120 has not yet read, that falls under the selected topic, and that presents user 120 with the appropriate challenge. However, reader application 125 does not prevent user 120 from selecting any book to read from book library 250. However, where user 120 selects a next book 252 that has a reading level that is beyond the current reading level of user 120 (e.g., where the selected book 252 has many new words that user 120 has not read before and that would be difficult for user 120), recommender 332 notifies one or more of user 120, teacher 292, and a parent or guardian of user 120, that the selected book 252 may be beyond the reading ability of user 120. Accordingly, the user may select a less difficult book, or one of the teacher, the parent or the guardian may intervene and advise user 120 to select a less difficult book. User 120 has a universal choice to select any text they wish to read, as is not designed to discourage a user from reading in any way by reader system 100.

[0092] In another example, recommender 332 recommends a new book 252 based on a recommendation by teacher 292. In yet another example, recommender 332 assigns a personalized score to one or more books 252, where the score is based on a current reading level of user 120 (e.g., based on past reading of user 120), how difficult the next book 252 is perceived to be for user 120 (e.g., based on a percentage of how many words in the next book 252 user 120 has already read correctly, and / or based at past exposure of user 120 to certain written words). In another example, recommender 332 determines the score based on how complicated the words are to pronounce in the next book 252 and / or a number of syllables in the words of the next book 252. In certain embodiments, recommender 332 also reviews how many words in the next book 252 are irregular or sight words that will be difficult for user 120 to read. In another example, recommender 332 uses a third party API to determine a difficulty of one or more challenge words to determine a difficulty level of the next book 252. Recommender 332 may also make recommendations to the user on which books 252 user 120 should consider selecting, based on already read books 252. For example, recommender 332 may select a next book 252 in a series. However, recommender 332 may include an algorithm that selects a next book 252 to advance reading skills of user 120.

[0093] When user 120 selects a new book 252 to read, recommender 332 and / or motivator 336 may indicate a number of classmates of user 120 that have read part or all of that book.

[0094] FIG. 5 shows GUI 352 of FIG. 3 displaying example page text 356 from book 252 for user 120 to read in an Oral (read out loud) mode, in embodiments. In this example, reading monitor 330 controls computing device 110 to display a prompt 502 in GUI 352indicating a next word for user 120 to read, and where user 120 is hesitant in reading, reading monitor 330 controls GUI 352 to display a reading aid 504, which may allow user 120 to skip the word or to play audio of an example pronunciation of the next word. When operating in a silent mode, reader application 125 does not display prompt 502 or reading aid 504.

[0095] User 120 may move to other pages of book 252, by “swiping” GUI 352 for example. In certain embodiments, reader application 125 controls computing device 110 to implement a “swipe lock” feature that limits the rate of page advancement by user 120 to help ensure user 120 reads all pages of book 252. For example, the swipe lock feature may use the clock of computing device 110 and / or a counter to limit page advancement by user 120 when reader application 125 determines that user 120 has not read page text 356.Vocabulary Virtuoso

[0096] Vocabulary virtuoso 338 is a software algorithm that is invoked by reader application 125 (e.g., by one or more of reading monitor 330, recommender 332, tracker 334, and motivator 336) to provide tools 344 to improve the reading of user 120. In certain embodiments, as user opens reader application 125 at the start of a next session, reader application 125 invokes vocabulary virtuoso 338. Vocabulary virtuoso 338 calculates, based on a reading speed and the session duration goal, an expected number of words the user is likely to read in the next session. Vocabulary virtuoso 338 processes the next session words to identify a set (e.g., five words) of the most difficult challenge words that user 120 has not yet encountered (e.g., not in user word graph 320). Challenge words are words that are expected to be difficult for user 120. For example, the difficulty of each word may be predefined, may be based on the number of sounds (e.g., English has 44 phonemes), a number of syllables, and / or may be based on the complexity of the sounds (e.g., words with more syllables are more challenging).

[0097] Having identified the set of challenge words occurring in the next session words, vocabulary virtuoso 338 generates a training session for user 120 to learn these challenge words prior to encountering them in the next session words. For example, vocabulary virtuoso 338 may generate flashcards with the challenge words on them and invoke tools 344 that control computing device 110 to present these flashcards interactively to user 120. Vocabulary virtuoso 338 also indicates the benefits of learning the words before they are encountered by indicating how many occurrences of each challenge word are in the rest of book 252. Advantageously, user 120 is made aware of the benefit of learning the words since they appear later in book 252. The training session may also identify root wordsfound in the challenge words, and provides training (e.g., flashcards) to allow user 120 to also learn other words derived from the root word, including their meaning. Further, vocabulary virtuoso 338 provides each of the challenge words and other words derived from the root words in context of book 252, such as where the root word derivatives are found in the remainder of book 252, such that user 120 may practice (e.g., reading aloud while being monitored by reading monitor 330) and master these words prior to encountering them in book 252. Advantageously, user 120 is made aware of the benefit of learning the root word and its meaning to help understand the meaning of its derivative words, and is motivated to follow the training to learn to read these words knowing that they occur later in book 252. This advantage is not provided by other tools.

[0098] FIG. 6 is a flowchart illustrating one example method 600 for teaching imminently encounterable new words to user 120, in embodiments. FIG. 7 is a block diagram illustrating example data used by a word analyzer 702 of vocabulary virtuoso 338 of FIG. 3, in embodiments. FIGs. 6 and 7 are best viewed together with the following description. In this example, when user 120 opens reader application 125 in computing device 110, reader application 125 invokes vocabulary virtuoso 338 to prompt user 120 to learn a set of challenge words before they are encountered in a next reading session. Advantageously, user 120 sees this warm up session as an advantage to reading book 252 rather than a disconnected learning exercise.

[0099] In block 602, method 600 determines a number of words expected in a next reading session. In one example of block 602, vocabulary virtuoso 338 multiplies reading goal 325 (e.g., in minutes of reading) by words / minute 474 to determine an expected word count 710 indicating a number of words the user is expected to read in this next reading session. In another example of block 602, vocabulary virtuoso 338 retrieves recent session metrics 322, calculates a daily average reading time from reading durations 406, and uses words / minute 474 to calculate expected word count 710. In another example of block 602, vocabulary virtuoso 338 determines a count of words in a next chapter of book 252.

[0100] In block 604, method 600 retrieves next session words from the book. In one example of block 604, vocabulary virtuoso 338 uses bookmark 326 and expected word count 710 to retrieve next session words 712 (e.g., a contiguous block of text) from book 252. In block 606, method 600 generates a list of unmastered challenge words in next session words. In one example of block 606, vocabulary virtuoso 338 generates unmastered challenge words 714 by processing next session words 712 against user word graph 320 to identify challengewords within next session words 712 that are not indicated within user word graph 320 as being mastered by user 120.

[0101] In block 608, method 600 selects a set of most difficult challenge words. In one example of block 608, vocabulary virtuoso 338 uses a third party API to determine a difficulty of each of unmastered challenge words 714 to determine a set of most difficult challenge words 716 that are ones of unmastered challenge words 714 determined to be the most difficult. Particularly, most difficult challenge words 716 are words within the next portion of book 252 about to be read by user 120 that have not yet been encountered or have been encountered but not mastered. Words that have been mastered by user 120, such as in previous reading sessions or other books, and not included in most difficult challenge words 716.

[0102] In block 610, method 600 determines a number of times each most difficult challenge word occurs in the rest of the book. In one example of block 610, vocabulary virtuoso 338 processes the remainder (e.g., from bookmark 326) of book 252 against most difficult challenge words 716 to generate most difficult challenge word counts 718 based on the occurrence of each of most difficult challenge words 716 in the remaining text of book 252.

[0103] In block 612, method 600 determines a definition of each of the most difficult challenge words. In one example of block 612, vocabulary virtuoso 338 generates most difficult challenge word definitions 720 by retrieving, from definition dictionary 260 for example, definition entries that correspond to each of most difficult challenge words 716.

[0104] In block 614, method 600 determines a root word of each of the most difficult challenge words. In one example of block 614, vocabulary virtuoso 338 sends each most difficult challenge words 716 to a third party API and receives root words 722 in response. For example, where one of most difficult challenge words 716 is “pedestal,” vocabulary virtuoso 338 determines the root word as “ped.”

[0105] In block 616, method 600 determines root word variants for each root word. In one example of block 616, vocabulary virtuoso 338 uses a third party API to determine root word variants 724 of root words 722. For example, for the root word “ped” vocabulary virtuoso 338 determines root word variants 724 as including “pedal” and “pedestrian.”

[0106] In block 618, method 600 determines a number of times each of the root word variants occurs in the rest of the book. In one example of block 616, vocabulary virtuoso 338 processes root word variants 724 against the remaining text of book 252 to determine root word variant count 726 defining a count of the number of times each of root word variant 724occurs in the remaining text of book 252. For example, vocabulary virtuoso 338 counts occurrences of each of “pedal” and “pedestrian” in the remaining text of book 252.

[0107] In block 620, method 600 determines a definition of each of the root word variants and the root word. In one example of block 620, vocabulary virtuoso 338 generates root word variant definitions 728 by retrieving, from definition dictionary 260, definition entries that correspond to each of root word variants 724.

[0108] In block 622, method 600 generates a flashcard for each of the most difficult challenge words. In one example of block 622, vocabulary virtuoso 338 generates at least one flashcard 730 for each of most difficult challenge words 716, where the at least one flashcard 730 displays the challenge word for user 120 to read aloud, displays the definition of the challenge word, displays the challenge word in a first context from book 252 for user 120 to read, and displays the challenge word in a second context from book 252 for user 120 to read. Accordingly, when user 120 reads the challenge word aloud correctly three times, that word will already be mastered before the word in encountered in book 252.

[0109] In block 624, method 600 generates a flashcard for each of the root words. In one example of block 624, vocabulary virtuoso 338 generates a flashcard 730 for one of root words 722, displaying a definition of the root word and a list of corresponding root word variants 724 and corresponding definitions. In certain embodiments, vocabulary virtuoso 338 includes context of each root word variant from the remainder of book 252.

[0110] FIG. 8 shows one example flashcard 802 listing the most difficult challenge words 716 found in a next portion of book 252, in embodiments. Flashcard 802 is one example of flashcard 730 of FIG. 7. Particularly, flashcard 802 provides both motivation and education to user 120 by teaching user 120 how many times each of most difficult challenge words 716(1 )-(5) occurs in the rest of book 252 and provides motivation to interactively learn each of the most difficult challenge words 716.

[0111] FIG. 9A shows one example flashcard 902 defining one challenge word 716(1) and providing context 904(1 )-context 904(3) for the challenge word 716(1) from each of three places in book 252, in embodiments. Flashcard 902 is one example of flashcard 730 of FIG. 7. Advantageously, flashcard 902 allows user 120 to learn most difficult challenge word definition 720(1), to learn how to pronounce challenge word 716(1) by selecting play button 906 that causes reader application 125 to play an audible clip of challenge word 716(1) being spoken, and three opportunities to practice reading challenge word 716(1) in context 904 taken directly from book 252. Flashcard 902 may display a checkmark to indicated when challenge word 716(1) is read correctly in each context 904.

[0112] FIG. 9B shows another example flashcard 952 defining one challenge word 716(1) and providing user 120 with an opportunity to practice reading the word three times, in embodiment. Flashcard 952 is one example of flashcard 730 of FIG. 7. Flashcard 952 is similar to flashcard 902, providing the corresponding most difficult challenge word definition 720(1), the opportunity to hear pronunciation of challenge word 716(1) by selection of play button 956 that causes reader application 125 to play an audible clip of challenge word 716(1) being spoken, and three opportunities to read challenge word 716(1). However, with flashcard 952, user 120 first reads challenge word 716(1) alone, then reads challenge word 716(1) in context 954 taken from book 252, and then reads challenge word 716(1) alone again. Flashcard 952 may display a checkmark to indicated when challenge word 716(1) is read correctly at each opportunity.

[0113] FIG. 10 shows one example flashcard 1002 defining root words 722 that correspond to the most difficult challenge words 716 and allowing user 120 to select each root word 722 to learn more, in embodiments.

[0114] FIG. 11 shows one example flashcard 1102 providing a definition of one of root words 722, indicating a count of how many time it is used in book 252, and listing root word variants 724 derived from the root word, in embodiments. Advantageously, user 120 learns the definition of the root word, how to pronounce it by pressing play button 1106, and may select each of root word variants 724 to learn more and practice reading and mastering it.

[0115] FIG. 12 shows one example flashcard 1202 defining one of root word variants 724 and provides context 1204(1) -context (3) for the root word variants from three places in book 252, in embodiments. Advantageously, user 120 learns root word variant definition 728(1), how to pronounce it by pressing play button 1206, and is provided with context directly from book 252 to practice reading and mastering it.Find & Fill Foundational Holes

[0116] FIG. 13 is a block diagram illustrating reader server 105 of FIGs. 1 and 2 further including reader artificial intelligence 1301 (reader Al 1301), in embodiments. Reader Al 1301 includes a pattern engine 1302 that detects patterns 1310 in user data 274 based on one or more pattern rules 1304 and a learning modeler 1306 the creates a learning model 1350 defining learning behavior of one or more users 120. Pattern engine 1302 processes user data 274 stored in user database 270 to detect patterns in user word graph 320.Pattern engine 1302 may operate on individual user data 274 to identify patterns for any oneuser 120. or may operate on user data 274 of multiple users 120 to detect patterns within groups of users, or a combination thereof. In certain embodiments, user data 274 is extracted and anonymized from user database 270 before being processed by pattern engine 1302.

[0117] In certain embodiments, reader Al 1301 is implemented as a neural network, that is trained to identify patterns within user word graphs 320 across multiple users 120. In another embodiment, pattern engine 1302 is trained using pattern rules 1304 that instruct pattern engine 1302 how to detect patterns within individual user data 274.

[0118] In one example of operation, pattern engine 1302 processes user data 274 of users within one user group 276 to identify patterns in user word graphs 320 for a class taught by teacher 292. Accordingly, detected patterns 1310 may be used to inform teacher 292 of class related reading performance anomalies (e.g., good and bad).

[0119] In certain embodiments, pattern engine 1302 generates multiple different patterns 1310 and may rank patterns 1310 based on importance. In one example, importance may be defined as a number of users 120 involved in the pattern, where a first pattern having a large number of users is more important than a second pattern that has fewer users.

[0120] In another example, referring to user word graph 320 of FIG. 4A, pattern engine 1302 identifies pattern 1310 from one or more incorrect use counts 426, 436, 446, and 456 being above a threshold value (e.g., 5) in user data 274, where the pattern 1310 indicates a correlation in one or more of encountered sounds 422, word parts 432, root words 442, and whole words 452. Pattern 1310 may indicate an anomaly related to a particular reading deficiency of multiple users 120. In another example of operation, pattern engine 1302 matches a pattern in reading deficiency for one user 120 to similar reading deficiencies in other users, where the pattern 1310 indicate that the one user has a similar deficiency to at least one other user.

[0121] Where pattern 1310 indicates multiple users 120 in group 276 (e.g., one school class) consistently mispronounce word parts ‘sion’ and / or ‘tion’ (e.g., that make the sound <shun>), reader server 105 may generate one or more mini-lessons 1322 for each of the multiple users 120 that address the found mispronunciation. For example, lesson generator 1320 creates at least one mini-lesson 1322 for each of the multiple users 120 that may be deployed by tools 344 within reader application 125 of the corresponding computing device 110. For example, tools 344 may include a flashcard display 704 that presents one or more flashcards addressing the mispronunciation to each user identified within pattern 1310. Alternatively, lesson generator 1320 may present information of the deficiency identified in pattern 1310 to teacher 292 such that user group 276 may be instructed collectively.Similarly, pattern 1310 may indicate a positive trend in user data 274 corresponding to teaching of teacher 292, whereby lesson generator 1320 informs teacher 292 of the positive trend.

[0122] Lesson generator 1320 generates at least one mini-lesson 1322 for certain ones of patterns 1310, where each mini-lesson 1322 targets at least one user 120. Mini-lesson 1322 is for example a set of flashcards that address certain aspects of pattern 1310. Continuing with the above example where pattern 1310 identifies mispronounced word parts ‘sion’ and / or ‘tion,’ lesson generator 1320 generates at least one mini-lesson 1322 teaching pronunciation of word parts ‘sion’ and / or ‘tion’ as <shun>. For example, lesson generator 1320 may generate one mini-lesson 1322 that is deployed to computing device 110 of relevant users 120.

[0123] Within each computing device 110, reader application 125 may customize mini-lesson 1322 to generate a related mini-lesson 1324 that is more relevant to book 252 being read by that user. For example, reader application 125 may determine a number of times the mispronounced word part occurs in page text 356 and / or in a remaining part of book 252 of that user and update related mini-lesson 1324 to reflect that data. Similarly, reader application 125 updates related mini-lesson 1324 to include context for practicing mispronounced word parts ‘sion’ and / or ‘tion’ from book 252 of user 120. Accordingly, relevant mini-lesson 1324 is more relevant to the user’s reading, and thereby the user is more likely to follow the teaching and learn better with practice examples using context directly from the user’s book. Reader application 125 controls computing device 1 10 to interactively present related mini-lesson 1324 to user 120. Advantageously, this direct reference to book 252 causes each user 120 to learn the advantage and value of using related mini-lesson 1324, such as to learn the mispronounced word parts in this example.

[0124] Related mini-lesson 1324 may further define use of visual cues for certain words within book 252 by reading monitor 330. Reading monitor 330 may show visual clues, within page text 356 as displayed within GUI 352 of FIG. 5, corresponding to related mini-lesson 1324. Continuing with the above example, related mini-lesson 1324 may indicate that words including mispronounced word parts ‘sion’ and / or ‘tion’ should have visual cues to aid learning by user 120. Accordingly, reading monitor 330 controls GUI 352 to render the next X words ending in either ‘sion’ or ‘tion’ with a visual cue such as one or more of a different color, different font, and / or a visual treatment such as sparkles. Advantageously, the visual cue informs user 120 of the challenge word such that the user tried harder to pronounce it correctly. In certain embodiments, when the user reads the wordcorrectly they receive a visual and or digital reward. For example, reader application 125 may control 110 to cause the word to sparkle off the page when the user reads the word correctly a certain number of times (e.g., ten). The visual reward may deliver a dopamine hit to the user that further motivates reading.

[0125] After the user successfully pronounces the word correctly a certain number of times, reading monitor 330 may only apply the visual clue half the time the word occurs, leaving the other half of the words in the standard font and color. Where user 120 reads the word correctly with or without the visual cue, reading monitor 330 may control computing device 110 to generate a visual or digital reward. Where the user does not demonstrate correct reading of the word, with or without the visual cues, reading monitor 330 may cause computing device 110 to offer a different mini-lesson 1322 to address the deficiency in another way. This different mini-lesson 1322 may result in similar use of visual cues when the mini-lesson is completed. In certain embodiments, the mini-lesson and visual cues persist even when user 120 switches books. That is, reader application 125 causes computing device 110 to maintain the learning environment of user 120 independent of the selected book 252.

[0126] Once user 120 reads the word correctly a certain number of time (e.g., ten times) without the visual cue, user word graph 320 is updated to indicate that the word is mastered, and reading monitor 330 may proceed with a next mini-lesson 1322 and / or next pattern 1310 corresponding to the user.

[0127] In certain embodiments, reader Al 1301 may be invoked to generate a short story based on user data 274, where the short story includes certain vocabulary words that provide reading practice for user 120. For example, reader Al 1301 may be controlled based on statistics from one or more of session metrics 322, user word graph 320 and proficiency data 470. For example, reader Al 1301 generates the short story based on the genre and / or content type of books read by user 120 such that the short story is of interest to user 120. For example, the short story may be part of mini-lesson 1322 and / or related mini-lesson 1324.

[0128] Certain concepts may require more practice than other that are grasped quickly. Further, certain users may require more practice as compared to other users, regardless of content. Accordingly, reader Al 1301 includes a learning modeler 1306 that maintains a learning model 1350 indicative of how much practice each user 120 needs to learn concepts included within mini-lessons 1322. Since reading monitor 330 continually evaluates performance of each user 120, updating user data 274 (e.g., user word graph 320 and session metrics 322 that are sent to reader server 105), learning modeler 1306 updates learning model 1350 based on feedback within user data 274. In certain embodiments,learning modeler 1306 maintains one learning model 1350 for each user 120. In other embodiments, learning modeler 1306 maintains one learning model 1350 for each type of user 120. Learning model 1350 tracks time spent on each mini-lesson 1322 by each user 120 receiving the mini-lesson and evaluates changes in user word graph 320 and / or session metrics 322 relating to mini-lesson 1322 over time to discern the effectiveness of mini-lesson 1322. In certain embodiments, learning modeler 1306 determines a type of user 120 based on one or both of user word graph 320 and session metrics 322. Lesson generator 1320 uses corresponding learning model 1350 when generating mini-lessons 1 22 for each user 120. For example, where learning model 1350 indicates user 120 needs to read a minimum of six practice sentences to learn a word or word part, lesson generator 1320 generates mini-lesson 1322 to include at least six context sentences. Advantageously, mini-lessons 1322 are tailored to each user’s learning ability and needs.

[0129] Information of patterns 1310 may be aggregated across classrooms, grades, schools or districts to provide educators with feedback on anomalies detected within reading abilities of multiple student. In one operational example, pattern rules 1304 control pattern engine 1302 to detects patterns 1310 in incorrect 426 or skipped count 428 errors found in encountered sounds 422, correlated to a number of syllables in the word containing the sounds and / or whether the error relates to a first encounter of the word in a sentence by the user. For example, words with more syllables are more challenging, but it may be ordering of the syllables and corresponding sounds 422 that causes problems for user 120. Further, a sentence with multiple previously unencountered words may be more problematic for user 120. Certain words are irregular in terms of pronunciation, where the word does not follow expected patterns of pronunciation. Accordingly, pattern rules 1304 may be defined to control pattern engine 1302 to indicate these anomalies with patterns 1310.

[0130] Lesson generator 1320 uses these detected patterns 1310 to generate one or more mini-lessons 1322, each having at least one flashcard 730, that explains concepts of the anomaly to user 120 using relevance of their own book 252, such as by indicating how many times that concept appears in the remainder of the book and providing context from book 252 for practicing reading the relevant words. Mini-lesson 1322 may also indicate where a particular lesson was not taught well enough, or may indicate where user 120 did not receive sufficient practice reading the word. In the first case, the lesson may be made clearer, and in the second case, user 120 is provided with more practice.Session Summaries

[0131] In certain embodiments, reader application 125 allows user 120 to generate audio or text summaries of passages they just read. For example, when user 120 finishes a current reading session, reader application 125 controls computing device 110 to prompt user 120 to create a summary of the passages that have just been read. In one example, reader application 125 controls computing device 110 to record a verbal description of the passages by user 120. In another example, reader application 125 controls computing device 110 to capture a text description of the passages from user 120 and / or transcribes the captured audio into text. User 120 may replay the recorded audio and / or display the saved text. For example, user 120 may replay the summary of previous passages prior to starting a next reading session.

[0132] In certain embodiments, reader application 125 may retrieve at least one question or prompt related to the read passages to encourage user 120 to provide the summary. For example, reader application 125 may invoke reader Al 1301 to generate one or more prompts based on the passages read.Predictions

[0133] In certain embodiments, at the end of a reading session, reader application 125 controls computing device 110 to invite user 120 to predict what will happen next in book 252. For example, reader application 125 may prompt user 120 to predict a next event in book 252 or to predict a consequence of an action in recently read passages. Reader application 125 may control computing device 110 to record the user’s prediction as audio and / or text. At the start of a subsequent reading session, reader application 125 encourages user 120 to review their previous predictions, thereby encouraging them to return to their book to read the subsequent passages and find out whether their prediction was correct (e.g., they nailed it!) or whether the author will surprise them. At the end of this second reading session, reader application 125 asks user 120 to rate their predictive performance. For example, reader application 125 may control computing device 110 to prompt user 120 with two buttons; “I nailed it,” and “no, the author surprised me!”

[0134] Advantageously, this additional interaction between reader application 125 and user 120 encourages user 120 to read more (e.g., to find out whether their predictions are correct) and to assimilate details from the passages being read. Thus, user 120 learns to focus on the content of the text and not just the words being read.Web Based Teacher Dashboard

[0135] As described for FIG. 2, teacher interface 280 provides teacher dashboard 282 for use by teacher 292, such as via a web interface, that allows teacher 292 to manage users, view metrics, and so on. FIG. 14 shows teacher dashboard 282 of FIG. 2 in further example detail. In certain embodiments, teacher dashboard 282 is web-based. User management 284 allows teacher 292 to manage book requests 1402. For example, user management 284 displays book requests 1402 with an approve button 1404 and a decline button 1406 that allows teacher 292 to respond to a request by user 120 for a new book. User management 284 also allows teacher 292 to make a book recommendation 1408 for one or more users 120. User management 284 also allow teacher 292 to monitor book spending 1410, such as by individual users 120 or by user groups 276.

[0136] User metrics 286 allows teacher 292 to see all student reading data including one or more of reading time 1420, reading time vs. reading goal 1422, reading accuracy rate 1424, reading persistence rate 1426, words per minute 1428, words read 1430, reading streaks 1432, books annotated 1434, high frequency word progress and mastery 1436, error words practice and mastery 1438, vocabulary growth 1440, student highlighting and annotation in text 1442, whole class reading relative to reading goal 1444. In certain embodiments, where a class reaches a class reading goal, reader server 105 may assign a prize to the class. Reader server 105 may allow teacher 292 to apply filters to user metrics 286, such as a time period and cohorts (e.g., individual students, groups of students, for the entire class).Reader Leader Board

[0137] FIG. 15 shows one example competition scenario 1500 where computing devices 110(l)-(M) are assigned to users 120 at a first school 1502(1), computing devices 110(M+l)-(N) are assigned to users 120 at a second school 1502(2), and computing devices 110(N+l)-(P) are assigned to users 120 at a third school 1502(3), in an embodiment. Although shows as a competition between schools 1502 in the following example, each competing entity could be one or more of an individual student, a groups of students within a class, the entire class, and the entire grade. For example, different classes may compete with each other in the same school or between schools. In another example, one grade level may compete with another grade level within the same or another school. In yet another example,school districts may compete. In certain embodiments, competition manager 262 allows one entity to challenge another entity, and tracks reding minutes accordingly.

[0138] Each computing device 1 10 tracks reading duration 406 as its corresponding user 120 reads aloud or silently. As described above, reading duration 406 is included within session metrics 322 that are sent to, and recorded by, reader server 105. Accordingly, competition manager 262 may determine a total number of minutes read by individuals within a predefined period (e.g., a competition period such as one school year, semester, trimester, etc.). Further, competition manager 262 may total these minutes for users 120 of each school 1502, where these schools 1502 are entered into a competition. Since reader application 125 accurately tracks when each user 120 is reading, ignoring time when the user is not paying attention, and therefore represents tracking of reading minutes independently of schools 1502 that are actively competing. Accordingly, competition manager 262 displays reader leader board 264 with competitor data 1510 for each competitor that includes validated minutes 1512 summed from individual reading minutes of users 120 tracked by reader application 125. In certain embodiments, the competition (e.g., a read-a-thon) is a regional or national competition where multiple schools within a region are grouped together as one competitive entity.

[0139] In certain embodiments, competition manager 262 may also invite one or more sponsors 1520(l)-(X) to promote the competition, where sponsors 1520 represent donors asked to purchase books for schools 1502 based on minutes read. Advantageously, reader system 100 provides an independently validated minute count for each school 1502 that sponsors 1520 may use when making a donation to each school.

[0140] FIG. 16 shows a diagrammatic representation of a computing device for a machine in the exemplary electronic form of a computer system 1600, within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. The computer system 1600 may be implemented within the computing device 110 and the reader server 105.

[0141] In various exemplary embodiments, the computing device operates as a standalone device or may be connected (e.g., networked) to other computing devices. In a networked deployment, the computing device may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The computing device may be a PC, a tablet PC, a set- top box, a cellular telephone, a digital camera, a portable music player (e.g., a portable hard drive audio device, such as an Moving Picture Experts Group Audio Layer 3 player), a webappliance, a network router, a switch, a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single computing device is illustrated, the term “computing device” shall also be taken to include any collection of computing devices or computers that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0142] The example computer system 1600 includes a processor or multiple processors 1602, a hard disk drive 1604, a main memory 1606, and a static memory 1608, which communicate with each other via a bus 1610. The computer system 1600 may also include a network interface device 1612. The hard disk drive 1604 may include a computer- readable medium 1620, which stores one or more sets of instructions 1622 embodying or used by any one or more of the methodologies or functions described herein. The instructions 1622 may also reside, completely or at least partially, within the main memory 1606 and / or within the processors 1602 during execution thereof by the computer system 1600. The main memory 1606 and the processors 1602 also constitute machine-readable media.

[0143] While the computer-readable medium 1620 is shown in one example embodiment to be a single medium, the term “computer-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The term “computer-readable medium” shall also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the present application, or that is capable of storing, encoding, or carrying data structures used by or associated with such a set of instructions. The term “computer-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media. Such media may also include, without limitation, hard disk drives 1604, floppy disks, NAND or NOR flash memory, digital video disks, Random Access Memory (RAM), Read-Only Memory (ROM), and the like.

[0144] The exemplary embodiments described herein may be implemented in an operating environment comprising computer-executable instructions (e.g., software) installed on a computer, in hardware, or in a combination of software and hardware. The computerexecutable instructions may be written in a computer programming language or may be embodied in firmware logic. If written in a programming language conforming to arecognized standard, such instructions may be executed on a variety of hardware platforms and for interfaces to a variety of operating systems.

[0145] In some embodiments, the computer system 1600 may be implemented as a cloud-based computing environment, such as a virtual machine operating within a computing cloud. In other embodiments, the computer system 1600 may itself include a cloud-based computing environment, where the functionalities of the computer system 1600 are executed in a distributed fashion. Thus, the computer system 1600, when configured as a computing cloud, may include pluralities of computing devices in various forms, as will be described in greater detail below.

[0146] In general, a cloud-based computing environment is a resource that typically combines the computational power of a large grouping of processors (such as within web servers) and / or that combines the storage capacity of a large grouping of computer memories or storage devices. Systems that provide cloud-based resources may be used exclusively by their owners, or such systems may be accessible to outside users who deploy applications within the computing infrastructure to obtain the benefit of large computational or storage resources.

[0147] The cloud may be formed, for example, by a network of web servers that comprise a plurality of computing devices, such as a client device, with each server (or at least a plurality thereof) providing processor and / or storage resources. These servers may manage workloads provided by multiple users (e.g., cloud resource consumers or other users). Typically, each user places workload demands upon the cloud that vary in real-time, sometimes dramatically. The nature and extent of these variations typically depends on the type of business associated with the user.

[0148] It is noteworthy that any hardware platform suitable for performing the processing described herein is suitable for use with the technology. The terms “computer- readable storage medium” and “computer-readable storage media” as used herein refer to any medium or media that participate in providing instructions to a CPU for execution. Such media may take many forms, including, but not limited to, non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as a fixed disk. Volatile media include dynamic memory, such as system RAM. Transmission media include coaxial cables, copper wire, and fiber optics, among others, including the wires that comprise one embodiment of a bus. Transmission media may also take the form of acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include,for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM disk, digital video disk, any other optical medium, any other physical medium with patterns of marks or holes, a RAM, a Programmable Read-Only Memory, an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory, a Flash EPROM, any other memory chip or data exchange adapter, a carrier wave, or any other medium from which a computer may read.

[0149] One skilled in the art will recognize that the Internet service may be configured to provide Internet access to one or more computing devices that are coupled to the Internet service, and that the computing devices may include one or more processors, buses, memory devices, display devices, input / output devices, and the like. Furthermore, those skilled in the art may appreciate that the Internet service may be coupled to one or more databases, repositories, servers, and the like, which may be used in order to implement any of the embodiments of the disclosure as described herein.

[0150] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present technology has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the present technology in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present technology. Exemplary embodiments were chosen and described in order to best explain the principles of the present technology and its practical application, and to enable others of ordinary skill in the art to understand the present technology for various embodiments with various modifications as are suited to the particular use contemplated.

[0151] Aspects of the present technology are described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present technology. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means forimplementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0152] These computer program instructions may also be stored in a computer readable medium that may direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0153] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0154] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present technology. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0155] In the preceding description, for purposes of explanation and not limitation, specific details are set forth, such as particular embodiments, procedures, techniques, etc. in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details.

[0156] Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the presentinvention. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” or “according to one embodiment” (or other phrases having similar import) at various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Furthermore, depending on the context of discussion herein, a singular term may include its plural forms and a plural term may include its singular form. Similarly, a hyphenated term (e.g., “on-demand”) may be occasionally interchangeably used with its non-hyphenated version (e.g., “on demand”), a capitalized entry (e.g., “Software”) may be interchangeably used with its non-capitalized version (e.g., “software”), a plural term may be indicated with or without an apostrophe (e.g., PE's or PEs), and an italicized term (e.g., “N+l”) may be interchangeably used with its non-italicized version (e.g., “N+l”). Such occasional interchangeable uses shall not be considered inconsistent with each other.

[0157] Also, some embodiments may be described in terms of “means for” performing a task or set of tasks. It will be understood that a “means for” may be expressed herein in terms of a structure, such as a processor, a memory, an VO device such as a camera, or combinations thereof. Alternatively, the “means for” may include an algorithm that is descriptive of a function or method step, while in yet other embodiments the “means for” is expressed in terms of a mathematical formula, prose, or as a flow chart or signal diagram.

[0158] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0159] It is noted at the outset that the terms “coupled,” “connected”, “connecting,” “electrically connected,” etc., are used interchangeably herein to generally refer to the condition of being electrically / electronically connected. Similarly, a first entity is considered to be in “communication” with a second entity (or entities) when the first entity electrically sends and / or receives (whether through wireline or wireless means) information signals (whether containing data information or non-data / control information) to the second entity regardless of the type (analog or digital) of those signals. It is further noted that variousfigures (including component diagrams) shown and discussed herein are for illustrative purpose only, and are not drawn to scale.

[0160] While specific embodiments of, and examples for, the system are described above for illustrative purposes, various equivalent modifications are possible within the scope of the system, as those skilled in the relevant art will recognize. For example, while processes or steps are presented in a given order, alternative embodiments may perform routines having steps in a different order, and some processes or steps may be deleted, moved, added, subdivided, combined, and / or modified to provide alternative or sub-combinations. Each of these processes or steps may be implemented in a variety of different ways. Also, while processes or steps are at times shown as being performed in series, these processes or steps may instead be performed in parallel, or may be performed at different times.

[0161] While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not limitation. The descriptions are not intended to limit the scope of the invention to the particular forms set forth herein. To the contrary, the present descriptions are intended to cover such alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims and otherwise appreciated by one of ordinary skill in the art. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments.Combination of Features

[0162] Features described above as well as those claimed below may be combined in various ways without departing from the scope hereof. The following enumerated examples illustrate some possible, non- limiting combinations:

[0163] (Al) A method for monitoring reading performance, includes: displaying, by an application running on a computing device and within a graphical user interface displayed, one or more written words from a book by the computing device; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user's spoken word matches the one or more written words; and updating a word graph based on the matching of the spoken word with the one or more written words; wherein the word graph defines accuracy and fluency of the user.

[0164] (A2) In embodiments of (Al), the application controls the computing device to prevent the user from changing a page with the written words when reading by the user is not been detected.

[0165] (A3) In either of embodiments (Al) or (A2), the word graph conforming to one or more of local standards, state standards, and national standards.

[0166] (A4) Any of the embodiments of (A1)-(A3) further including determining metrics of the user based on the word graph; and presenting, by the application, the metrics within a dashboard shown on the computing device.

[0167] (A5) Any of the embodiments of (A1)-(A4), further including incrementing, within the word graph and for a detected sound, a count of sounds spoken correctly by the user when the sound matches the one or more written words.

[0168] (A6) Any of the embodiments of (A1)-(A5) further including incrementing, within the word graph and for a detected word part, a count of word parts spoken correctly by the user when the word part matches the one or more written words.

[0169] (A7) Any of the embodiments of (A1)-(A6) further including incrementing, within the word graph and for a detected root word, a count of root words spoken correctly by the user when the root word matches the one or more written words.

[0170] (A8) Any of the embodiments of (A1)-(A7) further including incrementing, within the word graph and for a detected whole word, a count of whole words spoken correctly by the user when the whole word matches the one or more written words.

[0171] (A9) Any of the embodiments of (A1)-(A8) further including incrementing, within the word graph and for a detected sound, a count of sounds spoken incorrectly by the user when the sound does not match the one or more written words.

[0172] (A10) Any of the embodiments of (A1)-(A9) further including incrementing, within the word graph and for a detected word part, a count of word parts spoken incorrectly by the user when the word part does not match the one or more written words.

[0173] (Al l) Any of the embodiments of (Al)-(A10) further including incrementing, within the word graph and for a detected root word, a count of root words spoken incorrectly by the user when the root word does not match the one or more written words.

[0174] (A12) Any of the embodiments of (Al)-(Al 1) further including incrementing, within the word graph and for a detected whole word, a count of whole words spoken incorrectly by the user when the whole word does not match the one or more written words.

[0175] (Al 3) In the embodiments of (A1)-(A12), the word graph storing, for words encountered by the user in the book, a list of sounds, a list of word parts, a list of root words, and a list of whole words.

[0176] (A14) Any of the embodiments of (A1)-(A13) further including incrementing, within the word graph and for a detected sound, a count of sounds skipped by the user when the user skips the sound for the one or more written words.

[0177] (Al 5) Any of the embodiments of (A1)-(A14) further including incrementing, within the word graph and for a detected word part, a count of word parts skipped by the user when the user skips the word part for the one or more written words.

[0178] (Al 6) Any of the embodiments of (A1)-(A15) further including one or both of (a) incrementing, within the word graph and for a detected root word, a count of root words skipped when the user skips the one or more written words, and (b) incrementing, within the word graph and for a detected whole word, a count of whole words skipped by the user when the user skips the one or more written words.

[0179] (A17) Any of the embodiments of (A1)-(A16) further including determining a count of words read by the user in a current reading session; computing a difference between the count and a reading goal of the user; and generating at least one motivational notification based on the difference.

[0180] (B 1 ) A method for assisting reading of a book using a reader application, including: determining, by the reading application running on a computing device, next reading words of the book for a next reading session; generating a list of unmastered challenge words in the next reading words; selecting a most difficult challenge word from the list; determining a number of times the most difficult challenge word occurs in a remainder of the book; generating a flashcard presenting the most difficult challenge word and the number; presenting the flashcard on a display of the computing device; capturing, using a microphone of the computing device, audio of a user's spoken word when reading the most difficult challenge word; and processing the audio using speech recognition of the application to determine whether the user's spoken word matches the most difficult challenge word.

[0181] (B2) Any of the embodiments of (B 1 ) further including determining a root word of the most difficult challenge word; determining a root word variant of the root word; determining a number of times the root word variants occurs in the remainder of the book; determining a definition of the root word variant and the root word; generating a flashcard for the root word and the root word variant; presenting the flashcard on the display of the computing device; capturing, using the microphone, audio of the user's spoken word whenreading the root word and the root word variant; and processing the audio using speech recognition of the application to determine whether the user's spoken word matches the root word and the root word variant.

[0182] (Cl) A method for providing independent reading assistance, including displaying, by an application running on a computing device, one or more written words selected from a book; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user's spoken word matches the one or more written words; determining accuracy and fluency metrics based on the user's spoken word matching the one or more written words; and presenting, by the application, the metrics on a display of the computing device.

[0183] Changes may be made in the above methods and systems without departing from the scope hereof. It should thus be noted that the matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense. The following claims are intended to cover all generic and specific features described herein, as well as all statements of the scope of the present method and system, which, as a matter of language, might be said to fall therebetween.

Claims

CLAIMSWhat is claimed is:

1. A method for monitoring reading performance, comprising: displaying, by an application running on a computing device and within a graphical user interface displayed, one or more written words from a book by the computing device; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user's spoken word matches the one or more written words; and updating a word graph based on the matching of the spoken word with the one or more written words; wherein the word graph defines accuracy and fluency of the user.

2. The method of claim 1 , the application controlling the computing device to prevent the user from changing a page with the written words when reading by the user is not been detected.

3. The method of claim 1, the word graph conforming to one or more of local standards, state standards, and national standards.

4. The method of claim 1, further comprising: determining metrics of the user based on the word graph; and presenting, by the application, the metrics within a dashboard shown on the computing device.

5. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected sound, a count of sounds spoken correctly by the user when the sound matches the one or more written words.

6. The method of claim 1 , the updating further comprising incrementing, within the word graph and for a detected word part, a count of word parts spoken correctly by the user when the word part matches the one or more written words.

7. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected root word, a count of root words spoken correctly by the user when the root word matches the one or more written words.

8. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected whole word, a count of whole words spoken correctly by the user when the whole word matches the one or more written words.

9. The method of claim 1 , the updating further comprising incrementing, within the word graph and for a detected sound, a count of sounds spoken incorrectly by the user when the sound does not match the one or more written words.

10. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected word part, a count of word parts spoken incorrectly by the user when the word part does not match the one or more written words.

11. The method of claim 1 , the updating further comprising incrementing, within the word graph and for a detected root word, a count of root words spoken incorrectly by the user when the root word does not match the one or more written words.

12. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected whole word, a count of whole words spoken incorrectly by the user when the whole word does not match the one or more written words.

13. The method of claim 1, the word graph storing, for words encountered by the user in the book, a list of sounds, a list of word parts, a list of root words, and a list of whole words.

14. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected sound, a count of sounds skipped by the user when the user skips the sound for the one or more written words.

15. The method of claim 1, the updating further comprising incrementing, within the word graph and for a detected word part, a count of word parts skipped by the user when the user skips the word part for the one or more written words.

16. The method of claim 1, the updating further comprising one or both of (a) incrementing, within the word graph and for a detected root word, a count of root words skipped when the user skips the one or more written words, and (b) incrementing, within the word graph and for a detected whole word, a count of whole words skipped by the user when the user skips the one or more written words.

17. The method of claim 1, further comprising: determining a count of words read by the user in a current reading session; computing a difference between the count and a reading goal of the user; and generating at least one motivational notification based on the difference.

18. A method for assisting reading of a book using a reader application, comprising: determining, by the reading application running on a computing device, next reading words of the book for a next reading session; generating a list of unmastered challenge words in the next reading words; selecting a most difficult challenge word from the list; determining a number of times the most difficult challenge word occurs in a remainder of the book; generating a flashcard presenting the most difficult challenge word and the number; presenting the flashcard on a display of the computing device; capturing, using a microphone of the computing device, audio of a user's spoken word when reading the most difficult challenge word; and processing the audio using speech recognition of the application to determine whether the user's spoken word matches the most difficult challenge word.

19. The method of claim 18, further comprising: determining a root word of the most difficult challenge word; determining a root word variant of the root word; determining a number of times the root word variants occurs in the remainder of the book; determining a definition of the root word variant and the root word; generating a flashcard for the root word and the root word variant; presenting the flashcard on the display of the computing device; capturing, using the microphone, audio of the user's spoken word when reading the root word and the root word variant; and processing the audio using speech recognition of the application to determine whether the user's spoken word matches the root word and the root word variant.

20. A method for providing independent reading assistance, comprising: displaying, by an application running on a computing device, one or more written words selected from a book; capturing audio of a user's spoken word using a microphone of the computing device; processing the audio using speech recognition of the application to determine whether the user's spoken word matches the one or more written words; determining accuracy and fluency metrics based on the user's spoken word matching the one or more written words; and presenting, by the application, the metrics on a display of the computing device.

Images