Phonics-based educational system and method

Abstract

A phonics-based educational system and method is provided, the system including: at least one computing device; and a microphone associated with the at least one computing device. The at least one computing device is configured to: request a phoneme from a user; receive a sound from the user using the microphone; generate a spectrogram from the sound; and determine whether the sound corresponds to the requested phoneme using an artificial intelligence (AI) model and the spectrogram.

Description

PHONICS-BASED EDUCATIONAL SYSTEM AND METHODTECHNICAL FIELD

[0001] The present invention relates to teaching. In particular, although not exclusively, the present invention relates to phonics-based teaching of reading and writing.BACKGROUND ART

[0002] Reading and writing are foundational skills that support long-term educational achievement.

[0003] Phonics is a method for teaching reading and writing that teaches students the relationship between sounds (phonemes) and letters (graphemes) of a written language, such as English. Typically, sounds (phonemes) are taught, including how to read and pronounce each sound, as well as how to spell words containing the sounds.

[0004] Phonics has traditionally been taught in classrooms and similar environments, by trained educators. In short, students are taught to identify and reproduce sound (phoneme) and letter (grapheme) patterns, with feedback from the educator.

[0005] Several attempts have been made to enable students to practice phonics at home, e.g. using specialised books. A problem with such an approach, however, is that without a trained educator, the student may learn or reinforce incorrect patterns or sounds.

[0006] As such, there is clearly a need for improved phonics-based educational systems and methods.

[0007] It will be clearly understood that, if a prior art publication is referred to herein, this reference does not constitute an admission that the publication forms part of the common general knowledge in the art in Australia or in any other country.SUMMARY OF INVENTION

[0008] The present invention relates to phonics-based educational systems and methods, which may at least partially overcome at least one of the abovementioned disadvantages or provide the consumer with a useful or commercial choice.

[0009] With the foregoing in view, the present invention in one form resides broadly in a phonics-based educational system, including:at least one computing device; anda microphone associated with the at least one computing device;the at least one computing device configured to:request a phoneme from a user;receive a sound from the user using the microphone;generate a spectrogram from the sound; anddetermine whether the sound corresponds to the requested phoneme using an artificial intelligence (Al) model and the spectrogram.

[0010] Advantageously, the system enables accurate feedback to the user regarding the pronunciation of phonemes, without requiring evaluation by another person. In particular, the use of spectrograms and Al is more accurate at providing feedback in relation to pronunciation of phonemes than general speech recognition methods.

[0011] The use of such feedback simplifies the process of teaching phonics at home, when a trained educator is impractical, as it enables automatic feedback without the use of a trained educator.

[0012] In practical implementations, automatic phoneme assessment performed on a portable computing device is affected by technical constraints including microphone variability, ambient noise, inconsistent recording duration, variable input amplitude, limited memory and processing resources, and network latency where remote inference is used. These factors can reduce the reliability of phoneme classification if raw captured audio is analysed directly.

[0013] In preferred embodiments, the system addresses one or more of these technical constraints by controlling capture of the audio signal, pre-processing the captured signal to reduce noise and inter-sample variation, converting the captured signal into a machine-processable spectrogram representation, and applying a trained phoneme-classification model to the spectrogram rather than to unprocessed waveform data.

[0014] In some embodiments, the combination of controlled capture, signal normalisation, spectrogram generation and phoneme-specific inference reduces the effect of differences between recording devices and recording environments, and improves the consistency with which the system can automatically assess pronunciation using a portable computing device and / or a remote server.

[0015] In some embodiments, the system further improves machine operation by reducing the amount of data processed by the classification model, for example by trimming silence, normalising amplitude, segmenting a target phoneme region, and / or generating a spectrogram of predetermined size. This may reduce processing load, memory usage, transmissionbandwidth and / or inference latency.

[0016] The spectrogram may include a time axis and a frequency axis. The frequency axis may be defined according to a logarithmic scale.

[0017] Preferably, the spectrogram comprises a Mel spectrogram.

[0018] The spectrogram may comprise a 128x128 spectrogram.

[0019] The skilled addressee will, however, readily appreciate that the spectrogram may take any suitable form.

[0020] The sound may comprise a sound recording. The sound recording may comprise a predetermined length.

[0021] The sound recording may comprise a plurality of samples, sampled at a predefined frequency. Each sample may be of any suitable size, including 8 or 16 bits per sample. Similarly, the sampling frequency may comprise any suitable rate, including 44.1kHz.

[0022] The plurality of samples may be compressed according to one or more compression algorithms. Alternatively, the plurality of samples may be uncompressed.

[0023] The plurality of samples may comprise samples of an audio file.

[0024] The sound recording may be pre-processed prior to generating the spectrogram.

[0025] The sound recording may be filtered, to remove noise. Any suitable noise reduction algorithm may be used.

[0026] The sound recording may be trimmed to a pre-defined length.

[0027] The sound recording may be normalised, e.g. to a predefined amplitude. This is particularly useful in removing the impact of volume (amplitude) on analysis of the recording.

[0028] The Al model may take any suitable form. In a preferred embodiment, however, the Al model comprises an artificial neural network (ANN). The Al model may comprise a convolutional neural network (CNN). The CNN may be trained using a plurality of phonemes. The CNN may be phoneme specific.

[0029] The Al model may comprise a recurrent neural network (RNN). The Al model may comprise a CNN-RNN hybrid.

[0030] The Al model may return a label associated with the spectrogram.

[0031] The Al model may return a score associated with the spectrogram and a label. For example, the Al model may return a high score with a label when the label matches the spectrogram well, and a low score when the label matches the spectrogram poorly.

[0032] The Al model may be trained using a plurality of sample phonemes, and associated labels. The sample phonemes may comprise audio recordings of a plurality of persons saying the associated phonemes.

[0033] The sample phonemes may be pre-processed prior to being input into the Al model. As an illustrative example, the sample phonemes may be filtered, to remove noise therefrom.

[0034] A spectrogram may be generated from each of the sample phonemes, wherein the Al model is trained using the spectrogram.

[0035] The spectrogram may include a time axis and a frequency axis. The frequency axis may be defined according to a logarithmic scale.

[0036] Preferably, the spectrogram comprises a Mel spectrogram.

[0037] The spectrogram may comprise a 128x128 spectrogram.

[0038] The skilled addressee will, however, readily appreciate that the spectrogram may take any suitable form.

[0039] The labels may comprise graphemes associated with each of the phonemes.

[0040] The phoneme may comprise a phoneme of a plurality of predefined phonemes.

[0041] The plurality of predefined phonemes may comprise 44 phonemes associated with the English language.

[0042] The system may be configured to request different phonemes from the user at different points in time.

[0043] The system may be configured to provide an indication to the user indicating whether the sound from the user accurately represents the requested phoneme.

[0044] In case the sound does not accurately represent the requested phoneme, the system may be configured to request the phoneme again, and perform a new evaluation.

[0045] The system may be configured to provide aural instructions and / or feedback to the user. The system may be configured to provide written instructions and / or feedback to the user.

[0046] The system may include one or more characters, configured to interact with the user. The characters may comprise cartoon characters.

[0047] The system may include one or more games. The system may request phonemes from the user as part of a game.

[0048] The at least one computing device may include a portable computing device. The portable computing device may include the microphone.

[0049] The portable computing device may comprise a smartphone. The portable computing device may comprise a tablet computer.

[0050] The at least one computing device may comprise a single computing device.

[0051] The at least one computing device may comprise a plurality of computing devices. The plurality of computing devices may be interconnected using the Internet.

[0052] Preferably, the plurality of computing devices include a portable computing device, with which the user interacts, and at least one remote server. The Al model may be defined in the at least one remote server.

[0053] In some embodiments, the sounds comprise a word. The word may include one or more target phonemes. The one or more target phonemes may be extracted from the word, wherein the spectrogram is generated from the extracted phoneme(s).

[0054] The phoneme may be at the beginning of the word. The phoneme may be at the end of the word. The phoneme may be at the middle of the word.

[0055] In another form, the invention resides broadly in a phonics-based educational method, including:requesting, using at least one computing device, a phoneme from a user; receiving, using a microphone associated with the at least one computing device, a sound from the user;generating a spectrogram from the sound; anddetermining whether the sound corresponds to the requested phoneme using an artificial intelligence (Al) model and the spectrogram.

[0056] Any of the features described herein can be combined in any combination with anyone or more of the other features described herein within the scope of the invention.

[0057] The reference to any prior art in this specification is not, and should not be taken as an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge.BRIEF DESCRIPTION OF DRAWINGS

[0058] Various embodiments of the invention will be described with reference to the following drawings, in which:

[0059] Figure 1 illustrates a schematic of a phonics-based educational system, according to an embodiment of the present invention.

[0060] Figure 2 illustrates a schematic of the portable computing device of the system of Figure 1, according to an embodiment of the present invention.

[0061] Figure 3 illustrates a screenshot of a first screen of an educational application of the system of Figure 1 , according to an embodiment of the present invention.

[0062] Figure 4 illustrates a screenshot of a second screen of the educational application of the system of Figure 1 , according to an embodiment of the present invention.

[0063] Figure 5 illustrates a screenshot of a third screen of the educational application of the system of Figure 1 , according to an embodiment of the present invention.

[0064] Figure 6 illustrates a screenshot of a fourth screen of the educational application of the system of Figure 1 , according to an embodiment of the present invention.

[0065] Figure 7 illustrates a flow chart of an educational method, according to an embodiment of the present invention.

[0066] Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in any way.DESCRIPTION OF EMBODIMENTS

[0067] Embodiments of educational systems are provided below in the context of a phonicsbased reading system. The skilled addressee will, however, readily appreciate that the teachings may be applied to other scenarios, without deviating from the scope of the presentinvention.

[0068] Figure 1 illustrates a schematic of a phonics-based educational system 100, according to an embodiment of the present invention. The phonics-based educational system 100 is for teaching reading and writing, particularly to children. In particular, the system 100 provides structured lessons, activities and games to help students to understand letter-sound correspondence, the alphabetic principle, blending, segmenting, decoding and encoding.

[0069] The system 100 includes a portable computing device 105, with which a child 110 may interact. The portable computing device 105 includes a data interface, allowing it to connect to a remote server 115 through the Internet. The remote server 115 is in turn coupled to an Al model 120, which is used to provide feedback to the child 110, as outlined in further detail below.

[0070] The portable computing device 105 in the system 100 is in the form of a smartphone, but in other embodiments may take the form of a tablet computer, such as an iPad of Apple Inc of Cupertino, California, United States, a personal computer (PC), or any suitable computing device. Furthermore, in other embodiments, functionality of the server 115 and / or Al model 120 may be provided in the smartphone 105 or similar computing device, rather than separated across multiple devices.

[0071] In some embodiments, the portable computing device 105 and remote server 115 divide processing between them. For example, the portable computing device 105 may perform capture control, local signal conditioning and / or spectrogram generation, while the remote server 115 performs inference using the Al model 120.

[0072] In other embodiments, the Al model 120 or a reduced version thereof is stored in the memory 210 of the portable computing device 105 and executed by the processor 205 so that inference is performed locally, thereby reducing round-trip network delay and allowing operation when network connectivity is limited or unavailable.

[0073] In further embodiments, the system dynamically selects whether inference is performed locally or remotely based on one or more technical criteria including processor availability, memory availability, current network quality, latency, battery condition, and / or a selected application mode.

[0074] In some embodiments, where the portable computing device 105 transmits data to the remote server 115, the transmitted data comprises a processed representation of the sound recording, such as a spectrogram or reduced feature vector, rather than the entire raw waveform, thereby reducing bandwidth usage and / or server-side preprocessing requirements.

[0075] Initially, and generally prior to deployment of the system 100, the Al model 120 is trained using a plurality of samples (representing phonemes) 125 and a plurality of associated labels (representing graphemes) 130.

[0076] The samples 125 comprise audio files comprising speech. In order to avoid overfitting, the samples should comprise a range of speech from different persons and in different environments.

[0077] The samples 125 are initially cleaned, trimmed and normalised. In short, various processing techniques may be applied to remove noise, and ensure consistency between files, e.g. by trimming silence, and adjusting an amplitude of (normalising) the files. The skilled addressee will readily appreciate, however, that any suitable pre-processing of the samples 125 may be used.

[0078] In some embodiments, capture of the sound recording is controlled by the portable computing device 105 so that the recording begins in response to selection of the sound input button 310 and ends after expiry of a capture window, detection of silence, or detection that an amplitude envelope of the captured signal has fallen below a threshold for a predetermined period.

[0079] In some embodiments, pre-processing includes one or more of: DC offset removal, amplitude normalisation, clipping detection, band-pass filtering, background noise suppression, echo reduction, silence trimming, and resampling the captured audio to a predetermined sample rate.

[0080] In some embodiments, the pre-processing operations are selected to reduce interrecording variation arising from different microphones, different distances between the child 110 and the microphone 225, and different ambient acoustic environments, thereby providing a more consistent input to the Al model 120.

[0081] In some embodiments, the portable computing device 105 performs at least part of the pre-processing locally prior to transmission to the server 115, such that the amount of data transmitted across the data interface 230 is reduced and a more uniform signal representation is provided to the remote server 115.

[0082] Once the samples 125 have been cleaned, a spectrogram is generated for each sample 125. The spectrogram comprises a representation of the spectrum of frequencies of the sample as it varies over time. It has been found that spectrograms are particularly useful at distinguishing phonemes, and particularly vowels and nasals, which is very important in phonics.

[0083] In the system 100, the Mel spectrogram is used, where the frequency component of the spectrogram is mapped to the Mel scale. This results in a frequency axis more aligned to how humans perceive frequency differences. Further, the spectrogram is scaled to 128x128 pixels, which has been found to be a good balance between memory / processing load and recognition quality. The skilled addressee will, however, readily appreciate that any suitable spectrogram type and size may be used.

[0084] In some embodiments, generation of the spectrogram comprises windowing the sound recording into a plurality of overlapping frames, applying a transform to each frame to obtain frequency-domain information, and mapping the frequency-domain information to a perceptual scale, such as a Mel scale, to generate a two-dimensional feature representation having time and frequency dimensions.

[0085] In some embodiments, the spectrogram is stored as a matrix of intensity values and optionally scaled, cropped or padded to a predetermined input size for the Al model 120. By transforming the captured waveform into a standardised feature matrix, the system may reduce sensitivity of the downstream classifier to variation in raw waveform length and amplitude.

[0086] In some embodiments, the spectrogram generation parameters, including frame length, frame overlap, frequency-band count and output matrix dimensions, are selected to balance phoneme discrimination performance with memory consumption, processor load and / or network transmission cost on the portable computing device 105 and remote server 115.

[0087] The spectrograms are then used to train the Al model 120, together with the labels 130.

[0088] The Al model 120 may take any suitable form, but in the system 100 is a convolutional neural network (CNN).

[0089] Once the Al model 120 has been trained, it may be used to classify spectrograms corresponding to voice from the child 110, and to determine the likelihood each spectrogram corresponds to a particular phoneme (and thus an associated grapheme). This is then used to provide feedback to the child 110.

[0090] In some embodiments, the Al model 120 outputs, for a received spectrogram, a phoneme class label and an associated confidence value, probability, score or ranking.

[0091] In some embodiments, the system determines that the captured sound corresponds to the requested phoneme only when the confidence associated with the requested phoneme satisfies a threshold criterion. Where the threshold criterion is not satisfied, the system mayclassify the result as uncertain, reject the sample, or request a further recording from the user.

[0092] In some embodiments, application of the threshold criterion reduces false positive classifications arising from background noise, truncated recordings, microphone artefacts, and incomplete pronunciation events.

[0093] In some embodiments, the threshold criterion may vary according to the phoneme being tested, the age or proficiency setting of the user, a detected signal quality metric, and / or whether inference is performed using an isolated phoneme workflow or a phoneme-extracted-from-word workflow.

[0094] In use, an educational application is installed on the computing device 105. The educational application may take any suitable form, but advantageously comprises a combination of games and educational content, to ensure engagement with the child 110.

[0095] The application is interactive, and thus not only enables one-way teaching, but is able to confirm pronunciation of phonemes and the like, to ensure that the child 110 is not learning to incorrectly associate or pronounce phonemes.

[0096] Figure 2 illustrates a schematic of the portable computing device 105, according to an embodiment of the present invention.

[0097] The portable computing device 105 includes a processor 205, and a memory 210 coupled to the processor 205. The memory 210 includes instruction code executable by the processor 205 for performing a range of functions of the device 105, including running the educational application thereon.

[0098] The portable computing device 105 further includes a display 215, for displaying content to the user, a speaker 220, for playing sound, and a microphone 225 for recording sound (voice) from the user.

[0099] The display 215 comprises a touch screen display, and thus both allows the display of data, but also the receipt of commands as input, e.g. to the application. In short, the display may display a graphical user interface with elements which are selectable by the child 110, to make selections and perform various functions of the system 100.

[0100] Finally, the portable computing device 105 includes a data interface 230, on which data may be sent and received, e.g. to and from the server 115.

[0101] The functions described herein may be implemented by executable instruction codestored in the memory 210 and executed by the processor 205, by dedicated signal-processing circuitry, by one or more graphics processing units, neural processing units, digital signal processors, or by any suitable combination thereof.

[0102] In some embodiments, separate functional modules are provided for audio capture, pre-processing, feature generation, inference, confidence evaluation, and feedback generation. These modules may execute on the portable computing device 105, the remote server 115, or be distributed therebetween.

[0103] The skilled addressee will readily appreciate that the portable computing device 105 may take any suitable form, and include any suitable way for interacting with the user. As an illustrative example, the portable computing device 105 may include a trackpad, mouse or other input device.

[0104] As outlined above, the portable computing device 105 includes an educational application, for teaching reading and writing using phonics. Example screenshots of such application are described below.

[0105] The educational application generally teaches the child 110 to read and spell by teaching the relationship between sounds (phonemes) and letters (graphemes). This is typically done by focusing on one phoneme at a time. This is done, however, in a fun and playful way, using cartoon characters, aesthetically pleasing images, games, and the like.

[0106] Figure 3 illustrates a screenshot 300 of a first screen of the educational application, according to an embodiment of the present invention.

[0107] The first screen includes an image 305 of a monkey, and the application is configured to prompt (through voice output on the speaker) the child 110 to identify the image 305, e.g. by saying “what is the name of this picture?”. The first screen includes a sound input button 310, which is selectable by the child 110, to enable the child 110 to record their answer.

[0108] Upon selection of the sound input button 310, the microphone is configured to record sound, e.g. fora predetermined amount of time, or until silence is detected. The sound recording is then cleaned, trimmed and normalised, in a similar or identical manner to the sample data for training the Al model above.

[0109] The cleaned, trimmed and normalised sound recording is then uploaded to the server 115, where voice recognition is used to identify the word said in the recording by the child 110. The word is then provided back to the computing device 105, where it is compared to a word associated with the image 305 (e.g. “Monkey”).

[0110] If the word from the recording matches the word associated with the image 305, the child 110 is congratulated, and a new image 305 is displayed with a similar prompt. Otherwise, the child 110 is prompted to try again, potentially with hints.

[0111] The process is repeated for a number of words with similar phonemes, e.g. words starting with the / m / sound.

[0112] Figure 4 illustrates a screenshot 400 of a second screen of the educational application, according to an embodiment of the present invention.

[0113] The second screen displays thumbnails 405 corresponding to the images 305 of each of the previous screens (similar to that of Figure 3). This allows the child 110 to easily see a pattern between each of the images and words, in this case that they start with the same phoneme (in this case, the / m / sound).

[0114] The application then includes teaching around the phoneme, including identifying the sound (e.g. by referring to the “m” sound), and teaching the child how to say the phoneme.

[0115] Figure 5 illustrates a screenshot 500 of a third screen of the educational application, according to an embodiment of the present invention.

[0116] The third screen includes a video element 505, showing the shape of a person’s lips as they sound out the phoneme. The video element 505 illustrates the lips from the front, but in other embodiments, the lips may be shown from the side, or a combination of front and side.

[0117] The third screen includes playback elements, including a playback button 510, and a progress bar 515, enabling the child to pause and play the video, and navigate in the video.

[0118] The child 110 may be prompted to practice saying the phoneme as he or she is watching the video. The video may also be configured to repeat a set number of times.

[0119] Finally, once the child 110 has practiced saying the phoneme, the child 110 is evaluated and given feedback.

[0120] Figure 6 illustrates a screenshot 600 of a fourth screen of the educational application, according to an embodiment of the present invention.

[0121] The fourth screen includes an animated character 605, which interacts with the child 110. In this instance, the animated character 605 prompts the child to say the phoneme, e.g. by saying “Now it is your turn. Say ‘mmm’”. The fourth screen includes the sound input button 310, which is selectable by the child 110, to enable the child 110 to record their answer, similar to asillustrated in Figure 3.

[0122] Upon selection of the sound input button 310, the microphone is configured to record sound, which is cleaned, trimmed and normalised, in a similar or identical manner to the sample data for training the Al model above.

[0123] A 128x128 Mel spectrogram is generated for the cleaned, trimmed and normalised sound recording, again in a similar or identical manner to the sample data for training the Al model above.

[0124] The 128x128 Mel spectrogram is then uploaded to the server 115, where it is provided to the Al model to identify a phoneme and an associated grapheme. The Al may provide its result in any suitable form, including by returning an associated grapheme in case a phoneme is detected, indicating that no phoneme was detected, or by providing a matching score associated therewith. It is then compared with the requested phoneme (or grapheme).

[0125] In some embodiments, prior to or together with classification, the system evaluates a quality metric of the captured sound recording. The quality metric may indicate, for example, whether the recording includes sufficient signal energy, whether clipping is present, whether excessive background noise is present, whether silence occupies more than a threshold proportion of the capture window, and / or whether the duration of the captured utterance falls within an expected range.

[0126] Where the quality metric indicates that the captured sound recording is unsuitable for reliable classification, the system may prompt the user to repeat the utterance before the Al model 120 performs phoneme assessment, or may suppress presentation of a pronunciation result derived from the unsuitable recording.

[0127] In this way, the system may reduce incorrect feedback caused by poor signal acquisition, thereby improving operation of the phoneme assessment process.

[0128] In case the recorded sound matches the associated phoneme, or matches the associated phoneme, or produces a score above a threshold level, the child is congratulated. Otherwise, the child is prompted to try again, potentially with further guidance.

[0129] The process is then repeated for a range of phonemes, interspersed with games and other activities, to maintain engagement.

[0130] The above example relates to “first sounds”, i.e. the sounds at the beginning of a word. These are often taught first, as they are most easily identifiable, but the system will alsoinclude last sounds (i.e. the sounds at the ends of words), middle sounds (i.e. the sounds in the middle of words), syllabification, rhyming, segmenting and blending, including digraphs, diphthongs, consonant blends, the alphabetic principle, which connects aural sounds to their written representation.

[0131] In each case, the child may be prompted to record phonemes, regardless of their location in the words, upon which the Al model may be used to evaluate same.

[0132] As an illustrative example, in the case of middle sounds, words such as “yellow” and “umbrella” may be used to teach the / I / sound.

[0133] Furthermore, the system may include systematic, structured lessons that explicitly teach key vocabulary and phonemic awareness, including first, middle and last sounds; segmenting; blending and connecting aural sounds to their written representations.

[0134] Figure 7 illustrates a flow chart of an educational method 700, according to an embodiment of the present invention. The educational method is particularly useful for providing feedback in relation to pronunciation of phonemes.

[0135] The method 700 may be similar or identical to the method performed by the system 100, outlined above.

[0136] At step 705, a phoneme is requested from the user. The request may be an aural request (e.g. a voice on the speaker saying “please say the ‘mmm’ sound”), or a written request (e.g. text on a display stating “please say the ‘mmm’ sound”), for example.

[0137] At step 710, the user provides a sound recording of the user attempting to say the phoneme. The sound recording may be captured by a microphone in response to the user selecting an audio capture button, for example.

[0138] At step 715, a spectrogram is generated from the sound recording. The spectrogram may comprise a Mel spectrogram. Furthermore, in generating the spectrogram, the sound recording may be pre-processed, e.g. as outlined above.

[0139] At step 720, the spectrogram is provided to an Al model, such as the model 120, and a label is requested. The label may comprise a grapheme corresponding to the requested phoneme.

[0140] At step 725, the label is compared with the requested phoneme. In case of a match, the method ends at 730. The user may be congratulated at this stage.

[0141] In case of a mismatch, the phoneme is again requested from the user, and the process restarts at 705. The user may be prompted to try again, and encouraged to change their pronunciation.

[0142] While the above references evaluation of phonemes in isolation, in other embodiments, the system may extract one or more target phonemes from a word, and evaluate the extracted target phonemes.

[0143] In embodiments in which the captured sound comprises a spoken word, the system may identify a target phoneme region within the spoken word prior to classification.

[0144] Identification of the target phoneme region may comprise one or more of: locating a region of interest based on a known phoneme position within the word; aligning the spoken word to an expected phoneme sequence; identifying a likely phoneme boundary from changes in spectral energy overtime; and segmenting the recording to isolate a time interval corresponding to the target phoneme.

[0145] The segmented target phoneme region may then be used to generate a spectrogram or other feature representation for input to the Al model 120.

[0146] By segmenting a target phoneme region from a longer spoken word, the system may reduce irrelevant signal content provided to the classifier and improve classification consistency for first sounds, middle sounds and / or last sounds.

[0147] While the above is described with reference to the English language, the skilled addressee will readily appreciate that the teachings herein may be adapted to work with other languages.

[0148] Advantageously, the systems and methods provide an efficient way of teaching reading and writing, particularly to children, using phonics.

[0149] By capturing the child’s voice, and analysing the accuracy of the response (phoneme) using artificial intelligence, feedback is able to be provided in a simple, efficient and accurate manner. In case the response is accurate, encouraging (reinforcing) feedback may be provided, together with progression. In case the response is not accurate, guided correction may be used.

[0150] In preferred embodiments, the system improves operation of a computer-implemented phoneme assessment platform by converting variable speech input captured in uncontrolled environments into a standardised machine-readable representation suitable forautomated classification, and by using quality control and confidence-based evaluation to reduce unreliable classification outcomes.

[0151] The technical effect may include one or more of improved consistency of phoneme detection across different capture conditions, reduced computational burden by processing a reduced feature representation, reduced bandwidth in distributed implementations, and reduced latency in implementations that selectively perform local inference.

[0152] This ensures that the child receives tailored support, reinforcing the correct pronunciation of letter sounds and fostering independent learning, and simulates one-on-one instruction, promoting independent learning, accuracy, and confidence.

[0153] Further, by initially focusing on phonics and first sounds, a strong foundation is laid for reading and language skills. The emphasis on phonological awareness through activities such as active listening and recall helps children become more attuned to the sounds of language, which is crucial for reading fluency.

[0154] Finally, high engagement may be encouraged through songs, puzzles, videos, memory games, and storytelling.

[0155] In the present specification and claims (if any), the word ‘comprising’ and its derivatives including ‘comprises’ and ‘comprise’ include each of the stated integers but does not exclude the inclusion of one or more further integers.

[0156] Reference throughout this specification to ‘one embodiment’ or ‘an embodiment’ means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases ‘in one embodiment’ or ‘in an embodiment’ in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more combinations.

[0157] In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims (if any) appropriately interpreted by those skilled in the art.

Claims

CLAIMS1. A phonics-based educational system, including:at least one computing device; anda microphone associated with the at least one computing device;the at least one computing device configured to:request a phoneme from a user;receive a sound from the user using the microphone;generate a spectrogram from the sound; anddetermine whether the sound corresponds to the requested phoneme using an artificial intelligence (Al) model and the spectrogram.

2. The phonics-based educational system of claim 1, wherein the spectrogram includes a time axis and a frequency axis.

3. The phonics-based educational system of claim 2, wherein the frequency axis is defined according to a logarithmic scale.

4. The phonics-based educational system of claim 1, wherein the spectrogram comprises a Mel spectrogram.

5. The phonics-based educational system of claim 1 , wherein the sound comprises a sound recording having a predetermined length, and wherein the sound comprises a plurality of samples, sampled at a predefined frequency.

6. The phonics-based educational system of claim 1, wherein the sound is pre-processed prior to generating the spectrogram.

7. The phonics-based educational system of claim 1 , wherein the sound is trimmed to a pre-defined length.

8. The phonics-based educational system of claim 1 , wherein the sound is normalised, e.g. to a predefined amplitude.

9. The phonics-based educational system of claim 1, wherein the Al model comprises an artificial neural network (ANN), a convolutional neural network (CNN), a recurrent neural network(RNN) and / or a CNN-RNN hybrid.

10. The phonics-based educational system of claim 1 , wherein the Al model is trained using a plurality of audio recordings of phonemes and associated labels identifying the phonemes.

11. The phonics-based educational system of claim 10, wherein a spectrogram is generated from each of the audio recordings, wherein the Al model is trained using the spectrograms.

12. The phonics-based educational system of claim 1, wherein the Al model is phoneme specific.

13. The phonics-based educational system of claim 1 , wherein the Al model is configured to return a label associated with the spectrogram, the label associated with a phoneme.

14. The phonics-based educational system of claim 1 , wherein the Al model is configured to return a score associated with the spectrogram and a label.

15. The phonics-based educational system of claim 1, configured to request different phonemes from the user at different points in time, and provide an indication to the user indicating whether the sound from the user accurately represents the requested phoneme.

16. The phonics-based educational system of claim 15, wherein in case the sound does not accurately represent the requested phoneme, the system is configured to request the phoneme again, and perform a new evaluation.

17. The phonics-based educational system of claim 1, wherein the at least one computing device comprises a smartphone or tablet computer including the microphone.

18. The phonics-based educational system of claim 1, wherein the at least one computing device comprises a plurality of computing devices, interconnected using the Internet, the plurality of computing devices including a portable computing device, with which the user interacts, and at least one remote server, wherein the Al model is defined in the at least one remote server.

19. The phonics-based educational system of claim 1 , wherein the sound comprises a word including one or more target phonemes, wherein the one or more target phonemes are extracted from the word, wherein a spectrogram is generated from the extracted phonemes.

20. A phonics-based educational method, including:requesting, using at least one computing device, a phoneme from a user;receiving, using a microphone associated with the at least one computing device, asound from the user;generating a spectrogram from the sound; anddetermining whether the sound corresponds to the requested phoneme using an artificial intelligence (Al) model and the spectrogram.

Images