English learning support system based on precise voice recognition

Abstract

The present invention relates to an English learning support system based on precise voice recognition. The English learning support system comprises: a language analysis terminal for receiving English voice data spoken by a user, analyzing the pronunciation accuracy and the accuracy of sentences and transmitting same online, and receiving and outputting, with respect to the input English voice data, learning and training data required for English learning; a main server for storing the English voice data of the user received from the language analysis terminal, analyzing the level of pronunciation and the level of sentences of the user to evaluate the level of pronunciation and a sentence composition ability score of the user, and providing training data for English learning on the basis of same; a database management server for storing learning data including English pronunciation, the level of sentences, and learning progress information about the user; and a document-type server for storing the learning and training data required for learning and training as content or learning data. The main server is configured to: calculate the problem difficulty level and a problem rate on the basis of the ability score obtained by evaluating the pronunciation accuracy and sentence accuracy of the user input by means of the language analysis terminal and then provide a learning and training program; update the ability score of the user in real time; and adjust the problem difficulty level and the problem rate by reflecting the updated ability score and then provide the learning and training data again.

Description

English learning support system based on precision speech recognition

[0001] The present invention relates to a precision speech recognition-based English learning support system, and more specifically, to a precision speech recognition-based English learning support system that analyzes a user's voice data to provide an English learning training program by analyzing information such as speed, pronunciation, vocabulary, correct answers, and response time, and providing problem difficulty and problem speed under optimized conditions based on the user's current ability, including pronunciation level, speaking speed, native speaker intonation (or intonation), sentence comprehension, word level, and the user's English ability.

[0002] Existing English learning programs primarily provided static feedback to evaluate a user's learning progress. These programs were limited to providing personalized feedback on the text or sentences entered by the user, and there was a lack of systems that comprehensively evaluated pronunciation, grammar, and speed through real-time voice analysis to provide appropriate training.

[0003] Furthermore, while voice recognition systems simply recognized the user's voice to respond to specific keywords or sentences, there was no system that enhanced learning efficiency by comprehensively analyzing pronunciation accuracy, speech rate, sentence comprehension, and vocabulary level.

[0004] Furthermore, existing systems simply analyzed pronunciation and sentence comprehension individually and failed to provide an integrated evaluation; they also lacked the capability to provide real-time feedback by comprehensively evaluating pronunciation accuracy, speaking speed, and native speaker intonation.

[0005] In addition, the speech recognition system focused on analyzing the user's voice to recognize pronunciation or unique voice patterns.

[0006] Such technology improves learners' English pronunciation and sentence construction abilities through real-time pronunciation analysis and personalized feedback that existing learning systems could not provide. Furthermore, there is a demand in modern society for a learning and training environment that enables the achievement of pronunciation and sentence comprehension close to that of a native speaker.

[0007] [Prior Art Literature]

[0008] (Patent Document) KR Registered Patent No. 10-1888058

[0009] (Patent Document) KR Registered Patent No. 10-1025665

[0010] (Patent Document) KR Registered Patent No. 10-2031954

[0011] (Patent Document) KR Published Patent No. 10-2020-0093029

[0012] (Patent Document) KR Published Patent No. 10-2020-0015314

[0013] The present invention, aimed at solving the aforementioned problems, is intended to provide a more advanced language learning training program (learning training data) by comparing and analyzing the user's pronunciation and sentence level in real time.

[0014] In particular, the present invention aims to provide real-time feedback based on the user's pronunciation level and sentence structure, and to provide a function that evaluates how well the pronunciation matches the rhythm of a native speaker and the accuracy of the sentence comprehension.

[0015] In addition, the main purpose of the present invention is to provide a learning support system based on the user's current ability by providing feedback on ability scores measured in real-time during the learning process, such as the user's pronunciation level, sentence structure, speed, correct answers, and response time.

[0016] In addition, the present invention aims to provide customized English learning training for users by having AI (Artificial Intelligence) analyze the user's pronunciation level, speaking speed, native speaker rhythm, sentence comprehension, and vocabulary level in real time.

[0017] The present invention, for achieving the above-mentioned purpose, comprises: a language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; a main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; a database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and a document-type server that stores the learning training data necessary for learning training as content or learning data. The main server is configured to calculate problem difficulty and problem speed based on the ability score evaluated for the user's pronunciation accuracy and sentence accuracy input through the language analysis terminal, provide a learning training program, update the user's ability score in real time, adjust the problem difficulty and problem speed by reflecting the updated ability score, and then provide learning training data again.

[0018] In addition, the main server is configured to include a function that encrypts and processes user voice data to prevent hacking risks that may occur during the content retrieval and transmission process between the database management server and the document server, and securely transmits information necessary for learning training.

[0019] In addition, the main server comprises: an information collection unit that collects real-time voice or recorded voice data of the user received through the language analysis terminal and analyzes pronunciation and sentence levels; a learning analysis unit that deep-learns words and sentences pronounced by the user to analyze pronunciation accuracy and grammar accuracy, and calculates the user's ability score based on the analysis results; a storage unit that stores the user's pronunciation accuracy, sentence accuracy, and ability score analyzed by the learning analysis unit; a voice analysis matching unit that analyzes the user's English voice in real-time to provide feedback suitable for pronunciation and sentence levels and recommends learning training data necessary for learning training; and a document information processing unit that processes learning materials regarding the user's pronunciation accuracy and sentence accuracy in document format based on the analysis results analyzed by the learning analysis unit.

[0020] In addition, the language analysis terminal comprises: a voice analysis unit that receives user voice data in real time and analyzes pronunciation accuracy and sentence accuracy; and an analysis processing unit that analyzes the user's pronunciation and sentence accuracy in real time and provides customized learning training content based on learning data stored in a main server.

[0021] Additionally, the system is configured to include: a language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; a main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; a database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and a document-type server that stores the learning training data necessary for learning training as content or learning data. The main server is configured to determine the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty based on the ability score evaluated for the user's pronunciation accuracy, sentence accuracy, correct answer status, and correct answer response time input through the language analysis terminal, and then provide the determined learning training data; and to update the user's ability score in real time, and then re-determine the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty by reflecting the updated ability score, and then provide the learning training data.

[0022] In addition, the learning analysis unit calculates the ability score by calculating the correctness of the question, response time, and pronunciation accuracy using the following mathematical formula through the user's voice data collected by the information collection unit.

[0023] [Mathematical Formula]

[0024]

[0025] (Here, S ability,new is the updated ability score, τ is the feedback reflection coefficient (0 < τ < 1), C correct , T response , P pronunciation (Respectively, correct answer, response time, and pronunciation accuracy)

[0026] The present invention, configured and operating as described above, analyzes the user's voice in real time to evaluate the pronunciation level, speaking speed, native speaker rhythm, sentence comprehension, and vocabulary level, and based on this, can provide a customized English learning training program for the user, thereby maximizing the effectiveness of English learning by providing accurate feedback on the user's pronunciation and sentences, as well as customized training content.

[0027] Furthermore, the present invention analyzes a user's voice data in real time to evaluate pronunciation accuracy, speaking speed, and sentence comprehension, and automatically recommends appropriate training content accordingly. Through this, users can efficiently improve their English pronunciation and grammar levels, and there is the advantage of enabling customized English learning tailored to the individual characteristics of the learner.

[0028] In addition, the present invention has the effect of highly satisfying learning efficiency by measuring the user's ability score in real time, providing learning problems optimized for the user based on the ability score, and managing the learning progress.

[0029] FIG. 1 is an overall configuration diagram of a precision speech recognition-based English learning support system according to the present invention.

[0030] FIG. 2 is a detailed configuration diagram of a main server in a precision speech recognition-based English learning support system according to the present invention.

[0031] FIG. 3 is a detailed configuration diagram of a language analysis terminal in a precision speech recognition-based English learning support system according to the present invention.

[0032] FIG. 4 is an example diagram of ability score measurement in a precision speech recognition-based English learning support system according to the present invention.

[0033] Hereinafter, a precision speech recognition-based English learning support system according to the present invention will be described in detail with reference to the attached drawings.

[0034] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0035] The "terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminals via a network. Here, the computer may include, for example, a laptop, desktop, or notebook equipped with a web browser, or a VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.).

[0036] Here, VR HMDs include PC models (e.g., HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), mobile models (e.g., GearVR, DayDream, Storm Mirror, Google Cardboard, etc.), console models (PSVR), and stand-alone models implemented independently (e.g., Deepon, PICO, etc.). Portable terminals are wireless communication devices that ensure portability and mobility, and may include smartphones, tablet PCs, wearable devices, as well as various devices equipped with communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, ultrasonic, infrared, Wi-Fi, and Li-Fi.

[0037] In addition, "network" refers to a connection structure capable of exchanging information among respective nodes, such as terminals and servers, and includes local area networks (LAN), wide area networks (WAN), the internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television communication networks, etc. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC), and LiFi.

[0038] The English learning support system based on precision speech recognition according to the present invention comprises: a language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; a main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; a database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and a document-type server that stores learning training data necessary for learning training as content or learning data. The main server is configured to calculate problem difficulty and problem speed based on the ability score evaluated for the user's pronunciation accuracy and sentence accuracy input through the language analysis terminal, provide a learning training program, update the user's ability score in real time, adjust the problem difficulty and problem speed by reflecting the updated ability score, and then provide learning training data again.

[0039] In addition, the present invention comprises: a language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; a main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; a database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and a document-type server that stores learning training data necessary for learning training as content or learning data. The main server is configured to determine the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty based on the ability score evaluated for the user's pronunciation accuracy, sentence accuracy, correct answer status, and correct answer response time input through the language analysis terminal, and then provide the determined learning training data; and to update the user's ability score in real time, and then re-determine the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty by reflecting the updated ability score, and then provide the learning training data.

[0040] The precision speech recognition-based English learning support system according to the present invention processes speech spoken by a user in real time and provides customized English learning training to the user by providing appropriate feedback based on pronunciation accuracy and speed. In addition, by analyzing sentence comprehension and vocabulary level, it automatically recommends content of a difficulty level matching the learner's proficiency or provides feedback.

[0041] Furthermore, the present invention features software that performs a primary function of analyzing a user's pronunciation accuracy and sentence accuracy (or sentence level) in real time. It stores English audio input by the user and matches it with the words and sentences pronounced by the user to analyze pronunciation accuracy, sentence structure, and comprehension, and provides a training program tailored to the user's pronunciation level. This analysis is performed through an AI-based algorithm and is utilized to recommend appropriate learning content (learning data) that matches the learner's pronunciation level and sentence comprehension. Additionally, it evaluates the user's learning status in real time to track learning progress and provide necessary feedback.

[0042] Furthermore, the present invention analyzes the user's pronunciation accuracy, sentence comprehension, speaking speed, etc., to automatically provide learning content of different levels, and helps the learner to speak with pronunciation close to that of a native speaker. The present invention can provide a virtual space such as a metaverse, analyze the user's pronunciation level even in the virtual environment, and provide a function to provide real-time feedback on learning progress.

[0043] The present invention implements an AI analysis system that analyzes a user's voice in real time and compares pronunciation accuracy, grammatical accuracy, speaking speed, sentence comprehension, and native speaker rhythm with data stored in a DB.

[0044] In addition, the present invention analyzes the user's voice in real time to evaluate the user's pronunciation accuracy, speaking speed (English Speech Rate), native speaker rhythm, sentence comprehension, and vocabulary level, and provides customized learning training content based on this.

[0045] The objectives of the present invention are not limited to those mentioned above, and the learning content, which analyzes and provides pronunciation levels and sentence comprehension in real time to meet various user needs, can be adjusted and optimized to suit various educational environments in the future. Thus, the present invention achieves technological advancements that enhance the efficiency of English learning training and provides a customized learning experience tailored to the user's learning level.

[0046]

[0047] Figure 1 is an overall configuration diagram of a precision speech recognition-based English learning support system according to the present invention.

[0048] The English learning support system based on precise speech recognition according to the present invention comprises a language analysis terminal (100), a main server (200) that stores a user's voice and performs a command to provide feedback after analyzing English pronunciation, grammar, comprehension, etc. by comparing it with words or sentences pronounced by the user, a relational database management server (300) that includes learning data and user learning information, and a document server (400) that processes the user's learning data. The main server (200) analyzes the user's pronunciation and sentence comprehension in real time and automatically recommends customized English learning training content based on this.

[0049] The above language analysis terminal (100) receives language input from a user and analyzes it. In particular, the above language analysis terminal (100) analyzes pronunciation accuracy and sentence accuracy to analyze the user's level, and through this, receives the learning data required by the user from the above main server (200) and provides the learning data required for learning training.

[0050] The main server (200) encrypts the user's learning data and provides security technology capable of processing it in real time, in order to protect against the risk of hacking occurring during the transmission process, along with a content calling function between the relational database management server (300) and the document server (400).

[0051] In addition, according to the main technical features of the present invention, the main server (200) is configured to calculate problem difficulty and problem speed based on the ability score evaluated for the user's pronunciation accuracy and sentence accuracy input through the language analysis terminal, provide a learning training program, update the user's ability score in real time, adjust the problem difficulty and problem speed by reflecting the updated ability score, and then provide learning training data again.

[0052]

[0053] Figure 2 is a detailed configuration diagram of the main server in the English learning support system based on precise speech recognition according to the present invention.

[0054] The detailed configuration of the main server (200) is described as follows. The detailed configuration of the main server (200) according to the present embodiment includes an information collection unit (210), a learning analysis unit (220), a storage unit (230), a voice analysis matching unit (240), and a document information processing unit (250).

[0055] The above main server (200) includes an information collection unit (210) that analyzes the user's voice in real time and evaluates pronunciation accuracy, speed, native speaker rhythm, and sentence comprehension; a learning analysis unit (220) that deep learns the words and sentences pronounced by the user to analyze pronunciation accuracy, grammatical accuracy, sentence structure, etc., and provides training modules according to the learning progress; a storage unit (230) for storing the user's voice analysis data and providing real-time feedback; and a voice analysis matching unit (240) that compares and evaluates the user's pronunciation level, sentence comprehension, and learning status in real time, thereby accurately evaluating the level of pronunciation and sentences learned by the user.

[0056] In addition, the main server (200) manages training content through a document information processing unit (250) that documents training content based on the user's learning pattern.

[0057] More specifically, the configuration of the main server (200) is described in detail.

[0058] The above information collection unit (210) collects the voice of a word or sentence pronounced by the user and analyzes the pronunciation, accent (also called accent), rhythm, word level, etc. of the voice by performing calculus. Here, calculus means using spectrograms or frequency analysis.

[0059] Information regarding words or sentences entered by a user received through the above language analysis terminal (100) is collected in real time, and comparative analysis is performed based on native speaker voice data stored in the above storage unit (230).

[0060] The above learning analysis unit (220) deep learns the words and sentences pronounced by the user to analyze pronunciation accuracy, grammatical accuracy, sentence structure, etc.

[0061] In addition, the learning analysis unit (220) deep learns the words and sentences pronounced by the user to analyze pronunciation accuracy and grammar accuracy, and calculates the user's ability score based on the analysis results.

[0062] The above storage unit (230) stores user voice analysis data and real-time input voice data, and enables comparative analysis of the pronunciation and sentence comprehension level of the user's voice based thereon.

[0063] The voice analysis matching unit (250) compares the user's voice collected in real time with native speaker voice data stored in the server to analyze the matching of pronunciation accuracy, accent, sentence comprehension, rhythm, etc., and performs the function of recommending customized learning data (or learning content) based on this.

[0064] The above document information processing unit (250) processes the voice analysis and matching results into a documented form and generates learning materials to be provided to the user.

[0065] As described above, the language analysis function and matching function executed by the main server (200) perform the same function according to the specific description of the language analysis terminal (100) in Fig. 3 below, and the language analysis terminal (100) can be implemented independently through software installed in an offline state, and can provide analysis function, AI function, and feedback function together in real time in an online environment between the language analysis terminal (100) and the main server (200).

[0066] Meanwhile, the present invention includes a system that analyzes user pronunciation data through a deep learning model and processes the information in an encryption unit (260) to enhance storage and security.

[0067] The encryption unit (260) analyzes the input user's voice using deep learning and encrypts information such as pronunciation patterns, accents, and rhythms to process it securely. The encrypted data is exchanged with information stored in real-time within the main server (200) and includes a unique encryption system for comparing the user's pronunciation with the native speaker's pronunciation.

[0068]

[0069] Figure 3 is a detailed configuration diagram of a language analysis terminal in a precise speech recognition-based English learning support system according to the present invention.

[0070] The language analysis terminal (100) according to the present invention comprises a voice analysis unit (110) that analyzes pronunciation and sentence level input by a user, an analysis processing unit (120) that analyzes the user's pronunciation and sentence level information input in real time, compares it with learning data stored in the main server (200), and provides feedback for learning training, and a learning provision unit (130) that processes and provides learning data including pronunciation and sentence data required for learning training in real time.

[0071] A language analysis terminal (100) according to the present invention receives a user's voice input, analyzes the pronunciation level in real time, and collects learning data from the user based on the analyzed data.

[0072] The above language analysis terminal (100) compares the user's pronunciation with the native speaker's rhythm and evaluates the pronunciation accuracy and sentence comprehension by time-scaling in milliseconds. In addition, it analyzes the sentences and words pronounced by the user, compares them with data stored in a server to evaluate accuracy, speed, sentence comprehension, etc., and automatically recommends learning content that matches the results.

[0073] The voice analysis unit (110) collects voice input from the user in real time and analyzes it by voice pattern to identify pronunciation, accent, and sentence level.

[0074] The analysis processing unit (120) analyzes the user's real-time voice and compares it with native speaker pronunciation information stored on a server to analyze pitch, rhythm, accent, pronunciation accuracy, etc. The analyzed result is matched with the user's learning status and is used to recommend customized learning content.

[0075] Accordingly, the system is operated by analyzing the matching rate through the analysis processing unit (120) and providing feedback tailored to the user's learning level.

[0076] The above learning provision unit (130) provides learning data necessary for the user according to the analysis results, and provides learning content or a learning program.

[0077] A learning support system according to one embodiment of the present invention analyzes the accent, pronunciation accuracy, rhythm, and sentence comprehension of a word or sentence pronounced by a user in real time based on speech recognition, and automatically recommends content that can enhance the user's learning effect.

[0078] Through an algorithm that analyzes native speaker pronunciation, accent, rhythm, and word level for the words and sentences pronounced by the user, the user can gradually improve their pronunciation, grammar, and vocabulary. This analysis encourages the user to pronounce words accurately and construct sentences, and automatically provides appropriate learning content to maximize learning effectiveness.

[0079] Specifically, to explain the analysis function implemented in the learning support system of the present invention, the spectrogram applied as an analysis tool is a tool for visualizing sound or waves to understand their characteristics, and visualizes the features of waveforms and spectra as changes in frequency over time. In the waveform, changes in amplitude over time can be observed, and in the spectrum, changes in amplitude according to frequency can be confirmed.

[0080] Therefore, spectrograms can represent differences in amplitude along changes in the time and frequency axes as color intensity or concentration, helping to analyze users' speech patterns based on the surrounding environment. For example, differences in pronunciation due to ambient noise or weather changes can be visually identified.

[0081] A user's language patterns based on the surrounding environment can be identified by analyzing the emphasis and stress of speech. For example, when a user says "Hurry up," the emphasis pattern of the command may indicate an urgent situation. Normally, the ratio of the first to the second syllable may be 50:50, but in urgent situations, this ratio can change to 70:30. In relaxed situations, the ratio may be 40:60. The emphasis ratio can be calculated by analyzing the stress pattern of each syllable. Since the user's speaking speed and stress vary depending on the surrounding environment, this allows for the extraction of language patterns tailored to environmental changes. The formula for calculating the emphasis ratio is shown in Mathematical Formula 1 below.

[0082] [Mathematical Formula 1]

[0083]

[0084] Here, S1 is the stress on the first syllable (e.g., the "H" sound in "Hurry"), and S2 is the stress on the second syllable (e.g., the "urry" part of "Hurry").

[0085]

[0086] Information extraction through pitch and beat is based on changes in the user's pronunciation according to the surrounding environment. For example, when a user says "Play some music for a rainy day," the word "rainy" may have a descending pitch, and the pitch can generally become lower. As such, environmental factors such as weather or surrounding sounds can influence the user's speech and cause changes in pitch. Additionally, "rainy" and "day" receive stress depending on the environment, which can appear as strong frequency components in the spectrogram. Frequency levels are linked to the word level and can reflect changes in the environment. For example, the formula for calculating the frequency of "rainy" is as shown in Equation 2 below.

[0087] [Mathematical Formula 2]

[0088]

[0089] Here, is the fundamental pitch frequency, means an increase in frequency due to the emphasized syllable.

[0090] Through calculus-based modeling, language patterns based on the surrounding environment can be extracted by analyzing the user's speaking speed, linking (connected pronunciations), and intervals between words. For example, when a user says "I feel great," a smooth pitch curve may appear, and speaking speed or inconsistency may slow down depending on environmental changes. Speaking speed can be analyzed based on time per syllable (the ratio of time to the number of syllables). Additionally, the connection status between words can be identified through linking analysis. Speaking speed R can be calculated as shown in the following mathematical formula 3.

[0091] [Mathematical Formula 3]

[0092]

[0093] Here, n is the number of syllables, and t is the total time taken to speak.

[0094] Tone continuity evaluates the transition probability between consecutive pronunciations to reflect whether the connection between words is smooth or if there is a temporary pause.

[0095]

[0096] (Extraction of pronunciation characteristics through accent intensity analysis)

[0097] Accent is an important element in native pronunciation, a pronunciation style that places emphasis on specific words or syllables. By analyzing the intensity of the accent, it is possible to evaluate how a user's pronunciation differs from that of a native speaker. For example, when a user speaks an English sentence, the intensity of the accent can be measured by analyzing which parts are more prominent or weaker in stress. This analysis makes it useful for evaluating a user's pronunciation patterns regardless of their emotional state.

[0098] Accent intensity can be calculated through syllable-by-syllable stress and frequency analysis. For example, it includes a function that measures the frequency difference of stressed syllables and quantifies the accent intensity based on this. This analysis can provide a visualization of where the accent appears strongly or weakly when the user pronounces a specific word or sentence. The basic formula for calculating accent intensity is as shown in Equation 4 below.

[0099] [Mathematical Formula 4]

[0100]

[0101] Here, S1 is the stress on the first syllable (e.g., the "H" sound in "Hurry"), and S2 is the stress on the second syllable (e.g., the "urry" part of "Hurry").

[0102] Through this analysis, users can learn the intensity of their accent and correct weak points in their pronunciation to improve it into a more natural pronunciation.

[0103]

[0104] (Analysis of pronunciation patterns based on accent intensity)

[0105] The intensity of an accent significantly influences how a user pronounces a sentence. For example, when a user pronounces the sentence "Play some music for a rainy day," words like "rainy" exhibit pronounced accent intensity in the stressed syllables. In this process, frequency differences occur in the syllables where stress differs, allowing the intensity of the accent to be extracted. Through this process, the stress patterns and accent changes of each word are tracked via frequency analysis, and how the intensity of the accent changes according to the surrounding environment is identified. For instance, when a user pronounces "rainy," frequency differences in the stressed syllables can be observed, enabling a visual understanding of changes in accent intensity.

[0106] The formula for calculating the intensity of the accent is the same as the mathematical formula 4 described above.

[0107] The frequency change due to emphasis is calculated using the mathematical formula 2 described above.

[0108] In this way, it is a function that helps achieve natural pronunciation by analyzing the effect of accent intensity on pronunciation.

[0109]

[0110] (Pronunciation correction through accent intensity-based speech analysis)

[0111] Through calculus-based modeling, the intensity of accent can be quantified, and changes in stress and pitch can be evaluated by analyzing the user's pronunciation patterns. For example, when a user speaks a sentence such as "I feel great," the intensity of the accent on specific words can be tracked in real time, and the stress of the pronunciation can be corrected based on this data.

[0112] Accent intensity varies depending on speaking speed and linking analysis, thereby inducing natural pronunciation connections. For example, by analyzing the linking and accent intensity between words, it is possible to guide pronunciation to be connected naturally and consistently. Pronunciation speed and linking analysis based on accent intensity are processed in the following manner.

[0113] Speaking speed R can be calculated as shown in mathematical formula 3 above.

[0114] In addition, it can provide advice to the user for pronunciation improvement by providing real-time feedback on accent intensity and linking sounds. For example, it can suggest words that require more emphasis for parts where the accent intensity is insufficient, or provide feedback on parts where pronunciation is inconsistent.

[0115] In addition, as another major technical feature of the present invention, the learning analysis unit (220) calculates the difficulty of the problem and adjusts the problem speed when providing learning data according to the learning progress.

[0116] Problem difficulty calculation and problem speed adjustment are implemented through the following mathematical formulas 5 and 6.

[0117] [Mathematical Formula 5]

[0118]

[0119] Here, D problem The final difficulty of the problem, D basic is the basic difficulty level of the problem, is the difficulty adjustment coefficient.

[0120] If the user's ability score is low, the difficulty of the problem decreases, and if the ability score is high, the difficulty of the problem increases.

[0121] [Mathematical Formula 6]

[0122]

[0123] Here, v problem is the speed of the problem, v baseis the basic speed of the problem, is the speed adjustment coefficient.

[0124] In addition, the present invention updates the ability score by reflecting user feedback and performs difficulty adjustment based on this. The feedback includes whether the answer is correct, response time, and pronunciation accuracy.

[0125] The calculation for determining the ability score by reflecting the feedback is calculated using the following mathematical formula 7.

[0126] [Mathematical Formula 7]

[0127]

[0128] Here, S ability,new is the updated ability score, τ is the feedback reflection coefficient (0 < τ < 1), C correct , T response , P pronunciation These correspond to the correct answer, response time, and pronunciation accuracy, respectively.

[0129] As such, the present invention applies an ability score update algorithm based on problem difficulty, problem speed, and feedback reflection to analyze the user's learning data under optimized conditions and provide customized learning data.

[0130] Also, speed ( ), pronunciation difficulty( ), vocabulary difficulty( ) can be adjusted. The difficulty level for each ability score can be calculated using the following mathematical formula 8.

[0131] [Mathematical Formula 8]

[0132]

[0133] Each difficulty function is defined by adjusting the difficulty based on the ability score. For example, is a function that speeds up if the user has high ability and slows down if the user has low ability, and is calculated through the mathematical formula 9 below.

[0134] [Mathematical Formula 9]

[0135]

[0136] Here, If it is 50 or higher, the speed increases, and if it is lower, the speed decreases.

[0137] As such, the present invention applies an ability score update algorithm based on problem difficulty, problem speed, and feedback reflection to analyze the user's learning data under optimized conditions and provide customized learning data.

[0138]

[0139] The language analysis terminal (100) according to the present invention records and analyzes a user's voice in real time and transmits it to the main server (200). This device extracts information regarding frequency by performing calculus on the user's voice, and the voice information analyzed in this way is used to assist in analyzing the learner's pronunciation and to provide feedback necessary for pronunciation correction. In addition, through this, the learner can identify their pronunciation level and improve necessary parts.

[0140] The user's voice recording and storage proceed in real time, and the main server (200) performs voice recognition based on this. Through this analysis, not only the user's recording status but also their language ability can be evaluated. For example, by analyzing how the user pronounces English, feedback can be provided to correct stress patterns or improve irregular pronunciation.

[0141] Therefore, the present invention can analyze the user's pronunciation accuracy and emotional state, and in this process, provide real-time pronunciation correction and language proficiency evaluation necessary for English education. This system supports learners in speaking natural English and contributes to the improvement of their language abilities.

[0142] The main server (200) stores the user's voice and performs a command to automatically recommend content that matches the user's pronunciation accuracy, sentence comprehension, and learning pattern by comparing and analyzing the native speaker's accent, pronunciation, rhythm, word level, and sentence comprehension based on the words or sentences used by the user.

[0143] In addition, the main server (200) analyzes the user's voice data in real time and compares various elements such as the pronunciation, accent, rhythm, and sentence comprehension of the voice with native speaker standard information stored on the server to track the user's condition and recommend customized content accordingly.

[0144] The main server (200) according to the present embodiment protects user data by encrypting it with the same pronunciation analysis algorithm to prevent hacking risks that may occur during the process of calling and transmitting content between the relational database management server (200) and the document server (400).

[0145] The above database management server (300) manages the user's pronunciation characteristics, accent, rhythm, sentence comprehension, and content list. That is, it stores data according to pronunciation characteristics, accent, rhythm, sentence comprehension, etc.

[0146] The above document-type server (400) stores analyzed data and stores various content related to the user's learning progress status. The content referred to here is user-customized English learning content, which provides learning data stored according to the user's level.

[0147]

[0148] The language analysis terminal (100) according to the present invention records and analyzes a user's voice in real time and transmits it to the main server (200). This device extracts information regarding frequency by performing calculus on the user's voice, and the voice information analyzed in this way is used to assist in analyzing the learner's pronunciation and to provide feedback necessary for pronunciation correction. In addition, through this, the learner can identify their pronunciation level and improve necessary parts.

[0149] The user's voice recording and storage proceed in real time, and the main server (200) performs voice recognition based on this. Through this analysis, not only the user's recording status but also their language ability can be evaluated. For example, by analyzing how the user pronounces English, feedback can be provided to correct stress patterns or improve irregular pronunciation.

[0150] Therefore, the present invention can analyze the user's pronunciation accuracy and emotional state, and in this process, provide real-time pronunciation correction and language proficiency evaluation necessary for English education. This system supports learners in speaking natural English and contributes to the improvement of their language abilities.

[0151] The main server (200) stores the user's voice and performs a command to automatically recommend content that matches the user's pronunciation accuracy, sentence comprehension, and learning pattern by comparing and analyzing the native speaker's accent, pronunciation, rhythm, word level, and sentence comprehension based on the words or sentences used by the user.

[0152] In addition, the main server (200) analyzes the user's voice data in real time and compares various elements such as the pronunciation, accent, rhythm, and sentence comprehension of the voice with native speaker standard information stored on the server to track the user's condition and recommend customized content accordingly.

[0153] The main server (200) according to the present embodiment protects user data by encrypting it with the same pronunciation analysis algorithm to prevent hacking risks that may occur during the process of calling and transmitting content between the relational database management server (200) and the document server (400).

[0154] The above database management server (300) manages the user's pronunciation characteristics, accent, rhythm, sentence comprehension, and content list. That is, it stores data according to pronunciation characteristics, accent, rhythm, sentence comprehension, etc.

[0155] The above document-type server (400) stores analyzed data and stores various content related to the user's learning progress status. The content referred to here is user-customized English learning content, which provides learning data stored according to the user's level.

[0156]

[0157] The present invention, configured as described above, is, above all, an AI-based system that accurately analyzes and evaluates a user's pronunciation and sentence comprehension, thereby providing learners (users) with the opportunity to continuously improve their pronunciation and sentence structure to the level of a native speaker. Furthermore, by providing various learning modes and content, it is highly useful for learners who wish to improve their English conversation or pronunciation.

[0158] Furthermore, the present invention processes and analyzes voice data pronounced by a learner in real time to provide rapid feedback based on the voice input by the user. Consequently, learners can receive real-time feedback on pronunciation accuracy or speaking speed and immediately access customized learning content to improve their performance. This enables more effective pronunciation training than traditional English learning methods and is particularly advantageous for learners struggling with pronunciation correction.

[0159] Furthermore, because learners can analyze their pronunciation levels and receive feedback in real time, this system serves as a powerful learning tool for effective pronunciation correction and improving English sentence comprehension. Users receive specific evaluations of their pronunciation and are provided with more detailed learning content and training as needed.

[0160] Figure 4 is an example of measuring ability scores in a speech recognition-based English learning support system that analyzes language ability according to the present invention.

[0161] As explained above, the present invention measures the user's ability score in real time and thereby provides customized learning data optimized for the user's current English ability.

[0162] To this end, the main server (200) measures the ability score for English voice data input by the user in real time and provides learning data (e.g., learning training questions) based on the ability score. The ability score is measured based on the correct answer to the question, response time, and pronunciation accuracy, and the learning data is controlled in real time based on the user's ability score measured in real time.

[0163] The present invention, configured in this manner, can achieve very high learning efficiency by precisely analyzing the user's speech information and providing real-time customized learning data through this analysis.

[0164] The present invention, configured as described above, is, above all, an AI-based system that accurately analyzes and evaluates a user's pronunciation and sentence comprehension, thereby providing learners (users) with the opportunity to continuously improve their pronunciation and sentence structure to the level of a native speaker. Furthermore, by providing various learning modes and content, it is highly useful for learners who wish to improve their English conversation or pronunciation.

[0165]

[0166] Furthermore, the present invention processes and analyzes voice data pronounced by a learner in real time to provide rapid feedback based on the voice input by the user. Consequently, learners can receive real-time feedback on pronunciation accuracy or speaking speed and immediately access customized learning content to improve their performance. This enables more effective pronunciation training than traditional English learning methods and is particularly advantageous for learners struggling with pronunciation correction.

[0167] Furthermore, because learners can analyze their pronunciation levels and receive feedback in real time, this system serves as a powerful learning tool for effective pronunciation correction and improving English sentence comprehension. Users receive specific evaluations of their pronunciation and are provided with more detailed learning content and training as needed.

[0168]

[0169] The present invention, configured as described above, analyzes the user's voice in real time to evaluate the pronunciation level, speaking speed, native speaker rhythm, sentence comprehension, and vocabulary level, and based on this, provides a customized English learning training program for the user. This allows for the provision of accurate feedback on the words or sentences pronounced by the user and the automatic recommendation of necessary training content, thereby maximizing the effectiveness of English learning.

[0170] Furthermore, the present invention analyzes a user's voice data in real time to evaluate pronunciation accuracy, speaking speed, and sentence comprehension, and automatically recommends appropriate training content accordingly. Through this, users can efficiently improve their English pronunciation and grammar levels, and there is the advantage of enabling customized education tailored to the individual characteristics of the learner.

[0171] In addition, the present invention has the effect of highly satisfying learning efficiency by measuring the user's ability score in real time, providing learning problems optimized for the user based on the ability score, and managing the learning progress.

[0172] Although preferred embodiments have been described and illustrated to illustrate the principles of the present invention, the present invention is not limited to the configuration and operation as depicted and described. Rather, those skilled in the art will understand that numerous changes and modifications to the present invention are possible without departing from the spirit and scope of the appended claims. Accordingly, all such appropriate changes and modifications and equivalents should be deemed to be within the scope of the present invention.

[0173] [Explanation of the symbol]

[0174] 100 : Language analysis terminal

[0175] 110 : Voice Analysis Unit

[0176] 120 : Analysis processing unit

[0177] 130 : Learning Provider

[0178] 200 : Main Server

[0179] 210: Information Gathering Department

[0180] 220 : Learning Analytics

[0181] 230 : Storage unit

[0182] 240 : Voice Analysis Matching Unit

[0183] 250 : Document Information Processing Unit

[0184] 300: Database Management Server

[0185] 400 : Document-based server

Claims

1. A language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; A main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; A database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and It is configured to include a document-type server that stores learning training data required for learning training as content or learning data, and A precision speech recognition-based English learning support system configured such that the main server calculates problem difficulty and problem speed based on ability scores evaluated for the user's pronunciation accuracy and sentence accuracy input through the language analysis terminal, provides a learning training program, updates the user's ability score in real time, adjusts the problem difficulty and problem speed by reflecting the updated ability score, and then provides learning training data again.

2. In Paragraph 1, the main server is, A precision speech recognition-based English learning support system configured to include a function that encrypts and processes user voice data to prevent hacking risks that may occur during the content retrieval and transmission process between the database management server and the document-type server, and securely transmits information necessary for learning training.

3. In Paragraph 1, the main server is, An information collection unit that collects real-time voice or recorded voice data of the user received through the language analysis terminal and analyzes pronunciation and sentence level; A learning analysis unit that deep learns words and sentences pronounced by a user to analyze pronunciation accuracy and grammar accuracy, and calculates the user's ability score based on the analysis results; A storage unit that stores the user's pronunciation accuracy, sentence accuracy, and ability score analyzed by the above-mentioned learning analysis unit; A voice analysis matching unit that analyzes the user's English voice in real time to provide feedback tailored to pronunciation and sentence level, and recommends learning training data necessary for learning training; and A precise speech recognition-based English learning support system comprising: a document information processing unit that processes learning materials regarding the user's pronunciation accuracy and sentence accuracy in document format according to the analysis results analyzed by the learning analysis unit above.

4. In Paragraph 1, the language analysis terminal, A voice analysis unit that receives user voice data in real time and analyzes pronunciation accuracy and sentence accuracy; and A precision speech recognition-based English learning support system comprising: an analysis processing unit that analyzes the user's pronunciation and sentence accuracy in real time and provides customized learning training content based on learning data stored on a main server.

5. In Paragraph 3, the above-mentioned learning analysis unit, A speech recognition-based English learning support system that analyzes language ability by calculating an ability score through the following mathematical formulas based on user voice data collected by the information collection unit, determining whether the question is correct, response time, and pronunciation accuracy. [Mathematical Formula] (Here, S ability,new is the updated ability score, τ is the feedback reflection coefficient (0 < τ < 1), C correct , T response , P pronunciation (Respectively, correct answer, response time, and pronunciation accuracy) 6. A language analysis terminal that receives English voice data spoken by a user, analyzes pronunciation accuracy and sentence accuracy and transmits it online, and receives and outputs learning training data necessary for English learning based on the input English voice data; A main server that stores the user's English voice data received from the language analysis terminal, analyzes the user's pronunciation and sentence level to evaluate the user's pronunciation level and sentence construction ability score, and provides English learning training data based thereon; A database management server that stores learning data including the user's English pronunciation, sentence level, and learning progress information; and It is configured to include a document-type server that stores learning training data required for learning training as content or learning data, and A speech recognition-based English learning support system for analyzing language ability, configured such that the main server determines the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty based on the ability score evaluated for the user’s pronunciation accuracy, sentence accuracy, correct answer status, and correct answer response time input through the language analysis terminal, and then provides the determined learning training data, updates the user’s ability score in real time, and then re-determines the problem difficulty, problem speed, pronunciation difficulty, and vocabulary difficulty by reflecting the updated ability score, and then provides the learning training data.

Images