Intelligent english teaching interactive system and method based on speech recognition

Abstract

The application relates to an intelligent English teaching interactive system and method based on voice recognition, and relates to the technical field of voice recognition, and comprises the following steps: acquiring a voice signal and a dialogue context, analyzing an acoustic candidate list, a phoneme acoustic confidence and a semantic context representation; querying the confusion prior probability of each candidate based on a native language pronunciation confusion knowledge graph; fusing acoustic probability, semantic prior and confusion prior to calculate posterior probability, and taking the highest one as an intention recognition result; comparing an intention standard phoneme with an optimal output phoneme, and diagnosing pronunciation bias in combination with phoneme confidence; calculating reasoning confidence according to the posterior probability, performing multi-level interaction disambiguation in combination with bias diagnosis, generating teaching feedback and outputting. The application solves the technical problem that an existing English teaching voice recognition system lacks targeted processing ability for pronunciation deviation of non-native language learners caused by mother tongue interference, so that the system cannot correctly understand the real expression intention of the learners.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to an intelligent English teaching interactive system and method based on speech recognition. Background Technology

[0002] Speech recognition technology has made significant progress in recent years. A new generation of systems, represented by end-to-end deep learning models, has achieved near-native-speaker proficiency in standard speech transcription tasks. In the field of intelligent English teaching, existing systems are beginning to apply speech recognition technology to pronunciation assessment and interactive teaching, helping learners improve their oral skills through conversational exercises. Some products have already integrated basic pronunciation assessment functions.

[0003] In existing technologies, speech recognition systems and pronunciation assessment systems typically operate independently in English oral teaching dialogue scenarios. Speech recognition systems are trained using native speaker speech data. When non-native speakers produce speech with systematic accent shifts, the system treats it as random noise and handles it with tolerance, mapping it to the semantically most probable text output. This mechanism ensures semantic intelligibility but simultaneously masks pronunciation errors; the original pronunciation shift data is lost during transcription, making downstream pronunciation correction impossible. Pronunciation assessment systems calculate acoustic confidence scores for each phoneme through forced alignment, marking low-confidence phonemes as potential pronunciation errors. However, when the transcribed text itself contains errors due to accent shifts, the target sequence for forced alignment also becomes incorrect, resulting in pronunciation assessment based on errors. The two systems operate independently, lacking a unified mechanism to accurately understand learners' intentions while precisely locating pronunciation errors, making it difficult to balance dialogue fluency and error correction accuracy in teaching interactions. Summary of the Invention

[0004] This application provides an intelligent English teaching interaction system and method based on speech recognition. It improves upon existing technologies where, in English oral teaching dialogue scenarios, when non-native speakers pronounce English with a native accent, the speech recognition system treats the pronunciation deviations as random noise and ignores them. While maintaining basic semantic comprehensibility, this loses detailed information about pronunciation errors. Furthermore, the pronunciation assessment system only provides scores in isolation, failing to consider the learner's intended meaning within the context of the dialogue. This disconnect between the two functions makes it difficult to accurately diagnose and specifically correct learners' pronunciation problems while maintaining fluency in interactive teaching scenarios.

[0005] This application discloses the following technical solution: In a first aspect, this application provides an intelligent English teaching interactive system based on speech recognition, the system comprising: Speech analysis module: used to acquire learners' speech signals and corresponding dialogue context information in teaching dialogue scenarios, and to analyze and acquire acoustic recognition results, phoneme acoustic confidence information and semantic context representation. The graph query module is used to query and obtain the prior probability of knowledge graph confusion corresponding to each transcription candidate in the acoustic recognition result based on the pre-built pronunciation confusion knowledge graph for learners' native language. The intent reasoning module is used to combine the acoustic recognition result, the semantic context representation and the prior probability of confusion of the knowledge graph to calculate the posterior probability of each transcription candidate, and determine the transcription candidate with the highest posterior probability as the intent recognition result; The error diagnosis module is used to extract the standard phoneme sequence based on the intent recognition result and compare it phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result, and generate pronunciation error diagnosis results by combining the phoneme acoustic confidence information. The disambiguation feedback module is used to calculate the inference confidence based on the posterior probability of each transcription candidate, and to perform multi-level interactive disambiguation in conjunction with the pronunciation error diagnosis results, generating interactive teaching feedback and outputting it.

[0006] Secondly, this application provides an intelligent English teaching interaction method based on speech recognition, the method comprising: Acquire learners’ speech signals and corresponding dialogue context information in teaching dialogue scenarios, and analyze the acoustic recognition results, phoneme acoustic confidence information and semantic context representation accordingly. Based on the pre-built pronunciation confusion knowledge graph for learners' native language, the prior probability of knowledge graph confusion for each transcription candidate in the acoustic recognition result is obtained by querying. Combining the acoustic recognition results, the semantic context representation, and the prior probability of confusion in the knowledge graph, the posterior probability of each transcription candidate is calculated, and the transcription candidate with the highest posterior probability is determined as the intent recognition result; Based on the intent recognition result, the standard phoneme sequence is extracted and compared phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result. Combined with the phoneme acoustic confidence information, a pronunciation error diagnosis result is generated. The inference confidence is calculated based on the posterior probability of each transcription candidate, and multi-level interactive disambiguation is performed in conjunction with the pronunciation error diagnosis results to generate and output interactive teaching feedback.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: The technical solution of this application provides an intelligent English teaching interaction system based on speech recognition. First, by acquiring the learner's speech signals and dialogue context information in the teaching dialogue scenario, it simultaneously performs three parallel processing steps: acoustic recognition, phoneme-level confidence calculation, and semantic context extraction. The output includes acoustic recognition results containing transcription candidates and acoustic probabilities, phoneme acoustic confidence information containing the confidence scores of each phoneme position and the optimal produced phoneme, and semantic context representation containing semantic vectors and expected response range. This integrates the originally isolated speech signals, pronunciation details, and dialogue context into multi-dimensional input information required for subsequent joint reasoning, solving the problem that traditional systems only output a single optimal transcribed text and lose pronunciation offset information and contextual reference information.

[0008] Furthermore, by pre-constructing a pronunciation confusion knowledge graph for learners' native languages, based on the confusion pattern skeleton identified in native language transfer research, and after automatically extracting confusion pairs and statistically analyzing their frequencies from learners' speech sample data, a weighted fusion expansion is performed to construct a knowledge graph containing phoneme confusion edges and near-phonetic word pairs. The prior probability of confusion in the knowledge graph corresponding to each transcription candidate is obtained by querying, introducing structured prior knowledge of known pronunciation confusion rules for specific native language groups in the second language acquisition field into the system. This solves the problem that traditional speech recognition systems lack targeted perception methods for systematic pronunciation shifts in non-native language learners and cannot utilize known native language transfer rules to assist in intent judgment.

[0009] Furthermore, by multiplying and normalizing the acoustic probability in the acoustic recognition result, the semantic prior probability in the semantic context representation, and the knowledge graph confusion prior probability, the posterior probability of each transcription candidate is calculated. The transcription candidate with the highest posterior probability is determined as the intent recognition result. This achieves joint reasoning of acoustic evidence, semantic context, and learner's native language pronunciation rules under a unified probabilistic framework. It solves the problem that relying solely on acoustic models can easily lead to misjudgment of intent when there is accent shift, while relying solely on semantics or priors cannot independently complete accurate judgment.

[0010] Furthermore, by aligning and comparing the standard phoneme sequence corresponding to the intent recognition result with the optimal output phoneme sequence in the acoustic recognition result phoneme by phoneme, the position of each phoneme is determined to be consistent or biased. Combined with the acoustic confidence score of that position, a pronunciation error diagnosis result containing standard phonemes, biased phonemes, and bias confidence is generated. This achieves synchronous output from intent understanding to pronunciation error localization, solving the problems of separation between speech recognition error tolerance mechanism and pronunciation evaluation in traditional systems, and the error in the pronunciation diagnosis target benchmark caused by intent recognition error.

[0011] Finally, by calculating the inference confidence level and determining the confidence level based on the ratio of the optimal to the suboptimal value of the posterior probability of each transcription candidate, and by combining the pervasiveness of mother tongue transfer confusion corresponding to each error phoneme, the matching degree of the current teaching focus, and the learner's historical error frequency, a weighted teaching value score is calculated. Based on the confidence level and the teaching value score, the corresponding level of strategy among implicit paraphrasing disambiguation, explicit confirmation disambiguation, or explicit teaching disambiguation is selected to perform disambiguation processing and generate interactive teaching feedback. This achieves adaptive matching between inference uncertainty and teaching intervention intensity, transforming the intention disambiguation process itself into a pronunciation training activity with independent teaching value. This solves the problem that traditional systems abandon interaction when encountering ambiguity because they cannot confirm the intention or cannot generate teaching value in ambiguity resolution.

[0012] In summary, this application employs a three-pronged parallel processing approach: acoustic recognition, phoneme-level confidence calculation, and semantic context extraction of learners' speech signals. It utilizes a pronunciation confusion knowledge graph to provide prior knowledge of first language transfer, integrates acoustic, semantic, and graph-based prior knowledge within a Bayesian joint inference framework to determine intent, and simultaneously outputs pronunciation error diagnoses. Furthermore, it selects corresponding levels of disambiguation strategies based on inference confidence and the teaching value of the errors to generate interactive teaching feedback. This effectively solves the technical problem in existing English teaching systems where speech recognition tolerance and pronunciation correction are mutually contradictory, making it impossible to accurately locate pronunciation errors while accurately understanding intent. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of an intelligent English teaching interactive system based on speech recognition, provided as an embodiment of this application.

[0015] Figure 2 This application provides an embodiment of an adaptive disambiguation and graph update flowchart for an intelligent English teaching interaction method based on speech recognition.

[0016] Figure 3 This is a flowchart illustrating an intelligent English teaching interaction method based on speech recognition, provided as an embodiment of this application.

[0017] In the attached diagram, Figure 1 The components represented by each number are explained as follows: Speech Analysis Module 11, Graph Query Module 12, Intent Reasoning Module 13, Error Diagnosis Module 14, and Disambiguation Feedback Module 15. Detailed Implementation

[0018] This application provides an intelligent English teaching interaction system and method based on speech recognition, which addresses the technical problems in existing technologies where, in teaching dialogue scenarios, when learners' pronunciation deviates from the standard due to the influence of their mother tongue transfer, the system lacks the ability to comprehensively utilize the semantic context of the current dialogue and the known pronunciation confusion rules of the native language group for collaborative judgment. This results in the inability to accurately identify the learner's true expression when the acoustic signal is ambiguous and multiple possible intentions exist. Furthermore, it fails to effectively utilize the interactive advantage of the system's ability to ask learners questions for confirmation in teaching scenarios, thus failing to transform the intention disambiguation process into valuable teaching activities. Consequently, the system frequently experiences intention misjudgment and pronunciation error localization failures in complex teaching scenarios.

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that the numerical values ​​in the embodiments are for illustrative purposes only and do not constitute a limitation on this application.

[0020] Example 1, as shown in the appendix Figure 1 As shown, this application provides an intelligent English teaching interactive system based on speech recognition, the system comprising the following modules: Speech analysis module 11: used to acquire learners’ speech signals and corresponding dialogue context information in teaching dialogue scenarios, and to analyze and acquire acoustic recognition results, phoneme acoustic confidence information and semantic context representation.

[0021] In this embodiment, in an English oral teaching dialogue scenario, when a learner speaks English with a native accent, the system needs to simultaneously acquire the learner's speech signal and the contextual information of the current dialogue, and extract three types of information from them: acoustic transcription candidates, phoneme-level pronunciation confidence, and semantic-level contextual features. Traditional speech recognition systems only output a single optimal transcribed text, losing detailed information about pronunciation deviations and reference information about the dialogue context, and cannot provide sufficient multi-dimensional input for subsequent intent reasoning and pronunciation diagnosis.

[0022] In the system provided in this application embodiment, the voice analysis module is specifically used for: Speech recognition processing is performed on the speech signal to output multiple transcription candidates and their corresponding acoustic probabilities, constituting the acoustic recognition result. Using the transcription candidate with the highest acoustic probability as a benchmark, phoneme-level forced alignment is performed on the speech signal to determine the target phoneme sequence. The acoustic confidence score and optimal output phoneme at each target phoneme position in the target phoneme sequence are calculated, constituting the phoneme acoustic confidence information. Based on the dialogue context information, semantic vectors are extracted through a pre-trained semantic coding model and combined with the current teaching scenario to generate the expected response range, constituting the semantic context representation. Detailed explanation follows: In this embodiment, transcription candidates refer to multiple possible text sequences and their corresponding acoustic probabilities output by the speech recognition engine after decoding the input speech signal. Typically, N best candidates are selected to form an N-best candidate list. Each candidate is a text word sequence and does not contain phoneme information. The acoustic probability refers to the log probability value assigned by the speech recognition engine to each transcription candidate, reflecting the degree of matching between the candidate and the input speech signal in terms of acoustic features. The higher the acoustic probability, the more the acoustic evidence supports the candidate.

[0023] Phoneme-level forced alignment refers to using the optimal transcribed phoneme sequence as the target sequence to locate the start and end time boundaries of each target phoneme on the original speech signal, ensuring that each phoneme in the text precisely corresponds to its acoustic feature segment in the audio. The target phoneme sequence refers to the standard English phoneme sequence obtained by querying a pronunciation dictionary from the optimal transcribed text.

[0024] The phoneme acoustic confidence information is a matrix consisting of the acoustic confidence score for each target phoneme location and the most probable phoneme identified by the acoustic model. The acoustic confidence score uses the GOP (Goodness of Pronunciation) score, which is calculated by taking the logarithm of the ratio of the target phoneme's posterior probability in the acoustic model to the highest posterior probability among all phonemes. A higher score indicates that the actual pronunciation of that phoneme location is closer to the standard pronunciation. The most probable phoneme identified by the acoustic model refers to the phoneme with the highest posterior probability in a given acoustic feature segment; that is, the optimal phoneme produced by the learner in actual pronunciation.

[0025] Semantic encoding models refer to pre-trained language representation models that can encode variable-length dialogue history text into fixed-dimensional semantic vectors, extracting the semantic themes and intentions of the dialogue. The expected response range refers to the set of words a learner might use in the current context, based on the current dialogue history and the type of teaching scenario. In teaching-based dialogue scenarios, the expected response range can be directly determined by the dialogue script template; the system holds a complete dialogue script and can accurately predict the vocabulary students might use. In topic discussion scenarios, the expected response range is determined by the current topic keywords and the syllabus vocabulary. In free-flowing dialogue scenarios, the expected response range relies on the conditional prediction of the language model and is relatively broad.

[0026] In this module, to simultaneously extract acoustic, phonemic, and semantic information from the learner's speech signal, a pre-trained end-to-end speech recognition model is first used to decode the input speech signal, generating an N-best candidate list containing multiple transcription candidates and their acoustic probabilities, which constitutes the acoustic recognition result. The speech recognition model employs an attention-based encoder-decoder architecture, encoding the acoustic features of the input speech into a latent representation and then decoding it word by word. At each decoding step, the probability distribution of each word in the vocabulary is output, and the N candidate sequences with the highest probabilities are retained through beam search.

[0027] Then, using the transcription candidate with the highest acoustic probability as a benchmark, the standard phoneme sequence corresponding to the candidate is obtained by querying the pronunciation dictionary. A forced alignment tool is then used to locate the start and end time positions of each target phoneme on the original speech signal, ensuring a precise correspondence between the text and audio at the phoneme level. Next, for each target phoneme position, the GOP score at that position is calculated as an acoustic confidence score, and the phoneme that the acoustic model considers most likely at that position is recorded as the optimal output phoneme. The target phoneme, GOP score, optimal output phoneme, and the logarithmic probability value of the optimal output phoneme for each phoneme position are combined to construct the phoneme acoustic confidence information.

[0028] Simultaneously, based on the current dialogue context, the utterance texts from each round in the dialogue history are concatenated chronologically and input into a pre-trained semantic encoding model to extract semantic vectors. The generation method for the expected response range is determined according to the current teaching scenario type: for a teaching-based dialogue scenario, expected response keywords are extracted from the dialogue script template to construct the expected response range; for a topic discussion scenario, the expected response range is constructed using current topic keywords combined with the teaching syllabus vocabulary; for a free dialogue scenario, the expected response range is constructed by using the semantic vectors as conditions to predict the words with the highest probabilities from the language model. The semantic vectors and expected response ranges together constitute a semantic context representation. These three processing methods are executed in parallel, and the output results are aggregated for use by subsequent modules.

[0029] In summary, this module integrates the originally isolated speech signals, pronunciation details, and dialogue context into multi-dimensional input information required for subsequent joint reasoning by simultaneously performing acoustic recognition, phoneme-level confidence calculation, and semantic context extraction on the learner's speech signals. This solves the problem of traditional systems only outputting a single optimal transcribed text and losing pronunciation offset information and contextual reference information.

[0030] Graph query module 12: Used to query and obtain the prior probability of knowledge graph confusion corresponding to each transcription candidate in the acoustic recognition result based on the pre-built pronunciation confusion knowledge graph for learners' native language.

[0031] In this embodiment, based on the obtained acoustic recognition results and phoneme acoustic confidence information, and considering the systematic and regular pronunciation deviations of non-native language learners, the pronunciation confusion patterns of specific native language groups can be pre-summarized by applied linguistics research. To provide additional reference for intent discrimination when acoustic evidence is blurred due to accent shifts, this prior knowledge needs to be structured into a queryable knowledge graph, and the prior probability of confusion corresponding to each transcription candidate needs to be calculated accordingly. This addresses the problem that traditional speech recognition systems rely solely on acoustic and language models for transcription, lacking structured perception and utilization methods for known pronunciation confusion patterns of specific native language groups.

[0032] In the system provided in this application embodiment, the pre-construction of the pronunciation confusion knowledge graph in the graph query module includes: Based on literature on mother tongue transfer research, this study analyzes the English phoneme confusion patterns corresponding to learners' native languages ​​and establishes a skeleton graph containing phoneme confusion edges and near-homophone pairs. Each phoneme confusion edge represents the conditional probability that a target phoneme is produced as another phoneme in a specific phonemic context, and each near-homophone pair represents the phonological similarity between two words under a specific confusion pattern. Based on learners' speech sample data, multiple transcription candidates are generated for each speech segment, and phoneme-level acoustic confidence scores are calculated. Based on the phoneme-level acoustic confidence scores, confusion patterns are extracted from the phoneme differences between multiple transcription candidates corresponding to low-confidence phoneme segments, and the confusion probability is automatically calculated after frequency statistics. The probability values ​​of existing edges in the skeleton graph are weighted and fused with the confusion probability for updating, and the skeleton graph is expanded using data-driven methods to obtain the pronunciation confusion knowledge graph. Detailed explanation follows: In this embodiment, the pronunciation confusion knowledge graph is a graph database that structurally describes the pronunciation error patterns of a specific native speaker group in English. It consists of English target phoneme nodes, pronunciation output nodes, high-frequency teaching word nodes, and near-homophone pair nodes as entities, and phoneme confusion edges and near-homophone pair edges as relationships. There are approximately 44 English target phoneme nodes, each with attributes including IPA encoding, place of articulation, manner of articulation, and voicing. Pronunciation output nodes correspond to the phoneme variants actually produced by the learner and are connected to the target phoneme nodes via phoneme confusion edges. The high-frequency teaching word nodes consist of approximately 2000 to 3000 high-frequency words from the teaching syllabus.

[0033] The skeleton graph is an initial knowledge graph structure manually constructed based on literature on mother language transfer research. The construction method involves applied linguistics researchers manually identifying English pronunciation error patterns in the target native language group based on mother language transfer research literature, extracting high-frequency confused phoneme pairs, and establishing phoneme confusion edges for each pair. The data structure of each phoneme confusion edge includes a native language group identifier, the target phoneme, the actual produced phoneme, and the conditional probability under a specific phonemic environment and learning level. Phoneme confusion edges are directional and asymmetric. Simultaneously, near-phonetic word pairs caused by the above confusion in high-frequency teaching words are identified, and near-phonetic word pair edges are established, recording the phonological similarity scores of the two words under specific confusion patterns. The expected size of the skeleton graph is 600 to 800 phoneme confusion edges and 300 to 500 near-phonetic word pair edges.

[0034] Phoneme-level acoustic confidence score refers to the GOP score calculated for each phoneme position in each learner speech sample. Low-confidence phoneme segments refer to the acoustic feature segments corresponding to phoneme positions where the GOP score is below the decision threshold; these positions are precisely the signal points where accent shift occurs. GOP scores are generally low at low-confidence phoneme positions, and phoneme differences will appear among the N-best candidates output by the speech recognition engine at these positions. Confusion patterns can be automatically extracted from these differences. The frequency of occurrence of the same candidate confusion pair in all speech samples is counted, and the confusion probability is automatically calculated when the frequency exceeds a preset threshold. Weighted fusion update involves weighting the expert probability values ​​of existing edges in the skeleton graph with the confusion probabilities obtained from data-driven statistics according to preset weights. The initial value of the skeleton weight is 0.7 to prioritize expert knowledge, and it gradually decreases to 0.3 as the amount of data increases.

[0035] In this module, to establish a structured knowledge graph that can represent the pronunciation confusion patterns of a specific native language group, firstly, based on literature on mother tongue transfer research, applied linguistics experts manually sorted out the English phoneme confusion patterns corresponding to this native language group. For each pair of confused phonemes, a phoneme confusion edge was created, labeled with conditional probability values ​​and corresponding phonemic environmental constraints. Simultaneously, near-homophone pairs caused by phoneme confusion were sorted from high-frequency words in the teaching syllabus, and near-homophone pair edges were created to construct a skeleton graph. Then, using learner speech sample data, an N-best candidate list was generated for each speech segment using a speech recognition engine. The GOP score for each phoneme position was calculated, and low-confidence phoneme segments were extracted. The phoneme differences of the low-confidence phoneme segments in the N-best candidates were compared, candidate confusion pairs were extracted, and the confusion probability was automatically calculated after frequency statistics. The expert probability values ​​and data-driven statistical probabilities of the existing edges in the skeleton graph were updated by weighted averaging according to the current skeleton weights, completing the data-driven expansion of the knowledge graph.

[0036] Furthermore, in the system provided in this application embodiment, the map query module is also used for: After querying the pronunciation dictionary, the standard phoneme sequence corresponding to each transcription candidate is extracted. The standard phoneme sequence is then compared position-by-position with the optimal output phoneme sequence represented by the phoneme acoustic confidence information. This includes: for each phoneme position, if the standard phoneme matches the optimal output phoneme, the correct output probability for that phoneme position is determined based on the sum of all confusion edge probabilities in the pronunciation confusion knowledge graph; if the standard phoneme does not match the optimal output phoneme, the confusion edge from the standard phoneme to the optimal output phoneme in the pronunciation confusion knowledge graph is queried, and the conditional probability recorded on the confusion edge is obtained as the confusion probability of the phoneme position; the confusion probability and the correct output probability of each phoneme position are multiplied to obtain the prior confusion probability of the knowledge graph for each transcription candidate. A detailed explanation follows: In this embodiment, the standard phoneme sequence is the standardized English phoneme sequence obtained by querying a pronunciation dictionary after transcribing the candidate text. The optimal produced phoneme sequence is the phoneme sequence formed by sequentially constructing the optimal produced phonemes at each phoneme position in the phoneme acoustic confidence information. The correct production probability is the probability that a phoneme position is judged as correctly pronounced when the standard phoneme and the optimal produced phoneme are consistent, which is equal to 1 minus the sum of the probabilities of all confusion edges of the target phoneme in the knowledge graph. The confusion probability is the conditional probability value recorded on the confusion edge from the corresponding standard phoneme to the optimal produced phoneme found in the knowledge graph when the standard phoneme and the optimal produced phoneme are inconsistent. The prior confusion probability of the knowledge graph is the product value obtained by multiplying the confusion probability or correct production probability of each phoneme position digit by digit.

[0037] In this module, to retrieve the prior probability of confusion for each transcription candidate from the pronunciation confusion knowledge graph, the standard phoneme sequence corresponding to each transcription candidate text is first extracted by querying the pronunciation dictionary. The standard phoneme sequence is then compared position-by-position with the optimal produced phoneme sequence. For each phoneme position, if the standard phoneme matches the optimal produced phoneme, the probability values ​​of all confusion edges for that standard phoneme are retrieved from the knowledge graph and summed. The correct production probability equals 1 minus the total confusion probability.

[0038] If the standard phoneme and the optimal produced phoneme are inconsistent, the knowledge graph is queried for the obfuscated edges from the corresponding standard phoneme to the optimal produced phoneme. The query logic prioritizes matching obfuscated edges that precisely match the current phoneme environment conditions. If no such edge is found, the search is broadened to include general obfuscated edges without positional constraints. If no such edge is found, a preset minimum constant value is used to avoid zero probability leading to a zero product. The obfuscation probability or correct production probability of each phoneme position is multiplied digit by digit to obtain the prior knowledge graph obfuscation probability of the transcription candidate. The above query and calculation process is performed on each candidate in the N-best candidate list.

[0039] For example, consider the speech of a native Chinese learner with the actual phoneme sequence / sri: / . The N-best candidate list output by the speech recognition engine includes candidate 1 as "three" and candidate 2 as "free". The standard phoneme sequence of candidate three is / θri: / . The first phoneme position, the standard phoneme / θ / , is inconsistent with the optimal output phoneme / s / . Querying the knowledge graph for the confusion edges from / θ / to / s / of the native Chinese learner yields a confusion probability of 0.71. The other two phoneme positions are consistent with the optimal output phoneme, with correct output probabilities of 0.85 and 0.95 respectively. The prior confusion probability of candidate three in the knowledge graph is approximately 0.71 × 0.85 × 0.95 ≈ 0.573.

[0040] The standard phoneme sequence for the candidate "free" is / fri: / . The first phoneme position, the standard phoneme / f / , is inconsistent with the optimal output phoneme / s / . There is no corresponding confusion edge in the knowledge graph. Taking a minimum constant value of 0.01, the prior probability of confusion in the knowledge graph for the candidate "free" is approximately 0.01 × 0.85 × 0.95 ≈ 0.008. The calculation results show that, under the pronunciation confusion patterns of native Chinese speakers, when the actual output is / sri: / , the learner's intention is far more likely to be "three" than "free". This prior information will play a crucial role in subsequent joint inference.

[0041] In summary, this module combines prior knowledge from mother tongue transfer research with data-driven expansion by pre-constructing a pronunciation confusion knowledge graph containing phoneme confusion edges and near-phonetic word pairs. It queries and obtains the prior probability of confusion in the knowledge graph corresponding to each transcription candidate, providing a structured prior reference based on the pronunciation rules of a specific mother tongue group for subsequent Bayesian joint inference.

[0042] Intent reasoning module 13: is used to combine the acoustic recognition result, the semantic context representation and the prior probability of confusion of the knowledge graph to calculate the posterior probability of each transcription candidate, and determine the transcription candidate with the highest posterior probability as the intent recognition result.

[0043] In this embodiment, based on the acoustic probability of each transcription candidate in the acoustic recognition result, the semantic prior information in the semantic context representation, and the prior probability of confusion in the knowledge graph, it is necessary to fuse these three probabilistic information from different sources, calculate the posterior probability of each candidate being the learner's true intention under a unified framework, and determine the final intention recognition result based on the magnitude of the posterior probability. This addresses the problem that traditional speech recognition systems rely solely on acoustic and language models for transcription, lacking a probabilistic framework that jointly models acoustic evidence, semantic context, and pronunciation confusion patterns of specific native language groups. When acoustic signals are blurred due to accent shifts and multiple possible intentions exist, it is impossible to comprehensively utilize semantic context and native language transfer prior knowledge for collaborative disambiguation.

[0044] In the system provided in this application embodiment, the intent reasoning module is specifically used for: The acoustic probabilities corresponding to each transcription candidate are extracted from the acoustic recognition results; based on the semantic vectors and expected response range in the semantic context representation, the semantic prior probability corresponding to each transcription candidate is calculated; the acoustic probability, the semantic prior probability, and the knowledge graph confusion prior probability are multiplied together, and the product result of all transcription candidates is normalized to obtain the posterior probability of each transcription candidate; the transcription candidate with the highest posterior probability is determined as the intent recognition result. A detailed explanation follows: In this embodiment, acoustic probability refers to the probability value obtained by exponentially transforming and normalizing the logarithmic probability assigned by the speech recognition engine to each transcription candidate. The transformation method is as follows: the logarithmic probability of each candidate is exponentially calculated, and the sum of the exponential values ​​of all candidates is used as the normalized denominator. The acoustic probability of each candidate is equal to its exponential value divided by the normalized denominator. Acoustic probability reflects the degree of matching likelihood between acoustic features and candidate text; a higher acoustic probability indicates that the acoustic evidence strongly supports the candidate.

[0045] Semantic prior probability refers to the probability value obtained by matching the set of content words in the transcription candidate with the expected response range, and taking the arithmetic mean of the conditional probabilities of each content word under the dialogue history and scenario conditions. Content words include keywords carrying semantic information such as nouns, verbs, and adjectives in the candidates. The conditional probability is predicted by the language model based on the current dialogue history and teaching scenario. The formula for calculating semantic prior probability is: Semantic prior probability = This probability reflects the reasonableness of the candidate in the current semantic context; the larger the value, the more the candidate conforms to the current dialogue logic.

[0046] The prior probability of confusion in a knowledge graph is a probability value obtained from a pronunciation confusion knowledge graph, reflecting the likelihood that a learner from a specific native language group will actually produce the current acoustic observation given that a candidate represents the standard intent. The posterior probability is the probability value obtained by multiplying the acoustic probability, semantic prior probability, and knowledge graph confusion prior probability, and then normalizing the product of all candidates. The normalization method is as follows: calculate the three-factor product of each candidate, sum the products of all candidates to obtain the normalized denominator, and the posterior probability of each candidate is equal to the product of that candidate divided by the normalized denominator. The posterior probability represents the probability that, given the acoustic observation, semantic context, and native language pronunciation rules, the candidate represents the learner's true intent.

[0047] The intent recognition result is the transcription candidate with the highest posterior probability, indicating that the system, after integrating acoustic evidence, semantic context, and pronunciation confusion patterns within a specific native language group, determines that this candidate is most likely to represent the learner's true expressive intent. This result will serve as the foundational input for subsequent pronunciation error diagnosis and disambiguation feedback modules.

[0048] In this module, to fuse information from three different sources within a unified probabilistic framework and determine the learner's true intent, the acoustic probabilities of each transcription candidate are first extracted from the acoustic recognition results. The logarithmic probabilities are then transformed by exponentialization and normalization to obtain standardized acoustic probability values. Next, semantic vectors and expected response ranges are obtained from the semantic context representation. For each transcription candidate, content words are extracted, and the conditional probability of each content word under the dialogue history and scenario conditions is calculated. The arithmetic mean is taken to obtain the semantic prior probability of that candidate. Then, the acoustic probability, semantic prior probability, and knowledge graph confusion prior probability of each candidate are multiplied together to obtain the unnormalized posterior probability of that candidate. The unnormalized posterior probabilities of all candidates are summed, and the unnormalized posterior probability of each candidate is divided by this sum to obtain the normalized posterior probability. Finally, the posterior probability values ​​of all candidates are compared, and the transcription candidate with the highest posterior probability is selected as the intent recognition result output.

[0049] For example, continuing the previous example. A native Chinese learner speaks the speech with the actual phoneme sequence / sri: / . The speech recognition engine outputs three candidates: candidate 1 is "three", candidate 2 is "free", and candidate 3 is "tree". The acoustic probabilities of each candidate are extracted: the acoustic probability of candidate "three" after logarithmic probability transformation is 0.23, candidate "free" is 0.29, and candidate "tree" is 0.13. The current teaching scenario is ordering food in a restaurant. The expected response range in the semantic context includes words such as "course", "meal", and "order". The content word set of candidate "three" includes "three" and "course". "Three" has no direct match within the expected response range, so its conditional probability is 0.1, indicating a lower probability for words outside the expected range. The conditional probability of "course" being matched is 0.9, and the semantic prior probability is (0.1 + 0.9) ÷ 2 = 0.50. The content word set of candidate "free" includes "free" and "course". "Free" has no direct match within the expected response range, so its conditional probability is 0.1, and the semantic prior probability is (0.1 + 0.9) ÷ 2 = 0.50. The set of semantic terms for candidate tree includes tree and course. There is no direct match for tree within the expected response range, so the conditional probability is 0.1. The semantic prior probability is (0.1 + 0.9) ÷ 2 = 0.50.

[0050] The prior probabilities of knowledge graph confusion for each candidate have been calculated in the aforementioned module: 0.573 for candidate three, 0.008 for candidate free, and approximately 0.023 for candidate tree. Multiplying the three factors together: the unnormalized posterior probability of candidate three = 0.23 × 0.50 × 0.573 ≈ 0.066, candidate free = 0.29 × 0.50 × 0.008 ≈ 0.001, and candidate tree = 0.13 × 0.50 × 0.023 ≈ 0.001. The normalized denominator = 0.066 + 0.001 + 0.001 = 0.068. The normalized posterior probability of candidate three = 0.066 ÷ 0.068 ≈ 0.971, the posterior probability of candidate free = 0.001 ÷ 0.068 ≈ 0.015, and the posterior probability of candidate tree = 0.001 ÷ 0.068 ≈ 0.015. Candidate three has the highest posterior probability, and the intent recognition result is three.

[0051] In summary, this module obtains the posterior probability by multiplying and normalizing the acoustic probability, semantic prior probability, and knowledge graph confusion prior probability of each transcription candidate, and uses the candidate with the highest posterior probability as the intent recognition result, thus realizing joint reasoning of acoustic evidence, semantic context, and native language pronunciation rules under a unified Bayesian framework.

[0052] Error diagnosis module 14: Based on the intent recognition result, extract the standard phoneme sequence and compare it phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result, and combine the phoneme acoustic confidence to generate pronunciation error diagnosis result.

[0053] In this embodiment, based on the determined intent recognition result, the system needs to further determine whether the learner pronounces each phoneme correctly when expressing the intent, and locate the specific error location and type. Diagnosis of pronunciation errors requires a precise comparison between standard pronunciation and actual output, and also needs to combine phoneme-level acoustic confidence as a quantitative reference for the degree of error. This addresses the problems of traditional pronunciation assessment systems that only provide an overall score, cannot clearly locate the error location and direction at the phoneme level, and whose pronunciation assessment is based on an erroneous benchmark when the speech recognition-transcribed text itself contains errors due to accent shifts.

[0054] In the system provided in this application embodiment, the error diagnosis module is specifically used for: Based on a pronunciation dictionary, a standard phoneme sequence corresponding to the intent recognition result is extracted; the standard phoneme sequence is compared phoneme-by-phoneme with the optimal produced phoneme sequence, and each phoneme position is determined to be either consistent in pronunciation or deviated in pronunciation; for phoneme positions determined to be deviated in pronunciation, the corresponding acoustic confidence score is combined to generate the pronunciation deviation diagnosis result, which includes the standard phoneme, the deviated phoneme, and the deviation confidence score. A detailed explanation follows: In this embodiment, the pronunciation dictionary is a mapping table that converts text words into standard phoneme sequences. It stores the standard English phoneme composition corresponding to each word and serves as a standard reference source for phoneme-by-phoneme comparison. The pronunciation dictionary covers all high-frequency words in the teaching syllabus, and for words not included, supplementary phoneme sequences can be generated using a rule-based character-to-phoneme conversion algorithm.

[0055] A standard phoneme sequence refers to the standardized English phoneme sequence obtained by querying a pronunciation dictionary after the intent recognition result text is processed. The optimal output phoneme sequence is a phoneme sequence composed of the optimal output phonemes at each phoneme position in the phoneme acoustic confidence information, representing the learner's actual pronunciation.

[0056] Phoneme-by-phoneme alignment comparison compares the standard phoneme sequence with the optimal produced phoneme sequence position by position, independently judging whether the standard phoneme matches the actual output for each phoneme position. Pronunciation consistency means that the standard phoneme and the optimal produced phoneme are the same at a certain phoneme position, and the acoustic confidence score at that position is higher than the judgment threshold, indicating that the learner's actual pronunciation matches the standard pronunciation. Pronunciation deviation means that the standard phoneme and the optimal produced phoneme are different at a certain phoneme position, or that they appear to be consistent but the acoustic confidence score is lower than the judgment threshold, indicating that the learner's pronunciation deviates.

[0057] Error confidence score is the acoustic confidence score used to quantify the location of a phoneme identified as a pronunciation error; a higher score indicates greater confidence in the error. The pronunciation error diagnosis result is the output data set of the diagnostic module. Each error location record includes the standard phoneme, the error phoneme (i.e., the actually produced phoneme), and the error confidence score, which are then used by the subsequent disambiguation feedback module to determine the error type and its pedagogical value.

[0058] In this module, to further assess pronunciation accuracy and pinpoint error locations after determining the learner's true intent, the standard pronunciation of each word in the intent recognition result text is first extracted based on a pronunciation dictionary query, resulting in a complete standard phoneme sequence. Then, the standard phoneme sequence is compared and aligned position-by-position with the optimal produced phoneme sequence. For each phoneme position, if the standard phoneme matches the optimal produced phoneme and the acoustic confidence score at that position is higher than the judgment threshold, it is determined to be a pronunciation match; if the standard phoneme differs from the optimal produced phoneme, or if they appear to match but the acoustic confidence score is lower than the judgment threshold, it is determined to be a pronunciation error. For phoneme positions determined to be pronunciation errors, the standard phoneme at that position, the actual optimal produced phoneme, and the acoustic confidence score at that position are recorded as the error phoneme, respectively, as the error confidence score. The diagnostic information for all error positions is then summarized to generate a pronunciation error diagnosis result.

[0059] For example, continuing the previous example. The intent recognition result is three, course, meal. A phoneme query yields the standard phoneme sequence / θri:, kɔ:s, mi:l / . The optimal produced phoneme sequence is / sri:, kɔ:s, mi:l / . Phoneme alignment comparison: The first phoneme position differs between the standard phoneme / θ / and the optimal produced phoneme / s / , indicating a pronunciation error. The error confidence score is set to 4.2 for that position. For the remaining phoneme positions, the standard phoneme and the optimal produced phoneme are consistent, and the acoustic confidence scores are all above the judgment threshold, indicating consistent pronunciation. The pronunciation error diagnosis result contains one error record: the standard phoneme is / θ / , the error phoneme is / s / , and the error confidence score is 4.2.

[0060] In summary, this module compares the standard phoneme sequence of the intent recognition result with the optimal output phoneme sequence phoneme sequence phoneme by phoneme, and combines phoneme acoustic confidence information to determine whether the pronunciation is consistent or incorrect, generating a pronunciation error diagnosis result containing standard phonemes, incorrect phonemes, and error confidence, thus realizing the synchronous output from intent understanding to phoneme-level pronunciation error localization.

[0061] Disambiguation feedback module 15: used to calculate the inference confidence based on the posterior probability of each transcription candidate, and to perform multi-level interactive disambiguation in conjunction with the pronunciation error diagnosis results, generating interactive teaching feedback and outputting it.

[0062] In this embodiment, based on the obtained intent recognition results and pronunciation error diagnosis results, the system needs to determine whether and to what extent to correct the learner's pronunciation according to the level of inference confidence and the teaching value of the error. Different confidence levels and error importance correspond to different levels of teaching intervention intensity. Simultaneously, the confirmation data generated during the disambiguation process should be fed back to the knowledge graph to continuously improve system performance. This addresses the problems of traditional teaching systems that directly abandon interaction or mechanically request repetition when encountering recognition ambiguity, failing to integrate the disambiguation process with teaching activities, and lacking a tiered strategy for adaptively adjusting the intervention intensity based on inference certainty and the teaching value of the error. The adaptive disambiguation and knowledge graph update process in this step is as follows: Figure 2 As shown.

[0063] In the system provided in this application embodiment, the disambiguation feedback module is specifically used for: Based on the first posterior probability value of the transcription candidate with the highest posterior probability, and the ratio of the first posterior probability value to the second highest posterior probability value of the transcription candidate, the inference confidence is calculated and the confidence level is determined. Based on the prevalence of mother tongue transfer confusion, the matching degree of current teaching focus, and the learner's historical error frequency corresponding to each error phoneme in the pronunciation error diagnosis results, a weighted teaching value score is calculated for each error phoneme. Based on the confidence level and the teaching value score, the strategy at the corresponding level in the multi-level interactive disambiguation strategy is selected for disambiguation processing, generating interactive teaching feedback and outputting it. Detailed explanation follows: In this embodiment, the inference confidence is a comprehensive confidence index calculated based on the posterior probability of the best candidate and its ratio to the posterior probability of the second-best candidate. The calculation method is: Inference Confidence = Posterior Probability of Best Candidate × (Posterior Probability of Best Candidate ÷ Posterior Probability of Second-Best Candidate). This formula considers both absolute confidence and relative confidence. Absolute confidence is the posterior probability of the best candidate itself, and relative confidence is the ratio of the posterior probabilities of the best candidate to the second-best candidate. A low value in either dimension will lower the inference confidence.

[0064] The confidence level is determined by comparing the inference confidence with a preset threshold. The preset high and low confidence thresholds are determined through statistical analysis of the validation dataset. Specifically, the inference confidence value for each correct and incorrect inference is calculated on the validation dataset. The lower quartile of the inference confidence for correct inference is used as the high confidence threshold, and the upper quartile of the inference confidence for incorrect inference is used as the low confidence threshold. An inference confidence higher than or equal to the high confidence threshold is classified as a high level, an inference confidence lower than the low confidence threshold is classified as a low level, and an inference confidence in between is classified as a medium level.

[0065] The prevalence of mother tongue transfer confusion refers to the probability value of the confused side of the erroneous phoneme in the pronunciation confusion knowledge graph corresponding to the mother tongue group, which is obtained directly from the knowledge graph. The higher the probability value, the more prevalent the error is in that mother tongue group, and the higher the teaching value. The matching degree of current teaching focus refers to the matching relationship between the erroneous phoneme and the teaching target phoneme set in the current teaching unit, provided by the teaching target management module. If the erroneous phoneme is the focus of the current teaching unit, the value is 1; otherwise, the value is 0. The learner's historical error frequency refers to the proportion of the number of times the learner made errors with this phoneme in the total number of interactions in the historical interaction record, obtained by querying and statistically analyzing the user's historical interaction storage.

[0066] The teaching value score is calculated as follows: Prevalence of Mother Tongue Transfer Confusion × 0.3 + Matching Degree of Current Teaching Focus × ​​0.4 + Frequency of Learners' Historical Errors × 0.3. The three weighting coefficients were determined through statistical regression analysis of the teaching experts' ratings, with the weighting coefficients for Prevalence of Mother Tongue Transfer Confusion (0.3), Matching Degree of Current Teaching Focus (0.4), and Frequency of Learners' Historical Errors (0.3) all set at 0.

[0067] In this module, to adaptively select an appropriate disambiguation strategy based on the degree of inference uncertainty and the instructional value of errors, the posterior probabilities of the best and second-best candidates are first obtained from the intention inference module. The inference confidence is calculated by multiplying the posterior probability of the best candidate by the ratio of the posterior probability of the best candidate to that of the second-best candidate. The inference confidence is then compared with preset high and low confidence thresholds to determine the current confidence level. Next, each error phoneme is extracted from the pronunciation error diagnosis results. The knowledge graph is queried to obtain the prevalence of mother tongue transfer confusion corresponding to each error phoneme. The current teaching focus matching degree corresponding to each error phoneme is obtained from the instructional objective management module. The learner's historical error frequency corresponding to each error phoneme is obtained from the user's historical interaction storage. The values ​​of the three dimensions are weighted and summed according to preset weights to calculate the instructional value score of each error phoneme.

[0068] Furthermore, in the system provided in this application embodiment, the disambiguation feedback module is also used for: When the confidence level is high and there are no erroneous phonemes with a teaching value score exceeding the preset teaching value threshold, the first-level disambiguation strategy is selected. In the generated dialogue response, the keywords in the intent recognition result are naturally repeated with correct pronunciation, performing implicit disambiguation. When the confidence level is low, the second-level disambiguation strategy is selected. Based on the near-sounding word edges in the pronunciation confusion knowledge graph, confirmation candidates are selected. Confirmation intent is requested from the learner through dialogue follow-up questions to obtain confirmation intent labels. The standard phoneme sequence corresponding to the confirmation intent label is compared with the optimal produced phoneme sequence to locate the erroneous phoneme, achieving explicit confirmation disambiguation. When the confidence level is high and there are erroneous phonemes with a teaching value score exceeding the teaching value threshold, the third-level disambiguation strategy is selected. The erroneous phoneme is directly pointed out to the learner, and the pronunciation training of the smallest contrasting word containing the erroneous phoneme is matched, performing explicit teaching disambiguation. Detailed explanation follows: In this embodiment, the first-level disambiguation strategy is an implicit disambiguation method. The system naturally repeats the keywords from the intent recognition result with correct pronunciation in the generated dialogue response, and generates a speech response containing standard pronunciation through the text-to-speech module. Learners receive correct speech input in a natural dialogue flow without interrupting the conversation. The first-level disambiguation strategy is suitable for scenarios with high confidence and low error-based teaching value.

[0069] The second-level disambiguation strategy is explicit confirmation disambiguation. The selection of confirmation candidates follows these priority rules: first, it queries the near-phonetic edges in the knowledge graph to find words that form a near-phonetic relationship with the optimal candidate; second, it selects candidates with high probability ranking and semantic possibility from the N-best candidate list; the number of candidates is controlled to 2 to 3. The system presents the confirmation candidates to the learner in a dialogue-based manner and requests confirmation. After the learner replies with confirmation, the system obtains the confirmation intent label and compares the standard phoneme sequence corresponding to the confirmation intent label with the optimal produced phoneme sequence phoneme-by-phoneme to locate the erroneous phonemes. The second-level disambiguation strategy is suitable for scenarios with low confidence.

[0070] The third-level disambiguation strategy is an explicit instructional disambiguation method. The system directly points out the erroneous phoneme to the learner, and the generated feedback includes concise prompts on the articulation position and manner of the erroneous phoneme, as well as the minimal contrast word pair containing that erroneous phoneme for the learner to practice. The minimal contrast word pair is retrieved from the instructional vocabulary as two words that differ only in the position of the erroneous phoneme, and are identical in all other phonemes. For example, to distinguish between / θ / and / s / , "think" and "sink" are selected. The system uses a text-to-speech module to generate standard pronunciations for learners to imitate. The third-level disambiguation strategy is suitable for scenarios with high confidence and erroneous phonemes whose instructional value scores exceed the instructional value threshold.

[0071] The preset teaching value threshold is determined based on the effectiveness analysis of teaching intervention. By statistically analyzing the teaching expert rating data, the average of the lowest teaching value scores that teaching experts consider worthy of explicit intervention is taken as the teaching value threshold.

[0072] In this module, disambiguation strategies are selected based on confidence level and instructional value score. When the confidence level is high and the instructional value scores of all erroneous phonemes do not exceed the instructional value threshold, it indicates that the system has sufficient confidence in understanding the intent, but the errors are not worth interrupting the dialogue. The first-level disambiguation strategy is selected, naturally embedding the correct pronunciation in the dialogue response for implicit disambiguation. When the confidence level is low, it indicates that the system's understanding of the intent is uncertain. Regardless of the instructional value of the errors, the intent must be confirmed first. The second-level disambiguation strategy is selected, choosing confirmation candidates from the edges of near-homophones in the knowledge graph to confirm with the learner, and locating the erroneous phonemes after obtaining the confirmation intent label. When the confidence level is high and there are erroneous phonemes with instructional value scores exceeding the instructional value threshold, it indicates that the system has sufficient confidence in understanding the intent and there are errors worth instructing. The third-level disambiguation strategy is selected, directly pointing out the errors and guiding pronunciation training.

[0073] Furthermore, in the system provided in this application embodiment, the disambiguation feedback module is also used for: The confirmation intent tags obtained from the interactive teaching feedback are compared with the corresponding optimal output phoneme sequences at the phoneme level to extract phoneme-level confusion events. These phoneme-level confusion events are then matched with existing phoneme confusion edges in the pronunciation confusion knowledge graph. For successfully matched phoneme confusion edges, the corresponding conditional probability values ​​are updated using a sliding weighting method. For phoneme-level confusion events that fail to match and whose cumulative frequency reaches a new creation threshold, corresponding phoneme confusion edges are created and written into the pronunciation confusion knowledge graph. A detailed explanation follows: In this embodiment, a phoneme-level confusion event is a single pronunciation error event record extracted from the phoneme-level difference comparison between the standard phoneme sequence corresponding to the confirmed intent tag and the optimal produced phoneme sequence. It includes the target phoneme, the actual produced phoneme, and the corresponding phonemic context information. Sliding weighted update is a method of updating the conditional probability values ​​of existing phoneme confusion edges in the knowledge graph using a sliding window weighting method. The update formula is: New probability value = Current probability value × (1 - Learning rate) + Current event indication value × Learning rate. The learning rate is set relatively high in the early stages of system operation to accelerate adaptation and relatively low in the stable period to maintain stability. The creation threshold is the lower limit of the cumulative frequency for determining whether to create a new phoneme confusion edge in the knowledge graph. When the cumulative frequency of a certain phoneme-level confusion event reaches the creation threshold, the creation edge operation is triggered. The initial conditional probability value of the new edge is taken as the cumulative frequency of the confusion event.

[0074] In this module, to continuously optimize the knowledge graph using disambiguation confirmation data, after completing disambiguation feedback and obtaining confirmation intent labels, the standard phoneme sequence corresponding to the confirmation intent label is compared with the optimal produced phoneme sequence at the phoneme level, bit-by-bit difference. The target phoneme and the actual produced phoneme at each inconsistent phoneme position are extracted, generating phoneme-level confusion events. The extracted phoneme-level confusion events are matched with existing phoneme confusion edges in the knowledge graph. If a confusion edge from the corresponding target phoneme to the actual produced phoneme already exists in the knowledge graph, the conditional probability value of that edge is updated using a sliding weighted method. If no corresponding confusion edge exists in the knowledge graph, the cumulative frequency of the confusion event is recorded. When the cumulative frequency reaches a new creation threshold, a new phoneme confusion edge is created and written into the knowledge graph.

[0075] For example, continuing the previous example. The posterior probability of the best candidate "three" is 0.971, and the posterior probability of the second-best candidate "free" is 0.015. The inference confidence score = 0.971 × (0.971 ÷ 0.015) ≈ 62.8. The preset high confidence threshold is 5.0, and the low confidence threshold is 1.5. The inference confidence score of 62.8 is higher than the high confidence threshold, so the confidence level is high. In the error diagnosis results, the native language transfer confusion prevalence of the error phonemes / θ / to / s / is 0.71, the current teaching focus matching degree is 1.0, that is, / θ / is the key phoneme of the current teaching unit, and the learner's historical error frequency is 0.62. The teaching value score = 0.3 × 0.71 + 0.4 × 1.0 + 0.3 × 0.62 ≈ 0.799. The teaching value threshold is 0.6. 0.799 is greater than 0.6, triggering the third-level disambiguation strategy. The system points out the incorrect phoneme / θ / to the learner and matches the smallest contrasting word to train the pronunciation of "think" and "sink".

[0076] In another interaction, if the posterior probability of the best candidate is 0.45 and the posterior probability of the second-best candidate is 0.38, the inference confidence = 0.45 × (0.45 ÷ 0.38) ≈ 0.533, which is lower than the low confidence threshold of 1.5, resulting in a low confidence level and triggering the second-level disambiguation strategy. The system queries the edge pairs of near-phonetic words in the knowledge graph that form near-phonetic relationships with the best candidate as confirmation candidates, and confirms the learner's intention through dialogue-based follow-up questions.

[0077] After disambiguation, the system compares the standard phoneme sequence corresponding to the confirmed intent label with the optimal produced phoneme sequence, extracting phoneme-level confusion events, such as the target phoneme / θ / being produced as / s / . This confusion event is successfully matched with an existing phoneme confusion edge in the knowledge graph, and the conditional probability value of the edge is updated using a sliding weighted method. The learning rate is set to 0.05, and the new probability value = 0.71 × 0.95 + 1 × 0.05 ≈ 0.725.

[0078] In summary, this module calculates and grades inference confidence, and evaluates teaching value by weighting the prevalence of mother tongue transfer confusion of erroneous phonemes, the matching degree of teaching focus, and the frequency of historical errors. Based on the confidence level and teaching value score, it adaptively selects implicit paraphrasing disambiguation, explicit confirmation disambiguation, or explicit teaching disambiguation strategies to generate teaching feedback. After disambiguation, the confirmation data is written back to update the knowledge graph, thus achieving adaptive matching between inference uncertainty and the intensity of teaching intervention.

[0079] The embodiments of this application achieve the following technical effects through the specific implementation modules described above: This application proposes an intelligent interactive English teaching system based on speech recognition. The system employs a speech analysis module to extract acoustic recognition results, phoneme acoustic confidence information, and semantic context representation from learners' speech signals in parallel. A graph query module obtains the prior probability of each transcription candidate from a pre-built pronunciation confusion knowledge graph. An intent reasoning module fuses the acoustic probability, semantic prior probability, and knowledge graph confusion prior probability to calculate the posterior probability and determine the intent recognition result. An error diagnosis module compares the standard phoneme sequence of the intent result with the optimal output phoneme sequence phoneme-by-phone to generate pronunciation error diagnosis results. A disambiguation feedback module adaptively selects a three-level disambiguation strategy (implicit repetition, explicit confirmation, or explicit instruction) based on the reasoning confidence level and the error teaching value score to generate interactive teaching feedback, and writes the disambiguation confirmation data back to update the knowledge graph. These modules work collaboratively to effectively solve the technical problem in existing English teaching systems where speech recognition error tolerance and pronunciation error correction are contradictory, making it impossible to accurately locate pronunciation errors while accurately understanding intent.

[0080] Example 2, as shown in the appendix Figure 3 As shown, based on the inventive concept of an intelligent English teaching interactive system based on speech recognition provided in Embodiment 1, this application also provides an intelligent English teaching interactive method based on speech recognition, specifically including: Acquire learners’ speech signals and corresponding dialogue context information in teaching dialogue scenarios, and analyze the acoustic recognition results, phoneme acoustic confidence information and semantic context representation accordingly. Based on the pre-built pronunciation confusion knowledge graph for learners' native language, the prior probability of knowledge graph confusion for each transcription candidate in the acoustic recognition result is obtained by querying. Combining the acoustic recognition results, the semantic context representation, and the prior probability of confusion in the knowledge graph, the posterior probability of each transcription candidate is calculated, and the transcription candidate with the highest posterior probability is determined as the intent recognition result; Based on the intent recognition result, the standard phoneme sequence is extracted and compared phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result. Combined with the phoneme acoustic confidence information, a pronunciation error diagnosis result is generated. The inference confidence is calculated based on the posterior probability of each transcription candidate, and multi-level interactive disambiguation is performed in conjunction with the pronunciation error diagnosis results to generate and output interactive teaching feedback.

[0081] This involves acquiring learners' speech signals and corresponding dialogue context information in teaching dialogue scenarios, and analyzing the acquired acoustic recognition results, phoneme acoustic confidence information, and semantic context representations, including: Based on the speech signal, speech recognition processing is performed to output multiple transcription candidates and their corresponding acoustic probabilities, which constitute the acoustic recognition result. Based on the transcription candidate with the highest acoustic probability, the speech signal is subjected to phoneme-level forced alignment to determine the target phoneme sequence; The acoustic confidence score and the optimal produced phoneme at each target phoneme position in the target phoneme sequence are calculated to constitute the phoneme acoustic confidence information. Based on the dialogue context information, semantic vectors are extracted through a pre-trained semantic encoding model, and the expected response range is generated in combination with the current teaching scenario to form the semantic context representation.

[0082] Furthermore, the pre-construction of the pronunciation confusion knowledge graph includes: Based on literature on mother tongue transfer research, this study analyzes the English phoneme confusion patterns corresponding to learners' native languages ​​and constructs a skeleton map containing phoneme confusion edges and near-phonetic word pairs, where: Each of the aforementioned phoneme confusion edges represents the conditional probability that a target phoneme is produced as another phoneme in a specific phonemic environment, and each of the aforementioned near-phonetic word pairs represents the phonological similarity between two words in a specific confusion pattern. Based on learner speech sample data, multiple transcription candidates are generated for each speech segment and phoneme-level acoustic confidence scores are calculated. Based on the phoneme-level acoustic confidence score, confusion patterns are extracted from the phoneme differences among multiple transcriptional candidates corresponding to low-confidence phoneme segments, and the confusion probability is automatically calculated after counting the frequency. The probability values ​​of existing edges in the skeleton graph are weighted and fused with the confusion probability to update the skeleton graph, and the data-driven expansion is performed to obtain the pronunciation confusion knowledge graph.

[0083] Furthermore, based on a pre-built pronunciation confusion knowledge graph for learners' native language, the prior probability of knowledge graph confusion corresponding to each transcription candidate in the acoustic recognition result is retrieved, including: After consulting a pronunciation dictionary, the standard phoneme sequence corresponding to each of the transcription candidates is extracted; The standard phoneme sequence is compared bit-by-bit with the optimal produced phoneme sequence characterized by the phoneme acoustic confidence information, including: For each phoneme position, if the standard phoneme is consistent with the optimal output phoneme, the correct output probability of the phoneme position is determined based on the sum of all confusion edge probabilities of the phoneme in the pronunciation confusion knowledge graph. If the standard phoneme is inconsistent with the optimal produced phoneme, then query the confusion edge from the standard phoneme to the optimal produced phoneme in the pronunciation confusion knowledge graph, and obtain the conditional probability recorded on the confusion edge as the confusion probability of the phoneme position. The confusion probability and the correct output probability of each phoneme position are multiplied together to obtain the knowledge graph confusion prior probability of each transcription candidate.

[0084] Furthermore, combining the acoustic recognition results, the semantic context representation, and the prior probability of confusion with the knowledge graph, the posterior probability of each transcription candidate is calculated, and the transcription candidate with the highest posterior probability is determined as the intent recognition result, including: Extract the acoustic probability corresponding to each transcription candidate from the acoustic recognition results; Based on the semantic vector and expected response range in the semantic context representation, the semantic prior probability corresponding to each transcription candidate is calculated. The acoustic probability, the semantic prior probability, and the knowledge graph confusion prior probability are multiplied together, and the product result of all transcription candidates is normalized to obtain the posterior probability of each transcription candidate. The transcription candidate with the highest posterior probability is determined as the intent recognition result.

[0085] Furthermore, based on the intent recognition result, a standard phoneme sequence is extracted and compared phoneme-by-phoneme with the optimal output phoneme sequence from the acoustic recognition result. Combined with the phoneme acoustic confidence, a pronunciation error diagnosis result is generated, including: Based on the pronunciation dictionary, extract the standard phoneme sequence corresponding to the intent recognition result; The standard phoneme sequence is compared with the optimal produced phoneme sequence on a phoneme-by-phoneme alignment basis, and each phoneme position is determined to be either consistent in pronunciation or deviated in pronunciation. For the phoneme locations identified as pronunciation errors, the corresponding acoustic confidence scores are combined to generate the pronunciation error diagnosis results, which include the standard phoneme, the error phoneme, and the error confidence score.

[0086] Furthermore, the inference confidence is calculated based on the posterior probability of each transcription candidate, and multi-level interactive disambiguation is performed in conjunction with the pronunciation error diagnosis results to generate and output interactive teaching feedback, including: The inference confidence level is calculated and the confidence level is determined based on the first posterior probability value of the transcription candidate with the highest posterior probability and the ratio of the first posterior probability value to the second posterior probability value of the transcription candidate with the second highest posterior probability. The teaching value score of each phoneme is calculated by weighting the prevalence of mother tongue transfer confusion, the matching degree of current teaching focus, and the learner's historical error frequency for each phoneme in the pronunciation error diagnosis results. Based on the confidence level and the teaching value score, the strategy at the corresponding level in the multi-level interactive disambiguation strategy is selected to perform disambiguation processing, generate interactive teaching feedback, and output it.

[0087] Furthermore, based on the confidence level and the teaching value score, a strategy at the corresponding level from the multi-level interactive disambiguation strategy is selected for disambiguation processing, including: When the confidence level is high and there are no biased phonemes whose teaching value scores exceed the preset teaching value threshold, the first level of disambiguation strategy is selected. In the generated dialogue response, the keywords in the intent recognition result are naturally repeated with correct pronunciation to perform implicit disambiguation. When the confidence level is low, a second-level disambiguation strategy is selected. Based on the near-sounding words in the pronunciation confusion knowledge graph, confirmation candidates are selected. The learner is asked to confirm the intention through dialogue and questioning. The confirmation intention label is obtained. The standard phoneme sequence corresponding to the confirmation intention label is compared with the optimal output phoneme sequence to locate the erroneous phoneme and achieve explicit confirmation disambiguation. When the confidence level is high and there are erroneous phonemes with teaching value scores exceeding the teaching value threshold, the third-level disambiguation strategy is selected to directly point out the erroneous phonemes to the learner and match the pronunciation training of the smallest contrasting word containing the erroneous phonemes for explicit teaching disambiguation.

[0088] Furthermore, the methods also include: The confirmation intent tags obtained from the interactive teaching feedback are compared with the corresponding optimal output phoneme sequences at the phoneme level to extract phoneme-level confusion events. The phoneme-level confusion events are matched with existing phoneme confusion edges in the pronunciation confusion knowledge graph; For successfully matched phoneme confusion edges, the corresponding conditional probability values ​​are updated using a sliding weighting method; For phoneme-level obfuscation events that fail to match and whose cumulative frequency reaches the new threshold, a corresponding phoneme obfuscation edge is created and written into the pronunciation obfuscation knowledge graph.

[0089] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

Claims

1. A speech recognition based intelligent English teaching interactive system, characterized in that, include: Speech analysis module: used to acquire learners' speech signals and corresponding dialogue context information in teaching dialogue scenarios, and to analyze and acquire acoustic recognition results, phoneme acoustic confidence information and semantic context representation. The graph query module is used to query and obtain the prior probability of knowledge graph confusion corresponding to each transcription candidate in the acoustic recognition result based on the pre-built pronunciation confusion knowledge graph for learners' native language. The intent reasoning module is used to combine the acoustic recognition result, the semantic context representation and the prior probability of confusion of the knowledge graph to calculate the posterior probability of each transcription candidate, and determine the transcription candidate with the highest posterior probability as the intent recognition result; The error diagnosis module is used to extract the standard phoneme sequence based on the intent recognition result and compare it phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result, and generate pronunciation error diagnosis results by combining the phoneme acoustic confidence information. The disambiguation feedback module is used to calculate the inference confidence based on the posterior probability of each transcription candidate, and to perform multi-level interactive disambiguation in conjunction with the pronunciation error diagnosis results, generating interactive teaching feedback and outputting it.

2. The intelligent English teaching interactive system based on speech recognition according to claim 1, wherein, Acquire learners' speech signals and corresponding dialogue context information in teaching dialogue scenarios, and analyze the obtained acoustic recognition results, phoneme acoustic confidence information, and semantic context representations, including: Based on the speech signal, speech recognition processing is performed to output multiple transcription candidates and their corresponding acoustic probabilities, which constitute the acoustic recognition result. Based on the transcription candidate with the highest acoustic probability, the speech signal is subjected to phoneme-level forced alignment to determine the target phoneme sequence; The acoustic confidence score and the optimal produced phoneme at each target phoneme position in the target phoneme sequence are calculated to constitute the phoneme acoustic confidence information. Based on the dialogue context information, semantic vectors are extracted through a pre-trained semantic encoding model, and the expected response range is generated in combination with the current teaching scenario to form the semantic context representation.

3. The intelligent English teaching interactive system based on speech recognition according to claim 1, wherein, The pre-construction of the pronunciation confusion knowledge graph includes: Based on literature on mother tongue transfer research, this study analyzes the English phoneme confusion patterns corresponding to learners' native languages ​​and constructs a skeleton map containing phoneme confusion edges and near-phonetic word pairs, where: Each of the aforementioned phoneme confusion edges represents the conditional probability that a target phoneme is produced as another phoneme in a specific phonemic environment, and each of the aforementioned near-phonetic word pairs represents the phonological similarity between two words in a specific confusion pattern. Based on learner speech sample data, multiple transcription candidates are generated for each speech segment and phoneme-level acoustic confidence scores are calculated. Based on the phoneme-level acoustic confidence score, confusion patterns are extracted from the phoneme differences among multiple transcriptional candidates corresponding to low-confidence phoneme segments, and the confusion probability is automatically calculated after counting the frequency. The probability values ​​of existing edges in the skeleton graph are weighted and fused with the confusion probability to update the skeleton graph, and the data-driven expansion is performed to obtain the pronunciation confusion knowledge graph.

4. The intelligent English teaching interactive system based on speech recognition according to claim 1, wherein, Based on a pre-built pronunciation confusion knowledge graph for learners' native language, the prior probability of confusion in the knowledge graph corresponding to each transcription candidate in the acoustic recognition results is retrieved, including: After consulting a pronunciation dictionary, the standard phoneme sequence corresponding to each of the transcription candidates is extracted; The standard phoneme sequence is compared bit-by-bit with the optimal produced phoneme sequence characterized by the phoneme acoustic confidence information, including: For each phoneme position, if the standard phoneme is consistent with the optimal output phoneme, the correct output probability of the phoneme position is determined based on the sum of all confusion edge probabilities of the phoneme in the pronunciation confusion knowledge graph. If the standard phoneme is inconsistent with the optimal produced phoneme, then query the confusion edge from the standard phoneme to the optimal produced phoneme in the pronunciation confusion knowledge graph, and obtain the conditional probability recorded on the confusion edge as the confusion probability of the phoneme position. The confusion probability and the correct output probability of each phoneme position are multiplied together to obtain the knowledge graph confusion prior probability of each transcription candidate.

5. The intelligent English teaching interactive system based on speech recognition as described in claim 1, characterized in that, Combining the acoustic recognition results, the semantic context representation, and the prior probability of confusion with the knowledge graph, the posterior probability of each transcription candidate is calculated, and the transcription candidate with the highest posterior probability is determined as the intent recognition result, including: Extract the acoustic probability corresponding to each transcription candidate from the acoustic recognition results; Based on the semantic vector and expected response range in the semantic context representation, the semantic prior probability corresponding to each transcription candidate is calculated. The acoustic probability, the semantic prior probability, and the knowledge graph confusion prior probability are multiplied together, and the product result of all transcription candidates is normalized to obtain the posterior probability of each transcription candidate. The transcription candidate with the highest posterior probability is determined as the intent recognition result.

6. The intelligent English teaching interactive system based on speech recognition as described in claim 1, characterized in that, Based on the intent recognition result, a standard phoneme sequence is extracted and compared phoneme-by-phoneme with the optimal output phoneme sequence from the acoustic recognition result. Combined with the phoneme acoustic confidence level, a pronunciation error diagnosis result is generated, including: Based on the pronunciation dictionary, extract the standard phoneme sequence corresponding to the intent recognition result; The standard phoneme sequence is compared with the optimal produced phoneme sequence on a phoneme-by-phoneme alignment basis, and each phoneme position is determined to be either consistent in pronunciation or deviated in pronunciation. For the phoneme locations identified as pronunciation errors, the corresponding acoustic confidence scores are combined to generate the pronunciation error diagnosis results, which include the standard phoneme, the error phoneme, and the error confidence score.

7. The intelligent English teaching interactive system based on speech recognition as described in claim 1, characterized in that, The inference confidence is calculated based on the posterior probability of each transcription candidate, and multi-level interactive disambiguation is performed in conjunction with the pronunciation error diagnosis results to generate and output interactive teaching feedback, including: The inference confidence level is calculated and the confidence level is determined based on the first posterior probability value of the transcription candidate with the highest posterior probability and the ratio of the first posterior probability value to the second posterior probability value of the transcription candidate with the second highest posterior probability. The teaching value score of each phoneme is calculated by weighting the prevalence of mother tongue transfer confusion, the matching degree of current teaching focus, and the learner's historical error frequency for each phoneme in the pronunciation error diagnosis results. Based on the confidence level and the teaching value score, the strategy corresponding to the level in the multi-level interactive disambiguation strategy is selected to perform disambiguation processing, generate interactive teaching feedback, and output it.

8. The intelligent English teaching interactive system based on speech recognition according to claim 7, wherein, Based on the confidence level and the instructional value score, select the corresponding level of the multi-level interactive disambiguation strategy for disambiguation processing, including: When the confidence level is high and there are no biased phonemes whose teaching value scores exceed the preset teaching value threshold, the first level of disambiguation strategy is selected. In the generated dialogue response, the keywords in the intent recognition result are naturally repeated with correct pronunciation to perform implicit disambiguation. When the confidence level is low, a second-level disambiguation strategy is selected. Based on the near-sounding words in the pronunciation confusion knowledge graph, confirmation candidates are selected. The learner is asked to confirm the intention through dialogue and questioning. The confirmation intention label is obtained. The standard phoneme sequence corresponding to the confirmation intention label is compared with the optimal output phoneme sequence to locate the erroneous phoneme and achieve explicit confirmation disambiguation. When the confidence level is high and there are erroneous phonemes with teaching value scores exceeding the teaching value threshold, the third-level disambiguation strategy is selected to directly point out the erroneous phonemes to the learner and match the pronunciation training of the smallest contrasting word containing the erroneous phonemes for explicit teaching disambiguation.

9. The intelligent English teaching interactive system based on speech recognition according to claim 8, wherein, Also includes: The confirmation intent tags obtained from the interactive teaching feedback are compared with the corresponding optimal output phoneme sequences at the phoneme level to extract phoneme-level confusion events. The phoneme-level confusion events are matched with existing phoneme confusion edges in the pronunciation confusion knowledge graph; For successfully matched phoneme confusion edges, the corresponding conditional probability values ​​are updated using a sliding weighting method; For phoneme-level obfuscation events that fail to match and whose cumulative frequency reaches the new threshold, a corresponding phoneme obfuscation edge is created and written into the pronunciation obfuscation knowledge graph.

10. A voice recognition-based intelligent English teaching interaction method, characterized in that, The method is applied to the module implementation of the speech recognition-based intelligent English teaching interactive system according to any one of claims 1 to 9, and the method includes: Acquire learners’ speech signals and corresponding dialogue context information in teaching dialogue scenarios, and analyze the acoustic recognition results, phoneme acoustic confidence information and semantic context representation accordingly. Based on the pre-built pronunciation confusion knowledge graph for learners' native language, the prior probability of knowledge graph confusion for each transcription candidate in the acoustic recognition result is obtained by querying. Combining the acoustic recognition results, the semantic context representation, and the prior probability of confusion in the knowledge graph, the posterior probability of each transcription candidate is calculated, and the transcription candidate with the highest posterior probability is determined as the intent recognition result; Based on the intent recognition result, the standard phoneme sequence is extracted and compared phoneme by phoneme with the optimal output phoneme sequence in the acoustic recognition result. Combined with the phoneme acoustic confidence information, a pronunciation error diagnosis result is generated. The inference confidence is calculated based on the posterior probability of each transcription candidate, and multi-level interactive disambiguation is performed in conjunction with the pronunciation error diagnosis results to generate and output interactive teaching feedback.

Images