English teaching training system and method fusing semantic matching and cognitive evaluation

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an English teaching training system and method fusing semantic matching and cognitive assessment, and the method comprises the steps: synchronously capturing a text response, a voice intonation, an eye movement track, a facial micro-expression and a touch rhythm generated in a learning process; semantic deviation deconstruction and cognitive intention quantization processing are carried out on the learning interaction original sequence, and a bidirectional deep semantic matching network is adopted to carry out context alignment on student answers and target corpora; based on the word meaning divergence point set and the cognitive load multi-scale vector, extracting a nonlinear diffusion trajectory of a learning state by using a time gating multi-layer recursive trajectory evolution algorithm; the knowledge point nodes, the deviation type nodes and the emotion triggering nodes associated with the emotion instability candidate segments are fused to construct a local learning map; and forming an emotion cognition feedback result driven by learning interest based on the local learning map and the self-adaptive error correction intervention sequence. The method has the advantage of improving the learning interest of students.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an English teaching and training system and method integrating semantic matching and cognitive assessment. Background Art

[0002] With the rapid development of information technology and artificial intelligence, English teaching is gradually moving towards personalized and intelligent learning. Traditional English teaching methods focus on teacher explanation and student acceptance, ignoring individual differences among students, particularly in cognitive load, learning motivation, and emotional factors. This results in students' learning interest and efficiency being unable to be sustained. Students often encounter learning bottlenecks during the learning process, especially when their self-confidence is undermined, causing their learning efficiency to plummet, which in turn affects the quality of their learning. While existing intelligent education systems can evaluate students' responses through automated scoring and feedback mechanisms, they generally lack in-depth analysis and intervention of students' emotional states. Students may feel frustrated by frequent incorrect feedback or slow progress in answering questions, leading to a significant decrease in their learning interest and motivation. Therefore, it is necessary to design an English teaching and training system and method that integrates semantic matching and cognitive assessment to enhance students' learning interest. Summary of the Invention

[0003] In response to the deficiencies in the prior art, the present invention provides an English teaching and training system and method that integrates semantic matching and cognitive assessment, which has the advantage of enhancing students' learning interest and solves the problems in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose of enhancing students' learning interest, the present invention provides the following technical solution: an English teaching and training method integrating semantic matching and cognitive assessment, comprising the following steps:

[0005] Synchronously capture text responses, voice intonation, eye movement trajectories, facial micro-expressions, and touch rhythms generated during the learning process, and perform real-time alignment on cross-modal data streams to obtain the original sequence of learning interactions;

[0006] The original learning interaction sequence is subjected to semantic deviation deconstruction and cognitive intention quantification. A bidirectional deep semantic matching network is used to contextually align student answers with the target corpus. Combined with attention clustering-based eye movement hotspot mapping and emotion tensor spectrum, a set of word meaning divergence points and a multi-scale vector of cognitive load are generated.

[0007] Based on the word meaning divergence point set and the multi-scale vector of cognitive load, a time-gated multi-layer recursive trajectory evolution algorithm is used to extract the nonlinear diffusion trajectory of the learning state. The coupled threshold detection of trajectory distribution entropy and emotion polarity transition rate is then used to identify candidate segments of emotional instability.

[0008] The knowledge point nodes, deviation type nodes, and emotion trigger nodes associated with candidate emotional instability segments are integrated to construct a local learning graph. Furthermore, the adaptive error correction intervention sequence is deduced by combining the learning path dependency weights and error evolution link aggregation.

[0009] Based on the local learning map and adaptive error correction intervention sequence, combined with the motivational sensitivity parameters in the learner's historical self-efficacy model and the current cognitive load peak position, an emotional cognitive feedback result driven by learning interest is formed.

[0010] Preferably, the process of obtaining the original sequence of learning interaction is:

[0011] The text input, voice expression, touch operation, eye movement trajectory and facial micro-expression changes of students in the process of English learning are collected in real time through a multi-source data acquisition device;

[0012] The multimodal sequence alignment is achieved by using timestamp calibration and synchronous encoding mechanism for input data of different modalities;

[0013] Text input is segmented and semantically annotated using the natural language processing component. Voice data is extracted through voiceprint recognition and intonation curvature analysis to extract speech features. Eye movement trajectories are collected by an eye tracker and mapped to the question interface. Micro-expressions are converted into emotion feature tensors using the facial action coding system. Touch rhythm is quantified through response frequency and click location.

[0014] Finally, it is stored in a unified learning interaction raw sequence format.

[0015] Preferably, the process of implementing semantic deviation deconstruction and cognitive intention quantification processing on the original sequence of learning interaction is:

[0016] Based on a bidirectional deep semantic matching network, the student's answers are semantically aligned with the target corpus at the context level, and the semantic similarity and deviation vector between each unit sentence pair are calculated;

[0017] Combined with students' historical answer records under the same corpus structure, potential semantic misunderstandings are marked to form a trajectory of word meaning deviation;

[0018] At the same time, the information characteristics reflecting cognitive load in the original sequence of learning interaction are extracted, including eye movement jump frequency, intonation mutation rate, and touch rhythm fracture index. Through normalization and principal component analysis, a multi-dimensional vector of cognitive intention is constructed.

[0019] Preferably, the eye movement hotspot mapping and emotion tensor spectrum process based on attention clustering is:

[0020] The fixation point coordinates in the eye movement data are spatially mapped to the question interface structure, and the multi-head attention mechanism is used to perform weighted aggregation on the fixation time distribution to generate an eye movement hotspot area map;

[0021] Locations in the hotspot area that appear frequently but have large answer deviations are marked as high-risk cognitive nodes;

[0022] At the same time, the facial micro-expression sequence is converted into a time-labeled emotion tensor flow, and a spectral modeling structure is introduced to model the evolution trend of the emotion tensor throughout the learning task.

[0023] Preferably, the process of generating a word meaning divergence point set and a cognitive load multi-scale vector is as follows:

[0024] In the identified hot cognitive areas, further determine the semantic differences between the key words in the students' answers and the standard corpus;

[0025] By combining the word vector transfer model with syntactic graphs, we can identify core phrases with contextual conflicts or conceptual ambiguity and form a set of semantic divergence points.

[0026] At the same time, the cognitive load parameters are calculated at multiple time scales by combining eye movement indicators, speech indicators, and touch behavior indicators;

[0027] The cognitive load vectors of different modalities are temporally integrated through a multi-scale fusion mechanism to generate a multi-scale cognitive load vector.

[0028] Preferably, the nonlinear diffusion trajectory process of extracting the learning state using the time-gated multi-layer recursive trajectory evolution algorithm is:

[0029] Taking the word meaning divergence point set and the multi-scale vector of cognitive load as input, a multi-layer recurrent neural network model based on time gating mechanism is used for temporal modeling.

[0030] The output of each time step of the model is encoded into trajectory nodes, the state change rate and fluctuation amplitude are calculated, and a nonlinear diffusion trajectory is formed;

[0031] Introducing a gated attention mechanism during recursive training to identify historical behavior segments that contribute significantly to the current state;

[0032] The frequency entropy of trajectory distribution and the density estimation of state transition points are used to model the state stability and discreteness during the learning process.

[0033] Preferably, the process of identifying candidate emotional instability segments is as follows:

[0034] In the nonlinear diffusion trajectory, the emotional polarity transition rate and the magnitude of cognitive load gradient change in consecutive segments are calculated;

[0035] If there are multiple consecutive segments where the emotion polarity index changes from positive to negative and the cognitive load increases by more than the threshold, it is marked as a potential emotion instability interval;

[0036] The coupled threshold detection method is used to jointly analyze the emotion polarity transition rate and trajectory distribution entropy.

[0037] Preferably, the process of deducing the adaptive error correction intervention sequence is:

[0038] The knowledge point nodes, semantic deviation type nodes and emotion-triggering external condition nodes corresponding to the candidate emotional instability segments are integrated to construct a ternary relationship graph.

[0039] Model the dependencies between nodes in the learning path based on graph neural networks and quantify the impact factors of the error propagation chain;

[0040] By combining the aggregation degree and edge weight of each node in the graph, representative error patterns and intervention windows are identified;

[0041] Introducing historical data of learners' individual behavior paths, and using reinforcement learning strategies to dynamically plan the optimal error correction path;

[0042] Finally, a multi-step error correction intervention sequence is output.

[0043] Preferably, the process of forming the emotional cognitive feedback result driven by learning interest is:

[0044] Based on the adaptive error correction intervention sequence, combined with the learner historical self-efficacy model, learning motivation and incentive response are modeled;

[0045] Extract the motivational sensitivity parameter from the self-efficacy model and combine it with the current cognitive load peak position to determine the feedback intervention point;

[0046] Generate feedback content that matches the current emotional state based on the intervention point.

[0047] An English teaching and training system integrating semantic matching and cognitive assessment, comprising:

[0048] Interaction Collection Module: Synchronously captures multimodal data such as text, voice, eye movements, micro-expressions, and touch generated by students during the learning process, and aligns them in real time to generate a unified original sequence of learning interactions;

[0049] Feature modeling module: This module performs semantic deviation analysis and cognitive state extraction on the original learning interaction sequence, and generates word meaning divergence points and multi-scale cognitive load vectors by combining eye movement hotspots and emotion spectra.

[0050] Trajectory Recognition Module: Based on word meaning and cognitive features, it uses a time-gated neural network to extract the nonlinear diffusion trajectory of learning states and identify potential emotional instability segments;

[0051] Error correction deduction module: Constructs a local learning graph reflecting the correlation between emotional instability and derives personalized adaptive error correction intervention strategies based on the graph structure and path dependency;

[0052] Emotional feedback module: Integrates the historical self-efficacy model with the current learning status to generate motivational and personalized emotional cognitive feedback content.

[0053] Compared with the existing technology, the present invention provides an English teaching and training system and method that integrates semantic matching and cognitive assessment, which has the following beneficial effects:

[0054] By integrating multimodal learning behavior data with deep semantic analysis technology, this paper systematically constructs an intelligent closed-loop mechanism from learning state perception to emotional response intervention. The synchronous capture and time-series alignment of multimodal data achieves a comprehensive restoration of the learning process, providing a true dynamic foundation for subsequent state identification. Through semantic deviation deconstruction and cognitive load modeling, students' understanding breakpoints and potential cognitive bottlenecks are accurately located, enhancing the expressiveness of individualized learning profiles. Based on the extraction of nonlinear diffusion trajectories and the detection of emotional transition thresholds, it can effectively identify latent emotional instability nodes in the learning process and achieve early warning of emotional states. By leveraging local learning graph construction and graph neural inference mechanisms, a causal path between knowledge bias, emotional triggers, and error evolution is established, supporting the generation of strategic error correction intervention sequences. By integrating learners' historical self-efficacy and incentive response characteristics, feedback content is generated that matches the individual's emotional and motivational state, providing multimodal emotional support and cognitive guidance at the appropriate time. This not only enhances the acceptability and motivational effectiveness of feedback, but also promotes the sustained drive and deep internalization of learning interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the method of the present invention;

[0056] Figure 2 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] Example 1

[0059] See also Figure 1 As shown, an English teaching and training method integrating semantic matching and cognitive assessment according to an embodiment of the present invention includes the following steps:

[0060] S1: Synchronously capture the text responses, voice intonation, eye movement trajectories, facial micro-expressions, and touch rhythm generated during the learning process, and perform real-time alignment on cross-modal data streams to obtain the original sequence of learning interactions.

[0061] The process of obtaining the original sequence of learning interaction in S1 is:

[0062] The text input, voice expression, touch operation, eye movement trajectory and facial micro-expression changes of students in the process of English learning are collected in real time through a multi-source data acquisition device;

[0063] A multi-source data acquisition device is used to collect the following data in real time: text input: keyboard or handwriting input content; voice expression: recording reading or answering voice through a microphone; touch operation: recording click position, sliding trajectory, etc.; eye movement trajectory: capturing gaze point and gaze time through eye movement; facial micro-expression: capturing expression change frame sequence through camera.

[0064] The multimodal sequence alignment is achieved by using timestamp calibration and synchronous encoding mechanism for input data of different modalities;

[0065] Assign high-precision timestamps in a unified format to all types of modal data; use a synchronous encoding mechanism to align data from different sources in chronological order; form a unified timeline of multimodal interaction events to support subsequent cross-modal analysis.

[0066] Text input is segmented and semantically annotated through the natural language processing component. Voice data is extracted through voiceprint recognition and intonation curvature analysis to extract speech features. Eye movement trajectories are collected by an eye tracker and mapped to the question interface. Micro-expressions are converted into emotion feature tensors through the facial action coding system. Touch rhythm is quantified through response frequency and click position.

[0067] Capture the natural language content entered by students in the input box or answer area; use natural language processing components to perform the following: word segmentation: For example, using tools such as BERT or LTP for Chinese and English word segmentation; part-of-speech tagging: assigning grammatical categories to each word; named entity recognition: identifying entities such as names of people, places, and terms; dependency parsing: constructing sentence structure diagrams; semantic role labeling: identifying the semantic relationship between verbs and related arguments; output: structured text feature vector after semantic annotation;

[0068] Speech data processing: Extracting individual characteristics and language expression characteristics from students' speech. Specific process: Using a microphone to collect speech data in real time; performing the following processing steps: Speech segmentation: Segmenting the original audio by pauses or semantic boundaries; Voiceprint recognition: Extracting the student's individual identity feature vector; Intonation curvature analysis: Extracting prosodic parameters such as pitch, speaking rate, energy, and spectral variation; Identifying emotional tones in speech; Output: Speech prosodic feature vector + voiceprint vector;

[0069] Eye movement tracking processing: Identify the distribution and changes of students' visual attention. Specific process: Use an eye tracker to track gaze in real time; capture the following key information: gaze point coordinate sequence; gaze duration; number and amplitude of jumps; map gaze point coordinates to the learning interface structure to determine which question, sentence, or option the user is looking at; output: gaze track data with spatial labels;

[0070] Micro-expression processing: Extracting students' emotional clues and cognitive state changes. Specific process: Using a camera to record a frame sequence of students' facial expression changes, using the facial action coding system to analyze expressions: extracting action unit sequences; mapping the sequences to standard emotion labels; constructing an emotion feature tensor: encoding emotion intensity, change frequency, and duration on the timeline; output: a structured emotion tensor flow sequence;

[0071] Touch rhythm processing: Analyzes operation behavior and response rhythm to assist in cognitive load estimation. Specific process: Collects all touch behaviors on the screen; extracts the following features: response latency (the time from the appearance of the question to the first touch); click frequency and interval distribution; click location hotspot map; gesture type recognition; output: touch behavior rhythm vector;

[0072] Finally, it is stored in a unified learning interaction raw sequence format.

[0073] S2: Implement semantic deviation deconstruction and cognitive intention quantification on the original sequence of learning interactions, use a bidirectional deep semantic matching network to contextually align student answers with the target corpus, and combine eye movement hotspot mapping based on attention clustering and emotion tensor spectrum to generate a set of word meaning divergence points and a multi-scale vector of cognitive load.

[0074] The process of implementing semantic deviation deconstruction and cognitive intention quantification on the original sequence of learning interaction in S2 is as follows:

[0075] Based on a bidirectional deep semantic matching network, the student's answers are semantically aligned with the target corpus at the context level, and the semantic similarity and deviation vector between each unit sentence pair are calculated;

[0076] The student answer text and the target corpus are segmented, denoised, part-of-speech tagged and syntactically analyzed; a dependency analysis tool is used to extract a grammatical structure tree, and the two texts are input into a bidirectional deep semantic matching network; a contextual semantic representation vector is obtained; each pair of sentences or phrases is cross-compared to extract a matching score matrix; semantic similarity indicators and deviation vectors are calculated to obtain a sentence-level deviation matrix and a word pair matching confidence vector; parts with matching confidence below the threshold are marked as potential semantic deviation fragments.

[0077] Combined with students' historical answer records under the same corpus structure, potential semantic misunderstandings are marked to form a trajectory of word meaning deviation;

[0078] From the student's historical answer database, extract: previous answer texts of the same corpus / question type; their corresponding scoring feedback and deviation analysis records; repeat the steps of vector encoding and semantic matching to obtain the deviation vectors of previous answers; compare whether the semantic residual directions of the same keyword in multiple answers are consistent; if there are multiple consecutive deviations from the same meaning, mark them as "high-frequency understanding deviation points" to construct a word meaning deviation trajectory diagram: construct a time series meaning change trajectory for each word; output is a word meaning drift diagram.

[0079] At the same time, we extract information characteristics reflecting cognitive load from the original learning interaction sequence, including eye movement jump frequency, intonation mutation rate, and touch rhythm fracture index. Through normalization and principal component analysis, we construct a multi-dimensional vector of cognitive intention.

[0080] Identify gaze jump events from the gaze point sequence collected by the eye tracker; count the number of jumps per unit time to obtain a jump frequency index; a high jump frequency usually indicates distraction or increased cognitive stress; extract the intonation mutation rate: extract the intonation curve of the speech signal; calculate the first-order derivative of the curve to locate areas with large intonation changes; count the number of mutation segments to indicate emotional fluctuations or cognitive breaks in speech expression; extract the touch rhythm fracture index: extract the time series of touch operations; identify the number of fracture points with click intervals > 2 seconds, reflecting the pauses, hesitations, or mental burden of students during the operation; standardize or normalize the three extracted modal indicators to unify the numerical range and eliminate interference from different scales in the analysis; use the normalized features as input to perform principal component analysis: identify the correlation between features; extract the principal component vector, retain the main cognitive change dimensions, reduce redundant dimensions, and highlight the core factors of cognitive load changes; combine each principal component and its corresponding weight into a cognitive intention vector; this vector represents the learner's cognitive load state and cognitive focus characteristics in the current task, and provides quantitative input for subsequent learning state evolution modeling and emotional instability detection.

[0081] The eye movement hotspot mapping and emotion tensor spectrum process based on attention clustering in S2 is:

[0082] The fixation point coordinates in the eye movement data are spatially mapped to the question interface structure, and the multi-head attention mechanism is used to perform weighted aggregation on the fixation time distribution to generate an eye movement hotspot area map;

[0083] Obtain gaze point data from the eye tracker in the format of , which represents gaze position and time; align the gaze point coordinates and map them to the question interface structure; construct a spatial semantic grid or heat map structure, and spatially anchor the gaze data with the question area as the reference frame; use a multi-head attention mechanism to process gaze sequences: each "head" focuses on different time scales; the attention score uses gaze duration as the main weight factor to strengthen the expression of long gaze areas; after aggregating the attention results of different heads, a high-dimensional attention distribution map is obtained; and finally, an eye movement heat map is generated: the heat represents the degree to which the area attracts the learner's visual attention.

[0084] Locations in the hotspot area that appear frequently but have large answer deviations are marked as high-risk cognitive nodes;

[0085] Compare the heat map with the student response deviation data; mark those interface locations with longer gaze times but lower response accuracy; these areas are identified as high-risk cognitive nodes, indicating that learners have deep deviations in their understanding of this knowledge point or corpus.

[0086] At the same time, the facial micro-expression sequence is converted into a time-labeled emotion tensor flow, and a spectral modeling structure is introduced to model the evolution trend of the emotion tensor throughout the learning task.

[0087] The facial action coding system is used to extract the key action units of micro-expressions; each frame of facial image is converted into a tensor structure to encode different emotional states; the emotion tensor flow is annotated by timestamp and aligned with the learning task process; the time series modeling method is used to analyze the evolution trajectory of the emotion tensor throughout the entire learning task; an emotion spectrum diagram is constructed to depict the changing relationship of emotions at the beginning, middle and end of the task; key emotion transition nodes are identified and coupled with cognitive nodes for analysis to predict and intervene in emotional instability segments.

[0088] The process of generating the word meaning divergence point set and cognitive load multi-scale vector in S2 is as follows:

[0089] In the identified hot cognitive areas, further determine the semantic differences between the key words in the students' answers and the standard corpus;

[0090] The input sources are the eye movement hotspot map after attention clustering and the student's answer text; the core keywords that appear in the text position corresponding to the hotspot area in the student's answer are extracted; these keywords are used as candidate points to enter the word meaning divergence analysis process; the word vector transfer model is used to obtain the context embedding vectors of the student's answer and the corresponding words in the standard corpus; the syntactic dependency graph is introduced to analyze the syntactic structure of the keyword to determine the semantic distance between the same word in the student's answer and the target corpus. If the context embedding difference is large or the syntactic role is different, the word is marked as a word meaning divergence point; all identified divergent words constitute a word meaning divergence point set.

[0091] By combining the word vector transfer model with syntactic graphs, we can identify core phrases with contextual conflicts or conceptual ambiguity and form a set of semantic divergence points.

[0092] Use the pre-trained language model to perform contextual semantic encoding on student answers and standard answers; extract the contextual embedding representation of each corresponding word in the two corpora; calculate the cosine similarity to determine the degree of word meaning migration, and if it is lower than the set threshold, mark it as a semantic deviation point; construct a syntactic dependency graph: perform dependency syntactic analysis on student answers and standard answers; establish a syntactic structure tree to determine the role of keywords in the sentence and their modification relationship; if the syntactic structure of the same phrase is significantly different in the two contexts, further enhance the ambiguity judgment of the word; mark phrases with low word vector similarity and deviation in syntactic structure as "word meaning divergence points" and summarize all such words to form a set of word meaning divergence points in this round of learning interaction.

[0093] At the same time, the cognitive load parameters are calculated at multiple time scales by combining eye movement indicators, speech indicators, and touch behavior indicators;

[0094] Extract jump events from the gaze point sequence; count the number of jumps in different time windows and calculate the frequency value; different time scales correspond to different degrees of cognitive reaction speed; speech indicators: extract the curve of speech frequency change over time; calculate the first-order derivative of the curve to identify mutation points; calculate the number of mutations in each answering cycle to reflect emotional tension or cognitive instability; touch indicators: extract the timestamp sequence of click behavior; count the number of break points with a click interval greater than 2 seconds; the fracture density in different time windows can reflect the smoothness of operation and the degree of thinking jamming.

[0095] The cognitive load vectors of different modalities are temporally integrated through a multi-scale fusion mechanism to generate a multi-scale cognitive load vector.

[0096] The extracted multimodal features such as jump frequency, mutation rate, and fracture index are standardized to keep them in a unified numerical range. The features of different time windows are multi-dimensionally spliced ​​to construct a hierarchical input matrix. The modal information and temporal context structure are retained. The sequence features are temporally modeled and dimensionally compressed using an attention fusion mechanism, principal component analysis, or bidirectional analysis. Finally, a fixed-dimensional multi-scale vector of cognitive load is generated to reflect the cognitive intention characteristics of the current learning state.

[0097] S3: Based on the word meaning divergence point set and the multi-scale vector of cognitive load, a time-gated multi-layer recursive trajectory evolution algorithm is used to extract the nonlinear diffusion trajectory of the learning state. The coupled threshold detection of trajectory distribution entropy and emotion polarity transition rate is then used to identify candidate segments of emotional instability.

[0098] The process of extracting the nonlinear diffusion trajectory of the learning state using the time-gated multi-layer recursive trajectory evolution algorithm in S3 is:

[0099] Taking the word meaning divergence point set and the multi-scale vector of cognitive load as input, a multi-layer recurrent neural network model based on time gating mechanism is used for temporal modeling.

[0100] The divergence point of word meaning indicates the possible semantic deviation that students may have in the process of understanding the content; the cognitive load vector is composed of a fusion of multimodal indicators such as eye movement, voice, and touch, reflecting the individual's attention and cognitive resource usage; after the two are aligned on the time axis, a structured temporal input is formed; the network structure takes into account time interval information, enabling the model to distinguish the rhythm differences between behaviors; the multi-layer design is used to capture the evolution from low-level behavior to high-level strategies; during the training process, the network can automatically learn the deep laws and dynamic transformation process of the evolution of learning status over time.

[0101] The output of each time step of the model is encoded into trajectory nodes, the state change rate and fluctuation amplitude are calculated, and a nonlinear diffusion trajectory is formed;

[0102] Each node represents the student's comprehensive learning state at that moment; the changes between nodes reflect state jumps, cognitive fluctuations or strategy adjustments in the learning process; the trajectory does not unfold linearly, but presents an irregular, diffuse dynamic distribution, reflecting the complexity and diversity of learning behavior.

[0103] Introducing a gated attention mechanism during recursive training to identify historical behavior segments that contribute significantly to the current state;

[0104] This mechanism automatically highlights high-value historical segments, such as locations where cognitive dissonance, emotional abnormalities, or semantic confusion have occurred. It helps the model focus on the most representative past experiences when predicting states, improving modeling interpretability. It also provides a basis for subsequent personalized intervention or diagnosis, such as locating the recurring trigger points of a certain cognitive misunderstanding.

[0105] The frequency entropy of trajectory distribution and the density estimation of state transition points are used to model the state stability and discreteness during the learning process.

[0106] Frequency entropy describes the dispersion of a student's state throughout the learning process. A high frequency entropy indicates frequent state changes, potentially indicating cognitive fluctuations or unclear strategies. Transition point density reflects the frequency of sudden changes in state. More frequent transitions indicate discontinuities, repetitive rework, or cognitive derailment in the learning process. Ultimately, these two assessments can be used to determine the sustainability, coherence, and stability risks of a student's learning state, serving as input for strategy optimization.

[0107] The process of identifying candidate emotional instability segments in S3 is as follows:

[0108] In the nonlinear diffusion trajectory, the emotional polarity transition rate and the magnitude of cognitive load gradient change in consecutive segments are calculated;

[0109] The overall learning trajectory is divided into multiple continuous behavioral segments according to the time series. Each segment contains a learning state node, an emotion tensor, and a cognitive load vector. Each segment reflects the evolution of the student's emotions and cognitive states in a continuous interaction. These segments serve as basic units and enter the subsequent instability analysis process. The emotion tensor flow in each continuous segment is analyzed to extract the evolution of the emotion polarity label in the time series. The frequency and mutation rate of emotions changing from "positive or neutral" to "obviously negative" are counted. A high emotion transition rate indicates that there are obvious emotional fluctuations in the segment, which may be a precursor to emotional instability.

[0110] If there are multiple consecutive segments where the emotional polarity index changes significantly from positive to negative, and the increase in cognitive load exceeds the threshold, it is marked as a potential emotional instability interval;

[0111] A time series analysis of the multi-scale vectors of cognitive load in the same segment is performed to determine whether there is a sudden increase in cognitive load within the segment, with a focus on identifying the turning point from stable to drastic increase. If the load increase rate exceeds the set threshold, the segment is marked as a high-load segment. If a segment simultaneously meets the following requirements: a significant emotional polarity transition rate; and a high amplitude of cognitive load change, the segment is marked as a potential emotional instability interval. This joint judgment mechanism can eliminate false positive segments caused solely by voice intonation or short-term emotional fluctuations, thereby improving the effectiveness of candidate segments.

[0112] The coupling threshold detection method is used to jointly analyze the emotion polarity transition rate and trajectory distribution entropy to eliminate the interference of occasional emotion fluctuations.

[0113] Based on the labeling results, the trajectory distribution entropy is further introduced as an auxiliary filtering indicator to measure the rarity and volatility of the segment in the global state trajectory. If the segment distribution entropy is low, its fluctuation is considered to be within the normal range. Only segments that simultaneously meet the requirements of high emotional transition rate, high load gradient, and high distribution entropy are retained as the final candidate segments for emotional instability.

[0114] S4: The knowledge point nodes, deviation type nodes and emotion trigger nodes associated with the candidate emotional instability fragments are integrated to construct a local learning graph, and the adaptive error correction intervention sequence is deduced by combining the learning path dependency weight and error evolution link aggregation degree.

[0115] The process of deducing the adaptive error correction intervention sequence in S4 is:

[0116] The knowledge point nodes, semantic deviation type nodes and emotion-triggering external condition nodes corresponding to the candidate emotional instability segments are integrated to construct a ternary relationship graph.

[0117] The three core element nodes corresponding to each candidate emotional instability segment are extracted: knowledge point node: the specific knowledge unit involved in the student's current answer; semantic deviation type node: such as definition confusion, causal inversion, pronoun reference error, etc.; external condition node: external stimulus that triggers this state, such as question type conversion, time pressure, interface interaction complexity, etc.; use the above three types of nodes to construct triples, and all triples together form a heterogeneous graph structure, in which the node types are different and the edges represent semantic relationships such as co-occurrence, causality, and evolution.

[0118] Model the dependencies between nodes in the learning path based on graph neural networks and quantify the impact factors of the error propagation chain;

[0119] The graph constructed above is input into the graph neural network: the node vector is initialized from the word vector, knowledge point embedding or historical statistical features; the edge weight represents the strength of co-occurrence or mutual causality between nodes in the historical trajectory; the model aggregates neighbor information through multiple rounds of graph convolution to obtain the state representation vector of each node; the path of adjacent error nodes in the sequence is modeled, and the "influence factor" of the error propagation path is calculated, that is, whether one error is likely to cause the next error; which knowledge points are the "highly contagious" misunderstanding centers.

[0120] By combining the aggregation degree and edge weight of each node in the graph, representative error patterns and intervention windows are identified;

[0121] Analyze the aggregation degree and influence range of nodes in the graph: high-aggregation nodes → frequently appear in multiple error paths → priority intervention; combine edge weight strength and node activity to identify key patterns that "prone to chain errors"; intervention windows are determined by the following strategies: "jump nodes" on the graph structure; weak connection breakpoints within a certain time range before and after emotional instability occurs; and time periods with high frequency of commonalities with other learners in the error pattern cluster.

[0122] Introducing historical data of learners' individual behavior paths, and using reinforcement learning strategies to dynamically plan the optimal error correction path;

[0123] The graph structure and the learner's historical behavior path are used together as the state space. A reinforcement learning model is set up: state: current node characteristics + individual historical behavior state; action: choose whether to intervene and how to intervene; reward function: dynamically adjust based on feedback such as the student's error correction rate, emotional recovery speed, and task completion quality. After model iteration, the optimal intervention decision path can be planned for specific learners and specific error chains.

[0124] The final output is a multi-step error correction intervention sequence, including prompting method, timing, and content form.

[0125] S5: Based on the local learning map and adaptive error correction intervention sequence, combined with the motivational sensitivity parameters in the learner's historical self-efficacy model and the current cognitive load peak position, an emotional cognitive feedback result driven by learning interest is formed.

[0126] The process of forming the emotional cognitive feedback result driven by learning interest in S5 is as follows:

[0127] Based on the adaptive error correction intervention sequence, combined with the learner historical self-efficacy model, learning motivation and incentive response are modeled;

[0128] Integrate behavioral indicators such as learners' past learning records, task completion rate, error correction efficiency, and frequency of active help-seeking; establish an individualized self-efficacy dynamic model to characterize the coupling relationship between their learning motivation fluctuation trend and emotion-performance; extract incentive sensitivity parameters: identify learners' response preferences to positive feedback and negative prompts; generate a nonlinear response curve of learning motivation-feedback intensity as a regulatory basis for subsequent feedback generation.

[0129] Extract the motivational sensitivity parameter from the self-efficacy model and combine it with the current cognitive load peak position to determine the feedback intervention point;

[0130] Extract multi-scale curves of cognitive load from multimodal data; identify moments when cognitive load increases significantly during the learning process; match cognitive peaks with learners' motivational sensitivity, aligning cognitive critical zones with easily motivated windows; and finally, determine specific intervention nodes in terms of timing, ensuring neither premature interruption nor excessive slow response, thus ensuring emotional acceptance of feedback.

[0131] Generate feedback content that matches the current emotional state based on the intervention point, including correction suggestions, learning guidance, and motivational words;

[0132] Judge the learner's current emotional state based on real-time facial micro-expressions, voice emotional characteristics, behavioral rhythm, etc.; provide personalized correction plans based on previously identified semantic deviations; provide simplified explanations, analogy prompts or visual auxiliary information related to the current knowledge points; introduce language styles that adapt to the current emotional state, such as encouraging sentences, positive evaluations, empathetic sentences, etc.; ensure that the feedback discourse is coordinated with the current emotional state in terms of tone, rhythm and content structure to avoid rebellion or emotional aggravation.

[0133] Feedback content is presented in a multimodal format of text, voice, and animation;

[0134] The key content is presented in a concise and clear way using pictures and texts, which is suitable for learners with a visual preference. Feedback is output in a gentle and positive tone through speech synthesis, which is suitable for those with a high auditory acceptance. The emotional interactivity and fun are enhanced by combining animated characters, facial expression transitions and interactive mini-theaters. The dominant feedback presentation method is automatically adjusted according to the learner's preference for different modes.

[0135] Example 2: Figure 2 As shown, an English teaching and training system integrating semantic matching and cognitive assessment includes:

[0136] Interaction Collection Module: Synchronously captures multimodal data such as text, voice, eye movements, micro-expressions, and touch generated by students during the learning process, and aligns them in real time to generate a unified original sequence of learning interactions;

[0137] Feature modeling module: This module performs semantic deviation analysis and cognitive state extraction on the original learning interaction sequence, and generates word meaning divergence points and multi-scale cognitive load vectors by combining eye movement hotspots and emotion spectra.

[0138] Trajectory Recognition Module: Based on word meaning and cognitive features, it uses a time-gated neural network to extract the nonlinear diffusion trajectory of learning states and identify potential emotional instability segments;

[0139] Error correction deduction module: Constructs a local learning graph reflecting the correlation between emotional instability and derives personalized adaptive error correction intervention strategies based on the graph structure and path dependency;

[0140] Emotional feedback module: Integrates the historical self-efficacy model with the current learning status to generate motivational and personalized emotional cognitive feedback content.

[0141] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0142] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An English teaching and training method integrating semantic matching and cognitive assessment, characterized in that: The following steps are involved: Synchronously capture text responses, voice intonation, eye movement trajectories, facial micro-expressions, and touch rhythms generated during the learning process, and perform real-time alignment on cross-modal data streams to obtain the original sequence of learning interactions; The original learning interaction sequence is subjected to semantic deviation deconstruction and cognitive intention quantification. A bidirectional deep semantic matching network is used to contextually align student answers with the target corpus. Combined with attention clustering-based eye movement hotspot mapping and emotion tensor spectrum, a set of word meaning divergence points and a multi-scale vector of cognitive load are generated. Based on the word meaning divergence point set and the multi-scale vector of cognitive load, a time-gated multi-layer recursive trajectory evolution algorithm is used to extract the nonlinear diffusion trajectory of the learning state. The coupled threshold detection of trajectory distribution entropy and emotion polarity transition rate is then used to identify candidate segments of emotional instability. The knowledge point nodes, deviation type nodes, and emotion trigger nodes associated with candidate emotional instability segments are integrated to construct a local learning graph. Furthermore, the adaptive error correction intervention sequence is deduced by combining the learning path dependency weights and error evolution link aggregation. Based on the local learning map and adaptive error correction intervention sequence, combined with the motivational sensitivity parameters in the learner's historical self-efficacy model and the current cognitive load peak position, an emotional cognitive feedback result driven by learning interest is formed.

2. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 1, characterized in that: The process of obtaining the original sequence of learning interaction is: The text input, voice expression, touch operation, eye movement trajectory and facial micro-expression changes of students in the process of English learning are collected in real time through a multi-source data acquisition device; The multimodal sequence alignment is achieved by using timestamp calibration and synchronous encoding mechanism for input data of different modalities; Text input is segmented and semantically annotated using the natural language processing component. Voice data is extracted through voiceprint recognition and intonation curvature analysis to extract speech features. Eye movement trajectories are collected by an eye tracker and mapped to the question interface. Micro-expressions are converted into emotion feature tensors using the facial action coding system. Touch rhythm is quantified through response frequency and click location. Finally, it is stored in a unified learning interaction raw sequence format.

3. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 2 is characterized in that: The process of semantic deviation deconstruction and cognitive intention quantification of the original learning interaction sequence is as follows: Based on a bidirectional deep semantic matching network, the student's answers are semantically aligned with the target corpus at the context level, and the semantic similarity and deviation vector between each unit sentence pair are calculated; Combined with students' historical answer records under the same corpus structure, potential semantic misunderstandings are marked to form a trajectory of word meaning deviation; At the same time, the information characteristics reflecting cognitive load in the original sequence of learning interaction are extracted, including eye movement jump frequency, intonation mutation rate, and touch rhythm fracture index. Through normalization and principal component analysis, a multi-dimensional vector of cognitive intention is constructed.

4. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 3 is characterized in that: The process of eye movement hotspot mapping and emotion tensor spectrum based on attention clustering is: The fixation point coordinates in the eye movement data are spatially mapped to the question interface structure, and the multi-head attention mechanism is used to perform weighted aggregation on the fixation time distribution to generate an eye movement hotspot area map; Locations in the hotspot area that appear frequently but have large answer deviations are marked as high-risk cognitive nodes; At the same time, the facial micro-expression sequence is converted into a time-labeled emotion tensor flow, and a spectral modeling structure is introduced to model the evolution trend of the emotion tensor throughout the learning task.

5. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 4 is characterized in that: The process of generating word meaning divergence point set and cognitive load multi-scale vector is as follows: In the identified hot cognitive areas, further determine the semantic differences between the key words in the students' answers and the standard corpus; By combining the word vector transfer model with syntactic graphs, we can identify core phrases with contextual conflicts or conceptual ambiguity and form a set of semantic divergence points. At the same time, the cognitive load parameters are calculated at multiple time scales by combining eye movement indicators, speech indicators, and touch behavior indicators; The cognitive load vectors of different modalities are temporally integrated through a multi-scale fusion mechanism to generate a multi-scale cognitive load vector.

6. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 5, characterized in that: The nonlinear diffusion trajectory process of extracting the learning state using the time-gated multi-layer recursive trajectory evolution algorithm is: Taking the word meaning divergence point set and the multi-scale vector of cognitive load as input, a multi-layer recurrent neural network model based on time gating mechanism is used for temporal modeling. The output of each time step of the model is encoded into trajectory nodes, the state change rate and fluctuation amplitude are calculated, and a nonlinear diffusion trajectory is formed; Introducing a gated attention mechanism during recursive training to identify historical behavior segments that contribute significantly to the current state; The frequency entropy of trajectory distribution and the density estimation of state transition points are used to model the state stability and discreteness during the learning process.

7. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 6, characterized in that: The process of identifying candidate segments of emotional instability is as follows: In the nonlinear diffusion trajectory, the emotional polarity transition rate and the magnitude of cognitive load gradient change in consecutive segments are calculated; If there are multiple consecutive segments where the emotion polarity index changes from positive to negative and the cognitive load increases by more than the threshold, it is marked as a potential emotion instability interval; The coupled threshold detection method is used to jointly analyze the emotion polarity transition rate and trajectory distribution entropy.

8. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 7, characterized in that: The process of deducing the adaptive error correction intervention sequence is: The knowledge point nodes, semantic deviation type nodes and emotion-triggering external condition nodes corresponding to the candidate emotional instability segments are integrated to construct a ternary relationship graph. Model the dependencies between nodes in the learning path based on graph neural networks and quantify the impact factors of the error propagation chain; By combining the aggregation degree and edge weight of each node in the graph, representative error patterns and intervention windows are identified; Introducing historical data of learners' individual behavior paths, and using reinforcement learning strategies to dynamically plan the optimal error correction path; Finally, a multi-step error correction intervention sequence is output.

9. The English teaching and training method integrating semantic matching and cognitive assessment according to claim 8, characterized in that: The process of forming emotional cognitive feedback results driven by learning interest is as follows: Based on the adaptive error correction intervention sequence, combined with the learner historical self-efficacy model, learning motivation and incentive response are modeled; Extract the motivational sensitivity parameter from the self-efficacy model and combine it with the current cognitive load peak position to determine the feedback intervention point; Generate feedback content that matches the current emotional state based on the intervention point.

10. An English teaching and training system integrating semantic matching and cognitive assessment, characterized in that: The method according to any one of claims 1 to 9 comprises: Interaction Collection Module: Synchronously captures multimodal data generated by students during the learning process, including text, voice, eye movements, micro-expressions, and touch, and aligns them in real time to generate a unified original sequence of learning interactions. Feature modeling module: This module performs semantic deviation analysis and cognitive state extraction on the original learning interaction sequence, and generates word meaning divergence points and multi-scale cognitive load vectors by combining eye movement hotspots and emotion spectra. Trajectory Recognition Module: Based on word meaning and cognitive features, it uses a time-gated neural network to extract the nonlinear diffusion trajectory of learning states and identify potential emotional instability segments; Error correction deduction module: Constructs a local learning graph reflecting the correlation between emotional instability and derives personalized adaptive error correction intervention strategies based on the graph structure and path dependency; Emotional feedback module: Integrates the historical self-efficacy model with the current learning status to generate motivational and personalized emotional cognitive feedback content.

Images