Real-time interactive online Chinese language learning platform

Abstract

The invention belongs to the technical field of online learning, and discloses a real-time interactive online Chinese language learning platform. Comprising the steps of constructing a virtual culture interaction scene; in the dialogue interaction process, continuous dialogue fragments are collected in real time; performing multi-level language bias error analysis on the continuous dialogue segments, identifying language errors, making corresponding language correction suggestions according to the language errors, and feeding back the language correction suggestions to the user through a progressive feedback mechanism; collecting multi-modal data of a user, carrying out nonlinear feature mapping, generating a multi-dimensional state feature vector, and dynamically evaluating a cognitive load and a learning state; the cognitive load and the learning state are fused, interaction regulation and control strategy reasoning is carried out, and the interaction mode and the interaction difficulty in dialogue interaction are dynamically optimized according to an interaction regulation and control strategy; according to the method, accurate control and dynamic regulation and control of the whole Chinese language learning process of the user are realized, personalized online Chinese learning experience is provided, and comprehensive improvement of the language ability of the user is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of online learning, and more specifically, to a real-time interactive online Chinese language learning platform. Background Art

[0002] With the continuous improvement of China's international influence, the demand for Chinese language learning has been growing globally, and Chinese is gradually becoming a language with broad international influence. However, current Chinese language teaching still faces many challenges, such as uneven distribution of teaching resources, obvious geographical restrictions, lack of interactivity in traditional teaching methods, etc., making it difficult to meet diverse and personalized learning needs. In recent years, with the rapid development of real-time audio and video communication, artificial intelligence, and mobile Internet, new technical paths have been provided for language learning. Online teaching methods based on the network enable learners to be no longer restricted by offline classrooms and can achieve real-time interaction through the Internet to obtain a more flexible and immersive learning experience. Therefore, developing an online Chinese language learning platform with real-time interaction capabilities can effectively improve the efficiency of language learning, promote the transformation of Chinese language teaching from indoctrination to interaction, and inject new vitality into international Chinese education.

[0003] The patent with the publication number CN115292499A discloses a real-time interactive online Chinese language learning platform, including: a cloud storage reservation module, a user operation module, an online interaction processing module, and a data execution monitoring module. The cloud storage reservation module is used to input Chinese language learning content and conveniently classify and store the input Chinese language learning content according to keywords. The user operation module is used for users to log in and synchronously connect with the cloud. Through the cooperative design of the cloud storage reservation module, the user operation module, the online interaction processing module, and the data execution monitoring module, this invention enables the platform to facilitate the application of real-time interactive online Chinese language learning according to usage needs, and is convenient for the synchronous application of the learning end, the live broadcast end, and the chat end according to usage needs, thereby forming keyword-directed learning, live broadcast interaction, and classmate interaction, and facilitating the monitoring and differentiated storage of data in the learning platform.

[0004] However, although the above technology can achieve real-time interaction and online learning, it focuses on the functional stacking of module structures. Although it covers aspects such as storage, user operation, interaction processing, and data monitoring, it lacks a deep understanding of user learning behaviors and an intelligent response mechanism, and does not involve key links such as language error analysis and personalized feedback mechanisms, making it difficult to provide learners with an intelligent and personalized learning experience.

[0005] In view of this, the present invention proposes a real-time interactive online Chinese language learning platform to solve the above problems. Summary of the Invention

[0006] To overcome the above-mentioned deficiencies of the prior art and to achieve the above object, the present invention provides the following technical solution: A real-time interactive online Chinese language learning platform, comprising: A scenario construction module, configured to establish a multi-dimensional cultural element matrix and construct a virtual cultural interaction scenario based on the multi-dimensional cultural element matrix; An interaction collection module, configured to continuously collect consecutive dialogue segments in real time during the process of a user having a dialogue interaction with a virtual character in the virtual cultural interaction scenario; A correction feedback module, configured to perform multi-level language error analysis on the consecutive dialogue segments, identify language errors in the consecutive dialogue segments, formulate corresponding language correction suggestions for the language errors, and feedback them to the user through a progressive feedback mechanism; A state evaluation module, configured to collect multi-modal data of the user, perform non-linear feature mapping on the multi-modal data of the user to generate a multi-dimensional state feature vector, and dynamically evaluate the cognitive load and learning state based on the multi-dimensional state feature vector; An interaction optimization module, configured to fuse the cognitive load and learning state, perform interaction regulation strategy reasoning, and dynamically optimize the interaction mode and interaction difficulty in the dialogue interaction according to the interaction regulation strategy.

[0007] Furthermore, the method for continuously collecting consecutive dialogue segments includes: Continuously collect dialogue interaction information between the user and the virtual character, where the dialogue interaction information includes dialogue content and the corresponding dialogue time; wherein, the dialogue content is the natural language text output by the user or the virtual character during the dialogue interaction process, and the dialogue time is the output time of each piece of dialogue content; Obtain the historical duration, perform mean processing on the historical duration to obtain a preliminary response threshold; the historical duration is the response duration of the user during the dialogue interaction with the virtual character at a historical moment; obtain the interaction mode and interaction difficulty of the dialogue interaction, and dynamically adjust the preliminary response threshold based on the interaction mode and interaction difficulty to obtain a dynamic response threshold; after each round of the virtual character outputs natural language text, monitor whether the user outputs natural language text within the dynamic response threshold. If so, determine that the dialogue interaction continues and continue to collect dialogue interaction information. If not, determine that the dialogue interaction ends and use the collected dialogue content as a preliminary dialogue segment; Calculate the semantic similarity between every two adjacent pieces of dialogue content in the preliminary dialogue segment, and compare each semantic similarity with a preset similarity threshold; mark the dialogue content with a semantic similarity less than the similarity threshold as irrelevant content, and mark the dialogue content with a semantic similarity greater than the similarity threshold as continuous content; if the dialogue content is marked as both irrelevant content and continuous content, cancel the mark as continuous content; among the dialogue times corresponding to the irrelevant content, mark the earliest dialogue time as the earliest time and the latest time as the latest time; The continuous content with a conversation time earlier than the earliest time and the irrelevant content corresponding to the earliest time are regarded as a continuous conversation segment; the continuous content with a conversation time later than the latest time and the irrelevant content corresponding to the latest time are regarded as a continuous conversation segment; determine whether there is continuous content between any two pieces of irrelevant content; if so, regard the corresponding irrelevant content and the continuous content therebetween as a continuous conversation segment; if not, no operation is performed.

[0008] Further, the method for obtaining the dynamic response threshold includes: Preset a coefficient set, which includes a mode set and a difficulty set; the mode set includes adjustment coefficients corresponding to different interaction modes, and the difficulty set includes adjustment coefficients corresponding to different interaction difficulties; obtain the corresponding adjustment coefficient from the coefficient set according to the interaction mode and interaction difficulty of the dialogue interaction; take the product of the adjustment coefficient of the interaction mode and the adjustment coefficient of the interaction difficulty as the overall adjustment coefficient; take the product of the overall adjustment coefficient and the preliminary response threshold as the dynamic response threshold; The method for calculating the semantic similarity between two adjacent pieces of conversation content includes: Adopt the SimCSE model to convert each piece of conversation content into a corresponding conversation vector respectively; calculate the cosine similarity between the conversation vectors corresponding to every two adjacent pieces of conversation content in turn, and use it as the semantic similarity.

[0009] Further, the steps for identifying language errors in continuous conversation segments include: Step S101: Conduct a lexical error analysis on each continuous conversation segment to identify lexical errors; Input each continuous segment into a trained context-aware language model respectively, and calculate the masked prediction probability of each phrase in each continuous segment; preset a probability threshold, compare each masked prediction probability with the probability threshold respectively, mark the phrases with masked prediction probabilities less than the probability threshold as incorrect phrases, and do not mark the phrases with masked prediction probabilities greater than or equal to the probability threshold; combine the same incorrect phrases in the corresponding continuous conversation segment as the lexical errors of the corresponding continuous conversation segment; Step S102: Conduct a grammar error analysis on each continuous conversation segment to identify grammar errors; Input each continuous conversation segment into a trained grammar error correction model respectively to identify the grammar errors in each continuous conversation segment; Step S103: Conduct a pragmatic error analysis on each continuous conversation segment to identify pragmatic errors; Input each continuous conversation segment into a trained context analysis model respectively to identify the pragmatic errors in each continuous conversation segment; Step S104: Determine the language errors in each continuous dialogue segment by synthesizing vocabulary errors, grammar errors, and pragmatic errors; Synthesize the same vocabulary errors, grammar errors, and pragmatic errors in the corresponding continuous dialogue segments as the language errors in the corresponding continuous dialogue segments.

[0010] Further, the method for formulating corresponding language correction suggestions for language errors includes: Take each continuous dialogue segment and its corresponding language errors as a set of segment collections, and input each set of segment collections into the trained language correction model respectively to output the correction set corresponding to each set of segment collections; the correction set includes a set of suggestions corresponding to each language error, and the set of suggestions includes types of correction suggestions, where is an integer greater than 1; Randomly select one correction suggestion from each set of suggestions corresponding to each continuous dialogue segment, and construct a set of suggestion subsets, and construct sets of suggestion subsets for each continuous dialogue segment, where is an integer greater than 1; Use the SimCSE model to convert the correction suggestions in each set of suggestion subsets into corresponding suggestion vectors respectively; Take the dialogue vector corresponding to each continuous dialogue segment and its corresponding set of suggestion subsets as a set of analysis collections, and input each set of analysis collections into the trained correction analysis model respectively to predict the corresponding index set; among them, the index set includes language fluency and cultural fit, and the correction analysis model is a deep neural network model; A preset ratio set, where the ratio set includes the ratio coefficients corresponding to language fluency and cultural fit; According to the ratio set, perform weighted summation on the language fluency and cultural fit in each set of index sets to obtain the comprehensive score of each set of suggestion subsets; Compare the same comprehensive scores of the corresponding continuous dialogue segments, and take the suggestion subset with the highest comprehensive score as the language correction suggestion for the corresponding continuous dialogue segment.

[0011] Further, the step of feeding back to the user through the progressive feedback mechanism includes: Step S201: Feed back the language errors corresponding to each continuous dialogue segment to the user and obtain a first modified content; Step S202: Identify the language errors in the first modified content. If there are still language errors, go to step S203. If there are no language errors, the feedback ends; Step S203: Based on the pre-constructed language knowledge base, obtain the error reasons corresponding to the language errors identified in step S202; Step S204: Feed back the error reasons to the user and obtain a second modified content; Step S205: Identify language errors in the content of the secondary modification. If there are still language errors, proceed to Step S206; if there are no language errors, feedback ends. Step S206: Feedback the correction set of all consecutive dialogue segments and language correction suggestions to the user.

[0012] Furthermore, the user multi-modal data includes a sequence of facial images and an eye movement trajectory; the sequence of facial images includes the facial images collected during the process of the user's dialogue interaction with the virtual character, where \(n\) is an integer greater than 1; the eye movement trajectory includes fixation points, fixation durations, and saccade frequencies; The method for generating the multi-dimensional state feature vector includes: Using a trained emotion recognition model, sequentially recognize each facial image in the sequence of facial images, and output the recognition result of each facial image. The recognition result includes a category label and an emotion intensity; the category label is a digital label corresponding to an emotion category, and the digital labels corresponding to different emotion categories are all different; the range of the emotion intensity is , ; According to the category label in the recognition result, obtain the corresponding emotion category; count the number of each emotion category and mark it as the emotion quantity; compare all the emotion quantities, and take the emotion category with the largest emotion quantity as the user's emotion state; perform mean processing on all the emotion intensities corresponding to the emotion state to obtain the average emotion intensity; the emotion recognition model is a convolutional neural network model; Using a clustering algorithm to cluster all the fixation points in the eye movement trajectory to obtain multiple fixation regions; count the number of fixation regions and mark it as the fixation quantity; perform mean processing on all the fixation durations in the eye movement trajectory to obtain the average fixation duration; preset a weight set, and the weight set includes the weight coefficients corresponding to the reciprocal of the fixation quantity, the average fixation duration, and the reciprocal of the saccade frequency; perform standardization processing on the fixation quantity, the average fixation duration, and the saccade frequency, and perform weighted summation on the reciprocal of the standardized fixation quantity, the average fixation duration, and the reciprocal of the saccade frequency according to the weight set to obtain the attention concentration; Count the total number of lexical errors, grammar errors, and pragmatic errors in all consecutive dialogue segments and mark it as the language error quantity; count the number of times the progressive feedback mechanism gives feedback to the user and mark it as the feedback quantity; Generate a multi-dimensional state feature vector according to the emotion state, average emotion intensity, attention concentration, language error quantity, and feedback quantity.

[0013] Furthermore, the method for dynamically evaluating cognitive load and learning state includes: Take the category label corresponding to the emotional state, the average emotional intensity, and the attention concentration in the multi-dimensional state feature vector as the first analysis data, and take the number of language errors and the number of feedbacks in the multi-dimensional state feature vector as the second analysis data; dynamically evaluate the learning state according to the first analysis data, dynamically evaluate the cognitive load according to the second analysis data, and the method for dynamically evaluating the learning state according to the first analysis data is the same as the method for dynamically evaluating the cognitive load according to the second analysis data; The method for dynamically evaluating the learning state according to the first analysis data includes: Construct multiple fuzzy sets for each data in the first analysis data respectively; convert each data in the first analysis data into the membership degree of the corresponding fuzzy set through the fuzzification technology respectively; define fuzzy rules, match the fuzzified first analysis data with the fuzzy rules respectively, and use the fuzzy inference method to perform fuzzy inference to obtain the fuzzy inference result, and the fuzzy inference result is the membership degree of each learning state; compare each membership degree respectively, and take the learning state with the largest membership degree as the dynamically evaluated learning state.

[0014] Furthermore, the method for inferring the interactive regulation strategy includes: Obtain the corresponding load label according to the cognitive load, and obtain the corresponding state label according to the learning state; wherein, the load label is the digital label corresponding to the cognitive load, and the state label is the digital label corresponding to the learning state; input the load label and the state label into the trained direction judgment model, and predict the corresponding direction label, and the direction judgment model is a deep neural network model; wherein, the direction label is the digital label corresponding to the regulation direction, and the regulation direction includes positive regulation and negative regulation; Set the corresponding guidance intensity for each interaction mode respectively, sort all interaction modes according to the corresponding guidance intensity from high to low to generate a mode sequence; sort all interaction difficulties from low to high to generate a difficulty sequence; If the regulation direction is positive regulation, obtain the interaction modes ranked behind the current mode from the mode sequence, and obtain the interaction difficulties ranked behind the current difficulty from the difficulty sequence; if the regulation direction is negative regulation, obtain the interaction modes ranked in front of the current mode from the mode sequence; obtain the interaction difficulties ranked in front of the current difficulty from the difficulty sequence; wherein, the current mode is the interaction mode corresponding to the user's current dialogue interaction, and the current difficulty is the interaction difficulty corresponding to the user's current dialogue interaction; mark all the obtained interaction modes as regulation modes, and mark all the obtained interaction difficulties as regulation difficulties; Randomly select a regulation mode and a regulation difficulty to construct a regulation strategy, and a total of regulation strategies are constructed, is an integer greater than 0; calculate the regulation effects of each group of regulation strategies in turn, compare all the regulation effects, and take the regulation strategy with the largest regulation effect as the interactive regulation strategy.

[0015] Further, the method for calculating the regulation effects of each group of regulation strategies in turn includes: Set different digital tags for different interaction modes and mark them as mode tags; set different digital tags for different interaction difficulties and mark them as difficulty tags; use the multi-dimensional state feature vector, the mode tag of the current mode, the difficulty tag of the current difficulty, and the mode tag of the regulation mode and the difficulty tag of the regulation difficulty in a regulation strategy as a group of regulation data; input each group of regulation data into the trained effect prediction model respectively to predict the corresponding regulation effect; the effect prediction model is a deep neural network model.

[0016] Technical effects and advantages of an online Chinese language learning platform with real-time interaction of the present invention: By constructing a virtual cultural interaction scenario and integrating multi-dimensional historical and cultural elements, it provides an immersive language practice environment for users, enhances the understanding of Chinese traditional culture, and improves the interest and sense of participation in Chinese language learning; during the real-time conversation process, continuous conversation segments are collected and multi-level language error analysis is carried out to accurately identify various errors such as vocabulary, grammar, and pragmatics, and gradually formulated targeted language correction suggestions to effectively promote the systematic improvement of users' Chinese language level; collect users' facial expressions and eye movement data, dynamically analyze the cognitive load and learning state, and combine intelligent reasoning to optimize the interaction mode and difficulty in real time, making the learning process more in line with users' learning needs and cognitive characteristics; achieve precise control and dynamic regulation of the whole process of users' Chinese language learning, provide users with a personalized and efficient online Chinese learning experience, and promote the comprehensive improvement of users' language ability. Brief Description of the Drawings

[0017] Figure 1 It is a schematic diagram of an online Chinese language learning platform with real-time interaction according to Embodiment 1 of the present invention.

[0018] Figure 2 It is a flowchart of an online Chinese language learning platform with real-time interaction according to Embodiment 1 of the present invention. Detailed Embodiment

[0019] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] Example 1

[0021] Please refer to Figure 1 and Figure 2 As shown, an online Chinese language learning platform for real-time interaction in this embodiment includes a scenario construction module, an interaction collection module, a correction and feedback module, a status evaluation module, and an interaction optimization module; each module is connected by wired and / or wireless means to achieve data transmission between modules.

[0022] The scenario construction module is used to establish a multi-dimensional cultural element matrix and construct a virtual cultural interaction scenario based on the multi-dimensional cultural element matrix.

[0023] The multi-dimensional cultural element matrix is established by those skilled in the art according to Chinese history and culture. Each element in the multi-dimensional cultural element matrix includes but is not limited to dynasties, regions, language expressions, social scenarios, etc.; Dynasties are used to reflect the time clues and political and cultural characteristics in the evolution of Chinese history; for example, political systems (such as the prefecture and county system in the Qin Dynasty, the three departments and six ministries system in the Tang Dynasty, etc.), cultural styles (such as the prosperity of Confucianism in the Han Dynasty, the openness and diversity in the Tang Dynasty, the development of Neo-Confucianism in the Song Dynasty, etc.), social fashions (such as the pure conversation and admiration for mystery in the Wei and Jin Dynasties, the boldness and luxury in the Tang Dynasty, the refinement and restraint in the Song Dynasty, etc.); Regions are used to reflect the spatial distribution and regional characteristics of Chinese culture; for example, natural environments (such as arid plains in the north, water towns in the south of the Yangtze River, grasslands and gobi in the northwest, etc.), cultural circles (such as Guanzhong culture, Bashu culture, Lingnan culture, etc.), economic characteristics (such as handicrafts and commerce in the south of the Yangtze River, agriculture in the north, animal husbandry in the northwest, etc.); Language expressions are used to reflect the language usage characteristics in different historical periods and regions; for example, language habits (such as appellations, greeting methods, honorifics, etc.), literary styles (such as the boldness and gracefulness of Tang poetry, the elegance and restraint of Song ci-poems, the market-oriented expressions of Ming and Qing novels, etc.); Social scenarios are used to reflect the specific environments and interaction methods of social role activities; for example, market scenarios (such as teahouses, wine shops, streets, etc.), court scenarios (such as court meetings, hunting, etc.), academy scenarios (such as academy lectures, imperial examinations, etc.).

[0024] The method for constructing a virtual cultural interaction scenario based on the multi-dimensional cultural element matrix includes: For each element in the multi-dimensional cultural element matrix, a corresponding role system, interaction rule system, and intelligent dialogue system are designed, and technologies such as mixed reality and natural language processing are comprehensively used to construct a virtual cultural interaction scenario; both mixed reality and natural language processing are existing technologies, and the specific process will not be elaborated here; Among them, the method for designing the role system is: technical personnel in this field obtain the social role set corresponding to each element by consulting historical documents and cultural materials, and construct a corresponding role portrait for each social role in the social role set; the role portrait includes but is not limited to basic attributes (such as age, gender, occupation, etc.), personality traits (such as elegant and restrained, bold and confident, etc.), behavior patterns (such as daily activities, social habits, etc.), etc.; based on the role portraits of different social roles in the social role set, different virtual characters are formed, and all virtual characters are integrated to form a role system; The method for designing an interaction rule system is as follows: technical personnel in this field obtain the social norms and interaction modes corresponding to each element based on historical research and cultural studies, and construct an interaction rule system; social norms include but are not limited to etiquette norms (such as specific etiquette for visiting elders and paying homage to the king, etc.), social class behavioral norms (such as behavioral boundaries between different social roles such as civilians, literati, and officials), scene-specific behavioral norms (such as behavioral norms for specific scenes such as drinking tea in teahouses, lecturing in academies, and market transactions), etc.; interaction modes include but are not limited to social class interaction modes (such as communication methods between civilians and officials, communication methods between literati and merchants, etc.), scene-specific interaction modes (bargaining methods in market transactions, debate and discussion modes in academy lectures, etc.), etc.; The method for designing an intelligent dialogue system is as follows: based on the language expression corresponding to each element in the multi-dimensional cultural element matrix, a corresponding dialogue generation model is constructed. The dialogue generation model is a large language model based on the Transformer architecture (such as BERT, GPT, etc.); an intelligent dialogue system with historical language sense, cultural depth and contextual awareness is realized, and real-time natural interaction with low latency and high semantic fidelity is supported between users and virtual characters.

[0025] The interaction collection module is used to collect continuous dialogue segments in real time during the process of dialogue interaction between the user and the virtual character in the virtual cultural interaction scene.

[0026] Methods for collecting continuous conversation snippets in real time include: Continuously collect the conversation interaction information between the user and the virtual character, the conversation interaction information includes the conversation content and the corresponding conversation time; wherein the conversation content is the natural language text output by the user or the virtual character during the conversation interaction process; the conversation time is the output time of each conversation content, which is used to record the timing information of the conversation; Obtain the historical duration, perform mean processing on the historical duration, and obtain the preliminary response threshold; the historical duration is the response duration during the process of the user interacting with the virtual character at historical moments, and is obtained through the database built in the Chinese language learning platform; obtain the interaction mode and interaction difficulty of the dialogue interaction, and dynamically adjust the preliminary response threshold based on the interaction mode and interaction difficulty to obtain the dynamic response threshold; among them, the interaction mode is the structured way of interaction between the user and the virtual person, such as question-and-answer interaction (for example, the user asks a Neo-Confucianism concept to a teacher in a Song Dynasty academy, and the teacher gives a systematic answer), chatting interaction (such as the user chatting with the owner of a Ming Dynasty teahouse about the weather, current affairs, and neighborhood gossip), scenario task interaction (such as the user completing the task of purchasing specific goods in a Song Dynasty market and having to bargain with different vendors), role-playing interaction (such as the user playing a newly appointed Tang Dynasty Jinshi and participating in court discussions), etc.; the interaction difficulty is the complexity of the knowledge reserve, language skills, and cultural understanding depth required for the user to participate in the interaction mode, such as primary difficulty, intermediate difficulty, advanced difficulty, etc.; both the interaction mode and the interaction difficulty are directly obtained through the virtual cultural interaction scene; After each round of the virtual character outputs a natural language text, monitor whether the user outputs a natural language text within the dynamic response threshold. If so, determine that the dialogue interaction continues and continue to collect dialogue interaction information. If not, determine that the dialogue interaction ends and use the collected dialogue content as the preliminary dialogue segment; calculate the semantic similarity between every two adjacent pieces of dialogue content in the preliminary dialogue segment, and compare each semantic similarity with a preset similarity threshold, and the similarity threshold is preset by those skilled in the art according to the actual situation; mark the dialogue content with a semantic similarity less than the similarity threshold as irrelevant content, and mark the dialogue content with a semantic similarity greater than the similarity threshold as continuous content; if the dialogue content is marked as both irrelevant content and continuous content, cancel the mark as continuous content; among the dialogue times corresponding to the irrelevant content, mark the earliest dialogue time as the earliest time and the latest time as the latest time; Use the continuous content with a dialogue time earlier than the earliest time and the irrelevant content corresponding to the earliest time as a continuous dialogue segment; use the continuous content with a dialogue time later than the latest time and the irrelevant content corresponding to the latest time as a continuous dialogue segment; judge whether there is continuous content between any two pieces of irrelevant content; if so, use the corresponding irrelevant content and the continuous content between them as a continuous dialogue segment; if not, do nothing.

[0027] The method for obtaining the dynamic response threshold includes: A preset coefficient set, which includes a mode set and a difficulty set. The coefficient set is pre-set by those skilled in the art according to the actual situation. The mode set includes adjustment coefficients corresponding to different interaction modes, and the difficulty set includes adjustment coefficients corresponding to different interaction difficulties. Among them, in different interaction modes, the thinking and response speeds of users are different, so the corresponding adjustment coefficients are also different. For example, the adjustment coefficient for scenario task-based interaction should be greater than that for chat-based interaction, and the adjustment coefficient for chat-based interaction should be greater than that for Q&A-based interaction. Usually, more complex or professional dialogue content requires users more time to understand and conceive answers, that is, the greater the interaction difficulty, the longer the response duration of users, so the corresponding adjustment coefficients are different. For example, the adjustment coefficient for high difficulty should be greater than that for medium difficulty, and the adjustment coefficient for medium difficulty should be greater than that for low difficulty. According to the interaction mode and interaction difficulty of the dialogue interaction, obtain the corresponding adjustment coefficient from the coefficient set; multiply the adjustment coefficient of the interaction mode by the adjustment coefficient of the interaction difficulty as the overall adjustment coefficient; multiply the overall adjustment coefficient by the preliminary response threshold as the dynamic response threshold.

[0028] The method for calculating the semantic similarity between two adjacent pieces of dialogue content includes: Using the SimCSE model, convert each piece of dialogue content into a corresponding dialogue vector respectively; calculate the cosine similarity between the dialogue vectors corresponding to every two adjacent pieces of dialogue content in turn and use it as the semantic similarity. It should be noted that the construction method of the SimCSE model and the calculation method of the cosine similarity are both existing technologies, and the specific process will not be elaborated here. The training process of the SimCSE model includes: Pre-collect sentences where is the th sentence; input each sentence into the SimCSE model. For each sentence, use the same Transformer encoder (such as BERT), and through two dropout encodings, obtain two different but semantically similar vector representations and ; among them, is the vector representation after the first encoding of sentence , is the vector representation after the second encoding of sentence , is the th sentence, ; , ; In the formula, represents the Transformer encoder, Denote the Dropout mask randomly generated when passing through the encoder for the first time. Denote the Dropout mask randomly generated when passing through the encoder for the second time; Dropout is a regularization method in neural networks, and some neuron connections will be randomly discarded during each forward propagation. Based on the two vector representations corresponding to each sentence, calculate the vector similarity of each sentence; the expression of the vector similarity is: In the formula, is the vector similarity of sentence , denotes the dot product of and , denotes the modulus length of According to the vector similarity of each sentence, calculate the contrastive loss of each sentence; the expression of the contrastive loss is: In the formula, is the contrastive loss of sentence , is the temperature parameter (usually 0.05), is the indicator function, which takes the value of 1 only when (to avoid comparing the same sentence); add up the contrastive losses of each sentence in turn to obtain the overall loss. According to the overall loss, use the backpropagation algorithm to calculate the gradient of the first model parameters, and use an optimizer (such as Adam) to update the first model parameters; take minimizing the sum of all overall losses as the training objective, and repeat the above training process until the sum of the overall losses converges and then stop training.

[0029] The correction feedback module is used to perform multi-level language error analysis on continuous dialogue segments, identify language errors in continuous dialogue segments, and formulate corresponding language correction suggestions for the language errors, and feedback them to the user through a progressive feedback mechanism.

[0030] The steps to identify language errors in continuous dialogue segments include: Step S101: Perform lexical error analysis on each continuous dialogue segment to identify lexical errors. Each consecutive segment is separately input into a trained context-aware language model to calculate the masked prediction probability of each phrase in each consecutive segment; a probability threshold is preset, and the probability threshold is preset by those skilled in the art according to the actual situation; each masked prediction probability is respectively compared with the probability threshold, and the phrases with masked prediction probabilities less than the probability threshold are marked as incorrect phrases, while the phrases with masked prediction probabilities greater than or equal to the probability threshold are not marked; the same incorrect phrases in the corresponding consecutive dialogue segments are merged as the lexical errors of the corresponding consecutive dialogue segments. Exemplarily, in the scenario of a teahouse in the Song Dynasty, the user says, "Shopkeeper, please serve me a cup of excellent tea." Since the unit for weighing tea in the Song Dynasty should be zhan instead of cup, cup is identified as an incorrect phrase. In this embodiment, the context-aware language model is preferably the RoBERTa model, and the training process of the RoBERTa model includes: Pre-collect a large-scale text corpus to construct a training data set; based on the training data set, construct a vocabulary; take a segment of text in the training data set as an input text sequence, and denote each input text sequence as , where is the length of the input text sequence; For the input text sequence, first randomly mask a part of the words to obtain the set of masked word positions ; after the input text sequence passes through the word embedding layer and the position encoding, it is input into the same Transformer encoder to obtain the hidden vector representation: ; ; In the formula, is the vector representation input to the first layer of the Transformer encoder, is the word vector matrix obtained after the input text sequence passes through the word embedding layer, is the position encoding vector obtained after the input text sequence passes through the position encoding, is the output representation of the input Transformer encoder at the layer, that is, the hidden vector, , is the number of layers of the Transformer encoder, represents the single-layer operation of the Transformer encoder, including the multi-head self-attention mechanism and the feed-forward neural network; For each masked word position, calculate the corresponding conditional probability according to the hidden vector representation, and the expression of the conditional probability is: ; in the formula, represents that the second model parameter is and given the context In the case of (all words in the input text sequence except the th word), the RoBERTa model predicts the probability that the th word is a specific word in the vocabulary. is a function that converts a set of numerical values into a probability distribution. is the weight matrix of the output layer. is the bias vector of the output layer. Define the loss function as: ; where is the value of the loss function. According to the value of the loss function, use the backpropagation algorithm to calculate the gradient of the second model parameters, and use the optimizer to update the second model parameters; take minimizing the sum of all loss function values as the training objective, and repeat the above training process until the sum of the overall losses reaches convergence and then stop training.

[0031] Step S102: Perform grammatical error analysis on each continuous dialogue segment to identify grammatical errors. Input each continuous dialogue segment into the trained grammatical error correction model respectively to identify the grammatical errors in each continuous dialogue segment; Exemplarily, in the Tang Dynasty court scene, the user says to the emperor: "Your Majesty, I have already submitted the memorial", since "submitted to Your Majesty" is a complete verb-object structure and should not be separated by the word "le", so it is identified as a grammatical error. In this embodiment, the preferred grammatical error correction model is the GECToR model. The training process of the GECToR model includes: Pre-construct sentence pairs, denoted as ; where is the original sentence (including grammatical errors), is the correct sentence, is the number of training samples; Align each original sentence with the corresponding correct sentence, generate the editing operation label corresponding to each word in the original sentence, and mark it as the true label; Editing operation labels such as retain word, delete word, replace with new word, etc. Input each original sentence into the GECToR model in turn, use the BPE algorithm to perform word segmentation processing on each original sentence in turn to obtain the word sequence corresponding to each original sentence; Pass each word sequence through the embedding layer respectively to obtain the embedding vector; Input each embedding vector into the Transformer encoder respectively to obtain the context encoding representation of each word in each word sequence; Perform a linear transformation on the context encoding representation of each word. The expression of the linear transformation is: ; where is the vector obtained after the context encoding representation of the th word passes through the linear transformation, is the The context encoding representation of the word; for each context encoding representation of the word after linear transformation, use The function to obtain the probability that each word belongs to each editing operation label; Define the cross-entropy loss function as: ; In the formula, is the value of the cross-entropy loss function, is the true label of the word, is the probability that the GECToR model predicts that the word belongs to the true label, is the length of the original sentence; According to the value of the cross-entropy loss function, calculate the gradient of the third model parameter using the backpropagation algorithm, and update the third model parameter using the optimizer; Take minimizing the sum of all cross-entropy loss function values as the training objective, and repeat the above training process until the sum of the cross-entropy loss function values converges and then stop training.

[0032] Step S103: Perform pragmatic error analysis on each continuous dialogue segment to identify pragmatic errors; Input each continuous dialogue segment into the trained context analysis model respectively to identify the pragmatic errors in each continuous dialogue segment; Exemplarily, in the Tang Dynasty court scene, the user says to the emperor: "Your Majesty, I have written this memorial. Please take a look and don't forget to give your comments.", among which, "Please take a look" is too colloquial and casual in tone, not in line with the due humility and courtesy of an official to the emperor, and "Don't forget to give your comments" has a sense of urging, appearing disrespectful and even with a commanding tone, not in line with court etiquette; Therefore, "Please take a look" and "Don't forget to give your comments" are identified as pragmatic errors; In this embodiment, the preferred context analysis model is the fine-tuned RoBERTa model. The fine-tuning process of the RoBERTa model includes: Pre-collect multiple pragmatic error sentences, and use the BPE algorithm to tokenize each pragmatic error sentence in turn, and pre-set corresponding judgment labels for each word in each pragmatic error sentence; the judgment labels include correct labels and incorrect labels, set the correct label to 0, and set the incorrect label to 1; input each pragmatic error sentence into the RoBERTa model, and after being processed by multiple Transformer encoders, output the predicted judgment label sequence corresponding to each pragmatic error sentence, and the predicted judgment label sequence includes the probability that each word in the pragmatic error sentence output by the RoBERTa model belongs to each judgment label; according to the predicted judgment label sequence and the actual judgment label sequence of each pragmatic error sentence, calculate the fine-tuning cross-entropy loss function value corresponding to each pragmatic error sentence, and the actual judgment label sequence is the judgment label corresponding to each word in the pre-set pragmatic error sentence, and the calculation method of the fine-tuning cross-entropy loss function value is the same as the calculation method of the above cross-entropy loss function value; According to the fine-tuning cross-entropy loss function value, use the backpropagation algorithm to calculate the gradient of the fourth model parameter, and use the optimizer to update the fourth model parameter; take minimizing the sum of all fine-tuning cross-entropy loss function values as the training objective, and repeat the above training process until the sum of the cross-entropy loss function values converges and stop training.

[0033] Step S104: Synthesize lexical errors, grammatical errors, and pragmatic errors to determine the language errors in each continuous dialogue segment; Synthesize the same lexical errors, grammatical errors, and pragmatic errors corresponding to the continuous dialogue segments as the language errors in the corresponding continuous dialogue segments.

[0034] The method for formulating corresponding language correction suggestions for language errors includes: Take each continuous dialogue segment and its corresponding language error as a set of segment collections, and the segment collections correspond one-to-one with the continuous dialogue segments; input each set of segment collections into the trained language correction model respectively, and output the correction set corresponding to each set of segment collections; the correction set includes the suggestion set corresponding to each language error, and the suggestion set includes the types of correction suggestions, which is an integer greater than 1; In this embodiment, the preferred language correction model is the T5 model, and the training process of the T5 model includes: Pre-collect multiple training samples, and each training sample includes a language error sentence and the corresponding language error in the language error sentence and a target suggestion sequence composed of multiple correction suggestions; construct a corresponding input sequence according to the language error sentence and language error in each training sample; use the BPE algorithm to perform word segmentation processing on each input sequence in turn to obtain the word sequence corresponding to each input sequence; pass each word sequence through the embedding layer to obtain an embedding vector; input each embedding vector into the Transformer encoder to obtain the context encoding representation of each word in each word sequence; use the Transformer decoder, with the target suggestion sequence as the supervision signal, to gradually predict the probability of each word in the target suggestion sequence , so as to generate correction suggestions word by word; where is the th word in the target suggestion sequence, represents the first words that have been generated, is the fifth model parameter; Calculate the prediction deviation according to the probability of each word; The expression of the prediction deviation is: ; where is the prediction deviation, is the length of the target suggestion sequence; According to the prediction deviation, use the backpropagation algorithm to calculate the gradient of the fifth model parameter, and use the optimizer to update the fifth model parameter; take minimizing the sum of all prediction deviations as the training objective, and repeat the above training process until the sum of the prediction deviations converges and stop training.

[0035] Randomly select a correction suggestion from each suggestion set corresponding to each continuous dialogue segment, and construct a set of suggestion subsets, and construct sets of suggestion subsets for each continuous dialogue segment, is an integer greater than 1; Using the SimCSE model, convert the correction suggestions in each group of suggested subsets into corresponding suggestion vectors respectively; Take the dialogue vector corresponding to each continuous dialogue segment and the corresponding group of suggested subsets as a group of analysis sets, and the analysis sets correspond one-to-one with the continuous dialogue segments; Input each group of analysis sets into the trained correction analysis model respectively to predict the corresponding index set; Among them, the index set includes language fluency and cultural fit. Language fluency refers to the naturalness and smoothness of the expression of the continuous dialogue segment, and cultural fit refers to the continuous dialogue segment meeting the requirements of a specific cultural background, historical period, and communication etiquette; The correction analysis model is a deep neural network model, which includes an input layer, a hidden layer, and an output layer; Each hidden layer includes multiple neurons, and there are connections between each neuron and the neurons in the next layer. The connections contain weights, which determine the importance and influence of data transmission in the neural network; Apply an activation function to each neuron between the hidden layer and the output layer. The activation function introduces non-linearity and allows the network to learn more complex patterns and features; The training process of the correction analysis model includes: Pre-collect groups of different analysis sets, and set the corresponding index sets for each group of analysis sets in turn, is an integer greater than 1; Convert the analysis set and the corresponding index set into a corresponding group of feature vectors; The index set corresponding to the analysis set is collected by those skilled in the art during the process of formulating language correction suggestions in history groups of different analysis sets, and analyze each group of analysis sets in turn, evaluate the corresponding language fluency and cultural fit, and use them as the corresponding index sets. For groups of different analysis sets, set the corresponding index sets in turn; Take each group of feature vectors as the input of the correction analysis model. The correction analysis model takes a group of predicted index sets corresponding to each group of analysis sets as the output, and takes the actual index set corresponding to each group of analysis sets as the prediction target. The actual index set is the index set preset corresponding to the analysis set; Take minimizing the sum of the prediction errors of all analysis sets as the training target; Among them, the calculation formula of the prediction error is where is the prediction error, is the group number of the feature vector corresponding to the analysis set, is the group of predicted index sets corresponding to the group of analysis sets, is the group of actual index sets corresponding to the group of analysis sets; Train the correction analysis model until the sum of the prediction errors reaches convergence and stop training.

[0036] A preset ratio set, which includes ratio coefficients corresponding to language fluency and cultural fit. The ratio set is pre-set by those skilled in the art according to the actual situation; according to the ratio set, the language fluency and cultural fit in each set of index sets are weighted and summed to obtain the comprehensive score of each set of suggested subsets; the comprehensive scores corresponding to the same continuous dialogue segments are compared, and the suggested subset with the maximum comprehensive score is used as the language correction suggestion for the corresponding continuous dialogue segment.

[0037] The steps of feeding back to the user through the progressive feedback mechanism include: Step S201: Feed back all the language errors corresponding to each continuous dialogue segment to the user and obtain a first modified content; wherein, the first modified content is the natural language text output by the user after modifying the corresponding dialogue content according to the fed-back language errors. Step S202: Identify the language errors in the first modified content. If there are still language errors, go to step S203; if there are no language errors, the feedback ends. Step S203: Based on the pre-constructed language knowledge base, obtain the error reasons corresponding to the language errors identified in step S202; the language knowledge base includes, but is not limited to, historical and cultural knowledge bases (such as in the Song Dynasty, tea was called "zhan" instead of "bei"), grammar structure rule bases (such as the verb-object structure cannot be separated by "le"), pragmatic norm bases (such as court language style requirements, identity and rank discourse norms), etc., which are pre-constructed by those skilled in the art based on language knowledge and cultural common sense such as historical and cultural norms, grammar rule systems, and pragmatic etiquette guidelines, and are used to support the structured expression of background knowledge for error explanation. Step S204: Feed back the error reasons to the user and obtain a second modified content. The second modified content is the natural language text output by the user after modifying the corresponding dialogue content according to the fed-back error reasons. Step S205: Identify the language errors in the second modified content. If there are still language errors, go to step S206; if there are no language errors, the feedback ends. The steps of identifying the language errors in the first modified content and the second modified content are the same as the steps of identifying the language errors in the continuous dialogue segments. Step S206: Feed back the correction set of all continuous dialogue segments and the language correction suggestions to the user.

[0038] It should be noted that the beneficial effect of adopting the progressive feedback mechanism is that it provides a structured, in-depth and educational language learning process; by gradually identifying errors, providing specific reasons and guiding users to modify multiple times, it not only corrects surface language errors, but also helps users understand the cultural, grammatical or pragmatic knowledge behind the errors, thus promoting the improvement of deeper language abilities; the final correction set and language correction suggestions strengthen the learning results, enabling users to systematically master the correct usage, form long-term memory, and effectively improve the accuracy and cultural appropriateness of language expression.

[0039] A state evaluation module, which is used to collect multi-modal data of users, perform non-linear feature mapping on the multi-modal data of users, generate a multi-dimensional state feature vector, and dynamically evaluate the cognitive load and learning state based on the multi-dimensional state feature vector.

[0040] The multi-modal data of users includes a sequence of facial images and eye movement trajectories; the sequence of facial images includes the facial images collected during the process of the user's dialogue and interaction with the virtual character, where \(N\) is an integer greater than 1, and the sequence of facial images is obtained in real time through the camera built in the user terminal; the eye movement trajectories include fixation points, fixation durations, and saccade frequencies, which are used to depict the user's visual attention state; among them, the fixation point is the spatial position where the user's eyes stay relatively stably, reflecting the specific area where the user's visual attention is concentrated; the fixation duration is the time length that the user's eyes stay at a single fixation point, indicating the depth of the user's processing of the information in this area; the saccade frequency is the number of saccades per unit time, and a saccade is the process of the user's eyes quickly moving from one fixation point to another; a higher saccade frequency indicates that the user's attention is scattered, while a lower saccade frequency indicates that the user is in a state of deep thinking; by comprehensively applying the eye detection algorithm and the eye movement tracking algorithm, the sequence of facial images is analyzed to extract the user's eye movement trajectories; both the eye detection algorithm and the eye movement tracking algorithm are existing technologies, and the specific process will not be elaborated here.

[0041] The method for generating the multi-dimensional state feature vector includes: Using the trained emotion recognition model, each facial image in the sequence of facial images is sequentially recognized, and the recognition result of each facial image is output. The recognition result includes a category label and an emotion intensity; the category label is a digital label corresponding to the emotion category, and the digital labels corresponding to different emotion categories are all different. The emotion categories are, for example, happy, confused, frustrated, etc.; the range of the emotion intensity is , , and the specific numerical values of ; According to the category labels in the recognition results, obtain the corresponding emotion categories; count the number of each emotion category and mark it as the emotion quantity; compare all the emotion quantities, and take the emotion category with the largest emotion quantity as the user's emotion state; perform an average processing on all the emotion intensities corresponding to the emotion state to obtain the average emotion intensity; the emotion recognition model is a convolutional neural network model, and the convolutional neural network model is an extended form of a deep neural network, mainly including an input layer, multiple convolutional layers, pooling layers (downsampling layers), fully connected layers, and an output layer; among them, the convolutional layer consists of multiple convolutional kernels (filters), and each convolutional kernel performs a sliding operation with the input data to extract local features. The convolutional operation contains weight parameters for learning the importance of features; a pooling layer is usually connected after the convolutional layer, and the pooling layer is used to reduce the size of the feature map, reducing the computational complexity while retaining the main information; a fully connected layer is connected after the convolutional layer and the pooling layer to map the high-dimensional features to the output space; in layers such as the convolutional layer and the fully connected layer, each neuron usually applies an activation function to introduce non-linearity and enhance the model's expression ability and generalization ability for complex image patterns; the specific training process of the emotion recognition model includes: Pre-collect multiple facial images, annotate the patient's face in each facial image, and the annotation includes category labels and emotion intensities; divide the annotated facial images into a training set and a test set, use 70% of the facial images as the training set, and use 30% of the facial images as the test set; use the training set to train the emotion recognition model, and use the test set to test the emotion recognition model; preset an error threshold, and when the average prediction error of all facial images in the test set is less than the error threshold, output the emotion recognition model; among them, the calculation formula for the average prediction error is: , where is the prediction error, is the number of the facial image, is the predicted category label corresponding to the th group of facial images, is the actual category label corresponding to the th group of facial images, is the predicted emotion intensity corresponding to the th group of facial images, is the actual emotion intensity corresponding to the th group of facial images, and

[0042] is the number of facial images in the test set; the error threshold is preset according to the required accuracy of the emotion recognition model.Use clustering algorithms (such as K-means clustering algorithm, density-based spatial clustering algorithm, etc.) to cluster all fixation points in the eye movement trajectory to obtain multiple fixation regions; count the number of fixation regions and label it as the fixation count; calculate the mean of all gaze durations in the eye movement trajectory to obtain the average gaze duration; preset a weight set, which includes the weight coefficients corresponding to the reciprocal of the fixation count, the average gaze duration, and the reciprocal of the video skipping rate. The weight set is preset by those skilled in the art according to the actual situation; perform normalization processing on the fixation count, the average gaze duration, and the video skipping rate (such as methods like Z-score normalization, Min-Max normalization, etc.), and perform weighted summation on the reciprocal of the normalized fixation count, the average gaze duration, and the reciprocal of the video skipping rate according to the weight set to obtain the attention concentration degree; Count the total number of lexical errors, grammar errors, and pragmatic errors in all consecutive dialogue segments and label it as the language error count; count the number of times the progressive feedback mechanism provides feedback to the user and label it as the feedback count; Generate a multi-dimensional state feature vector based on the emotional state, average emotional intensity, attention concentration degree, language error count, and feedback count.

[0043] The method for dynamically evaluating cognitive load and learning state includes: Take the category label corresponding to the emotional state, the average emotional intensity, and the attention concentration degree in the multi-dimensional state feature vector as the first analysis data, and take the language error count and the feedback count in the multi-dimensional state feature vector as the second analysis data; dynamically evaluate the learning state according to the first analysis data, dynamically evaluate the cognitive load according to the second analysis data, and the method for dynamically evaluating the learning state according to the first analysis data is the same as the method for dynamically evaluating the cognitive load according to the second analysis data; The method for dynamically evaluating the learning state according to the first analysis data includes: Construct multiple fuzzy sets for each data in the first analysis data respectively; for example: the fuzzy sets corresponding to the category label are positive, neutral, negative, etc.; the fuzzy sets corresponding to the average emotional intensity are high intensity, medium intensity, low intensity, etc., and the fuzzy sets corresponding to the attention concentration degree are high concentration, medium concentration, low concentration, etc.; convert each data in the first analysis data into the membership degree of the corresponding fuzzy set respectively through fuzzy technology. Fuzzy technology is the process of converting accurate numerical values into the membership degrees of fuzzy sets. Fuzzy technologies include, for example, triangular membership functions, trapezoidal membership functions, etc.; for example, if the value of the average emotional intensity is low, then it is inferred that the membership degree of high intensity is 0.1, the membership degree of medium intensity is 0.3, and the membership degree of low intensity is 0.8; Define fuzzy rules, which are defined based on expert knowledge or relevant literature. For example, if the category labels are positive, high intensity, and high concentration, it is inferred that the learning state belongs to a high membership degree of positive; if the category labels are negative, high intensity, and low concentration, it is inferred that the learning state belongs to a high membership degree of negative. Match the first analyzed data after fuzzification with the fuzzy rules respectively, and use fuzzy inference methods (such as Mamdani fuzzy inference model, Sugeno fuzzy inference model, etc.) to perform fuzzy inference to obtain the fuzzy inference results. The fuzzy inference results are the membership degrees of each learning state, and the learning states include but are not limited to positive, stable, negative, etc. The fuzzy inference results are, for example, the membership degree of positive is 0.6, the membership degree of neutral is 0.8, and the membership degree of negative is 0.1. Compare each membership degree respectively, and take the learning state with the largest membership degree as the dynamically evaluated learning state.

[0044] The interaction optimization module is used to fuse the cognitive load and the learning state, and perform interactive regulation strategy inference, and dynamically optimize the interaction mode and interaction difficulty in the dialogue interaction according to the interactive regulation strategy.

[0045] The methods for performing interactive regulation strategy inference include: Obtain the corresponding load label according to the cognitive load, and obtain the corresponding state label according to the learning state. Among them, the load label is the digital label corresponding to the cognitive load, and different cognitive loads correspond to different load labels; the state label is the digital label corresponding to the learning state, and different cognitive loads correspond to different digital labels. Input the load label and the state label into the trained direction judgment model to predict the corresponding direction label. The direction judgment model is a deep neural network model, and the specific training process of the direction judgment model is the same as that of the above correction analysis model. The difference is that the input of the direction judgment model is the load label and the state label, and the output is the direction label. Among them, the direction label is the digital label corresponding to the regulation direction, and different regulation directions correspond to different direction labels. The regulation directions include positive regulation and negative regulation. Positive regulation indicates that the user's current learning state is good and the cognitive load is low, and the learning motivation of the user can be further stimulated and the learning effect can be strengthened by increasing the difficulty and deepening the interaction; negative regulation indicates that the user's current learning state is poor and the cognitive load is high, and it is necessary to reduce the difficulty and increase the guidance to help the user recover the state and prevent cognitive overload and learning frustration. Set corresponding guiding intensities for each interaction mode respectively, sort all interaction modes according to the corresponding guiding intensities from high to low to generate a mode sequence; Exemplarily, the question-and-answer interaction is dominated by a virtual character with clear content, so the guiding intensity is relatively high; In the role-playing interaction, users have a greater degree of autonomy, the interaction method is flexible, and the explorability is high, so the guiding intensity is relatively low; The guiding intensity is set by those skilled in the art according to the actual situation; Sort all interaction difficulties from low to high to generate a difficulty sequence; If the regulation direction is positive regulation, obtain the interaction modes ranked after the current mode from the mode sequence, and obtain the interaction difficulties ranked after the current difficulty from the difficulty sequence; If the regulation direction is negative regulation, obtain the interaction modes ranked before the current mode from the mode sequence; Obtain the interaction difficulties ranked before the current difficulty from the difficulty sequence; Wherein, the current mode is the interaction mode corresponding to the user's current dialogue interaction, and the current difficulty is the interaction difficulty corresponding to the user's current dialogue interaction; Mark all the obtained interaction modes as regulation modes, and mark all the obtained interaction difficulties as regulation difficulties; Randomly select one regulation mode and one regulation difficulty to construct a regulation strategy, and a total of regulation strategies are constructed, where is an integer greater than 0; Calculate the regulation effects of each group of regulation strategies in turn, compare all the regulation effects, and take the regulation strategy with the largest regulation effect as the interaction regulation strategy.

[0046] The method for calculating the regulation effects of each group of regulation strategies in turn includes: Set different digital tags for different interaction modes and mark them as mode tags; Set different digital tags for different interaction difficulties and mark them as difficulty tags; Use the multi-dimensional state feature vector, the mode tag of the current mode, the difficulty tag of the current difficulty, and the mode tags of the regulation modes and the difficulty tags of the regulation difficulties in a regulation strategy as a group of regulation data; Input each group of regulation data into the trained effect prediction model respectively to predict the corresponding regulation effect; The regulation effect reflects the improvement of the user's learning state and cognitive load under the action of the current regulation strategy; The effect prediction model is a deep neural network model, and the specific training process of the effect prediction model is the same as that of the above-mentioned correction analysis model, the difference is that the input of the effect prediction model is regulation data and the output is the regulation effect.

[0047] This embodiment provides an immersive language practice environment for users by constructing a virtual cultural interaction scene and integrating multi-dimensional historical and cultural elements, thereby enhancing their understanding of traditional Chinese culture and improving the fun and participation of Chinese language learning. It collects continuous dialogue segments during real-time dialogue and conducts multi-level language error analysis to accurately identify various types of errors such as vocabulary, grammar, and pragmatics, and formulates targeted and progressive language correction suggestions to effectively promote the systematic improvement of users' Chinese language proficiency. It collects users' facial expressions and eye movement data, dynamically analyzes cognitive load and learning status, and optimizes interaction modes and difficulty in real time in combination with intelligent reasoning, so that the learning process is more in line with users' learning needs and cognitive characteristics. It achieves precise control and dynamic regulation of the entire process of users' Chinese language learning, provides users with personalized and efficient online Chinese learning experience, and promotes the comprehensive improvement of users' language ability.

[0048] Embodiment 2

[0049] The present application also provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories store computer-readable codes, and when the computer-readable codes are executed by the one or more processors, they can execute a real-time interactive online Chinese language learning platform as described above.

[0050] The method or system according to the implementation mode of the present application can also be implemented with the help of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, can store a real-time interactive online Chinese language learning platform provided by the present application. Further, the electronic device may also include a user interface. Of course, the architecture shown in the present application is only exemplary. When implementing different devices, one or more components in the electronic device shown in the present application may be omitted according to actual needs.

[0051] Embodiment 3

[0052] One embodiment of the present application discloses a computer-readable storage medium. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by a processor, a real-time interactive online Chinese language learning platform according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0053] In addition, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be run by a processor to execute instructions corresponding to the method steps provided by the present application, such as: a real-time interactive online Chinese language learning platform. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed.

[0054] The foregoing is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0055] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element.

[0056] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0057] In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0058] In the description of the present invention, the meaning of "several" is one or more, and the meaning of "a large number" is two or more.

[0059] In the description of this specification, the description with reference to terms such as "an embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0060] For the formulas in this specification, only the numerical values are calculated after dimensionless processing. The formulas are obtained by collecting a large amount of data and performing software simulations to get a formula that is closest to the actual situation. The preset parameters and threshold values in the formulas are set by those skilled in the art according to the actual situation.

[0061] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A real-time interactive online Chinese language learning platform, characterized in that, including: A scene construction module, configured to establish a multi-dimensional cultural element matrix and construct a virtual cultural interaction scene based on the multi-dimensional cultural element matrix; An interaction collection module, configured to continuously collect consecutive dialogue segments in real time during the process of a user's dialogue interaction with a virtual character in the virtual cultural interaction scene; A correction feedback module, configured to perform multi-level language error analysis on the consecutive dialogue segments, identify language errors in the consecutive dialogue segments, formulate corresponding language correction suggestions for the language errors, and feedback them to the user through a progressive feedback mechanism; A state evaluation module, configured to collect multi-modal data of the user, perform non-linear feature mapping on the multi-modal data of the user to generate a multi-dimensional state feature vector, and dynamically evaluate the cognitive load and learning state based on the multi-dimensional state feature vector; An interaction optimization module, configured to fuse the cognitive load and learning state, perform interaction regulation strategy reasoning, and dynamically optimize the interaction mode and interaction difficulty in the dialogue interaction according to the interaction regulation strategy.

2. The real-time interactive online Chinese language learning platform according to claim 1, wherein The method for continuously collecting consecutive dialogue segments includes: Continuously collecting dialogue interaction information between the user and the virtual character, where the dialogue interaction information includes dialogue content and corresponding dialogue time; wherein, the dialogue content is the natural language text output by the user or the virtual character during the dialogue interaction process, and the dialogue time is the output time of each piece of dialogue content; Obtaining the historical duration, performing an average process on the historical duration to obtain a preliminary response threshold; the historical duration is the response duration of the user during the dialogue interaction with the virtual character at a historical moment; obtaining the interaction mode and interaction difficulty of the dialogue interaction, dynamically adjusting the preliminary response threshold based on the interaction mode and interaction difficulty to obtain a dynamic response threshold; after each output of natural language text by the virtual character, monitoring whether the user outputs natural language text within the dynamic response threshold, if so, determining that the dialogue interaction continues and continuing to collect dialogue interaction information, if not, determining that the dialogue interaction ends and taking the collected dialogue content as a preliminary dialogue segment; Calculating the semantic similarity between every two adjacent pieces of dialogue content in the preliminary dialogue segment, and comparing each semantic similarity with a preset similarity threshold; marking the dialogue content with a semantic similarity less than the similarity threshold as irrelevant content, and marking the dialogue content with a semantic similarity greater than the similarity threshold as consecutive content; if the dialogue content is marked as both irrelevant content and consecutive content, canceling the marking as consecutive content; among the dialogue times corresponding to the irrelevant content, marking the earliest dialogue time as the earliest time and the latest time as the latest time; Regarding the consecutive content with a dialogue time earlier than the earliest time and the irrelevant content corresponding to the earliest time as a consecutive dialogue segment; regarding the consecutive content with a dialogue time later than the latest time and the irrelevant content corresponding to the latest time as a consecutive dialogue segment; judging whether there is consecutive content between any two pieces of irrelevant content; if so, regarding the corresponding irrelevant content and the consecutive content therebetween as a consecutive dialogue segment; if not, no operation is performed.

3. An online Chinese language learning platform for real-time interaction according to claim 2, characterized in that, The method for obtaining the dynamic response threshold includes: A preset coefficient set, which includes a mode set and a difficulty set; the mode set includes adjustment coefficients corresponding to different interaction modes, and the difficulty set includes adjustment coefficients corresponding to different interaction difficulties; according to the interaction mode and interaction difficulty of the dialogue interaction, obtain the corresponding adjustment coefficient from the coefficient set; take the product of the adjustment coefficient of the interaction mode and the adjustment coefficient of the interaction difficulty as the overall adjustment coefficient; take the product of the overall adjustment coefficient and the initial response threshold as the dynamic response threshold. The method for calculating the semantic similarity between two adjacent pieces of dialogue content includes: Using the SimCSE model, convert each piece of dialogue content into a corresponding dialogue vector respectively; calculate the cosine similarity between the dialogue vectors corresponding to every two adjacent pieces of dialogue content in turn, and use it as the semantic similarity.

4. An online Chinese language learning platform with real-time interaction according to claim 3, characterized in that, The steps for identifying language errors in continuous dialogue segments include: Step S101: Conduct a lexical error analysis on each continuous dialogue segment to identify lexical errors; Input each continuous segment into a trained context-aware language model respectively, and calculate the masked prediction probability of each phrase in each continuous segment; preset a probability threshold, compare each masked prediction probability with the probability threshold respectively, mark the phrases with masked prediction probabilities less than the probability threshold as error phrases, and do not mark the phrases with masked prediction probabilities greater than or equal to the probability threshold; combine the same error phrases in the corresponding continuous dialogue segment as the lexical errors in the corresponding continuous dialogue segment. Step S102: Conduct a grammatical error analysis on each continuous dialogue segment to identify grammatical errors; Input each continuous dialogue segment into a trained grammatical error correction model respectively to identify the grammatical errors in each continuous dialogue segment. Step S103: Conduct a pragmatic error analysis on each continuous dialogue segment to identify pragmatic errors; Input each continuous dialogue segment into a trained context analysis model respectively to identify the pragmatic errors in each continuous dialogue segment. Step S104: Combine lexical errors, grammatical errors, and pragmatic errors to determine the language errors in each continuous dialogue segment; Combine the same lexical errors, grammatical errors, and pragmatic errors in the corresponding continuous dialogue segment as the language errors in the corresponding continuous dialogue segment.

5. An online Chinese language learning platform with real-time interaction according to claim 4, characterized in that, The method for formulating corresponding language correction suggestions for language errors includes: Taking each consecutive dialogue segment and its corresponding language error as a set of segments, input each set of segments into the trained language correction model, and output the correction set corresponding to each set of segments; the correction set includes a set of suggestions corresponding to each language error, and the set of suggestions includes types of correction suggestions, which is an integer greater than 1; From each set of suggestions corresponding to each consecutive dialogue segment, a correction suggestion is randomly selected, and a subset of suggestions is constructed. A subset of suggestions is constructed for each consecutive dialogue segment. A subset of suggestions is an integer greater than 1; using the SimCSE model, the correction suggestions in each subset of suggestions are respectively converted into corresponding suggestion vectors; the dialogue vector corresponding to each consecutive dialogue segment and the corresponding subset of suggestions are used as a set of analysis sets, and each set of analysis sets is respectively input into the trained correction analysis model to predict the corresponding index set; wherein, the index set includes language fluency and cultural fit, and the correction analysis model is a deep neural network model. Preset a ratio set, which includes ratio coefficients corresponding to language fluency and cultural fit; according to the ratio set, perform weighted summation on language fluency and cultural fit in each group of index sets to obtain the comprehensive score of each group of suggestion subsets; compare the same comprehensive scores in the corresponding continuous dialogue segments, and take the suggestion subset with the highest comprehensive score as the language correction suggestion for the corresponding continuous dialogue segment.

6. An online Chinese language learning platform with real-time interaction according to claim 5, characterized in that, The steps for feeding back to the user through a progressive feedback mechanism include: Step S201: Feed back all the language errors corresponding to each continuous dialogue segment to the user and obtain a first modification content. Step S202: Identify the language errors in the first modification content. If there are still language errors, go to Step S203; if there are no language errors, the feedback ends. Step S203: Based on the pre - constructed language knowledge base, obtain the error reasons corresponding to the language errors identified in Step S202; Step S204: Feed back the error reasons to the user and obtain the content of the second - round modification; Step S205: Identify the language errors in the content of the second - round modification. If there are still language errors, proceed to Step S206; if there are no language errors, feedback ends; Step S206: Feed back the correction set of all consecutive dialogue segments and language correction suggestions to the user.

7. An online Chinese language learning platform with real-time interaction according to claim 6, characterized in that, The user multimodal data includes a facial image sequence and an eye movement trajectory; the facial image sequence includes the facial images collected during the process of the user's dialogue interaction with the virtual character, where is an integer greater than 1; the eye movement trajectory includes fixation points, fixation duration, and saccade frequency; The method for generating the multi - dimensional state feature vector includes: Using the trained emotion recognition model, each facial image in the facial image sequence is sequentially recognized to output the recognition result of each facial image. The recognition result includes a category label and an emotion intensity. The category label is a digital label corresponding to an emotion category, and the digital labels corresponding to different emotion categories are all different. The range of the emotion intensity is , ; According to the category label in the recognition result, the corresponding emotion category is obtained; the number of each emotion category is counted and marked as the emotion quantity; all emotion quantities are compared, and the emotion category with the largest emotion quantity is used as the user's emotion state; the average emotion intensity is obtained by averaging all the emotion intensities corresponding to the emotion state; the emotion recognition model is a convolutional neural network model; Using a clustering algorithm to cluster all fixation points in the eye movement trajectory to obtain multiple fixation regions; counting the number of fixation regions and marking it as the fixation count; averaging all the fixation durations in the eye movement trajectory to obtain the average fixation duration; presetting a weight set, where the weight set includes the weight coefficients corresponding to the reciprocal of the fixation count, the average fixation duration, and the reciprocal of the video - skipping rate; normalizing the fixation count, the average fixation duration, and the video - skipping rate, and performing weighted summation on the reciprocal of the normalized fixation count, the average fixation duration, and the reciprocal of the video - skipping rate according to the weight set to obtain the attention concentration; Count the total number of lexical errors, grammar errors, and pragmatic errors in all consecutive dialogue segments and mark it as the language error count; count the number of times the progressive feedback mechanism provides feedback to the user and mark it as the feedback count; Generate a multi - dimensional state feature vector based on the emotional state, average emotional intensity, attention concentration, language error count, and feedback count.

8. An online Chinese language learning platform for real-time interaction according to claim 7, characterized in that, The method for dynamically evaluating cognitive load and learning state includes: Taking the category label corresponding to the emotional state, the average emotional intensity, and the attention concentration in the multi - dimensional state feature vector as the first analysis data, and taking the language error count and the feedback count in the multi - dimensional state feature vector as the second analysis data; dynamically evaluating the learning state according to the first analysis data, dynamically evaluating the cognitive load according to the second analysis data, and the method for dynamically evaluating the learning state according to the first analysis data is the same as the method for dynamically evaluating the cognitive load according to the second analysis data; The method for dynamically evaluating the learning state according to the first analysis data includes: Constructing multiple fuzzy sets for each data in the first analysis data respectively; converting each data in the first analysis data into the membership degree of the corresponding fuzzy set through a fuzzification technique respectively; defining fuzzy rules, matching the fuzzified first analysis data with the fuzzy rules respectively, and using a fuzzy inference method for fuzzy inference to obtain a fuzzy inference result, where the fuzzy inference result is the membership degree of each learning state; comparing each membership degree respectively, and taking the learning state with the largest membership degree as the dynamically evaluated learning state.

9. An online Chinese language learning platform with real-time interaction according to claim 8, characterized in that, The method for performing interactive regulation strategy inference includes: Obtain the corresponding load label according to the cognitive load, and obtain the corresponding status label according to the learning status; among them, the load label is the digital label corresponding to the cognitive load, and the status label is the digital label corresponding to the learning status; input the load label and the status label into the trained direction judgment model, and predict the corresponding direction label, where the direction judgment model is a deep neural network model; among them, the direction label is the digital label corresponding to the regulation direction, and the regulation direction includes positive regulation and negative regulation; Set the corresponding guidance intensity for each interaction mode, sort all interaction modes according to the corresponding guidance intensity from high to low to generate a mode sequence; sort all interaction difficulties from low to high to generate a difficulty sequence; If the regulation direction is positive regulation, obtain the interaction mode ranked behind the current mode from the mode sequence, and obtain the interaction difficulty ranked behind the current difficulty from the difficulty sequence; if the regulation direction is negative regulation, obtain the interaction mode ranked in front of the current mode from the mode sequence; obtain the interaction difficulty ranked in front of the current difficulty from the difficulty sequence; among them, the current mode is the interaction mode corresponding to the user's current conversation interaction, and the current difficulty is the interaction difficulty corresponding to the user's current conversation interaction; mark the obtained interaction modes as regulation modes, and mark the obtained interaction difficulties as regulation difficulties; Randomly select a regulation mode and a regulation difficulty, and construct a regulation strategy. A total of regulation strategies are constructed, where is an integer greater than 0; calculate the regulation effects of each group of regulation strategies in turn, compare all the regulation effects, and take the regulation strategy with the largest regulation effect as the interactive regulation strategy.

10. An online Chinese language learning platform for real-time interaction according to claim 9, characterized in that, The method for calculating the regulation effect of each group of regulation strategies in sequence includes: Set different digital labels for different interaction modes and mark them as mode labels; set different digital labels for different interaction difficulties and mark them as difficulty labels; use the multi-dimensional state feature vector, the mode label of the current mode, the difficulty label of the current difficulty, and the mode label of the regulation mode and the difficulty label of the regulation difficulty in a regulation strategy as a group of regulation data; input each group of regulation data into the trained effect prediction model respectively to predict the corresponding regulation effect; the effect prediction model is a deep neural network model.

Images