A Chinese learning method and system based on multi-agent

Through multi-agent collaboration, a personalized Chinese learning environment and path is provided, which solves the problems of insufficient interactivity, lack of emotional communication and difficulty in data tracking in online Chinese teaching, and improves learning experience and efficiency.

CN118796041BActive Publication Date: 2024-11-26GUOKAI ONLINE EDUCATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411273637.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-11-26
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

The existing Chinese online teaching platform is not interactive and lacks emotional communication, making it difficult for learners to obtain a language experience in a real context in a digital environment, communication and interaction needs are not met, learning data tracking is difficult, and learning paths are single.

Method used

Using a Chinese learning method based on multi-agents, constructing agents, learning task agents, role dialogue agents and learning support agents through scenarios to generate personalized virtual scenarios, learning tasks, dialogue content and learning paths to realize users Emotional interaction and data tracking provide personalized learning paths.

Benefits of technology

It improves the learners' situational experience and interactivity, enhances the communication and interaction ability of Chinese learning, realizes dynamic tracking of learning data and personalized path customization, and improves learning efficiency and experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118796041B_ABST
    Figure CN118796041B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a Chinese learning method and system based on multi-agents. The method includes: obtaining the learning scenario and learning goal expected by the user, as well as the user's current learning level and current emotional state and sharing them; calling the scenario construction agent to generate a virtual scenario suitable for the user based on the shared content and sharing it; calling the learning task agent to generate a learning task suitable for the user based on the shared content and the user's emotional response in the virtual scenario and sharing it; calling the role dialogue agent to generate and share dialogue content for guiding the user's learning based on the shared content and the user's emotional response to the learning task; calling the learning support agent to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the dialogue content. This embodiment solves the problems of insufficient interactivity and lack of emotional communication in Chinese online learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of smart education technology, and in particular to a Chinese learning method and system based on multi-agent. Background Art

[0002] With the development of digital intelligence technology, online learning platforms have gradually achieved full coverage of language skills learning, language elements learning and teaching links. However, the current Chinese online teaching still has pain points such as insufficient interactivity and lack of emotional communication.

[0003] In the prior art, CN116739077B discloses a multi-agent deep reinforcement learning method and device based on curriculum learning, and CN111581470B discloses a multimodal fusion learning analysis method and system for dialogue system scenario matching, neither of which can solve the above problems. Summary of the invention

[0004] The embodiments of the present invention provide a Chinese learning method and system based on multi-agent to solve the above problems.

[0005] In a first aspect, an embodiment of the present invention provides a Chinese learning method based on multi-agent, comprising:

[0006] Obtain the user's desired learning scenario and learning goal, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent, where each agent is built based on the Chinese corpus knowledge base;

[0007] Calling the scenario construction agent, generating a virtual scenario suitable for the user according to the shared content, and sharing the virtual scenario with other agents;

[0008] Calling the learning task agent to generate a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and sharing the learning task and emotional response with other agents;

[0009] Calling the role dialogue agent to generate dialogue content for guiding the user to learn according to the shared content and the user's emotional response to the learning task, and sharing the dialogue content and emotional response to other agents;

[0010] The learning support agent is called to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.

[0011] In a second aspect, an embodiment of the present invention provides a Chinese learning system based on multiple agents, including: an information collection module, a scenario construction agent, a learning task agent, a role dialogue agent and a learning support agent; wherein,

[0012] The information collection module is used to obtain the user's desired learning scenario and learning goals, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent, wherein each agent is constructed based on the Chinese corpus knowledge base;

[0013] The scenario construction agent is used to generate a virtual scenario suitable for the user according to the shared content, and share the virtual scenario with other agents;

[0014] The learning task agent is used to generate a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and share the learning task and emotional response with other agents;

[0015] The role dialogue agent is used to generate dialogue content for guiding the user to learn according to the shared content and the user's emotional response to the learning task, and share the dialogue content and emotional response with other agents;

[0016] The learning support agent is used to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.

[0017] In summary, the embodiment of the present invention provides a Chinese learning method based on multi-agent, which can solve the following technical problems:

[0018] 1. The scenario-based Chinese learning dialogue in the embodiment of the present invention can solve the problem of insufficient learning scenario experience. The current digital platform for Chinese learning ignores the use of language to communicate in a real or simulated context in the learner's scenario experience. Learners may not be able to obtain an immersive Chinese usage environment through the digital platform. This embodiment provides a scenario-based Chinese learning dialogue to solve the problem of insufficient learning scenario experience and realize intelligent dialogue between learners and the system.

[0019] 2. The Chinese learning Q&A assistant based on companion learning in the embodiment of the present invention can solve the communication and interaction needs of learning. At present, the digital platform for Chinese learning lacks online intelligent Q&A, may focus more on one-way teaching, lacks instant feedback and interactive mechanisms, and learners may need more time to wait for the teacher's reply, which will prolong the learning process and reduce the learning experience. The Chinese learning Q&A assistant based on companion learning in this embodiment aims to correct learners' incorrect expressions, oral expression skills and listening comprehension skills in the process of Chinese learning by simulating the interaction and collaboration between different roles in a real language learning environment.

[0020] 3. The data collection based on the learning support agent in the embodiment of the present invention can solve the problem of difficulty in tracking learning data. It is difficult to track the learning progress, and learners may find it difficult to accurately grasp the learning progress and effect, which affects the adjustment and optimization of the learning plan. In the process of autonomous learning, learners often find it difficult to accurately and systematically track their learning progress and effect, which directly affects the formulation, adjustment and optimization of the learning plan, and may further affect the learning motivation and the final learning results. This embodiment is based on the learning support agent to dynamically track the learning data of students at all stages, and centrally manage and analyze the learners' learning data.

[0021] 4. The precise customization of personalized learning paths in the embodiments of the present invention can solve the problem of single learning tasks. Based on big data processing and NLP technology, the device can deeply analyze the learning behavior and preferences of learners and accurately formulate personalized learning paths for them. This personalized learning path not only takes into account the current level of the learner, but also factors such as their learning style and interests, thereby improving the pertinence and effectiveness of learning.

[0022] In addition, the multi-agent-based Chinese learning method provided by the embodiment of the present invention can also achieve the following beneficial effects:

[0023] 1. Simple algorithm and shared environment. This embodiment uses independent learning of each agent, and can alleviate stability through some techniques, so as to achieve simple algorithm, shared environment and strong scalability. At the same time, multiple agents can achieve centralized training through interactive strategies, thus having the ability of distributed decision-making.

[0024] 2. Complex pattern recognition. The multi-agent-based Chinese learning method and device can cope with complex environments and learners' interaction patterns. Through the complex nonlinear mapping of deep learning, it has powerful pattern recognition capabilities, thus having situational awareness and providing more accurate decision-making for the intelligent agent.

[0025] 3. The contextual nature of learning content: Contextual learning emphasizes the use of language in specific contexts to improve the ability to apply the language in practice.

[0026] 4. Multi-agents achieve optimal model utility. Compared with the traditional optimization of the utility function of a single agent, the use of various research methods such as model-free reinforcement learning and model-based reinforcement learning makes model optimization more efficient.

[0027] 5. Combining with digital human technology can realize AI teachers. With the application of digital humans in various industries and the deep integration of artificial intelligence technology represented by large models and education, the new generation of digital humans has the ability to create a sense of presence and immersive learning experience. With the continuous emergence of the Chinese learning craze, the AI ​​intelligent body system in the image of a digital teacher will be more deeply involved in the Chinese learning process, significantly improving the Chinese learning experience and learning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 It is a technical framework diagram of a multi-agent-based Chinese learning system provided by an embodiment of the present invention;

[0030] Figure 2 is a structural diagram of a multi-agent-based Chinese learning system provided by an embodiment of the present invention;

[0031] Figure 3 is a flow chart of a multi-agent-based Chinese learning method provided by an embodiment of the present invention;

[0032] Figure 4 It is a structural diagram of an Actor-Critic network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0034] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0035] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0036] The embodiment of the present invention provides a multi-agent based Chinese learning method. In order to illustrate the method, a multi-agent based Chinese learning system for implementing the method is first introduced. Figure 1 is a technical framework diagram of a multi-agent-based Chinese learning system provided by an embodiment of the present invention, such as Figure 1 The technical framework of the system is shown in the figure below. The system is a cluster of multiple agents, each of which represents a specific role or task in the learning process, such as vocabulary memory agent, grammar comprehension agent, listening training agent, situation model agent, etc. These agents form a complex and dynamic interactive network by sharing environmental information, exchanging learning results or competing for learning resources. The system integrates high-performance computing units, big data processing modules and advanced natural language processing (NLP) technology to ensure the efficient operation and accurate analysis of the multi-agent system.

[0037] Specific, combined Figure 1 At the AI ​​capability level, the system makes full use of advanced AI technologies, including but not limited to natural language processing (NLP), machine learning (ML), and deep learning (DL), as the underlying support for the entire system. Through API interfaces or internal integration, the system can efficiently call AI capabilities such as large language model capabilities, multimodal large model capabilities, and knowledge graph capabilities, providing a strong AI capability foundation for model training and intelligent agent construction at the model level, ensuring that the system can process complex language data in real time and understand learners' needs and feedback.

[0038] At the model level, the system further builds a customized proprietary model for Chinese learning based on the call of AI capabilities. The model is based on a massive Chinese corpus, and through pre-training, fine-tuning, testing, running and other steps, it continuously optimizes the understanding and recognition capabilities of Chinese language features. At the same time, the system also introduces advanced technologies such as reinforcement learning (RL) and transfer learning (TL), so that the model can continuously optimize itself and accumulate and reuse knowledge assets.

[0039] At the agent level, the system builds multiple agents based on trained models, each with unique functions, including but not limited to learning resource generation, scenario content creation, virtual scenario generation, oral dialogue partners, etc., which together constitute a collaborative Chinese learning ecosystem. In the process of building agents, the system fully considers the learners' learning style, interest preferences and current level, and ensures that each agent can provide the most suitable learning experience for learners through personalized customization. In addition, the system also realizes the flexible calling and dynamic adjustment of agents, automatically switching or combining different agents according to changes in the learning process to cope with various learning challenges.

[0040] At the application level, the system provides learners with a one-stop Chinese learning solution based on the ability call provided by the AI ​​capability layer, model training at the model layer, and the construction and integration of the intelligent agent layer. In this system, learners can use multiple intelligent agents to provide suitable learning paths and real-time interactive communication based on the learners' needs and interests. The system can not only provide rich learning resources, accurate feedback and guidance, but also tailor personalized learning plans for learners through intelligent evaluation and recommendation.

[0041] In short, the system provides learners with a more intelligent, efficient and personalized way to learn Chinese through the call of AI capabilities, the construction and optimization of training models, the construction and flexible call of intelligent agents, and the final application implementation.

[0042] Figure 2 FIG. 1 is a schematic diagram of a Chinese learning system based on a multi-agent provided by an embodiment of the present invention, and divides the system structure from the perspective of hardware devices and functional modules. Figure 2 As shown, the system includes 6 modules, namely binocular vision image acquisition device, face recognition layer, facial expression recognition layer, body movement recognition layer, AI computing model layer and data storage layer.

[0043] Among them, the binocular vision image acquisition device adopts advanced binocular stereo vision technology, which can capture high-definition video images of learners in real time.

[0044] Face recognition layer, based on deep learning algorithms, can quickly identify learners’ facial features from video streams and perform identity verification and preliminary emotion analysis.

[0045] The facial expression recognition layer uses refined facial expression recognition technology to accurately identify various subtle changes in facial expressions, such as smiling, frowning, surprise, etc., so as to more accurately evaluate learners' learning attitudes and interests.

[0046] The body movement recognition layer uses a 3D posture estimation algorithm to capture and analyze learners’ body movements in real time during the learning process. By identifying learners’ gestures, postures, and body movements, the system can determine their level of engagement, understanding, and potential learning barriers.

[0047] The AI ​​computing model layer is composed of multiple agents working together to process data from each module, conduct in-depth analysis, and generate personalized learning plans and learning reports based on this data.

[0048] The data storage layer is responsible for safely and efficiently storing learners' personal information, learning records, emotional data and other key information. The combination of cloud storage and local cache not only ensures the real-time and accessibility of data, but also takes into account the security and privacy protection of data.

[0049] Based on the above system, Figure 3 1 is a flowchart of a Chinese learning method based on multi-agents provided by an embodiment of the present invention. The method is applicable to the context of Chinese learning and is completed by the above system calling various hardware devices and agents. Figure 3 As shown, the method specifically includes:

[0050] S110, calling the content agent to build a Chinese learning corpus knowledge base.

[0051] This embodiment constructs a Chinese learning corpus knowledge base for the Chinese context. The collection, update and use of data in the library can be completed by content agents. Among them, the Chinese learning corpus knowledge base involves collecting a large amount of text data from various sources, and the data sources may include the Internet, libraries, newspapers, journals, etc. The data includes Chinese learning scenarios, Chinese grammar, daily conversations, business exchanges, and appreciation of literary works. Various specific data are classified according to the scenarios of Chinese learning, such as daily shopping, traveling, business conversations, conversation simulations, etc., and are reasonably divided.

[0052] Optionally, corpus collection uses automated programs to crawl text data from the Internet. These data are cleaned and preprocessed to remove irrelevant information and noise, providing datasets for training and testing machine learning and deep learning models. They can also help developers understand the statistical characteristics and usage patterns of language.

[0053] After the data is collected, it can be processed and handled, including cleaning, labeling, and word segmentation of text data to improve the quality and accuracy of data use. Optionally, the collected raw data may contain noise, such as garbled characters, spaces, special symbols, etc. Web crawlers should implement data cleaning steps to handle missing values ​​and outliers during data processing to improve the quality of the corpus.

[0054] Optionally, the Chinese learning corpus knowledge base is dynamically changing, so the collected corpus needs to be updated regularly to reflect the latest language usage. Specifically, the dynamic change of the Chinese learning corpus knowledge base refers to the reuse of knowledge assets formed by existing Chinese corpora, structured data records, and conversation records in the process to achieve workflow reuse, thereby providing knowledge-based suggestions and decisions.

[0055] Of course, the Chinese learning corpus knowledge base may also be constructed in other ways or artificially, or directly adopt an existing corpus knowledge base, which is not specifically limited in this embodiment.

[0056] S120. Obtain the user's expected learning scenario and learning goals, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent.

[0057] After the corpus knowledge base is built, the system UI can be used to receive basic information input by the user, including learning scenarios, learning goals, the user's current learning level, and current emotional state. This information will be shared with each intelligent agent in the system as basic environmental information for each intelligent agent to understand.

[0058] Optionally, the system allows learners to input or select their personal learning scenarios, self-assessed existing learning abilities (such as elementary, intermediate, advanced, vocabulary, grammar mastery, etc.), specific learning goals (such as mastering a certain concept, improving problem-solving speed, fluent daily communication, etc.), and desired difficulty requirements (such as basic, advanced, challenging, etc.).

[0059] At the same time, the system can use binocular visual image acquisition equipment to obtain the user's facial expressions and body movements; use audio equipment to obtain the user's voice intonation; and automatically identify the user's current emotional state, such as pleasure, calmness, disgust, sadness, anger, etc., based on the facial expressions, body movements and voice intonation.

[0060] S120, calling the scenario construction agent, generating a virtual scenario suitable for the user according to the shared content, and sharing the virtual scenario with other agents.

[0061] This step calls the scenario construction agent to generate a virtual learning scenario that matches the user based on the basic information input by the user. Specifically, the scenario construction agent is a database and algorithm module that contains a variety of learning scenarios and situations, which can provide users with a virtual learning environment that matches the learning scenario, such as a business meeting environment, a library learning environment, etc. The agent intelligently screens and matches the most suitable virtual scenario based on the user's basic situation. This process may involve analyzing the complexity of the learning scenario, the required skill level, and the degree of fit with the learner's current learning goals. Through algorithm optimization, it is ensured that the generated or extracted scenarios can not only meet the learner's difficulty requirements, but also effectively promote the achievement of learning goals.

[0062] Optionally, the scenario agent can use the integrated natural language processing and understanding module to perform detailed analysis and deconstruction of the target scene input by the user. The target learning scene input by the user is semantically split to obtain entity information (specific objects, places, times, events, etc.), attribute information (describes the specific attributes of entity information: including color, size, position, age, etc.), relationship information (describes the relationship between entities, including belonging, occurrence, and location), event information (involving the dynamic process of entity participation, such as meetings, travel, and purchases), and scenario information (involving the scenario information of entities, attributes, relationships, and event applications, including regular activities in the scenario, the location of items, etc.). The semantic analysis process not only involves capturing the direct meaning of the sentence, but also mining the contextual relationship of the sentence and possible implicit knowledge needs.

[0063] Optionally, for the target learning context input by the user, the situational agent will try to add connective modifiers to the sentence for parts that the machine may not be able to directly recognize, such as non-standard vocabulary, metaphors, abbreviations, or words with multiple meanings, such as adding connective modifiers to the sentence to enhance the clarity and logic of the sentence. These modifiers may include time adverbials, conditional adverbials, causal conjunctions, etc. They can help the machine better understand the sequence, conditional restrictions, and causal relationships of the sentence, so as to accurately grasp the user's true intention. For example, if the user's input is a vague learning goal, such as "learn how to shop", the agent may expand it to "in the next conversation, in a shopping mall, systematically communicate with the shopping guide and reach the level of being able to ask prices."

[0064] Optionally, the scenario agent can call the Chinese learning corpus according to the learning scenario expected by the learner, and dynamically match the corresponding virtual scenario according to the difficulty of the learning task and the emotional state. For example, when the learner is in a happy state, the scenario agent can output learning scenarios and virtual scenarios that are higher than the learner's current learning difficulty level; and when the learner is in a sad mood, the scenario construction agent can output a scenario that is comparable to the current learner's emotional state and is interesting, so as to mobilize the learner's learning enthusiasm.

[0065] At the same time, the scenario construction agent shares the generated virtual scenario with other agents, and the agents collaborate with each other through interactive interfaces to ensure the basis for each agent to understand and adapt to its operating environment.

[0066] S130, calling the learning task agent, generating a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and sharing the learning task and emotional response with other agents.

[0067] This step calls the learning task agent to further refine the learning tasks that match the user based on the basic information shared in S110 and the virtual scenario shared in S120. These tasks can include vocabulary learning, grammar practice, reading comprehension or listening training of a specific difficulty, etc., which are designed to help learners practice and consolidate what they have learned in a specific scenario. Similarly, the system can adjust the difficulty or type of learning tasks based on the learner's emotional response in the virtual scenario to stimulate the learner's interest and enthusiasm. For example, for learners who are depressed, some relaxing and interesting exercises can be provided.

[0068] Optionally, specific learning tasks can be generated by knowledge point matching. The learning task agent can match knowledge points suitable for the user in the corpus knowledge base based on the shared content and the user's emotional response in the virtual scenario; and then generate learning tasks suitable for the user based on the matched knowledge points.

[0069] The generated learning tasks and the user's emotional response in the virtual scenario will be shared with other agents as inputs of other agents, or shared with other agents in the form of agent training feedback, as the basis for each agent to understand the current basic environment. The sharing method of training feedback will be described in detail in subsequent embodiments and will not be repeated here.

[0070] S140, calling the role dialogue agent, generating dialogue content for guiding the user's learning according to the shared content and the user's emotional response to the learning task, and sharing the dialogue content and emotional response to other agents.

[0071] This step calls the role-playing agent to design the language dialogue content based on the basic information, virtual scenarios and learning tasks shared in the above steps, and conduct role-playing dialogues with learners. Specifically, the role-playing agent can use natural language processing technology to design and generate realistic dialogue content based on learning scenarios and learning tasks. These dialogues not only include knowledge point explanations, but also incorporate role-playing elements to make the learning process more vivid and interesting. By simulating dialogue exchanges in real scenarios, learners can improve their language application ability, problem-solving ability and critical thinking in the interaction.

[0072] After generating the scenario dialogue content, the system can present the designed dialogue to learners in the form of text, audio or video. The output interface is intuitive and friendly, supporting learners to adjust the reading speed, volume and other settings as needed. In addition, the system can also provide pause, playback and other functions so that learners can review and consolidate the learning content at any time.

[0073] S150, calling a learning support agent to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.

[0074] The learning support agent can run continuously throughout the learning process and is responsible for collecting and analyzing the learner's conversation data, including conversation turns, response time, answer accuracy, etc. This data is crucial for evaluating the learner's learning progress, identifying learning difficulties, and adjusting subsequent teaching strategies. Based on this data, the system can intelligently recommend more suitable learning resources, adjust the difficulty of learning tasks, or provide personalized learning suggestions to achieve a more efficient learning experience.

[0075] Furthermore, the above-mentioned multiple agents can be trained using deep reinforcement learning technology to continuously acquire the ability to interact accurately with users and communicate emotionally. Among them, deep reinforcement learning is a technology that combines deep learning technology with reinforcement learning methods, uses deep learning to perceive environmental features, and uses reinforcement learning methods to obtain strategies. The core of deep reinforcement learning is to use deep neural networks to process and understand complex environmental information, thereby extracting useful features and patterns; these features are then used in reinforcement learning algorithms to evaluate and select the optimal action strategy, with the goal of maximizing long-term accumulated rewards. The deep learning part can automatically learn high-level abstract representations of data from raw data by building a multi-layer neural network structure, which makes the deep learning model perform well in areas such as image recognition, speech processing, and natural language understanding. The reinforcement learning part, through interaction with the environment, enables the agent to learn how to choose the best action in a given state to achieve a specific goal or task. In deep reinforcement learning, the agent gradually optimizes its strategy through continuous trial and error and learning to adapt to the ever-changing environment and challenges. The interaction between the agent and the environment can be described by the Markov decision process. In particular, the essential difference between single-agent learning and multi-agent learning is that the state transfer and reward of a single-agent is only related to its current action and state, while the decision-making between multi-agents is not only related to its own action and state, but also to the actions of other agents. This makes the environment not steady from the perspective of a single-agent, and multi-agent technology can well solve the problem of personalized path formulation for Chinese learning and data tracking throughout the learning process for current learners.

[0076] In this embodiment, each agent shares the basic environment, which includes basic information about the user, the interaction content between each agent and the user, and the emotional changes of the user. The decision of each agent is related to the actions of all agents, and the return (or reward) obtained by each agent is related to its own actions and other agents. Therefore, each agent can optimize the model utility through interactive collaboration, and can solve key problems such as customized learning scenarios, customized learning paths, and real-time collection of learning data in Chinese learning.

[0077] In a specific implementation, under the technical framework of deep learning, each agent can be represented as an Actor-Critic network, which includes an action network (Actor) and a comment network (Critic). The network structure is as follows: Figure 4 As shown, this embodiment provides a method for multi-agent collaborative optimization and updating based on the network structure and the interaction of each agent in the learning link. Figure 4Let's introduce the basic algorithm of Actor-Critic (A2C algorithm for short). This algorithm combines the policy gradient and value function estimation methods to solve the decision-making problem in discrete or continuous action space. Specifically, the A2C algorithm introduces an advantage function to measure the pros and cons of taking a certain action relative to the average level. The Actor network update formula can be expressed as follows:

[0078]

[0079] in, and Respectively represent the network parameters of the Actor network at time t+1 and time t, represents the learning rate, Indicates in status Select each action The probability vector of Indicates status and actions When the advantage value is positive, it means that the current action is feasible, and its probability of being selected is gradually increased through gradient ascent. On the contrary, when the advantage value is negative, the action probability will decrease. is defined as follows:

[0080]

[0081] In the above formula, the state action value is included and state value To simplify the calculation process, the A2C algorithm uses the Critic network to estimate the state value function and converts it into The function is approximately expressed as:

[0082]

[0083] in, Represents the reward value at the current moment.

[0084] So, the advantage function It can be expressed as:

[0085]

[0086] The loss function of the Critic network at this time can be expressed as:

[0087]

[0088] The Critic network uses this loss function to update the network parameters.

[0089] Based on the above basic algorithm, each agent can optimize and update its own network parameters using the data learned by each user each time. Optionally, in the reward function of each agent, this embodiment considers the user's emotional response to the output content of the agent; if the user's emotional response to the output content of the agent changes in a positive direction, a positive reward is given; if the user's emotional response to the output content of the agent changes in a negative direction, a negative penalty is given; if the emotional state does not change, no reward is given in terms of emotional response. Exemplarily, the emotional states of pleasure, calmness, disgust, sadness, anger, etc. mentioned in the above embodiments gradually change from positive to negative. These emotional states can be assigned different scores of 10, 9, 8, 7, 6..., and the emotional score of the user facing a certain agent output is subtracted from the emotional score of the user not facing the output of the agent, and the difference in scores is added as an emotional reward item to the reward function of the entire agent network. At the same time, since the current emotional response is the result of the combined effect of the current agent and the upstream agent, after the user shows an emotional response to the output of a certain agent, in addition to rewarding the agent itself, reward feedback can also be given to the upstream agent of the current agent.

[0090] In a specific embodiment, after a virtual scene matching the user is generated in S120, a binocular visual image device and an audio device may be used to identify the user's emotional response to the virtual scene, and a reward value r is calculated based on the emotional response. 1 The reward value r 1 It can be added to the reward function of the scenario construction agent, and the parameters of the scenario construction agent are updated according to the update method of the Actor-Critic network. 1 It can be added to the reward function of the scenario building agent in a weighted manner. The added reward value for:

[0091]

[0092] in, is the original reward value, and is the weighting coefficient.

[0093] Subsequently, after generating a learning task suitable for the user in S130, the user's emotional response to the learning task can be identified in the same manner, and the reward value r can be calculated based on the emotional response. 2 The reward value r 2 On the one hand, it can be added to the reward function of the learning task agent to update the parameters of the learning task agent. The specific adding method can refer to formula (6). On the other hand, according to the reward value r 2A reward for the upstream scenario building agent can be constructed as a separate reward or added to the reward function of the scenario building agent to perform a secondary parameter update on the scenario building agent. Optionally, the update process can include the following steps:

[0094] Step 1: Cluster all users according to the learning scenario, learning goal, current learning level and current emotional state. That is, cluster all users who have used the system and are currently using the system according to the basic information input by the users.

[0095] Optionally, during the clustering process, each user’s learning scenario, learning goal, current learning level, and current emotional state can be first converted into four embedding vectors: ,in, represents the embedding vector of the learning scene, The embedding vector representing the learning target, The embedding vector representing the current learning level, Embedding vector representing the current emotional state.

[0096] Then, for each embedding vector: perform principal component analysis on the embedding vectors of all users, extract the first k feature vectors as principal component vectors, and decompose the embedding vector of each user along each principal component vector to obtain a new vector of dimension k, where k is a preset fixed constant. Specifically, for the first embedding vector For example, for all users By performing principal component analysis, we can obtain multiple mutually perpendicular eigenvectors, each of which represents The main component of data change in the vector space, the larger the eigenvalue corresponding to the eigenvector, the The more obvious the change in the feature vector is, the more obvious the change in the feature vector is. The principal component vector of each user Decompose along each principal component vector respectively, The components along each principal component vector form a new k-dimensional vector ,in, Respectively Components along the 1st, 2nd, ...kth principal component vectors. After performing the above operations on each embedding vector in the same way, each embedding vector will correspond to its own k principal component vectors. The four types of vectors for each user are can be converted into a new k-dimensional vector .

[0097] Next, for the same user: The eigenvalue of the kth principal component vector ,right Perform weighted fusion to obtain the fusion vector of the user :

[0098]

[0099] The fused vector will be used as the final vector representation of the user. This is because the input modalities of the four data, learning scenario, learning goal, current learning level and current emotional state, are not completely consistent. Some may be text data, while others may be image or audio data. Therefore, the methods of converting the four data into embedded vectors are not completely consistent. The vector dimensions are not completely consistent. This embodiment decomposes the principal components of the four vectors and re-represents the four vectors using the first k main components of the four vectors. On the one hand, it filters out the invalid components in the four vector spaces, and on the other hand, it converts the four vectors into a unified k dimension, which is convenient for weighted fusion of each vector. The larger the eigenvalue corresponding to the kth principal component vector of a certain vector, the greater the differentiated information of this vector in its own k principal component dimensions, and the better this vector can reflect the characteristics of the user, so this vector is given a greater weight.

[0100] After performing the above operations on each user, each user can obtain a fusion vector. These fusion vectors are clustered, and the clustering results represent the clustering characteristics of all users.

[0101] Step 2: For the users in the cluster that are grouped together with the current user, record the number of times all users in the cluster generate various learning tasks in the virtual scenario. For the sake of distinction and description, in this embodiment, the virtual scenario generated by the system for the current user in this learning is called the target virtual scenario, and all users in the cluster that are grouped together with the current user are called target users. This step records the virtual scenario generated by each target user in each learning. If the virtual scenario of some target users in some learning times is consistent with or similar to the target virtual scenario, continue to check the learning tasks generated by these target users in these learning times, and count the number of times each learning task is generated.

[0102] Step 3: Calculate the probability of generating this learning task in the target virtual scenario according to the number of times corresponding to various learning tasks. Optionally, the probability of generating this learning task in the target virtual scenario can be calculated according to the following formula: :

[0103]

[0104] in, and They respectively represent the number of times learning tasks with indexes i and j are generated in the target virtual scenario, and n1 represents the total number of types of learning tasks generated by all target users in the target virtual scenario; the learning task with index i is the learning task generated by the current user in the target virtual scenario, which is called the target learning task.

[0105] Furthermore, considering the volume of historical data, when the volume of historical data is small, the probability calculated using formula (8) may not be accurate. Therefore, the total number of times that users in all clusters generate learning tasks in the target virtual scenario can be calculated as When it is greater than or equal to the set threshold, the above probability is calculated using formula (8); and the total number of times users in all clusters generate learning tasks in the target virtual scenario is When it is less than the set threshold, the probability of generating the target learning task in the virtual scenario is calculated according to the following formula: :

[0106]

[0107] in, , s is any even number greater than 0, Represents the factorial of m. This formula can convert the number of times obtained by statistics into a probability value between 0 and 1. Each probability value still maintains the size relationship of the original number of times, but the intervals between specific values ​​are more uniform, which can alleviate the problem of unbalanced probability value distribution caused by insufficient historical data.

[0108] Step 4: Use reward value With the probability Multiply them together, and the product is used as the reward for the upstream scenario building agent, and the parameters of the scenario building agent are updated twice. Specifically, the product can represent the reward value The ability to influence the scenario-building agent can be used as a separate reward item to update the parameters of the scenario-building agent alone, or it can be used after the scenario-building agent generates a virtual scenario at a certain time (such as this time, or the next time closest to the current time). When updating parameters, add them to the reward function of the entire scenario agent to update the parameters. The added reward value It can be expressed as:

[0109]

[0110] in, is the weighting coefficient, The value of can be different from that in formula (6).

[0111] The above completes the dual parameter update of the learning task agent and the upstream agent based on the user's emotional response to the learning task. Afterwards, as the learning process progresses, after the dialogue content for guiding the user's learning is generated in S140, the user's emotional response during the interaction with the dialogue content can be collected in the same way, and the reward value can be calculated based on the emotional response The reward value On the one hand, it can be added to the reward function of the role dialogue agent to update the parameters of the role dialogue agent. The specific adding method can refer to formula (6). On the other hand, the reward value Rewards for the upstream learning task agent and scenario building agent are constructed separately, and the learning task agent is updated twice and the scenario building agent is updated three times according to the two rewards. Optionally, the update process may include the following steps:

[0112] Step 1. According to the above clustering results, for the target users clustered into one category with the current user, record the virtual scenarios and learning tasks generated by each target user in each learning session; if the virtual scenarios and learning tasks of some target users in certain learning sessions are respectively consistent with or similar to the target virtual scenarios and the target learning tasks, continue to check the conversation contents generated by these target users in these learning sessions, and count the number of times various conversation contents are generated.

[0113] Step 2: Calculate the probability of generating this dialogue content under the target virtual scenario and target learning task according to the number of times corresponding to various dialogue contents Optionally, the following two formulas are still used to calculate the probability: :

[0114]

[0115] in, They respectively represent the number of times dialogue contents with indexes i and j are generated in the target virtual scenario, and n2 represents the total number of types of dialogue contents generated by all target users under the target virtual scenario and target learning task; the dialogue content with index i is the content of this dialogue, which is called the target dialogue content.

[0116] Similarly, the total number of conversations among users in all clusters under the target virtual scenario and target learning task can be calculated as When is greater than or equal to the set threshold, the above probability is calculated using formula (11); When it is less than the set threshold, the probability is calculated according to the following formula :

[0117]

[0118] in, .

[0119] Step 3: Use reward value With probability Multiply them together, and use the product as a reward for the upstream learning task agent, and update the parameters of the learning task agent twice. Specifically, the product can represent the learning reward value The ability to influence the task agent can be used as a separate reward to update the parameters of the task learning agent, or it can be used as a reward after the learning task agent generates a learning task. When updating the parameters, they are added to the reward function of the entire task agent to update the parameters. The specific adding method is similar to formula (10).

[0120] Step 4: Use reward value With probability , Multiply them together, and use the product as the reward for the upstream scenario building agent, and update the parameters of the scenario building agent three times. Specifically, the product can represent the reward value The ability to influence the scenario building agent can be used as a separate reward to update the parameters of the scenario building agent alone, or it can be used as a reward after the scenario building agent generates a virtual scenario. When updating parameters, add them to the reward function of the scenario building agent as a whole to update the parameters. The added reward value It can be expressed as:

[0121]

[0122] in, is the weighting coefficient. More specifically, if the above and The reward items constructed are related to a certain The rewards are concentrated in the same parameter update, which can be:

[0123]

[0124] Similarly, in formulas (6), (10), (13), and (14) and The value of can be adjusted as needed, and this embodiment does not limit it. At the same time, in order to facilitate distinction and description, the reward value can also be They are called the first reward value, the second reward value and the third reward value respectively, and the probability and They are called the first probability and the second probability respectively.

[0125] It is worth mentioning that the entire training process mentioned above is a way of "sharing emotional responses to other agents through the form of agent training feedback" mentioned above. In summary, it can be seen that the basic user information, decision-making actions of each agent and user response in this embodiment all belong to the basic environment of multiple agents. This embodiment directly inputs part of them into each agent, and realizes the sharing of environmental information through input. The other part is added to the training process of each agent as a differentiated reward, and the sharing of environmental information is realized through training feedback, so that multiple agents can make joint decisions and joint training based on the shared environment, improving the stability of the environment and the accuracy of decision-making.

[0126] In summary, this embodiment provides a Chinese learning method based on multi-agent, which can solve the following technical problems:

[0127] 1. The scenario-based Chinese learning dialogue in this embodiment can solve the problem of insufficient learning scenario experience. The current digital platform for Chinese learning ignores the use of language to communicate in a real or simulated context in the learner's scenario experience. Learners may not be able to obtain an immersive Chinese usage environment through the digital platform. This embodiment provides a scenario-based Chinese learning dialogue to solve the problem of insufficient learning scenario experience and realize intelligent dialogue between learners and the system.

[0128] 2. The Chinese learning Q&A assistant based on companion learning in this embodiment can solve the communication and interaction needs of learning. At present, the digital platform for Chinese learning lacks online intelligent Q&A, may focus more on one-way teaching, lacks instant feedback and interactive mechanisms, and learners may need more time to wait for teachers' responses, which will prolong the learning process and reduce the learning experience. The Chinese learning Q&A assistant based on companion learning in this embodiment aims to correct learners' incorrect expressions, oral expression skills and listening comprehension skills in the process of Chinese learning by simulating the interaction and collaboration between different roles in a real language learning environment.

[0129] 3. The data collection based on the learning support agent in this embodiment can solve the problem of difficulty in tracking learning data. It is difficult to track the learning progress, and learners may find it difficult to accurately grasp the learning progress and effect, which affects the adjustment and optimization of the learning plan. In the process of autonomous learning, learners often find it difficult to accurately and systematically track their learning progress and effect, which directly affects the formulation, adjustment and optimization of the learning plan, and may further affect the learning motivation and the final learning results. This embodiment is based on the learning support agent to dynamically track the learning data of students at all stages, and centrally manage and analyze the learners' learning data.

[0130] 4. Based on the precise customization of personalized learning paths in this embodiment, the problem of single learning tasks can be solved. Based on big data processing and NLP technology, the device can deeply analyze the learning behavior and preferences of learners and accurately formulate personalized learning paths for them. This personalized learning path not only takes into account the current level of the learner, but also factors such as their learning style and interests, thereby improving the pertinence and effectiveness of learning.

[0131] In addition, the multi-agent-based Chinese learning method provided in this embodiment can also achieve the following beneficial effects:

[0132] 1. Simple algorithm and shared environment. This embodiment uses independent learning of each agent, and can alleviate stability through some techniques, so as to achieve simple algorithm, shared environment and strong scalability. At the same time, multiple agents can achieve centralized training through interactive strategies, thus having the ability of distributed decision-making.

[0133] 2. Complex pattern recognition. The multi-agent-based Chinese learning method and device can cope with complex environments and learners' interaction patterns. Through the complex nonlinear mapping of deep learning, it has powerful pattern recognition capabilities, thus having situational awareness and providing more accurate decision-making for the intelligent agent.

[0134] 3. The contextual nature of learning content: Contextual learning emphasizes the use of language in specific contexts to improve the ability to apply the language in practice.

[0135] 4. Multi-agents achieve optimal model utility. Compared with the traditional optimization of the utility function of a single agent, the use of various research methods such as model-free reinforcement learning and model-based reinforcement learning makes model optimization more efficient.

[0136] 5. Combining with digital human technology can realize AI teachers. With the application of digital humans in various industries and the deep integration of artificial intelligence technology represented by large models and education, the new generation of digital humans has the ability to create a sense of presence and immersive learning experience. With the continuous emergence of the Chinese learning craze, the AI ​​intelligent body system in the image of a digital teacher will be more deeply involved in the Chinese learning process, significantly improving the Chinese learning experience and learning efficiency.

[0137] It should be noted that the user data involved in this application (including face data, facial expression data, body movement data, high-definition video images of learners, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0138] Combination Figure 1 and Figure 2 The embodiment of the present invention also provides a Chinese learning system based on multiple agents. The system includes: an information collection module, a scenario construction agent, a learning task agent, a role dialogue agent and a learning support agent.

[0139] Among them, the information collection module is used to obtain the user's expected learning scenarios and learning goals, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent.

[0140] The scenario construction agent is used to generate a virtual scenario suitable for the user according to the shared content, and share the virtual scenario with other agents.

[0141] The learning task agent is used to generate a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and share the learning task and emotional response with other agents.

[0142] The role dialogue agent is used to generate dialogue content for guiding user learning based on the shared content and the user's emotional response to the learning task, and share the dialogue content and emotional response with other agents.

[0143] The learning support agent is used to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.

[0144] The system of this embodiment is based on the same inventive concept as the above method embodiment. After the features in any of the above method embodiments are applied to the system of this embodiment, the system of this embodiment can achieve the same technical effects as any of the above method embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A Chinese learning method based on multi-agent, characterized in that: include: Obtain the user's desired learning scenario and learning goal, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent, where each agent is built based on the Chinese corpus knowledge base; Calling the scenario construction agent, generating a virtual scenario suitable for the user according to the shared content, and sharing the virtual scenario with other agents; Calling the learning task agent to generate a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and sharing the learning task and emotional response with other agents; Calling the role dialogue agent to generate dialogue content for guiding the user to learn according to the shared content and the user's emotional response to the learning task, and sharing the dialogue content and emotional response to other agents; The learning support agent is called to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.

2. The method according to claim 1, characterized in that: The obtaining of the user's desired learning scenario and learning goal, as well as the user's current learning level and current emotional state, includes: Using binocular vision image acquisition equipment to obtain the user's facial expressions and body movements; Using audio equipment to obtain the user's voice intonation; The user's current emotional state is identified based on the facial expressions, body movements and voice intonation.

3. The method according to claim 1, characterized in that The calling of the scenario construction agent to generate a virtual scenario suitable for the user according to the shared content includes: Call the scenario building agent to perform the following operations: When the current emotional state is a happy state, generating a virtual scenario with a difficulty higher than the current learning level; When the current emotional state is a sad state, an interesting virtual scenario is generated.

4. The method according to claim 1, characterized in that: The calling of the learning task agent to generate a learning task suitable for the user according to the shared content and the emotional response of the user in the virtual scenario includes: Calling a learning task agent to match knowledge points according to the shared content and the user's emotional response in the virtual scenario; Generate learning tasks suitable for users based on the matched knowledge points.

5. The method according to claim 1, characterized in that After generating a virtual scenario suitable for the user according to the shared content, the method further includes: calculating a first reward value according to the emotional response of the user in the virtual scenario, and updating parameters of the scenario construction agent according to the first reward value; Correspondingly, after generating the learning task suitable for the user, the method further includes: calculating a second reward value according to the user's emotional response to the learning task, and updating the parameters of the learning task agent according to the second reward value; performing a secondary parameter update on the scenario construction agent according to the second reward value and the first probability of generating the learning task in the virtual scenario; Correspondingly, after generating the dialogue content for guiding the user's learning, it also includes: calculating a third reward value based on the interaction process between the user and the dialogue content, and updating the parameters of the role dialogue agent according to the third reward value; updating the parameters of the learning task agent twice according to the third reward value and the second probability of generating the dialogue content under the virtual scenario and learning task; and updating the parameters of the scenario construction agent three times according to the third reward value, the first probability and the second probability.

6. The method according to claim 5, characterized in that The performing secondary parameter updating on the scenario construction agent according to the second reward value and the first probability of generating the learning task in the virtual scenario comprises: Clustering all users according to the learning scenario, learning goal, current learning level and current emotional state; For users in the cluster that are clustered together with the current user, record the number of times all users in the cluster generate various learning tasks in the virtual scenario; According to the number of times corresponding to various learning tasks, a first probability of generating the learning task in the virtual scenario is calculated.

7. The method according to claim 6, characterized in that The clustering of all users according to the learning scenario, learning goal, current learning level and current emotional state includes: Each user’s learning scenario, learning goal, current learning level, and current emotional state are converted into four embedding vectors respectively; For the same embedding vector: perform principal component analysis on the embedding vectors of all users, and decompose the embedding vector of each user along the first k principal component vectors to obtain a new k-dimensional vector for each user; According to the eigenvalues ​​of the kth principal component vectors of various embedding vectors, four k-dimensional new vectors of the same user are fused to obtain a fused vector of the same user; All users are clustered according to their fusion vectors.

8. The method according to claim 6, characterized in that The calculating, according to the number of times corresponding to various learning tasks, a first probability of generating the learning task in the virtual scenario includes: When the total number of times that all users in the cluster generate learning tasks in the virtual scenario is less than a set threshold, the first probability of generating the learning task in the virtual scenario is calculated according to the following formula: : ; When the total number of times that all users in the cluster generate learning tasks in the virtual scenario is greater than or equal to the set threshold, the first probability of generating the learning task in the virtual scenario is calculated according to the following formula: : ; in, and They represent the number of times learning tasks with indexes i and j are generated, and n1 represents the total number of learning tasks; s is any even number greater than 0, is the factorial of m.

9. The method according to claim 1, characterized in that: Before obtaining the learning scenario and learning goal expected by the user, the method further includes: calling a content agent to construct a Chinese corpus knowledge base; The calling of the scenario construction agent to generate a virtual scenario suitable for the user according to the shared content includes: calling the scenario construction agent, and the scenario construction agent collaborates with the content agent to generate a virtual scenario suitable for the user according to the shared content.

10. A Chinese learning system based on multi-agent, characterized in that: include: Information collection module, scenario construction agent, learning task agent, role dialogue agent and learning support agent; among them, The information collection module is used to obtain the user's desired learning scenario and learning goals, as well as the user's current learning level and current emotional state, and share them with the scenario construction agent, learning task agent, role dialogue agent and learning support agent, wherein each agent is constructed based on the Chinese corpus knowledge base; The scenario construction agent is used to generate a virtual scenario suitable for the user according to the shared content, and share the virtual scenario with other agents; The learning task agent is used to generate a learning task suitable for the user according to the shared content and the user's emotional response in the virtual scenario, and share the learning task and emotional response with other agents; The role dialogue agent is used to generate dialogue content for guiding the user to learn according to the shared content and the user's emotional response to the learning task, and share the dialogue content and emotional response with other agents; The learning support agent is used to recommend a personalized learning path to the user based on the shared content and the user's interaction process with the conversation content.