A method and system for personalized learning path adaptive recommendation and academic early warning

Abstract

The present application relates to the technical field of education technology and artificial intelligence, and in particular to a personalized learning path adaptive recommendation and academic early warning method and system. The method comprises: initializing a cluster of intelligent agents driven by multiple LLMs under cold start conditions, and each intelligent agent obtains an initial Prompt through a small pool sampling strategy; the Prompt, strategy network and communication mechanism of the intelligent agent are optimized through evolutionary reinforcement learning, the adaptive evolution of role self-emergence and collaboration mode is realized; the team performance is evaluated based on a multi-dimensional reward function system, and feedback is given to the EvoRL optimization process; and finally, accurate education recommendation and academic early warning solutions are output. The system can realize self-organization and collaboration of the intelligent agent group without fine-tuning the LLM parameters, effectively solve the cold start problem, improve the personalization of the recommendation and the timeliness of the early warning, and is suitable for scenarios such as smart education platforms and online learning systems.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of educational technology and artificial intelligence, specifically to a method and system for personalized learning path adaptive recommendation and academic early warning. Background Technology

[0002] With the deepening development of educational informatization, personalized learning and academic early warning have become research hotspots in the field of education. However, most existing personalized learning systems mainly rely on preset rules, expert knowledge, or recommendation algorithms based on historical data, and their limitations are becoming increasingly apparent. For example, traditional collaborative filtering or content recommendation algorithms struggle to effectively capture learners' dynamically changing interests and cognitive states, and cannot flexibly adapt to the "cold start" problem of new courses or new learning models. Furthermore, these systems often lack a deep understanding of complex interactive behaviors during the learning process, making it difficult to achieve truly adaptive learning path recommendation.

[0003] In terms of academic risk warning, existing systems mostly employ threshold-based or statistical model-based methods, analyzing explicit data such as learners' grades and attendance to assess academic risk. However, these methods are often lagging, failing to capture early signs of academic risk, and neglecting to fully consider implicit behavioral data during the learning process (such as study time, mastery of knowledge points, and frequency of online interaction) and the complex relationships between different learners. More importantly, existing systems typically lack the ability to dynamically adjust warning strategies, failing to adaptively optimize based on changes in the learning environment or individual learner circumstances, resulting in insufficient accuracy and timeliness of warnings.

[0004] Therefore, solving the problem of self-organization and collaboration among multiple agents under "cold start" conditions without preset roles or fixed division of labor, and how to break through the traditional "majority voting" paradigm to achieve spontaneous emergence of division of labor and collaboration among agents, remains an urgent technical challenge. Summary of the Invention

[0005] Embodiments of the present invention provide a method for personalized learning path adaptive recommendation and academic warning, comprising the following steps:

[0006] Under cold start conditions, an agent cluster consisting of N agents driven by large language models (LLMs) is initialized, where N is an integer greater than 1. Each agent obtains an initial prompt word through a small-pool sampling strategy.

[0007] By using evolutionary reinforcement learning to collaboratively optimize the agent's Prompt, policy network, and communication mechanism, role self-emergence and adaptive evolution of cooperative modes are achieved.

[0008] The team performance of the agent cluster is evaluated based on various metrics and fed back into the EvoRL optimization process.

[0009] The Evolutionary Reinforcement Learning (EvoRL) mechanism iteratively evolves the Prompt, policy network, and communication mechanism to output educational recommendations and academic warning schemes.

[0010] The steps involved in achieving role emergence and adaptive evolution of cooperative modes through evolutionary reinforcement learning to collaboratively optimize the agent's Prompt, policy network, and communication mechanism include:

[0011] During the independent problem-solving phase, each agent generates a preliminary solution and corresponding reasoning process data based on the current Prompt and internal state.

[0012] During the multi-round free interaction phase, each agent shares the initial solutions and reasoning process data of all agents through message passing without a preset protocol, and dynamically adjusts its own behavior and prompt.

[0013] In the self-evolution phase driven by EvoRL, after multiple rounds of interaction, a consensus solution is generated. The EvoRL engine optimizes the Prompt, policy network parameters, and communication mechanisms of each agent based on the quality of the consensus solution and the performance during team collaboration.

[0014] In some embodiments, each agent obtains an initial Prompt through a small-pool sampling strategy, including:

[0015] Each agent is randomly assigned an initial prompt from a pre-defined pool of general prompts.

[0016] In some embodiments, during the EvoRL-driven self-evolution phase, after multiple rounds of interaction, a consensus solution is generated. The EvoRL engine optimizes the Prompt, policy network parameters, and communication mechanisms for each agent based on the quality of the consensus solution and the performance during team collaboration. The optimization of the Prompt for each agent includes:

[0017] The fitness of each agent's Prompt is assessed based on the quality of the consensus solution. High-fit Prompts are selected and mutated and crossovered to generate the next generation of Prompts.

[0018] In some embodiments, during the self-evolution phase driven by EvoRL, after multiple rounds of interaction, a consensus solution is generated. The EvoRL engine optimizes the Prompt, policy network parameters, and communication mechanisms for each agent based on the quality of the consensus solution and the performance during team collaboration. The optimization of the communication mechanism for each agent includes:

[0019] Based on team reward data, each agent dynamically adjusts its communication mechanism.

[0020] In some embodiments, the step of achieving role emergence and adaptive evolution of cooperative modes by collaboratively optimizing the agent's Prompt, policy network, and communication mechanism through evolutionary reinforcement learning includes:

[0021] A multi-dimensional reward function system is used to evaluate the team performance of the agent cluster, and the Prompt, policy network and communication mechanism of each agent are dynamically adjusted. The multi-dimensional reward function system includes a main reward and multiple secondary rewards.

[0022] In some embodiments, the primary reward is based on the accuracy of the team’s final output, while secondary rewards include teamwork and the diversity of agent roles.

[0023] In some embodiments, team collaboration is assessed by consensus convergence speed, and the diversity of division of labor is assessed by the difference in the types of contributions made by agents.

[0024] On the other hand, a system for personalized learning path adaptive recommendation and academic warning is provided, including:

[0025] The agent initialization and management module initializes an agent cluster consisting of N LLM-driven agents under cold start conditions, where N is an integer greater than 1. Each agent obtains an initial prompt through a small-pool sampling strategy.

[0026] The EvoRL multi-agent collaboration and self-evolution module optimizes the agent's Prompt, policy network, and communication mechanism through evolutionary reinforcement learning, achieving role self-emergence and adaptive evolution of collaboration modes.

[0027] The multi-dimensional reward function system module evaluates the team performance of the agent cluster based on various dimensional indicators and feeds it back to the EvoRL optimization process.

[0028] The recommendation and early warning module uses the EvoRL mechanism to iteratively evolve the Prompt, policy network, and communication mechanism to generate educational recommendation and academic early warning schemes.

[0029] On the other hand, an apparatus for personalized learning path adaptive recommendation and academic warning is provided, including a memory and a processor. The memory stores at least one program, and the processor executes the at least one program to implement the aforementioned method for personalized learning path adaptive recommendation and academic warning.

[0030] The above technical solution has the following technical effects:

[0031] This application achieves dual adaptive evolution of agent behavior and cooperation patterns by initializing an LLM-driven agent cluster without pre-defined roles, combining evolutionary algorithm optimization of prompts with reinforcement learning optimization of interaction strategies. This method effectively overcomes the limitations of traditional systems that rely on pre-defined rules, struggle with cold start problems, and are ill-suited for dynamic environments. It enables the agent group to self-organize and emerge with efficient division of labor without fine-tuning the massive parameters of the LLM, thereby generating highly personalized and precise learning paths for learners and providing timely and reliable academic risk warnings. This significantly improves the intelligence, adaptability, and deployment economy of educational support systems. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating a method for personalized learning path adaptive recommendation and academic warning according to an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of another embodiment of the present application. Detailed Implementation

[0034] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0035] Multi-agent systems and reinforcement learning have shown great potential in complex decision optimization. However, applying these technologies to personalized learning and academic early warning systems, particularly addressing the self-organizing collaboration of multi-agent systems under "cold start" conditions without pre-defined roles or fixed divisions of labor, and how to break through the traditional "majority voting" paradigm to achieve spontaneous emergence of division of labor and collaboration among agents, remains a pressing challenge. Existing research largely focuses on multi-agent learning under fixed roles or predefined collaborative models, which struggles to cope with the highly dynamic and uncertain complexities of real-world educational scenarios.

[0036] In view of this, this application proposes a cold-start multi-agent self-evolution method based on evolutionary reinforcement learning (EvoRL), aiming to overcome the limitations of existing technologies in personalized learning path recommendation and academic risk warning. This invention, by introducing evolutionary reinforcement learning (EvoRL), achieves automatic optimization of agent capabilities and self-evolution of group collaboration patterns, thereby providing learners with a more accurate and adaptive personalized learning experience, and enabling more timely and effective identification and warning of academic risks.

[0037] like Figure 1 As shown, one embodiment of this application provides a personalized learning path adaptive recommendation and academic warning method, including the following steps:

[0038] S101: Initialize an agent cluster consisting of N LLM-driven agents under cold start conditions, where N is an integer greater than 1, and each agent obtains an initial prompt through a small-pool sampling strategy.

[0039] For example, "You are a learning assistant, please help students solve problems." This approach aims to provide the agent with initial behavioral guidance while preserving sufficient room for evolution.

[0040] S102: Through evolutionary reinforcement learning, the agent's Prompt, policy network, and communication mechanism are collaboratively optimized to achieve role emergence and adaptive evolution of cooperative modes.

[0041] S103: Evaluate the team performance of the agent cluster based on various metrics and provide feedback to the EvoRL optimization process.

[0042] S104: Through the EvoRL mechanism, the Prompt, policy network, and communication mechanism are iteratively evolved to output educational recommendation and academic early warning schemes.

[0043] Step S102 specifically includes:

[0044] During the independent problem-solving phase, each agent generates a preliminary solution and corresponding reasoning process data based on the current Prompt and internal state.

[0045] During the multi-round free interaction phase, each agent shares the initial solutions and reasoning process data of all agents through message passing without a preset protocol, and dynamically adjusts its own behavior and prompt.

[0046] In the self-evolution phase driven by EvoRL, after multiple rounds of interaction, a consensus solution is generated. The EvoRL engine optimizes the Prompt, policy network parameters, and communication mechanisms of each agent based on the quality of the consensus solution and the performance during team collaboration.

[0047] After the agents obtain the initial Prompt through a small-pool sampling strategy, they continuously optimize the internal state of the agent cluster through evolutionary reinforcement learning, including but not limited to the current Prompt. Studying history (Records the agent's performance and interactions in past tasks), collaborative experience (It records the patterns and effects of cooperation between agents and other agents), as well as the policy network parameters obtained through the EvoRL process. It is worth emphasizing that the "training-free" characteristic of this invention is reflected in the fact that the method of this application does not directly fine-tune the model parameters of LLM, but improves the capabilities and collaborative efficiency of the agent cluster by optimizing the prompts, interaction methods and communication mechanisms of each agent. This greatly reduces training costs and deployment difficulties, and enhances the flexibility and adaptability of the method of this application.

[0048] In the above embodiments, after multiple rounds of evolution, the system outputs a final educational recommendation and academic warning report for educators and / or learners to refer to. Of course, in outputting the final educational recommendation and academic warning report, the system can selectively output a separate educational recommendation report or an academic warning report, or it can output a comprehensive report containing both.

[0049] This embodiment uses evolutionary reinforcement learning (EvoRL) as its core optimization mechanism, comprehensively leveraging the global exploration capabilities of evolutionary algorithms and the efficient policy optimization capabilities of reinforcement learning to achieve adaptive evolution of the agent cluster's capabilities and collaborative modes. For example, by defining a multi-agent interaction environment simulating an educational scenario, learners, course content, and teachers are modeled as individual agents within the environment, clearly defining the state space, action space, environmental dynamics, and reward signals. Externally, evolutionary algorithms optimize the diversity and collaborative strategies of the agent cluster, while internally, reinforcement learning guides individual agents in trial-and-error exploration and policy updates within the environment, thereby driving the overall evolution and capability improvement of the team. Under cold-start conditions without preset roles and fixed divisions of labor, through collaborative optimization using EvoRL, personalized learning paths that match individual learner characteristics, interests, and learning progress are dynamically generated and optimized, while simultaneously achieving accurate early warning of academic risks.

[0050] In the above embodiments, LLM, or Large Language Model, is an artificial intelligence model trained on massive amounts of text data. It can understand, generate, and process human language. For ease of understanding, it can be imagined as a super-powerful brain with vast knowledge, having read almost every book and article on the internet, but without subjective consciousness; it is merely a highly complex probabilistic prediction tool. A Prompt, on the other hand, is an instruction, question, or text message provided by the user to the LLM, designed to guide it to produce the expected response. It serves as the bridge and interface for communication between the user and the LLM.

[0051] This application formalizes personalized learning recommendation and academic early warning tasks as a multi-agent reinforcement learning environment. For example, this environment is described as a quintuple E=(S,A,P,R,N), where S is the environment state space, containing multi-dimensional features such as profiles of all learners, historical learning data, and knowledge mastery levels; A is the action space, where agents can choose actions based on the current state, including recommending specific learning paths, issuing risk warnings, and adjusting learning plans; P is the state transition function, defining the dynamic evolution of the environment state after the agent takes an action; R is the reward function, combining learning effectiveness, recommendation accuracy, warning timeliness, and team collaboration to provide real-time feedback on the quality of agent and team behavior; and N is the agent cluster, where the system supports parallel collaboration of multiple heterogeneous and / or homogeneous agents.

[0052] In step S101, under cold start conditions, the LLM-driven agent cluster is initialized in a state of "ignorance" or "semi-ignorance," thereby providing maximum exploration space for subsequent self-evolution. For example, the agents simulate various roles that may exist in the educational ecosystem, such as learning consultants, curriculum designers, academic tutors, and psychological counselors. However, in the initial stage, no explicit role labels or responsibilities are assigned to each agent; instead, these roles are allowed to emerge naturally during the collaboration process. Specifically, based on the preset number of agents N (for example, for a class or grade of students, N can be set to 2 to 100 agents to simulate different dimensions of educational support), N Large Language Model (LLM) agents are instantiated, and each agent is assigned a unique identifier ID. i The initial prompt (catch word) setting for each agent is crucial to its "cold start" characteristic. This application employs a small-pool sampling strategy to obtain the initial prompt (catch word).

[0053] The LLM agent is initialized in an "unconscious" or "semi-unconscious" state during a cold start, and an initial prompt is allocated using a small-pool sampling strategy. The main technical effects are reflected in the following aspects:

[0054] Avoiding pre-defined role restrictions: The system does not assign fixed roles or tasks to agents in the initial stage, which avoids the lack of flexibility caused by role pre-defined roles in traditional systems and provides ample room for exploration of subsequent role emergence and adaptive evolution of collaborative modes.

[0055] Enhance the system's generalization ability: By randomly assigning initial instructions from a set of general and open prompts through a small-pool sampling strategy, the agent is provided with weak behavioral guidance, which accelerates early evolution convergence, while avoiding the limitation of strong task pre-set on the system's adaptability, enabling it to better cope with diverse educational scenarios and task types.

[0056] Reduced cold start difficulty: Even in the absence of historical data or prior knowledge, the system can still quickly form effective cooperation through free interaction and evolution among agents, significantly improving the system's usability and practicality under cold start conditions.

[0057] Supports training-free optimization: By optimizing the Prompt instead of fine-tuning LLM parameters, the computational resource requirements and deployment costs are significantly reduced, while maintaining the system's flexibility and scalability.

[0058] In some embodiments, each agent is randomly assigned an initial prompt from a predefined, general prompt pool. Each agent's initial prompt is randomly selected from a predefined, small, general, and open-ended pool of instructions. For example, the prompt pool might contain: "You are a learning assistant, please help students solve problems," "You are an analyst, please analyze data and provide insights," or "You are an advisor, please give advice as appropriate," and so on. This approach maintains the cold-start characteristic while providing each agent with a weak initial direction, helping to accelerate early evolutionary convergence, but still avoiding strong presuppositions for specific roles or tasks. This prompt sampling can be represented as:

[0059]

[0060] in, This represents the Prompt obtained from sampling. It is the first in the predefined Prompt pool A Prompt, and This represents the total number of Prompts in the Prompt pool. This strategy guides the agent's initial behavior to some extent, but its openness still allows the agent to break through these initial settings in subsequent evolution, providing a foundation for subsequent self-evolution.

[0061] The agent randomly assigns an initial prompt from a pre-set general prompt pool, which has the following technical effects:

[0062] The balance between guidance and openness: Instructions in the general Prompt pool, such as "You are a learning assistant" or "You are an analyst," provide the agent with an initial behavioral framework, guiding it toward a direction beneficial to educational tasks, while maintaining sufficient openness to allow the agent to break through the initial settings in subsequent interactions and spontaneously evolve into a more professional role.

[0063] Promoting Emergent Diversity: The random assignment mechanism avoids the uniformity of initial behavior among all agents, increases diversity within the team, and lays the foundation for role differentiation and functional complementarity in subsequent collaboration.

[0064] Accelerated convergence and stability: A weak initial direction helps reduce the stochastic exploration time in the early evolution process and improves the system's convergence speed. At the same time, by controlling the generality of the Prompt, system rigidity or overfitting caused by excessive specificity is avoided.

[0065] Suitable for cold start scenarios: In the absence of historical data support, the intelligent agent is quickly initialized through a simple and reusable Prompt pool, ensuring that the system can be put into use immediately. It is especially suitable for rapid adaptation to new courses, new students or new educational scenarios.

[0066] In this embodiment, evolutionary reinforcement learning (EvoRL) models the educational recommendation and academic warning tasks as a multi-agent interactive environment. For example, the state of the multi-agent interactive environment includes each learner's individual profile, learning history, real-time knowledge mastery, and task progress; the agent's action space includes learning path recommendations, risk warning suggestions, and knowledge supplementation plans. Each agent's decision affects the transition of the environment state and receives reward feedback centered on learning effectiveness, warning accuracy, and team collaboration performance. An outer evolutionary algorithm is used to optimize the agent team's structure, strategy diversity, and collaborative division of labor, enhancing overall exploration capabilities. In each evolutionary iteration, each agent within the group acts as a reinforcement learning subject, continuously optimizing its decision-making strategy based on environmental state observations using RL algorithms (such as policy gradient, Actor-Critic, etc.) to maximize the cumulative rewards for individuals and the team in a specific environment. After the inner RL training is completed, high-fitness individuals are selected for crossover and mutation according to the evolutionary algorithm to generate a new generation of agent clusters, thereby driving the entire agent cluster to adaptively evolve towards a better collaborative mode.

[0067] In the above embodiments, fitness The calculation formula is as follows:

[0068]

[0069] in, Indicates the first The fitness of an agent at generation t is determined by the prompt. The fitness function considers two dimensions: the team reward signal and the difference in agent contribution. The formula is as follows: Indicates by prompt Guided agent policy distribution It is an intelligent agent exist The time-prompt, which is the input condition used to guide the generation of the agent's policy, is a task description and contextual information in natural language. A team reward signal (defined below). That is, 1 minus the first The cosine similarity between the output of an individual agent and the average output of the team of agents is used to measure the difference in independent contributions of the agents. Indicates in At this moment Given the policy distribution adopted by each agent, the expected value of the team reward for all possible generated trajectories. This is a hyperparameter used to balance overall team performance with individual diversity. Fitness is sorted from high to low, and a threshold is set to select fitness; fitness values ​​above this threshold are considered high fitness. This threshold is set empirically.

[0070] In the above embodiments, during the independent problem-solving phase, each agent generates a preliminary solution and corresponding reasoning process data based on the current prompt and internal state. This process can be proactively implemented by the agent cluster or passively implemented based on the needs of the user or learner. For example, upon receiving a learner's personalized learning needs or academic warning request, the independent problem-solving phase is triggered. At this time, each agent in the agent cluster team... Firstly, based on its current... and internal state Each agent independently analyzes the problem. At this stage, each agent generates an initial solution. (For example, preliminary learning path suggestions, potential risk point analysis) and the corresponding reasoning process. This stage emphasizes the independent thinking and initial contributions of individual intelligent agents, laying the foundation for subsequent team collaboration. Intelligent Agent The initial output can be represented as:

[0071]

[0072] in, It is an intelligent agent The initial output includes its independent insights into the problem and the logical chain supporting those insights.

[0073] When entering the multi-round free interaction phase, the initial solutions of all intelligent agents With reasoning process This will be fully shared with all other agents in the team. This means that each agent can access the initial thought processes of other team members. Based on this, the team enters a highly free, multi-round interactive phase without a fixed process. This phase is crucial for the self-evolution of the collaborative model among agents; it simulates the process of free discussion, debate, and revision in a human team. The interactive process can be represented as a sequence. ,in It refers to the first Intelligent agents in wheels Send to the agent The messages are open-ended, including statements of opinion, questions, supplementary information, rebuttals and challenges, suggestions and corrections. This interaction is completely free, without pre-defined communication protocols or role divisions. Agents dynamically adjust their behavior and current prompts by observing the messages and behaviors of other agents and the progress of the team's tasks. For example, in an academic warning scenario, an agent might initially focus on analyzing student grades, but upon discovering that other agents emphasize student attendance and classroom participation, it might adjust its focus, even taking on a more "data integrator" role in subsequent interactions, and its prompt might evolve towards "multimodal data analysis." This self-organization and role emergence are key to this invention's breakthrough from the traditional "majority voting" paradigm. It enables agent teams to spontaneously form efficient collaborative models based on the complexity and dynamism of the task, rather than simply integrating individual answers.

[0074] After multiple rounds of interaction, the team will attempt to reach a final consensus solution. (For example, a complete personalized learning path plan, and / or a detailed academic risk analysis report and intervention recommendations). The EvoRL engine will... The quality and performance during team collaboration, for each agent Policy network parameters And the communication mechanisms are optimized. This optimization is multi-layered and collaborative, aiming to maximize the overall effectiveness of the team.

[0075] In some embodiments, optimizing the Prompt for each agent includes: assessing the fitness of the Prompt for each agent based on the quality of the consensus solution, selecting the Prompt with high fitness, and performing mutation and crossover to generate the next generation of Prompt.

[0076] The fitness here is at the team level, meaning that optimizing a single agent's Prompt serves the collaborative effect of the entire agent cluster, rather than simply improving individual performance. Prompt mutation operations can involve random modification, insertion, or deletion of the Prompt text, or they can utilize LLM's own generation capabilities to intelligently rewrite the Prompt based on the current agent's performance and team goals; for example, evolving "You are a learning assistant" into "You are a learning assistant focused on math tutoring." Crossover operations can combine portions of two high-performance Prompts to form a new Prompt. This evolutionary process can be formalized as:

[0077]

[0078] in, It is an intelligent agent exist Prompt of the moment Based on team agent collaboration, the Prompt is obtained The fitness function enables agents to adaptively adjust their behavioral tendencies and task understanding, thereby better integrating into team collaboration; It is a reward signal received by the intelligent agent cluster team.

[0079] In some embodiments, optimizing the communication mechanism for each agent includes: dynamically adjusting the communication mechanism of each agent based on team reward data.

[0080] In the above embodiments, each agent possesses its own policy network, which guides the agent's behavioral choices during interactions; for example, when to send a message, what kind of message to send, how to respond to other agents, and when to proactively raise objections or consolidate information. These policy networks do not directly control the internal parameters of the LLM, but rather act as "external controllers" or "behavioral preference processors" of the LLM, indirectly guiding its behavior by influencing the LLM's input or output format. The evolution of the policy network can be achieved through reinforcement learning agents, with the goal of maximizing team rewards. For example, if an agent discovers through reinforcement learning that proactively offering a summary statement effectively promotes consensus when team discussions reach an impasse, its policy network reinforces this behavioral pattern. The update of the policy network can be represented as:

[0081]

[0082] in, It is an intelligent agent The strategy network parameters, It's the learning rate. For parameters The gradient operator is used to represent "gradient ascent along the parameter space with respect to the objective function", which can also be understood as being used to calculate the performance objective function. about The partial derivatives; It is the strategic objective function, designed to maximize team rewards.

[0083] In some embodiments, optimizing the communication mechanism for each agent includes: dynamically adjusting the communication mechanism of each agent based on team reward data.

[0084] In the above embodiments, the communication method between agents is also an important component of EvoRL optimization. Traditional communication mechanisms are relatively fixed or simple, while this application allows the communication mechanism to evolve adaptively to adapt to different collaborative tasks and team dynamics. For example, agents can learn to dynamically adjust the frequency of their message sending (increasing communication frequency at critical moments), the level of detail in the messages (adjusting information granularity for different recipients), and selectively send messages to which agents (identifying key information transmitters). This evolution of the communication mechanism can be achieved by optimizing a communication policy network. To achieve this, the network determines the intelligent agent. Towards The probability or intensity of message transmission. For example, in an academic warning scenario, a "data analysis" agent might learn that conveying complex statistical results in a concise and clear manner to a "psychological counseling" agent is more conducive to team decision-making than directly sending raw data. Updates to communication strategies are also based on team rewards.

[0085]

[0086] in, It's the learning rate. It is the communication objective function.

[0087] In some embodiments, the step of achieving role emergence and adaptive evolution of cooperative modes by collaboratively optimizing the agent's Prompt, policy network, and communication mechanism through evolutionary reinforcement learning includes:

[0088] A multi-dimensional reward function system is used to evaluate the team performance of the agent cluster, and the Prompt, policy network and communication mechanism of each agent are dynamically adjusted. The multi-dimensional reward function system includes a main reward and multiple secondary rewards.

[0089] In the above embodiments, the multi-dimensional reward function system serves as a "compass" for the EvoRL-driven self-evolution process. It is used to accurately evaluate the performance of the multi-agent collaborative framework in personalized learning path recommendation and academic warning tasks, and guide the self-evolution direction of the agent team. This system not only focuses on the quality of the final task completion but also emphasizes the emergent behavior, diversity, robustness, and generalization ability of the system in real educational environments during the collaborative process. The design of the multi-dimensional reward function is crucial to the effectiveness of EvoRL, as it directly determines the learning objectives and evolutionary direction of the agent team. By employing a multi-dimensional reward function system to evaluate the team performance of the agent cluster and dynamically adjusting the Prompt, policy network, and communication mechanism of each agent, the system's adaptability and robustness in complex and dynamic educational environments are significantly improved. Furthermore, multi-objective optimization avoids overfitting or behavioral rigidity under a single metric, thereby enhancing the overall decision-making quality and generalization ability. Meanwhile, the real-time feedback of reward signals drives the agent to continuously self-organize and evolve at the Prompt, policy, and communication levels, promoting the spontaneous emergence of more accurate role division and collaboration strategies, ultimately effectively improving the accuracy of personalized learning path recommendations and the timeliness and reliability of academic warnings.

[0090] In some embodiments, the reward function system proposed in this application includes a primary reward and multiple secondary rewards.

[0091] In the above embodiments, the primary reward is the team final answer accuracy, which directly reflects the goal achievement capability of the method in this application. The primary reward is defined as measuring the effectiveness and accuracy of the educational recommendations and academic warning schemes ultimately output by the intelligent agent cluster team. For example, in a personalized learning path recommendation scenario, the degree of matching between the recommended path and the learner's actual learning outcomes (such as grade improvement, knowledge mastery, and increased learning interest) can be evaluated; in an academic warning scenario, the accuracy (whether academic risks are successfully predicted) and recall (whether at-risk students are missed) of the warning are evaluated. For example, for academic warnings, indicators such as F1 score and AUC can be used to quantify their accuracy. Primary Reward It can be calculated in the following ways:

[0092]

[0093] in, and These are the performance metrics used to evaluate the team's final output on the validation set; they are key metrics for measuring the actual effectiveness of the system in practical applications.

[0094] In the above embodiments, the supplementary rewards aim to guide the intelligent agent cluster team to form a more efficient, diverse, and robust collaborative model, and improve the adaptability of this application in complex educational environments. Supplementary rewards may include, but are not limited to, the following aspects:

[0095] Team collaboration measures the efficiency and quality of information sharing, viewpoint integration, and conflict resolution within an intelligent agent team during interactions. Highly collaborative teams reach consensus faster and more effectively, fully leveraging the collective wisdom of their members. This can be calculated by measuring the convergence speed of team consensus, for example, by evaluating the trend of the similarity between the team's intermediate solutions and the final consensus solution over time during multiple rounds of interaction. The calculation can be expressed as:

[0096]

[0097] in, These are the interaction rounds required to reach a consensus. yes The team's intermediate solution after rounds of interaction. This is the final consensus solution.

[0098] In some embodiments, the secondary reward also includes agent division of labor diversity. This reward encourages agent teams to spontaneously form diverse roles and divisions of labor, avoiding all agents performing the same or similar tasks, thereby fully leveraging the advantages of multi-agent teams and improving the overall efficiency and problem-solving capabilities of the team. Agent division of labor diversity It can be evaluated by analyzing the differences in the content, focus, and type of contribution (e.g., proposing new viewpoints, refuting errors, integrating information, providing data support, and soothing emotions) of the intelligent agent during the interaction.

[0099]

[0100] in, It is the number of intelligent agents. and It is an intelligent agent and The final solution.

[0101] On the other hand, this application also provides a system for personalized learning path adaptive recommendation and academic warning, including:

[0102] The agent initialization and management module initializes an agent cluster consisting of N LLM-driven agents under cold start conditions, where N is an integer greater than 1. Each agent obtains an initial prompt through a small-pool sampling strategy.

[0103] The EvoRL multi-agent collaboration and self-evolution module optimizes the agent's Prompt, policy network, and communication mechanism through evolutionary reinforcement learning, achieving role self-emergence and adaptive evolution of collaboration modes.

[0104] The multi-dimensional reward function system module evaluates the team performance of the agent cluster based on various dimensional indicators and feeds it back to the EvoRL optimization process.

[0105] The recommendation and early warning module uses the EvoRL mechanism to iteratively evolve the Prompt, policy network, and communication mechanism to generate educational recommendation and academic early warning schemes.

[0106] like Figure 2 As shown, this application also provides a device for personalized learning path adaptive recommendation and academic warning, comprising a processor 201, a memory 202, and a bus 203 in a computing device. The memory stores a computer program, which, when executed by the processor, implements the aforementioned personalized learning path recommendation and academic warning method. This device can be a server, a cloud computing platform, or an edge computing device, supporting use by educational institutions or online learning platforms.

[0107] Furthermore, as an executable solution, the device for personalized learning path adaptive recommendation and academic warning can be a computer unit, which can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0108] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.

[0109] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0110] In some embodiments, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the methods described above in the embodiments of the present invention.

[0111] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0112] In some embodiments, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0113] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for personalized learning path adaptive recommendation and academic early warning, characterized in that, The method comprises the following steps: initializing an agent cluster consisting of N LLM-driven agents under cold start conditions, wherein N is an integer greater than 1, each agent obtains an initial Prompt through a small pool sampling strategy, collects environmental state space information, including learner profile, historical learning data, knowledge point mastery and task progress characteristics; co-optimizing the Prompt, strategy network and communication mechanism of the agent through evolutionary reinforcement learning to realize self-emergence of the role and adaptive evolution of the collaboration mode; evaluating the team performance of the agent cluster according to various indicators and feeding back to the evolutionary reinforcement learning optimization process; outputting an education recommendation and academic warning scheme through the iterative evolution of the Prompt, strategy network and communication mechanism by the evolutionary reinforcement learning mechanism; In the step of co-optimizing the Prompt, strategy network and communication mechanism of the agent through evolutionary reinforcement learning to realize self-emergence of the role and adaptive evolution of the collaboration mode, it comprises: The agent cluster initiatively triggers an independent problem-solving stage, or passively triggers the independent problem-solving stage by the demand of the user or the learner; In the independent problem-solving stage, each agent generates a preliminary solution and corresponding reasoning process data based on the current Prompt and internal state; In the multi-round free interaction stage, the agents completely share all the preliminary solutions and reasoning process data of the agents through message passing without preset protocol, and dynamically adjust their own behavior and Prompt; In the self-evolution stage driven by the evolutionary reinforcement learning, a consensus solution is generated after the multi-round interaction, and the evolutionary reinforcement learning engine optimizes the Prompt, strategy network parameters and communication mechanism of each agent according to the quality of the consensus solution and the performance in the team collaboration process; The optimization of the Prompt, strategy network parameters and communication mechanism of each agent further comprises: A multi-dimensional reward function system is used to evaluate the team performance of the agent cluster, and the Prompt, strategy network and communication mechanism of each agent are dynamically adjusted, the multi-dimensional reward function system includes a main reward and multiple auxiliary rewards, the main reward is based on the effectiveness and accuracy of the education recommendation and academic warning scheme output by the agent cluster team, and the auxiliary rewards include team collaboration degree and agent division diversity. 2.The method of claim 1, wherein, The initial Prompt of each agent is obtained through a small pool sampling strategy, which comprises: Each agent randomly allocates an initial Prompt from a preset general Prompt pool. 3.The method of claim 1, wherein, In the self-evolution stage driven by the evolutionary reinforcement learning, a consensus solution is generated after the multi-round interaction, and the evolutionary reinforcement learning engine optimizes the Prompt, strategy network parameters and communication mechanism of each agent according to the quality of the consensus solution and the performance in the team collaboration process, the optimization of the Prompt of each agent comprises: According to the quality evaluation of the consensus solution, the fitness of the Prompt of each agent is evaluated, the high fitness Prompt is selected and mutated and crossed to generate the next generation Prompt. 4.The method of claim 1, wherein, The evolution reinforcement learning driven self-evolution stage generates a consensus solution after the end of the multiple rounds of interaction. The evolution reinforcement learning engine optimizes the Prompt, strategy network parameters, and communication mechanism of each agent according to the quality of the consensus solution and the performance in the team collaboration process. The optimization of the communication mechanism of each agent includes: Based on the team reward data, each agent dynamically adjusts its own communication mechanism. 5.The method of claim 1, wherein, The team collaboration degree is evaluated by the consensus convergence speed, and the division diversity is evaluated by the difference in the contribution types of the agents.

6. A system for personalized learning path adaptive recommendation and academic early warning, characterized in that, The method comprises: The agent initialization and management module initializes an agent cluster composed of N LLM driven agents under cold start conditions, where N is an integer greater than 1. Each agent obtains an initial Prompt through a small pool sampling strategy, collects environmental state space information, including learner profile, historical learning data, knowledge point mastery, and task progress characteristics. The multi-agent evolution reinforcement learning collaboration and self-evolution module optimizes the Prompt, strategy network, and communication mechanism of the agents through evolution reinforcement learning, realizes role self-emergence and adaptive evolution of the collaboration mode, and specifically comprises: The agent cluster actively triggers the independent problem solving stage, or passively triggers the independent problem solving stage by the needs of the user or the learner. In the independent problem solving stage, each agent generates a preliminary solution and corresponding reasoning process data based on the current Prompt and internal state. In the multiple rounds of free interaction stage, the agents completely share all the preliminary solutions and reasoning process data of the agents through message passing without pre-set protocols, and dynamically adjust their own behaviors and Prompts. In the evolution reinforcement learning driven self-evolution stage, a consensus solution is generated after the end of the multiple rounds of interaction. The evolution reinforcement learning engine optimizes the Prompt, strategy network parameters, and communication mechanism of each agent according to the quality of the consensus solution and the performance in the team collaboration process. The optimization of the Prompt, strategy network parameters, and communication mechanism of each agent further includes: A multi-dimensional reward function system is used to evaluate the team performance of the agent cluster, and the Prompt, strategy network, and communication mechanism of each agent are dynamically adjusted. The multi-dimensional reward function system includes a main reward and multiple auxiliary rewards. The main reward is based on the effectiveness and accuracy of the education recommendation and academic warning solution output by the agent cluster team. The auxiliary rewards include team collaboration degree and agent division diversity. The multi-dimensional reward function system module evaluates the team performance of the agent cluster according to the dimensional indicators and feeds back to the evolution reinforcement learning optimization process. The recommendation and warning module generates an education recommendation and academic warning solution by iteratively evolving the Prompt, strategy network, and communication mechanism through the evolution reinforcement learning mechanism.

7. An apparatus for personalized learning path adaptive recommendation and academic early warning, characterized in that, The system comprises a memory and a processor. The memory stores at least one program, and the processor executes the at least one program to implement the personalized learning path adaptive recommendation and academic warning method of any one of claims 1 to 5.

Images