Multi-agent based knowledge graph learning path planning method

Abstract

The application discloses a knowledge graph learning path planning method based on multiple agents, and belongs to the technical field of intelligent optimization. In order to solve the problem that planning constraint conflicts are prone to occur when multiple agents plan learning paths on a knowledge graph, and it is difficult to reach an agreed path in real time, the application encodes the knowledge graph through a graph transformer, generates a candidate learning path by using centralized training and decentralized execution of multiple agent reinforcement learning, constructs an incremental satisfiability judgment problem for conflict detection and outputs an unsatisfiable core and a conflict type, performs consistency arbitration update based on a segmented consensus variable driven by a conflict kernel by using an alternating direction multiplier method, and adjusts a penalty parameter according to an original residual error, a dual residual error and a conflict type. Meanwhile, the application learns and compresses transmission of the consensus variable update amount, and controls the number of rounds, so that the technical effects of outputting a consistent learning path satisfying constraints, reducing the number of iteration rounds and communication overhead, and improving real-time performance are achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent optimization, and more particularly to a knowledge graph learning path planning method based on multi-agent systems. Background Technology

[0002] With the development of online education and intelligent learning systems, knowledge graph-based learning path planning has gradually become an important technical approach for personalized teaching. Existing technologies typically construct knowledge graphs based on knowledge points and their prerequisites, and then recommend learning sequences on the graph by combining learning objectives, learner mastery, and the availability of learning resources. To improve path quality and adaptability, various technical solutions have emerged in research and industry practice, such as rule-based and heuristic search-based path generation methods, constraint planning methods based on integer programming and satisfiability solutions, and methods that utilize graph neural networks or graph transformers to obtain graph representations and then further employ reinforcement learning for sequence decision-making. For large-scale knowledge graphs and multi-objective constraint scenarios, some solutions also attempt to introduce multi-agent division of labor to generate candidate paths or use distributed optimization frameworks for consistency coordination.

[0003] However, existing technologies still have the following shortcomings:

[0004] 1. In scenarios where multiple agents divide the work to generate candidate learning paths, there is a lack of effective conflict detection and consistency arbitration mechanisms among the agents. This can easily lead to inconsistencies in constraints such as priority order, target coverage, resource matching, and budget, making it difficult to output globally feasible and consistent learning paths.

[0005] 2. Existing constraint verification and conflict handling often involve full reconstruction and re-solving, lacking incremental updates and conflict location capabilities. This makes it difficult to quickly identify key constraint combinations that cause unsatisfaction, resulting in high iteration costs and low repair efficiency.

[0006] 3. In path planning for real-time interaction, distributed coordination is usually accompanied by high communication overhead and iteration rounds. The lack of effective control mechanisms for message transmission and stopping strategies makes it difficult for the system to converge stably and output results in a timely manner under communication budget and latency constraints.

[0007] Therefore, a knowledge graph learning path planning method is needed to address the shortcomings of existing technologies. Summary of the Invention

[0008] One objective of this invention is to propose a multi-agent knowledge graph learning path planning method. Addressing the problems of existing multi-agent division-of-labor planning methods, which easily generate constraints such as pre-learning order, target coverage, resource matching, and learning budget conflicts, and suffer from low conflict location and repair efficiency and high distributed coordination communication overhead, making it difficult to meet real-time requirements, the following technical solution is proposed: Obtain the knowledge graph, learning objective, learner state, and planning constraint set; encode the knowledge graph using a graph transformer to obtain representation information; based on the representation information, call the multi-agent policy model to generate a candidate learning path set according to the preset division of labor; construct the candidate learning path set as a satisfiability judgment problem and use incremental solution for conflict detection, outputting the unsatisfiable core and conflict type when not satisfied; segment the candidate learning path and establish segmented consensus variables; perform consensus arbitration updates on the segment to be updated based on the conflict core-driven alternating direction multiplier method, and adaptively adjust the penalty parameters according to the original residual, dual residual, and conflict type; perform learning-based compressed transmission of the consensus variable update amount and combine residual and communication budget for round control, outputting a consistent learning path. This invention has the technical advantages of quickly locating conflicts, reducing invalid updates and iteration rounds, reducing communication volume and improving real-time performance, and can output a consistent learning path that satisfies constraints.

[0009] This invention provides a multi-agent knowledge graph learning path planning method, including:

[0010] S1. Obtain the knowledge graph, learning objectives, learner states, and planning constraint set. The knowledge graph includes knowledge point nodes, prerequisite relation edges, and association edges between learning resources and knowledge points. Encode the knowledge graph using a graph coding model to obtain knowledge graph representation information. S2. Based on the knowledge graph representation information and combined with the learning objectives and learner states, invoke a multi-agent policy model to enable multiple agents to generate candidate learning paths on the knowledge graph according to a preset division of labor. Collect these candidate learning paths to obtain a set of candidate learning paths. Each candidate learning path includes a sequence of knowledge points arranged in learning order and a corresponding sequence of learning resources. S3. For the candidate learning path set, construct a satisfiability judgment problem based on the planning constraint set, and invoke the constraint solver to perform satisfiability judgment, outputting conflict detection results. When the conditions are not met, the conflict detection result includes the unsatisfiable core and the determined conflict type; S4, the candidate learning path set is segmented and segmented consensus variables are established for each learning path segment. Based on the conflict detection result, the learning path segment to be updated is determined. The segmented consensus variables corresponding to the learning path segment to be updated are iteratively updated using the alternating direction multiplier method to generate a consistent learning path. The original residual and dual residual are calculated, and the penalty parameter is adaptively adjusted based on the original residual, dual residual, and conflict type; S5, the update amount of the segmented consensus variables is compressed, encoded, and transmitted. The update amount is decoded and restored at the receiving end and used for subsequent iterative updates using the alternating direction multiplier method. Round control is performed based on the original residual and dual residual. When the stopping condition is met, the consistent learning path is output as the learning path planning result.

[0011] Optionally, S1 includes:

[0012] After obtaining the knowledge graph, a set of target knowledge point nodes is determined based on the learning objective, and the knowledge graph to be encoded is extracted along the prior relation edge within a preset number of hops, starting from the set of target knowledge point nodes.

[0013] For each knowledge point node in the knowledge graph to be encoded, an initial node vector is generated. The generation of the initial node vector includes concatenating the identifier embedding vector corresponding to the knowledge point node identifier, the type embedding vector corresponding to the knowledge point node type, and the mastery feature vector extracted from the learner's state.

[0014] For each relation in the knowledge graph to be encoded, an initial relation vector is generated. The generation of the initial relation vector includes combining the type embedding vector corresponding to the relation type with the relation direction feature vector.

[0015] Calculate the topological hierarchy of each knowledge point node on the directed acyclic graph formed by the pre-modified relation edges and generate the topological position code;

[0016] An attention mask is generated based on the reachability of the pre-modified edges;

[0017] The graph transformer model is used to encode the knowledge graph to be encoded under the constraints of topological position encoding and attention masking, and outputs the node representation vector corresponding to each knowledge point node and the relation representation vector corresponding to each relation as the knowledge graph representation information.

[0018] Optionally, S2 includes:

[0019] According to the preset division of labor, each agent is assigned a subset of the target knowledge point node set or a subgraph range of the knowledge graph, and a candidate knowledge point node set corresponding to the subset or the subgraph range is determined based on the knowledge graph representation information.

[0020] For each agent, target knowledge point nodes are determined based on the learning objectives, and mastered knowledge point nodes and knowledge point nodes to be learned are determined based on the learner's state.

[0021] During the candidate learning path generation process, an agent state is constructed for each agent at each decision moment. The agent state includes the node representation vector corresponding to the current knowledge point node, the node representation vector corresponding to the target knowledge point node, and the mastery feature determined by the learner state.

[0022] The multi-agent policy model is invoked, and action results are output according to the state of the agent. The action results include the next knowledge point node selection result and the learning resource selection result corresponding to the next knowledge point node.

[0023] Based on the action results, the knowledge point sequence and the learning resource sequence corresponding to the knowledge point sequence are gradually expanded, and a candidate learning path is output when a preset termination condition is met. The preset termination condition includes reaching the target knowledge point node or reaching a preset path length.

[0024] The candidate learning paths output by multiple agents are aggregated to obtain the candidate learning path set;

[0025] Furthermore, the preset division of labor includes dividing the knowledge graph into subgraphs based on graph partitioning, with the goal of minimizing the number of prerequisite relation edges across subgraphs. In subsequent iterations, when the conflict detection result is unsatisfactory, the subgraph range of at least one agent is adjusted according to the identifier of the prerequisite relation edge across subgraphs appearing in the unsatisfactory core, so as to reduce the probability of cross-agent conflict.

[0026] Optionally, S3 includes:

[0027] The candidate learning path set is converted into constraint variables and constraint expressions that can be processed by the constraint solver. For each candidate learning path, an order variable is set to represent the nodes of each knowledge point in the knowledge point sequence, and a selection variable is set to represent the learning resources in the learning resource sequence.

[0028] The constraint expression is generated based on the planning constraint set, including: the prior order constraint defined by the prior relation edge, the resource matching constraint defined by the association edge between learning resources and knowledge points, the target coverage constraint defined by the learning objective, and the budget constraint defined by the preset learning budget.

[0029] The candidate learning path set of the current iteration round is compared with the candidate learning path set of the previous iteration round. Based on the difference results, constraint expressions to be added, deleted or replaced incrementally are generated, and the corresponding constraints are updated in the solution state, so that the constraint solver performs satisfiability determination while retaining the solution information of the previous iteration round.

[0030] When the satisfiability determination result is satisfied, output the conflict detection result representing satisfaction;

[0031] When the satisfiability determination result is not satisfied, the unsatisfiable core is output, and the knowledge point node identifier and the prior relation edge identifier involved in the conflict are extracted from the unsatisfiable core. At the same time, the conflict type is determined according to the constraint category to which each constraint expression in the unsatisfiable core belongs, and the conflict detection result containing the unsatisfiable core, the knowledge point node identifier, the prior relation edge identifier and the conflict type is output.

[0032] Furthermore, when the determination result is unsatisfied, a candidate repair set is generated based on the unsatisfied core. The candidate repair set includes swapping the knowledge point order variables in the candidate learning path, replacing the learning resource selection variables, or adjusting the upper bound of the budget constraint. At least one item in the candidate repair set is added as an additional constraint to the constraint expression of subsequent iteration rounds to shorten the iteration rounds to reach the satisfied state.

[0033] Optionally, S4 includes:

[0034] Each candidate learning path in the candidate learning path set is segmented according to the preset segmentation rules. The preset segmentation rules include segmenting according to the topological level corresponding to the pre-repaired relation edges or segmenting according to the preset path length, resulting in multiple learning path segments.

[0035] For each learning path segment, segmented consensus variables are established, including sequence consensus variables and resource consensus variables. Local variables corresponding to the segmented consensus variables are also established for each candidate learning path in the candidate learning path set.

[0036] In the first iteration, initial values ​​are assigned to the segmented consensus variables of each learning path segment based on the candidate learning path set, and the dual variables corresponding to the segmented consensus variables are initialized. Learning path segments to be updated are determined based on conflict detection results. When the conflict detection results contain a satisfying identifier, all learning path segments are identified as learning path segments to be updated. When the conflict detection results contain a dissatisfaction identifier, learning path segments containing the knowledge point node identifier or the prior relation edge identifier are identified as learning path segments to be updated based on the knowledge point node identifier and the prior relation edge identifier in the unsatisfiable core, and the remaining learning path segments are identified as non-updating learning path segments. In each iteration, the alternating direction multiplier method is used to sequentially perform local variable updates, segmented consensus variable updates, and dual variable updates. The segmented consensus variables and their corresponding dual variables from the previous iteration are used as the initial values ​​for this iteration. For the learning path segments to be updated, the local variable updates, segmented consensus variable updates, and dual variable updates are performed, while for the non-updating learning path segments, the segmented consensus variables remain unchanged.

[0037] After completing the update of the dual variable, the original residual and the dual residual are calculated, and the adjustment coefficient of the penalty parameter is determined based on the original residual, the dual residual and the conflict type, so as to adaptively adjust the penalty parameter.

[0038] The segmented consensus variables corresponding to each learning path segment are combined to generate a consistent learning path, and the segmented consensus variable update amount, the original residual, and the dual residual of this iteration are output.

[0039] Optionally, S5 includes:

[0040] For the segmented consensus variable update amount, generate an update amount message that corresponds one-to-one with each learning path segment, and attach a learning path segment identifier and an iteration round identifier to each update amount message;

[0041] The learning-based compression model is used to encode each update message to obtain a compressed message. The encoding includes quantizing the update amount of the segmented consensus variable and outputting a quantization index.

[0042] The compressed message is transmitted to the receiving end;

[0043] The receiving end decodes the compressed message to recover the segmented consensus variable update amount, and writes the recovered segmented consensus variable update amount into the corresponding segmented consensus variable according to the learning path segment identifier, so as to serve as the segmented consensus variable update input for the next iteration round;

[0044] After each iteration round, round control is performed based on the original residual and the dual residual. When the original residual is less than or equal to the first threshold and the dual residual is less than or equal to the second threshold, a convergence stop instruction is generated to stop the iterative update of the alternating direction multiplier method. When the cumulative amount of data of the compressed message that has been transmitted reaches the preset communication budget, a budget stop instruction is generated to stop the iterative update of the alternating direction multiplier method.

[0045] In response to the convergence stop command or the budget stop command, output a consistent learning path as the final learning path planning result;

[0046] Furthermore, the learning-based compression model also includes sparsification processing for encoding each update message. The sparsification processing involves selecting the top few elements with the largest amplitude from the segmented consensus variable update and quantizing only these elements. An error feedback mechanism is introduced at the receiving end to accumulate the residuals corresponding to the untransmitted parts into the segmented consensus variable update of the next iteration, so as to improve the consistency convergence accuracy under the same communication budget.

[0047] Optionally, the acquisition of the multi-agent policy model includes the following training process: constructing an interactive environment based on the knowledge graph, using the learning objective, learner state, and knowledge graph representation information as state input, using candidate learning paths as action output, and constructing a reward function based on the planning constraint set, wherein the reward function includes learning objective coverage reward, learning cost penalty, and constraint conflict penalty;

[0048] During training, the conflict type and size of the unsatisfiable cores output by the constraint solver are used as training feedback to update the model parameters of the multi-agent policy model, so as to obtain the multi-agent policy model.

[0049] The beneficial effects of this invention are:

[0050] 1. By constructing the candidate learning path set as a satisfiability determination problem and using incremental solution to output the unsatisfiable core and conflict type, it is possible to quickly locate the key constraint combination and related knowledge points that lead to unsatisfiability and to pre-trim the edges, thereby reducing the computational overhead caused by full reconstruction and repeated solution, improving the directionality of conflict detection and repair, and thus shortening the iteration rounds to reach a feasible solution.

[0051] 2. By segmenting candidate learning paths and establishing segmented consensus variables, the unsatisfiable core driver performs alternating direction multiplier consensus arbitration only on the path segments to be updated. At the same time, the consensus variables from the previous round and the dual variables are used for hot start. The penalty parameters are adaptively adjusted according to the original residual, the dual residual and the conflict type. This can reduce the range of cross-agent conflict propagation, improve the consensus convergence speed and stability, and output a globally consistent learning path that satisfies the constraints.

[0052] 3. By learning-based compressed encoding and transmission of segmented consensus variable updates, and combining the original residual, dual residual, and communication budget for round control, the amount of transmitted data and the number of iterative communication rounds can be reduced under limited communication and latency constraints, thereby improving the real-time performance and deployability of multi-agent collaborative planning. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0054] Figure 1 This is a flowchart of the knowledge graph learning path planning method based on multi-agent systems proposed in this invention;

[0055] Figure 2 This is a flowchart of step S4, consistency arbitration update, of the present invention.

[0056] Figure 3 This is a flowchart of the conflict detection process in step S3 of the present invention. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0058] refer to Figure 1-3 A multi-agent knowledge graph learning path planning method includes:

[0059] S1. Obtain the knowledge graph, learning objectives, learner states, and planning constraint set. The knowledge graph includes knowledge point nodes, prerequisite relation edges, and association edges between learning resources and knowledge points. Encode the knowledge graph using a graph coding model to obtain knowledge graph representation information. S2. Based on the knowledge graph representation information and combined with the learning objectives and learner states, invoke a multi-agent policy model to enable multiple agents to generate candidate learning paths on the knowledge graph according to a preset division of labor. Collect these candidate learning paths to obtain a set of candidate learning paths. Each candidate learning path includes a sequence of knowledge points arranged in learning order and a corresponding sequence of learning resources. S3. For the candidate learning path set, construct a satisfiability judgment problem based on the planning constraint set, and invoke the constraint solver to perform satisfiability judgment, outputting conflict detection results. When the conditions are not met, the conflict detection result includes the unsatisfiable core and the determined conflict type; S4, the candidate learning path set is segmented and segmented consensus variables are established for each learning path segment. Based on the conflict detection result, the learning path segment to be updated is determined. The segmented consensus variables corresponding to the learning path segment to be updated are iteratively updated using the alternating direction multiplier method to generate a consistent learning path. The original residual and dual residual are calculated, and the penalty parameter is adaptively adjusted based on the original residual, dual residual, and conflict type; S5, the update amount of the segmented consensus variables is compressed, encoded, and transmitted. The update amount is decoded and restored at the receiving end and used for subsequent iterative updates using the alternating direction multiplier method. Round control is performed based on the original residual and dual residual. When the stopping condition is met, the consistent learning path is output as the learning path planning result.

[0060] In this specific embodiment, S1 includes:

[0061] The processor reads the knowledge graph data structure from the storage medium. The knowledge graph is stored in a triplet table and contains knowledge point nodes, learning resource nodes, prerequisite relation edges, and association edges between learning resources and knowledge points. The processor also receives the learning objective and the learner status. The learner status includes the mastery value of each knowledge point node, the most recent learning interval, and the cumulative number of practice sessions.

[0062] The processor parses the learning objective to obtain a set of target knowledge point nodes, and then performs a breadth-first traversal along the prior relation edges starting from this set to extract its prior dependency subgraph. The preset hop count range is set to... During traversal, for each visited knowledge point node, record the shortest hop count to any target knowledge point node, and only retain nodes with a hop count not greater than [a certain value]. The knowledge point nodes and their prerequisite relationship edges are included, and the learning resource nodes and their corresponding relationship edges that are associated with the retained knowledge point nodes are also included in the knowledge graph to be encoded.

[0063] Subsequently, an initial node vector is constructed for each node in the knowledge graph to be encoded. For knowledge point nodes, the identifier embedding vector corresponding to the node identifier, the type embedding vector corresponding to the node type, and the mastery feature vector extracted from the learner's state are concatenated to obtain the following:

[0064] ;

[0065] in This represents a knowledge point node in the knowledge graph to be encoded. Representing knowledge point nodes The initial vector of the node, This represents the identifier embedding vector obtained by looking up the node identifier in a table, and its dimension is fixed. This represents a type embedding vector obtained by looking up the node type in a table, and its dimension is fixed. This represents a mastery feature vector generated from the learner's state, with a fixed dimension of 48. The symbol [;] indicates a vector concatenation operation. The total dimension is fixed as The size of the identifier embedding lookup matrix is ​​fixed at 1. The integer encoding of the node identifier in the knowledge graph is used as the row index, and the size of the type embedding lookup matrix is ​​fixed. And using node type enumeration encoding as row index, we can grasp the degree feature vector. It is obtained by embedding and concatenating three discrete features, which are the mastery values ​​in closed intervals. Bucket numbering for 16 equally spaced bins, and the most recent learning interval. Day, 1 day, 2 days, 3 days, 4 days, 5 days, 6 days sky Map and further subdivide into 16 bucket numbers, and the cumulative number of practice sessions are as follows: The mapping is further subdivided into 16 bucket numbers, with the three bucket numbers fixed by three dimensions. The embedding table is used to look up three 16-dimensional vectors and concatenate them into a 48-dimensional vector;

[0066] For learning resource nodes, the same splicing structure is used to generate the node initial vector and the mastery feature vector is set to a vector with all zeros to ensure consistent dimensionality.

[0067] Simultaneously, an initial relation vector is generated for each relation in the knowledge graph to be encoded. The relation types are limited to two categories: prerequisite relations and resource association relations, and each is assigned a fixed size. The relation type embedding table yields a 32-dimensional type embedding vector. Relationship direction features are represented by two-dimensional one-hot vectors indicating forward or reverse direction. These vectors are then concatenated with the type embedding vectors and mapped to 32 dimensions through a fully connected layer to obtain the initial relation vector. The size of the fully connected weight matrix is ​​fixed. And the bias vector dimension is fixed at 32;

[0068] On a directed acyclic graph consisting only of prior relation edges, the processor performs topological sorting and calculates the topological level for each knowledge node in the knowledge graph to be encoded. The topological level is defined as the length of the longest path from any knowledge node with an in-degree of zero to that knowledge node, and the topological position code is obtained by looking up a table using the level integer as an index. The size of the topological position code table is fixed. And when the number of levels exceeds 63, it is truncated to 63;

[0069] The processor further constructs an attention mask based on the reachability relations of the prior repaired relation edges. The attention mask is indexed by node pairs and specifies that the attention mask is defined if and only if the node... Nodes reachable under the transitive closure of pre-fixed relation edges. or Time allows from attention calculation polymerization If the information is not available, the corresponding attention score will be set to a fixed, minimal constant. To achieve shielding;

[0070] After completing the above input construction, the processor calls the graph transformer model to represent and encode the knowledge graph to be encoded. The graph transformer model is composed of... The structure consists of stacked layers, each layer containing a masked multi-head self-attention sublayer and a feedforward network sublayer, with a fixed number of multi-heads. Furthermore, the dimension of each head is fixed at 16, the hidden dimension of the model is fixed at 128, the feedforward network adopts a two-layer fully connected structure with the middle dimension fixed at 256 and the activation function fixed at ReLU. The node input vector is obtained by adding the node initial vector and the corresponding topological position code element by element. During the self-attention calculation, the attention mask is applied to the attention score and the edge bias obtained by linear mapping of the relation initial vector is added to the attention score of the corresponding node pair to inject relation semantics.

[0071] The graph converter model is complete. After layer computation, the node representation vector corresponding to each node is output while maintaining a dimension of 128. For each relation, the initial relation vector is concatenated with the node representation vectors of the nodes at both ends of the relation and then input into a fully connected network with an output dimension of 128 to obtain the relation representation vector, thus forming the knowledge graph representation information.

[0072] In this specific embodiment, S2 includes:

[0073] After obtaining the node representation vectors of each node in the knowledge graph to be encoded, the processor initiates the candidate learning path generation process for multi-agent division of labor and fixes the number of agents to a certain value. Each agent corresponds to a policy execution process and shares the same set of policy network parameters to meet the distributed execution requirements under the centralized training and distributed execution framework.

[0074] The processor first performs subgraph partitioning on the directed graph of prior relations formed by knowledge point nodes in the knowledge graph to be encoded, to form a preset division of labor. The subgraph partitioning aims to minimize the number of prior relation edges across subgraphs and outputs the mapping from knowledge point nodes to agent numbers. ,in Representing knowledge point nodes The assigned agent ID, the subgraph partitioning implementation adopts a three-stage deterministic process. The first stage converts the directed graph with prior relations into an undirected graph by ignoring direction and resets the weight of each edge to 1. The second stage performs multi-level graph coarsening and reduces the number of nodes to a fraction of the original graph. As a termination condition, the third stage constructs an initial quartile on the coarsened graph and uses a greedy exchange strategy to refine the edge cut number during the layer-by-layer reflection, and takes the fact that the edge cut number no longer decreases after two consecutive iterations as the termination condition.

[0075] The processor then determines the set of target knowledge point nodes based on the learning objective, and maps them accordingly. The target knowledge point node set is divided into four target subsets and assigned to four agents respectively. At the same time, based on the learner status, the knowledge point nodes in the knowledge graph to be encoded are divided into mastered knowledge point nodes and knowledge point nodes to be learned. The criteria for mastered knowledge point nodes are that the mastery value of the knowledge point node is not less than 0.8 and the most recent learning interval is not greater than 7 days.

[0076] Before each round of candidate learning path generation begins, if the previous round's unsatisfiable core contains a prior relation edge identifier across the subgraph, then the processor performs prior relation edge processing on each cross-subgraph path. The division of labor was adjusted and the knowledge points and nodes of the preliminary stage were reviewed. Migration to subsequent knowledge point nodes The intelligent agent in question, i.e., Meanwhile, during the generation process, migration lock flags are set for migrated knowledge point nodes to ensure that they are not migrated again in the same round, thereby reducing the probability of cross-agent conflict and maintaining the determinism of division of labor adjustment.

[0077] After the division of labor is completed, the processor is for each intelligent agent. The candidate knowledge point node set is determined to be all nodes that satisfy the following conditions: The knowledge point nodes that belong to the knowledge point nodes to be learned, and the learning resource nodes that have a learning resource and knowledge point association edge with any knowledge point node in the candidate knowledge point node set constitute the candidate learning resource set of the agent.

[0078] During the candidate learning path generation process, the processor performs a process for each agent. At every decision moment Construct an agent state vector and input it into a multi-agent policy model to output action results. The agent state vector is:

[0079] ;

[0080] in Represents intelligent agents At the moment of decision State vector, symbol This represents the vector concatenation operation. Represents intelligent agents At the moment of decision The corresponding current knowledge point node, This indicates that the assignment is made to the intelligent agent. The target knowledge point nodes are determined by the principle of maximizing the topological hierarchy within their target subset. Indicates the current knowledge point node The node representation vector with dimension 1 Represents the target knowledge point node The node representation vector with dimension 1 This indicates that the learner's current knowledge point node is represented by their current state. The generated mastery feature vector has a dimension of 48, thus The dimension is fixed at 304;

[0081] The multi-agent policy model comprises four decentralized agent policy subnetworks and a centrally trained value evaluation subnetwork. Each agent policy subnetwork uses a two-layer fully connected structure with a fixed input dimension of 304, a first hidden layer dimension of 256, a second hidden layer dimension of 256, and a ReLU activation function. Two output branches, a knowledge point selection head and a resource selection head, are set after the second hidden layer. The knowledge point selection head outputs a selection score for the agent's set of candidate knowledge point nodes. During score calculation, action masks are applied to candidate knowledge point nodes that do not meet the prerequisite reachability requirements and have not yet satisfied all their direct prerequisite knowledge points, thus fixing their selection scores. After the knowledge point selection is completed, the resource selection head outputs the selection score of the candidate learning resources connected to the selected knowledge point node through the association edge and applies the same action mask to the non-associative learning resources. The processor performs Softmax normalization on the two output branches respectively and uses the maximum probability deterministic selection to obtain the action result to form the next knowledge point node selection result and the corresponding learning resource selection result.

[0082] The processor progressively expands based on the action results to obtain a knowledge point sequence for each agent and a learning resource sequence corresponding to each knowledge point sequence. When a preset termination condition is met, the processor outputs a candidate learning path for that agent. The preset termination condition is fixed as reaching the target knowledge point node. Or the path length reaches Furthermore, during the expansion process, access identifiers are set for knowledge point nodes that have already entered the knowledge point sequence of the agent to prevent repeated entry;

[0083] After all agents have output candidate learning paths, the processor will combine the four candidate learning paths to form a candidate learning path set.

[0084] In this specific embodiment, S3 includes:

[0085] The processor receives a set of candidate learning paths and maintains an incrementally updatable solution state in memory. The solution state consists of the context object of the constraint solver, the constraint activation dictionary, and a snapshot of the candidate learning path set from the previous iteration. The constraint solver adopts an SMT solver that supports incremental assertions and unsatisfiable core extraction and enables the unsatisfiable core function based on the assumption literal.

[0086] The processor converts the set of candidate learning paths into constraint variables and constraint expressions that can be processed by the constraint solver. For each candidate learning path, a knowledge point order variable and a learning resource selection variable are set. The knowledge point order variable uses "path identifier and sequence number" as the key and takes the integer code of the knowledge point node identifier as the value. The learning resource selection variable uses "path identifier and sequence number" as the key and takes the integer code of the learning resource identifier as the value. At the same time, for each candidate learning path, the knowledge point sequence and the learning resource sequence are written into the equality constraint to fix the candidate solution for this round.

[0087] The processor generates constraint expressions based on the planning constraint set and assigns a unique constraint identifier and activation literal to each constraint expression. These constraint expressions include prerequisite order constraints, resource matching constraints, target coverage constraints, and budget constraints. The prerequisite order constraint is constructed by forcing any pair of knowledge point nodes with a prerequisite relationship in the candidate learning path to satisfy the "precedence first" relation in their indices within the path. The resource matching constraint is constructed by forcing the learning resource selection variable at each index in the candidate learning path to have an association edge between it and the knowledge point order variable at the same index in the knowledge graph. The target coverage constraint is constructed by forcing the target knowledge point node corresponding to the learning target to appear at least once in the knowledge point order variable set of the candidate learning path. The budget constraint is constructed by summing the resource costs corresponding to the learning resource selection variables of the candidate learning path and limiting them to not exceeding a preset learning budget, and is written as:

[0088] ;

[0089] Among the symbols This represents the summation operation. This represents the summation index, rounded to the nearest integer. The path identifier is The length of the candidate learning path, and taken as a positive integer. The path identifier represents the candidate learning path. The path identifier is And the serial number is The value of the learning resource selection variable is assigned to an integer code that represents a learning resource identifier. This represents the resource cost function, which is derived from the learning duration and difficulty coefficient in the learning resource metadata through a fixed linear conversion. This indicates the upper limit of the preset learning budget, which is in minutes and fixed at 600.

[0090] To achieve incremental satisfiability determination, the processor compares the candidate learning path set of the current iteration with the candidate learning path set of the previous iteration and generates a constraint update operation set. The difference comparison uses "path identifier and sequence number" to locate changes in the knowledge point order variable and the learning resource selection variable, and generates constraint expressions to be added, deleted, or replaced incrementally. The constraint expressions to be deleted are not directly removed from the solver, but their corresponding activation literals are set to inactive in the hypothesis set of this round to achieve logical deletion. The constraint expressions to be replaced simultaneously perform logical deletion of old constraints and incremental addition of new constraints. The newly added constraint expressions are asserted to the solution state in the form of "activation literal implying constraint expression" and written into the constraint activation dictionary to establish a mapping of "constraint identifier - activation literal - constraint category - related knowledge point node identifier and prior relation edge identifier".

[0091] The processor calls the constraint solver to perform a satisfiability determination under the current round's hypothesis set. When the determination result is satisfied, it outputs the conflict detection result representing satisfaction and solidifies the snapshot of the candidate learning path set in the solution state for the next round's difference comparison. When the determination result is not satisfied, the constraint solver returns an unsatisfiable core, which consists of a set of activation literals. The processor decomposes the activation literals into a set of constraint identifiers corresponding to the unsatisfiable core based on the constraint activation dictionary and extracts the knowledge point node identifiers and prior relation edge identifiers involved in the conflict. At the same time, it determines the conflict type according to the constraint category corresponding to each constraint identifier in the unsatisfiable core and outputs a unique conflict type according to the fixed priority of prior order constraints, resource matching constraints, target coverage constraints, and budget constraints.

[0092] After outputting the unsatisfiable cores and conflict types, the processor generates a candidate repair set based on the unsatisfiable cores and adds at least one item from the candidate repair set as an additional constraint to the constraint expression of the next iteration round. Specifically, when the conflict type is a priority order conflict, the candidate repair set includes swapping the order variables of two knowledge points within the same path involved in the unsatisfiable core, and using the swapped equality constraint as an additional constraint to force the next round's candidate learning path to satisfy the priority order. When the conflict type is a resource matching conflict, the candidate repair set includes replacing the learning resource selection variables involved in the unsatisfiable core, using the replaced equality constraint as an additional constraint, and the replaced resources are taken from the knowledge graph that have an associated edge with the corresponding knowledge point node and whose resource cost function... The learning resource with the smallest value, when the conflict type is budget conflict, the candidate repair set includes adjusting the upper bound of the budget and the adjusted upper bound of the budget. The updated budget constraint is written as an additional constraint to the solution state. When the conflict type is target coverage conflict, the candidate repair set includes the missing target knowledge point node in the unsatisfiable core being forcibly added to the end of the same path and the equality constraint of the corresponding knowledge point order variable as an additional constraint. The processor selects one constraint expression from the candidate repair set to add to the next iteration round according to the fixed rule of "minimum number of changed variables" to shorten the iteration round to reach the satisfied state and outputs the conflict detection result including the unsatisfiable core, the knowledge point node identifier involved in the conflict, the prior repair relation edge identifier involved in the conflict, and the conflict type.

[0093] In this specific embodiment, S4 includes:

[0094] The processor receives the conflict detection results and the set of candidate learning paths and performs a consensus arbitration update.

[0095] The processor segments each candidate learning path in the candidate learning path set according to its path length and fixes the segment length as 1. The 1st to 5th knowledge points and their corresponding learning resources are divided into the 1st learning path segment, the 6th to 10th knowledge points and their corresponding learning resources are divided into the 2nd learning path segment, the 11th to 15th knowledge points and their corresponding learning resources are divided into the 3rd learning path segment, and the 16th to 20th knowledge points and their corresponding learning resources are divided into the 4th learning path segment. When the actual length of the candidate learning path is less than 20, only the learning path segment corresponding to the generated position is retained.

[0096] For each learning path segment Establish segmented consensus variables and fix them to include two parts: sequence consensus variables and resource consensus variables. The sequence consensus variables have a length of [missing information]. An integer vector stores the integer encoding of the knowledge point node identifier at each position in this segment. Resource consensus variables are in units of length [missing information]. An integer vector stores the integer encoding of the learning resource identifier at each position in the segment, and also represents the integer encoding of each candidate learning path in the candidate learning path set. In each learning path segment Establish local variables that are isomorphic to the segmented consensus variables and establish isomorphic dual variables;

[0097] In the first iteration round, the processor directly copies the local variables of the first candidate learning path in each learning path segment as the initial values ​​of the segmented consensus variables of the corresponding learning path segment and initializes all dual variables to zero vectors to satisfy reproducible initialization;

[0098] The processor determines the learning path segments to be updated based on the conflict detection results. When the conflict detection results contain a satisfying flag, all learning path segments are identified as learning path segments to be updated. When the conflict detection results contain a non-satisfied flag, the processor locates the path position of each knowledge point node in the candidate learning path set for each unsatisfied core and calculates the learning path segment identifier accordingly. For each prerequisite relation edge in the unsatisfied core, the processor locates the positions of the knowledge point nodes at both ends of the edge in the path and adds the learning path segment identifier containing the position of any endpoint to the set of learning path segments to be updated, thus obtaining the learning path segments to be updated and identifying the remaining learning path segments as non-learning path segments to be updated.

[0099] The processor performs iterative updates of the learning path segments to be updated using the alternating direction multiplier method, while keeping the segmented consensus variables and their dual variables unchanged for non-updating learning path segments. The iteration round index is denoted as... And in the first round In each iteration, local variable updates are performed first, and a discrete proximal solution based on enumeration is used to determine the feasible set of local variables. Generated by a fixed editing operator and containing only two types of edits, the first type of edit being in the learning path segment The first type of editing involves exchanging the knowledge point node identifiers at two locations and simultaneously exchanging the corresponding learning resource identifiers. The second type of editing involves editing the learning path segments. At any given location, the learning resource identifier is replaced with the learning resource identifier in the knowledge graph that has a learning resource-knowledge point association edge with that knowledge point node and has the minimum resource cost. The processor then... For each candidate segment, the objective function value is calculated and the minimum value is taken as the local variable update result for that segment. The local variable update is implemented in the following form:

[0100] ;

[0101] in The path identifier is And the learning path segment is identified as In the The local variables obtained from the round of iterations are concatenated with the sequential local variables and the resource local variables in a fixed order to form a one-dimensional vector. This represents the vector used to find the minimum candidate segment configuration and is then compared with... isomorphism The path identifier is And the learning path segment is identified as The locally feasible set, Represent the local cost function and fix it to the configuration of candidate segments. The weighted sum of the number of changes in knowledge point node identifiers and the number of changes in learning resource identifiers relative to the original candidate learning path segment of this path, with the weights fixed as knowledge point change weight 1 and resource change weight 1. Indicates the first The penalty parameter for each iteration is fixed at its initial value. The learning path segment is identified as In the Segmented consensus variables in rounds of iteration and according to The same concatenation order forms a one-dimensional vector. The path identifier is And the learning path segment is identified as In the The dual variables of the round iteration and their union Same dimension, The L2 norm is used to measure the difference between segment configuration vectors;

[0102] After completing the local variable updates for all paths on the learning path segments to be updated, the processor performs segmented consensus variable updates and updates each learning path segment to be updated. Construct a temporary mean vector for each path The element-wise average is then used to perform deterministic projection on the temporary mean vector to obtain new segmented consensus variables. The projection of the sequential consensus variables adopts position-level majority voting, and selects the knowledge point node identifier that minimizes the sum of the local cost function when the number of votes is the same. The projection of the resource consensus variables adopts the learning resource identifier that minimizes the resource cost and satisfies the association edge between the learning resource and the knowledge point under the constraint of the knowledge point node identifier determined by majority voting. After projection, deduplication correction is performed within the segment to ensure that the knowledge point node identifiers are not repeated in the same learning path segment. When a repetition occurs, the repeated position is replaced with the knowledge point node identifier that does not appear in the candidate knowledge point node set of the agent and whose direct prerequisite knowledge point has been satisfied in the previous position of the current segment in the order from back to front.

[0103] The processor then performs dual variable updates and updates for each path. With each learning path segment to be updated The dual variable is updated according to the rule of "the difference between the dual variable of the previous round and the local variable of the current round and the segmented consensus variable of the current round" to complete one round of iteration of the alternating direction multiplier method, and the segmented consensus variable of the previous round and the corresponding dual variable are used as the initial values ​​of the next round of iteration to achieve hot start.

[0104] After updating the dual variables, the processor calculates the original and dual residuals. The original residual is calculated based on the maximum L2 norm of the difference between local variables and segmented consensus variables on all learning path segments to be updated. The dual residual is calculated based on the maximum L2 norm of the difference between the current and previous round segmented consensus variables on all learning path segments to be updated, multiplied by the penalty parameter. The penalty parameter is adaptively adjusted based on the original, dual, and conflict types. For conflict types of prior order conflict, the penalty parameter adjustment coefficient is fixed at 2.0; for resource matching conflict, it is fixed at 1.5; for target coverage conflict, it is fixed at 1.2; and for budget conflict, it is fixed at 1.3. Further adjustments are made using residual balancing rules. When the original residual is greater than 10 times the dual residual, the penalty parameter is multiplied by 2; when the dual residual is greater than 10 times the original residual, the penalty parameter is divided by 2. This yields the penalty parameter for the next iteration. ;

[0105] The processor concatenates the segmented consensus variables corresponding to each learning path segment in segment order to generate a consensus learning path and outputs the segmented consensus variable update amount of this iteration relative to the previous iteration, the original residual, and the dual residual.

[0106] In this specific embodiment, S5 includes:

[0107] The processor organizes the segmented consensus variable update amounts of each learning path segment into data to be transmitted and performs compressed transmission and round control. The segmented consensus variable update amounts are then processed in the [number]th round. In the round of iteration, it is denoted as ,in Indicates the iteration round index of the alternating direction multiplier method. Indicates the learning path segment identifier. It is a one-dimensional vector, obtained by concatenating the sequential consensus variable update values ​​and resource consensus variable update values ​​in a fixed order, and the vector length is fixed. and the segment length in S4 Consistent;

[0108] The processor targets each learning path segment Generate an update message and attach a learning path segment identifier to the update message. With iteration round identifier In addition, to ensure verifiable consistency at the receiving end, a message length field and a verification field are added to the message header, and the verification field adopts CRC16 and is verified at the receiving end using the same polynomial.

[0109] The sending end encodes each update message using a learning-based compression model to obtain a compressed message. This learning-based compression model consists of a sparsification module, a quantization module, and an error feedback module, and its parameters are fixed before deployment. The sparsity module... Perform amplitude sorting and retain only the largest amplitude values. Each element is assigned a position index and its corresponding value, where the magnitude is expressed in absolute value and the position index is from 1 to... The integer number of the element is not retained and is considered as the part that was not transmitted in this round;

[0110] The error feedback module is for each learning path segment Maintain a with Residual cumulative vector of the same dimension Before performing sparsification, the untransmitted residuals from the previous round are accumulated into the update amount of the current round to form the input for encoding. After quantization and sparsification are completed, the elements that were not retained and the reconstruction error caused by quantization are written back to the residual accumulation vector of the next round to ensure consistent convergence accuracy under the same communication budget.

[0111] The quantization module uses codebook quantization and outputs the quantization index, denoted as . And includes Each codeword is stored in a fixed-point format in the same address space at both the sending and receiving ends. Each codeword corresponds to a representable update magnitude and covers an interval. The sending end maps the value of each element retained after sparsification to the codeword with the smallest Euclidean distance and outputs the quantization index of that codeword. The compressed message payload is written in a fixed order. Each location index and its corresponding Each quantization index is represented by a 16-bit unsigned integer and each quantization index is represented by an 8-bit unsigned integer, forming a determinately parsable compressed message, which is then sent to the receiving end via the communication link;

[0112] After receiving the compressed message, the receiving end first verifies the CRC16 and checks the iteration round identifier. If the iteration number is consistent with the local iteration number, and the verification passes and the iteration number is consistent, it is identified by the learning path segment. Locate the locally maintained segmented consensus variables and decode each pair of position indices and quantization indices in the load. During decoding, the codebook is consulted based on the quantization index. The corresponding codeword is retrieved as the update amount for that position and written into the corresponding segmented consensus variable update buffer. The update amount for positions that do not appear in the message is set to zero and together with the written non-zero update amount, they form the recovery update amount for the learning path segment in this round. Then, the receiving end adds the recovery update amount to the local segmented consensus variable of the previous round to obtain the segmented consensus variable input required for the next round of alternating direction multiplier method iteration and provides this input to S4 to continue to perform local variable update and dual variable update.

[0113] After each iteration, the processor performs round control and determines the stopping condition at the arbitration node based on the original residual and the dual residual, where the original residual is denoted as... And the dual residual is denoted as ,when and A convergence stop instruction is generated and subsequent compressed transmission and iterative updates using the alternating direction multiplier method are stopped, wherein the first threshold is... Fixed as And the second threshold Fixed as ;

[0114] The processor also maintains and records the cumulative amount of compressed message data that has been transmitted. ,when Reaching the preset communication budget The system generates a budget stop instruction and halts subsequent compressed transmissions and iterative updates using the alternating direction multiplier method, wherein a preset communication budget is provided. Fixed at 1,048,576 bytes and cumulative data volume It is obtained by summing the actual number of bytes in the compressed message of all learning path segments in each round;

[0115] In response to the convergence stop instruction or the budget stop instruction, the processor reads the segment consensus variables of each learning path segment maintained by the receiver in the current round and concatenates them in segment order to obtain a consistent learning path as the final learning path planning result.

[0116] In this specific embodiment, the multi-agent policy model is obtained through a training process that is centrally trained and then executed in a decentralized manner. The training process is completed on an offline training server and the model parameters are fixed before deployment.

[0117] The processor constructs an interactive environment based on the knowledge graph to be encoded. At the beginning of each training round, the interactive environment reads a task record from the training task set and determines the learning objective and learner state. Then, it calls S1 to encode the knowledge graph to be encoded corresponding to the task record to obtain the node representation vector and use it as the static observation base of the environment.

[0118] At each decision-making moment, the interactive environment outputs an agent state vector consistent with S2 to each agent. It also receives the action results output by each agent. The action results are composed of the next knowledge point node selection result and the learning resource selection result, and the action mask rule is consistent with S2 to ensure that the prior accessibility and resource correlation are satisfied at the action space level.

[0119] After receiving the action result, the interactive environment updates the current knowledge point node of each agent and records the access identifier to prevent repeated entry. Simultaneously, it accumulates learning resource costs and reaches the target knowledge point node or path length. The sequence generation of the agent ends when the round ends, so that multiple candidate learning paths corresponding to the agents are obtained at the end of the round and aggregated into a candidate learning path set.

[0120] At the end of the round, the interactive environment calls S3 to perform a satisfiability determination on the candidate learning path set and obtains the satisfiability determination result, the size of unsatisfiable cores, and the conflict type. The size of unsatisfiable cores is defined as the number of activated literals in the unsatisfiable cores and denoted as . Conflict type It is limited to one of four categories: priority conflict, resource matching conflict, target coverage conflict, and budget conflict, and is represented by a four-value enumeration.

[0121] The interactive environment injects the conflict type and unsatisfiable core size as training feedback into the reward function and writes the round reward into the trajectory cache in the manner of the last step feedback, so that each agent at the decision time... Rewards received The following relationship must be satisfied:

[0122] ;

[0123] in Indicates the agent's ID. Indicates the index of the decision moment. Represents intelligent agents At the moment of decision The scalar of the reward received. The learning objective coverage reward weight is fixed at 1. This indicates an indicator function that takes the value 1 if the logical expression within the parentheses is true, and otherwise takes the value 0. Represents intelligent agents At the moment of decision Selected knowledge point node identifiers This represents the set of target knowledge point nodes obtained from the parsing of learning objectives. Represents intelligent agents At the moment of decision The set of previously visited knowledge point nodes, The learning cost penalty weight is represented and fixed at 1. This indicates the resource cost function in S3. The calculated learning resource cost scalar, Represents intelligent agents At the moment of decision Selected learning resource identifier, Indicates the constraint conflict penalty weight and fixes it to 1. This indicates the action taken by the agent in this round. The termination time is triggered by reaching the target knowledge point node or reaching the maximum path length. UNSAT indicates an unsatisfiable flag, and is true if the judgment result is unsatisfiable, otherwise false. Indicates by conflict type The penalty coefficients for the determined conflict types are fixed as follows: 5.0 for priority order conflicts, 3.0 for resource matching conflicts, 4.0 for target coverage conflicts, and [missing value] for budget conflicts. This indicates that the core size penalty coefficient cannot be satisfied and is fixed at 1. This indicates that the core size cannot be satisfied and is a positive integer;

[0124] The multi-agent policy model employs an agent policy sub-network structure and uses a centrally trained value evaluation sub-network as the central value function during training. The input of the value evaluation sub-network is the state vector of all agents. The concatenated vectors and the input dimension is fixed. The value evaluation subnetwork adopts a two-layer fully connected structure with the first hidden layer dimension fixed at 512, the second hidden layer dimension fixed at 256, the activation function fixed at ReLU, and outputs a state value scalar.

[0125] The training algorithm employs a proximal policy optimization based on shearing targets and updates parameters using the Adam optimizer. The discount factor is fixed at 0.99, the generalized advantage estimation parameter is fixed at 0.95, the policy shearing coefficient is fixed at 0.2, and the learning rate is fixed at [value missing]. Each parameter update collects 256 rounds of trajectories and performs gradient updates for the same batch of trajectories for 10 training rounds. The total number of training rounds is fixed at 20,000, and the agent policy sub-network parameters are exported as a multi-agent policy model after training.

[0126] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0127] This invention addresses the technical problem of "multi-agent collaborative learning path planning on a knowledge graph and achieving rapid consensus under constraints." It constructs a closed loop from representation, generation, and detection to arbitration and communication control: First, a graph transformer is used to uniformly represent knowledge point nodes, prerequisite relationships, and learner mastery, enabling the multi-agent policy model to generate more suitable candidate learning paths under structured semantics and prerequisite reachability constraints. Then, the candidate learning paths, along with planning constraints such as prerequisite order, resource matching, target coverage, and budget, are transformed into a satisfiability judgment problem. A constraint solver is used to verify the computability of conflicts, thus avoiding infeasible path outputs caused by relying solely on heuristic rules. Based on this, the alternating direction multiplier method is used to arbitrate the local path results of the multi-agents, and combined with compressed transmission and round control of segmented consensus variable updates, stable convergence and real-time output are achieved under communication budget and latency constraints, ensuring that the final learning path simultaneously meets the requirements of constraint consistency and real-time performance.

[0128] This invention addresses the aforementioned technical problems by making engineering-deployable improvements to the algorithm structure: First, it employs an incremental solution method to maintain the satisfiability determination state, and outputs the unsatisfiable core and conflict type when satisfiability is not met, transforming conflict localization from "full recalculation" to "incremental update plus conflict core localization," thereby reducing solution overhead and improving repair directionality. Second, it introduces segmented consensus variables and uses the unsatisfiable core to drive the selection of the path segment to be updated, enabling the alternating direction multiplier method to update only conflict-related segments, while utilizing the previous round's consensus variables and dual variables for hot-start, reducing invalid iterations. Third, it adaptively adjusts the penalty parameter based on the original residual, dual residual, and conflict type, improving the consistency convergence speed and stability under different conflict patterns. Through these improvements, this invention can generate consistent learning paths that satisfy planning constraints more quickly, effectively reduce iteration rounds and communication overhead, and further enhance real-time performance and availability.

Claims

1. A knowledge graph learning path planning method based on multi-agent systems, characterized in that, include: S1. Obtain the knowledge graph, learning objectives, learner status, and planning constraint set. The knowledge graph includes knowledge point nodes, prerequisite relationship edges, and association edges between learning resources and knowledge points. After obtaining the knowledge graph, a set of target knowledge point nodes is determined based on the learning objective, and the knowledge graph to be encoded is extracted along the prior relation edge within a preset number of hops, starting from the set of target knowledge point nodes. For each knowledge point node in the knowledge graph to be encoded, an initial node vector is generated. The generation of the initial node vector includes concatenating the identifier embedding vector corresponding to the knowledge point node identifier, the type embedding vector corresponding to the knowledge point node type, and the mastery feature vector extracted from the learner's state. For each relation in the knowledge graph to be encoded, an initial relation vector is generated. The generation of the initial relation vector includes combining the type embedding vector corresponding to the relation type with the relation direction feature vector. Calculate the topological hierarchy of each knowledge point node on the directed acyclic graph formed by the pre-modified relation edges and generate the topological position code; An attention mask is generated based on the reachability of the pre-modified edges; The graph transformer model is used to encode the knowledge graph to be encoded under the constraints of the topological position encoding and the attention mask, and outputs the node representation vector corresponding to each knowledge point node and the relation representation vector corresponding to each relation as the knowledge graph representation information. S2. Each candidate learning path includes a sequence of knowledge points arranged in learning order and a corresponding sequence of learning resources. On the knowledge graph, each agent is assigned a subset of the target knowledge point node set or a subgraph range of the knowledge graph according to the preset division of labor, and a candidate knowledge point node set corresponding to the subset or the subgraph range is determined based on the knowledge graph representation information. For each agent, target knowledge point nodes are determined based on the learning objectives, and mastered knowledge point nodes and knowledge point nodes to be learned are determined based on the learner's state. During the candidate learning path generation process, an agent state is constructed for each agent at each decision moment. The agent state includes the node representation vector corresponding to the current knowledge point node, the node representation vector corresponding to the target knowledge point node, and the mastery feature determined by the learner state. Based on knowledge graph representation information and combined with learning objectives and learner states, a multi-agent policy model is invoked, and action results are output according to the agent states. The action results include the next knowledge point node selection result and the learning resource selection result corresponding to the next knowledge point node. Based on the action results, the knowledge point sequence and the learning resource sequence corresponding to the knowledge point sequence are gradually expanded, and a candidate learning path is output when a preset termination condition is met. The preset termination condition includes reaching the target knowledge point node or reaching a preset path length. The candidate learning paths output by multiple agents are aggregated to obtain a set of candidate learning paths; S3. Convert the candidate learning path set into constraint variables and constraint expressions that can be processed by the constraint solver. For each candidate learning path, set an order variable to represent each knowledge point node in the knowledge point sequence and a selection variable to represent each learning resource in the learning resource sequence. The constraint expression is generated based on the planning constraint set, including: the prior order constraint defined by the prior relation edge, the resource matching constraint defined by the association edge between learning resources and knowledge points, the target coverage constraint defined by the learning objective, and the budget constraint defined by the preset learning budget. The candidate learning path set of the current iteration round is compared with the candidate learning path set of the previous iteration round. Based on the difference results, constraint expressions to be added, deleted or replaced incrementally are generated, and the corresponding constraints are updated in the solution state, so that the constraint solver performs satisfiability determination while retaining the solution information of the previous iteration round. When the satisfiability determination result is satisfied, output the conflict detection result representing satisfaction; When the satisfiability determination result is not satisfied, the unsatisfiable core is output, and the knowledge point node identifier and the prior relation edge identifier involved in the conflict are extracted from the unsatisfiable core. At the same time, the conflict type is determined according to the constraint category to which each constraint expression in the unsatisfiable core belongs, and the conflict detection result containing the unsatisfiable core, the knowledge point node identifier, the prior relation edge identifier and the conflict type is output. S4. Segment each candidate learning path in the candidate learning path set according to a preset segmentation rule. The preset segmentation rule includes segmenting according to the topological level corresponding to the pre-repaired relation edge or segmenting according to the preset path length, to obtain multiple learning path segments. For each learning path segment, segmented consensus variables are established, including sequence consensus variables and resource consensus variables. Local variables corresponding to the segmented consensus variables are also established for each candidate learning path in the candidate learning path set. In the first iteration, initial values ​​are assigned to the segmented consensus variables of each learning path segment based on the candidate learning path set, and the dual variables corresponding to the segmented consensus variables are initialized. Learning path segments to be updated are determined based on conflict detection results. When the conflict detection results contain a satisfying identifier, all learning path segments are identified as learning path segments to be updated. When the conflict detection results contain a dissatisfaction identifier, learning path segments containing the knowledge point node identifier or the prior relation edge identifier are identified as learning path segments to be updated based on the knowledge point node identifier and the prior relation edge identifier in the unsatisfiable core, and the remaining learning path segments are identified as non-updating learning path segments. In each iteration, the alternating direction multiplier method is used to sequentially perform local variable updates, segmented consensus variable updates, and dual variable updates. The segmented consensus variables and their corresponding dual variables from the previous iteration are used as the initial values ​​for this iteration. For the learning path segments to be updated, the local variable updates, segmented consensus variable updates, and dual variable updates are performed, while for the non-updating learning path segments, the segmented consensus variables remain unchanged. After completing the update of the dual variable, the original residual and the dual residual are calculated, and the adjustment coefficient of the penalty parameter is determined based on the original residual, the dual residual and the conflict type, so as to adaptively adjust the penalty parameter. The segmented consensus variables corresponding to each learning path segment are combined to generate a consistent learning path, and the segmented consensus variable update amount, the original residual, and the dual residual of this iteration are output. S5. The update amount of the segmented consensus variable is compressed, encoded and transmitted. The update amount is decoded and restored at the receiving end and used for subsequent iterative updates using the alternating direction multiplier method. Round control is performed based on the original residual and the dual residual. When the stopping condition is met, the consensus learning path is output as the learning path planning result.

2. The knowledge graph learning path planning method based on multi-agent systems according to claim 1, characterized in that, S5 include: For the segmented consensus variable update amount, generate an update amount message that corresponds one-to-one with each learning path segment, and attach a learning path segment identifier and an iteration round identifier to each update amount message; A learning-based compression model is used to encode each update message to obtain a compressed message. The encoding includes quantizing the update amount of the segmented consensus variable and outputting a quantization index. The compressed message is transmitted to the receiving end; The receiving end decodes the compressed message to recover the segmented consensus variable update amount, and writes the recovered segmented consensus variable update amount into the corresponding segmented consensus variable according to the learning path segment identifier, so as to serve as the segmented consensus variable update input for the next iteration round; After each iteration round, round control is performed based on the original residual and the dual residual. When the original residual is less than or equal to the first threshold and the dual residual is less than or equal to the second threshold, a convergence stop instruction is generated to stop the iterative update of the alternating direction multiplier method. When the cumulative amount of data of the compressed message that has been transmitted reaches the preset communication budget, a budget stop instruction is generated to stop the iterative update of the alternating direction multiplier method. In response to the convergence stop command or the budget stop command, a consistent learning path is output as the final learning path planning result.

3. The knowledge graph learning path planning method based on multi-agent systems according to claim 1, characterized in that, The acquisition of the multi-agent policy model includes the following training process: constructing an interactive environment based on the knowledge graph, using the learning objective, learner state, and knowledge graph representation information as state inputs, using candidate learning paths as action outputs, and constructing a reward function based on the planning constraint set. The reward function includes learning objective coverage reward, learning cost penalty, and constraint conflict penalty. During training, the conflict type and size of the unsatisfiable cores output by the constraint solver are used as training feedback to update the model parameters of the multi-agent policy model, so as to obtain the multi-agent policy model.

4. The knowledge graph learning path planning method based on multi-agent systems according to claim 1, characterized in that, The pre-defined division of labor includes dividing the knowledge graph into subgraphs based on graph partitioning, with the goal of minimizing the number of prerequisite relation edges across subgraphs. In subsequent iterations, when the conflict detection result is unsatisfactory, the subgraph range of at least one agent is adjusted according to the identifier of the prerequisite relation edge across subgraph that appears in the unsatisfactory core, so as to reduce the probability of cross-agent conflict.

5. The knowledge graph learning path planning method based on multi-agent systems according to claim 1, characterized in that, When the determination result is not satisfied, a candidate repair set is generated based on the unsatisfiable core. The candidate repair set includes swapping the knowledge point order variables in the candidate learning path, replacing the learning resource selection variables, or adjusting the upper bound of the budget constraint. At least one item in the candidate repair set is added as an additional constraint to the constraint expression of subsequent iteration rounds to shorten the iteration rounds to reach the satisfied state.

6. The knowledge graph learning path planning method based on multi-agent systems according to claim 2, characterized in that, The learning-based compression model encodes each update message and also includes sparsification processing. The sparsification processing involves selecting the top few elements with the largest amplitude from the segmented consensus variable update amounts and quantizing only these elements. Furthermore, an error feedback mechanism is introduced at the receiving end to accumulate the residuals corresponding to the untransmitted portions into the segmented consensus variable update amounts of the next iteration round, thereby improving the consistency convergence accuracy under the same communication budget.

Images