Multi-agent decision enhancement method based on expert experience guidance and federal collaboration
By constructing an expert experience base and a federated collaborative learning framework, dynamically adapting thresholds and introducing fault tolerance mechanisms, and generating behavioral constraint anchor points, the deviation problem between simulation and real-world working conditions in multi-agent decision-making models is solved, thereby improving the safety, robustness, and adaptability of multi-agent decision-making.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-09
Smart Images
Figure CN122174019A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multi-agent decision-making technology, specifically to a multi-agent decision-making enhancement method based on expert experience guidance and federated collaboration. Background Technology
[0002] With the development of deep reinforcement learning, multi-agent systems have achieved remarkable success in simulation environments. Their core value lies in completing complex decision-making tasks that a single agent cannot accomplish through the collaborative interaction of multiple agents.
[0003] However, in the actual implementation of current multi-agent decision-making technology, there are still inherent deviations between the simulation environment and real-world working conditions. This results in poor real-world adaptability and insufficient credibility of the multi-agent decision-making model, which seriously restricts its reliability and promotion value in complex real-world working conditions.
[0004] Therefore, to meet existing needs, a multi-agent decision-making enhancement method based on expert experience guidance and federated collaboration is proposed. Summary of the Invention
[0005] The purpose of this invention is to provide a multi-agent decision enhancement method based on expert experience guidance and federated collaboration. By dynamically adapting working condition characteristics and constraint priorities, setting thresholds in layers, and introducing coupling fault tolerance and conflict optimization mechanisms, it not only solves the problems of fixed thresholds and poor working condition adaptation in traditional methods, but also achieves a dynamic balance among multiple constraints such as safety, efficiency, and compliance. It can generate accurate and reliable behavioral constraint anchor points, correct the deviation between simulation and reality from the root, and significantly improve the safety, robustness, and adaptability of multi-agent decision-making, thus solving the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A multi-agent decision enhancement method based on expert experience guidance and federated collaboration includes the following steps:
[0008] S1. Collect multi-domain expert experience related to the multi-agent decision-making target scenario and build an expert experience base; based on the expert experience base, extract the key constraints of multi-agent decision-making and establish behavioral constraint anchor points to correct the deviation between the simulation environment and the real working conditions.
[0009] S2. Build a multi-agent decision-making collaboration framework based on federated learning, including a central aggregation node and multiple local nodes. Each local node is used to store multi-agent decision data and local decision models in this scenario. The central aggregation node is responsible for the aggregation of model parameters, knowledge fusion and collaborative scheduling of each local node.
[0010] S3. Embed the constraint anchor points into the local decision model as hard constraints for training the local decision model.
[0011] S4. Each local node encrypts and uploads the trained local decision model parameters to the central aggregation node. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters of each local node in encrypted form to generate global decision model parameters.
[0012] S5. Combine the cross-scenario knowledge transmitted by each local node to optimize the parameters of the global decision model and extract the optimal cross-scenario decision strategy.
[0013] The optimized global model parameters are encrypted and distributed to each local node. Each local node then fine-tunes its local decision-making model based on the global model parameters, combined with local scenario data and expert experience.
[0014] Repeat steps S1-S5 until the performance of the global decision model and each local decision model reaches the preset threshold.
[0015] Furthermore, it also includes the following steps:
[0016] S6. Deploy the trained and optimized multi-agent decision-making model to a real-world working scenario, and call the behavioral constraint anchor points of the expert experience base in real time to verify and constrain the decision-making behavior of the agents.
[0017] A multi-scenario performance verification platform was built, and comparison groups were set up for simulated and real-world conditions, with and without expert guidance, and with and without federated collaboration to verify the performance of the multi-agent decision-making model. Based on the verification results, the behavioral constraint anchor points of the expert experience base, the privacy protection parameters of the federated learning framework, and the model training parameters were adjusted.
[0018] Furthermore, in S1, the expert experience base is constructed, including the following steps:
[0019] Collect expert experience from different fields in the target scenario, including demonstration data, safety rules or key constraints, including state safety boundaries, prohibited action areas, and key behavior sequences;
[0020] The collected expert experience is deduplicated and noise-reduced to remove invalid and contradictory experience data; natural language processing technology is used to transform unstructured expert experience into structured data.
[0021] Construct a layered architecture for the expert experience base, including an experience storage layer, an experience parsing layer, and an experience retrieval layer;
[0022] Based on the decision-making rules and behavioral norms in the expert experience base, the core constraint dimensions of multi-agent decision-making are extracted, including security constraints, efficiency constraints, compliance constraints, and robustness constraints.
[0023] For each constraint dimension, set a quantification threshold and constraint conditions to form behavioral constraint anchor points.
[0024] Furthermore, for each constraint dimension, quantization thresholds and constraints are set, including the following steps:
[0025] The core operating condition features of multi-agent decision-making scenarios are extracted, and the unstructured operating condition features are transformed into structured quantitative indicators using the analytic hierarchy process.
[0026] Based on the decision rules in the expert experience base, priority experience of constraint dimensions under each working condition is extracted, and the fuzzy comprehensive evaluation method is used to assign initial priority weights to safety constraints, efficiency constraints, compliance constraints, and robustness constraints.
[0027] A working condition feature priority linkage correction model is constructed, and an influence coefficient that adapts to various working condition features is introduced. The initial weight ratio is corrected in real time by combining the quantitative value of the real-time working condition features to obtain a dynamic priority weight that adapts to the current working condition.
[0028] Furthermore, for each constraint dimension, setting quantization thresholds and constraints also includes the following steps:
[0029] Based on the safety rules and industry safety standards in the expert experience database, determine the minimum safety baseline for safety constraints;
[0030] Based on core thresholds and expert experience, early warning boundaries are set. When a decision-making behavior approaches the core threshold, an early warning is triggered to guide the local decision-making model to adjust the decision-making strategy in advance.
[0031] A dynamic adjustment mechanism for the fault tolerance threshold is introduced, which dynamically adjusts the fault tolerance boundary based on the safety risk level under real-time operating conditions.
[0032] Furthermore, for each constraint dimension, setting quantization thresholds and constraints also includes the following steps:
[0033] Based on the efficiency target experience and historical best decision data in the expert experience base, statistical analysis is used to take the historical best efficiency index as the core threshold.
[0034] Set an early warning threshold based on a specified multiple of the core threshold; when the actual efficiency indicator is between the core threshold and the early warning threshold, trigger an efficiency early warning and adjust the decision-making strategy.
[0035] Furthermore, for each constraint dimension, setting quantization thresholds and constraints also includes the following steps:
[0036] A coupled fault tolerance mechanism is introduced, which links the fault tolerance threshold with the priority weights of security constraints and compliance constraints. The fault tolerance threshold is set by combining the weights of efficiency constraints, security constraints, and compliance constraints.
[0037] When the priority of security constraints and compliance constraints is increased, the sum of the weights of security constraints and compliance constraints increases, and the fault tolerance threshold decreases accordingly.
[0038] Under normal operating conditions, as the weight of efficiency constraints increases, the fault tolerance threshold also increases.
[0039] Furthermore, after obtaining the dynamic priority weights adapted to the current operating conditions, the following steps are included:
[0040] Based on dynamic priority weights, calculate the conflict coefficient between thresholds of each constraint dimension;
[0041] When the conflict coefficient exceeds the conflict threshold value calibrated by expert experience, a constraint conflict is determined to exist.
[0042] Construct a coupled optimization objective and set the optimization direction;
[0043] The particle swarm optimization algorithm is used to solve the coupled optimization objective, and the optimized three-layer thresholds for each constraint dimension are obtained. The thresholds are integrated into a standardized vector form to form behavioral constraint anchors, and the specific values of the three-layer thresholds for safety constraints, efficiency constraints, compliance constraints, and robustness constraints are defined.
[0044] Real-time collection of multi-agent decision-making results and real-world working condition data; calculation of the deviation between decision-making results and behavioral constraint anchor points.
[0045] When the deviation continues to exceed the preset range, the threshold re-optimization process is triggered.
[0046] Furthermore, in S3, embedding constraint anchors into the local decision model includes the following steps:
[0047] The raw decision data stored locally is cleaned, normalized, and its features are extracted. Abnormal data is removed and missing values are filled in, transforming the raw data into feature data suitable for training the local decision model.
[0048] Based on the characteristics of local working conditions, a simulation dataset and a real-world dataset for local scenarios are constructed for training local decision-making models and verifying bias corrections.
[0049] Initialize the multi-agent local decision-making model, which includes a state input layer, a feature fusion layer, a decision output layer, and a constraint verification layer. Construct a loss function that integrates expert experience constraints. The loss function consists of three parts: decision error loss, behavioral constraint loss, and generalization error loss.
[0050] Furthermore, in S3, embedding constraint anchors into the local decision model also includes the following steps:
[0051] The multi-agent local decision-making model is trained based on the locally preprocessed feature data, simulation dataset, and real dataset.
[0052] By using the experience call interface of the local node, the behavioral constraint anchor points in the expert experience base are called in real time to verify and correct the model decision results.
[0053] An iterative training method is used to compare the model's decision results on simulation datasets and real datasets, and to calculate the deviation between simulation and reality.
[0054] The model parameters are adjusted based on behavioral constraint anchors to gradually reduce the bias until the model's decision results on the real dataset meet the requirements of expert experience and the bias is within the preset threshold range.
[0055] Compared with the prior art, the beneficial effects of the present invention are:
[0056] 1. By extracting core operating condition features and transforming them into structured quantitative indicators, and combining the fuzzy comprehensive evaluation method with the operating condition feature priority linkage correction model, dynamic adaptation of constraint dimension priority weights is achieved, effectively solving the problems of fixed priorities and poor adaptability to real-time operating conditions in traditional threshold settings; by setting core thresholds, early warning thresholds, and dynamic fault tolerance thresholds for each constraint dimension in a hierarchical manner, and combining safety rules, industry standards, expert experience, and historical data, both the decision-making bottom line is established, and early warning of decision deviations and dynamic adjustment of fault tolerance space can be achieved, taking into account both the rigid constraints and flexible adaptability of decision-making.
[0057] 2. By introducing a coupling fault-tolerance mechanism, a dynamic balance between efficiency and safety / compliance constraints is achieved, avoiding the pursuit of efficiency at the expense of safety and compliance. Through conflict identification, coupling optimization, and closed-loop update mechanisms, conflicts between thresholds of various constraint dimensions are effectively resolved, generating standardized and precise behavioral constraint anchor points. This ensures that the anchor points always align with expert experience and complex real-world conditions, providing reliable guidance for multi-agent decision-making. It corrects the deviation between the simulation environment and real-world conditions at its root, further improving the safety, efficiency, compliance, and robustness of multi-agent decision-making, while reducing manual maintenance costs and enhancing the adaptability and practicality of the decision-making system. Attached Figure Description
[0058] Figure 1 This is a flowchart of the multi-agent decision enhancement method based on expert experience guidance and federated collaboration of the present invention. Detailed Implementation
[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] To address the inherent discrepancies between simulation environments and real-world conditions in the practical application of existing multi-agent decision-making technologies, which lead to poor real-world adaptability and insufficient reliability of multi-agent decision-making models, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution:
[0061] A multi-agent decision enhancement method based on expert experience guidance and federated collaboration includes the following steps:
[0062] S1. Collect multi-domain expert experience related to multi-agent decision-making target scenarios, including but not limited to intelligent transportation, industrial control, autonomous driving, and expert decision-making rules, behavioral norms, risk prediction experience, and anomaly handling strategies under complex working conditions; standardize and structure the expert experience to build an expert experience base; based on the expert experience base, extract key constraints for multi-agent decision-making, establish behavioral constraint anchor points to correct deviations between the simulation environment and real-world working conditions, and guide multi-agent decision-making to conform to the safety requirements of real-world scenarios and expert experience; including the following steps:
[0063] Through expert interviews, literature reviews, and historical decision-making case analyses, we collect expert experience from different fields within the target scenario, including demonstration data, safety rules, and key constraints. These constraints include state safety boundaries, prohibited action areas, and key behavior sequences. The collected expert experience is deduplicated and noise-reduced to remove invalid and contradictory data. Natural language processing (NLP) technology is used to transform unstructured expert experience, such as decision rules described in natural language, into structured data, such as decision rule vectors and behavior constraint matrices, facilitating subsequent retrieval and parsing.
[0064] The expert experience base is constructed with a layered architecture, including an experience storage layer, an experience parsing layer, and an experience retrieval layer. The experience storage layer adopts a distributed storage method to store structured expert experience data, supporting rapid retrieval and updating of experience. The experience parsing layer has a built-in experience parsing algorithm to parse the corresponding expert experience constraints based on the multi-agent decision-making scenario and current working conditions. The experience retrieval layer provides a standardized calling interface to support real-time retrieval of expert experience by multi-agent decision-making models and federated learning frameworks. Based on the decision-making rules and behavioral norms in the expert experience base, the core constraint dimensions of multi-agent decision-making are extracted, including safety constraints, efficiency constraints, compliance constraints, and robustness constraints.
[0065] For each constraint dimension, quantification thresholds and constraints are set to form behavioral constraint anchors. These anchors are represented in vector form and include, but are not limited to, safety scoring functions for state-action pairs, used to correct action guidance vectors or trigger rule logic for safety interventions. The process includes the following steps:
[0066] The core operating condition features of multi-agent decision-making scenarios are extracted, such as congestion level, weather level, and traffic density in intelligent transportation scenarios, and equipment operating load, production cycle, and environmental interference intensity in industrial control scenarios. The analytic hierarchy process (AHP) is used to transform unstructured operating condition features into structured quantitative indicators, such as converting severe weather into weather levels quantified from 1 to 5, where level 1 is sunny and level 5 is blizzard / rainstorm. Based on decision rules in the expert experience base, priority experience of constraint dimensions under each operating condition is extracted. For example, in emergency operating conditions, safety constraints have higher priority than efficiency constraints and compliance constraints. The fuzzy comprehensive evaluation method is used to assign initial priority weights to safety constraints, efficiency constraints, compliance constraints, and robustness constraints, clarifying the initial weight ratio of each constraint dimension.
[0067] A working condition feature priority linkage correction model is constructed, introducing an influence coefficient adapted to various working condition features. This coefficient can be obtained by joint training with expert experience and historical decision data, clarifying the degree of influence of different working condition features on the priority of each constraint. The initial weight ratio is corrected in real time by combining the quantitative value of real-time working condition features, resulting in a dynamic priority weight adapted to the current working condition. For example, in intelligent transportation scenarios, when encountering severe weather such as blizzards, i.e., when the weather level is quantified to the highest level, the influence coefficient of the corresponding safety constraint triggers a weight increase, and the actual weight of the safety constraint increases significantly compared to the initial weight, realizing the dynamic improvement of the priority of safety constraints under severe weather conditions and solving the problem of insufficient threshold adaptability caused by the traditional fixed priority.
[0068] Based on safety rules and industry safety standards in the expert experience database, such as vehicle safety distance standards in the transportation industry and equipment safety operation thresholds in the industrial industry, the minimum safety baseline of safety constraints is determined. For example, in autonomous driving scenarios, the core threshold for safe vehicle distance is determined by combining expert experience and vehicle dynamics characteristics, clarifying the minimum safe vehicle distance standard at different driving speeds. When the actual vehicle distance is less than the core threshold, an emergency braking decision is immediately triggered, and it is strictly forbidden to exceed the threshold.
[0069] Based on core thresholds and expert experience, a warning boundary is set. When the decision-making behavior approaches the core threshold, a warning is triggered, guiding the local decision-making model to adjust its decision-making strategy in advance. The warning threshold is determined by combining the core threshold and the warning coefficient. The warning coefficient is calibrated by expert experience and ranges from 1 to 1.5 to ensure that the warning threshold is slightly higher than the core threshold, forming a reasonable warning range. For example, the safe distance warning threshold is set to 1.2 times the core threshold. When the actual distance is between the core threshold and the warning threshold, the model triggers a warning and outputs a deceleration prompt.
[0070] A dynamic adjustment mechanism for the fault tolerance threshold is introduced, which dynamically adjusts the fault tolerance boundary based on the safety risk level under real-time operating conditions. The fault tolerance threshold is a reasonable fluctuation range of the core threshold, which is determined by combining the core threshold, the fault tolerance base coefficient, and the quantitative value of the safety risk level, with a value range between 0.05 and 0.1. When the risk level reaches level 10 (extremely high risk), the fault tolerance threshold approaches 0, and the fault tolerance space is significantly reduced to avoid decision-making errors. When the risk level is level 1 (extremely low risk), the fault tolerance threshold reaches its maximum value to ensure the flexibility of the local decision-making model. The safety risk level is predicted by combining quantitative data of operating conditions and expert experience, and is quantified into levels 1-10, with higher levels indicating greater risk.
[0071] Based on efficiency target experience and historical best decision data from the expert experience base, statistical analysis is used to take the historical best efficiency index as the core threshold to ensure that the efficiency target is achievable and not extreme. For example, in industrial control scenarios, the core threshold for equipment operation and maintenance efficiency is the fault repair time, which is taken as 85% of the historical best repair time. When the actual repair time exceeds this core threshold, efficiency optimization decisions are triggered. An early warning threshold is set based on 1.1-1.2 times the core threshold. When the actual efficiency index is between the core threshold and the early warning threshold, an efficiency early warning is triggered, and the decision-making strategy is adjusted to avoid further decline in efficiency. For example, the congestion diversion efficiency early warning threshold is set to 5.5 minutes, which is 1.1 times the core threshold of 5 minutes. When the diversion time reaches 5.2 minutes, an early warning is triggered.
[0072] The beneficial effects achieved by the above are as follows: By extracting core operating condition features and transforming them into structured quantitative indicators, and combining the fuzzy comprehensive evaluation method with the operating condition feature priority linkage correction model, dynamic adaptation of the priority weights of constraint dimensions is realized, effectively solving the problems of fixed priorities and poor adaptability to real-time operating conditions in traditional threshold settings; by setting core thresholds, early warning thresholds, and dynamic fault tolerance thresholds for each constraint dimension in a hierarchical manner, and combining safety rules, industry standards, expert experience, and historical data, both the bottom line of decision-making is established, and early warning of decision deviations and dynamic adjustment of fault tolerance space can be achieved, taking into account both the rigid constraints and flexible adaptability of decision-making.
[0073] A coupled fault-tolerance mechanism is introduced, linking the fault-tolerance threshold to the priority weights of safety and compliance constraints. This, combined with the weights of efficiency, safety, and compliance constraints, sets the fault-tolerance threshold to ensure it aligns with the constraint priority requirements of the current operating condition. When the priority of safety and compliance constraints increases, such as in emergency situations, the sum of their weights increases, and the fault-tolerance threshold decreases accordingly, shrinking the fault-tolerance space and preventing the pursuit of efficiency at the expense of safety and compliance. Under normal operating conditions, the weight of efficiency constraints increases, and the fault-tolerance threshold increases accordingly, providing the local decision-making model with more room for efficiency optimization and achieving a dynamic balance between efficiency and safety / compliance.
[0074] After obtaining the dynamic priority weights adapted to the current working conditions, the process includes the following steps:
[0075] Based on dynamic priority weights, the conflict coefficient between thresholds of each constraint dimension is calculated to clarify the degree of mutual constraint between thresholds of different constraint dimensions. When the conflict coefficient exceeds the conflict threshold value calibrated by expert experience, it is determined that there is a constraint conflict. For example, the safety threshold requires an increase in vehicle distance, while the efficiency threshold requires a decrease in vehicle distance, which is a conflict. With the goal of each constraint threshold satisfying its own core requirements, minimizing conflicts, and adapting to dynamic working conditions, a coupled optimization objective is constructed, and an optimization direction is set. For example, the optimization direction is to reduce the conflict between each constraint threshold, ensuring that the optimized threshold does not deviate from the core range calibrated by the initial expert experience, and taking into account both conflict resolution and experience adaptability.
[0076] The particle swarm optimization algorithm is used to solve the coupled optimization objective, resulting in three-layer thresholds for each constraint dimension: core threshold, early warning threshold, and fault tolerance threshold. These thresholds are integrated into a standardized vector form to create behavioral constraint anchors. The specific values of the three-layer thresholds for safety constraints, efficiency constraints, compliance constraints, and robustness constraints are clearly defined, providing a clear behavioral constraint basis for multi-agent decision-making. The decision-making results of multi-agents are collected in real time along with real-world operating data, and the deviation between the decision results and the behavioral constraint anchors is calculated. When the deviation continuously exceeds a preset range, such as when the decision deviation exceeds the fault tolerance threshold for five consecutive times, a threshold re-optimization process is triggered, enabling dynamic self-updating of the behavioral constraint anchors and ensuring that the thresholds always adapt to complex real-world operating conditions and the needs of updating expert experience.
[0077] The beneficial effects achieved by the above are as follows: The introduction of a coupling fault-tolerance mechanism achieves a dynamic balance between efficiency and safety / compliance constraints, preventing the pursuit of efficiency at the expense of safety and compliance. Through conflict identification, coupling optimization, and closed-loop update mechanisms, conflicts between thresholds in various constraint dimensions are effectively resolved, generating standardized and precise behavioral constraint anchor points. This ensures that the anchor points always align with expert experience and complex real-world conditions, providing reliable guidance for multi-agent decision-making. It fundamentally corrects deviations between the simulation environment and real-world conditions, further enhancing the safety, efficiency, compliance, and robustness of multi-agent decision-making, while reducing manual maintenance costs and enhancing the adaptability and practicality of the decision-making system.
[0078] S2. Build a multi-agent decision-making collaboration framework based on federated learning, including a central aggregation node and multiple local nodes. Each local node corresponds to an application scenario and is used to store multi-agent decision data and local decision models in that scenario. The central aggregation node is responsible for the aggregation of model parameters, knowledge fusion and collaborative scheduling of each local node to ensure that the original data of each scenario does not leave the domain and to protect data privacy and security.
[0079] S3. Embed constraint anchor points into the local decision-making model as hard constraints for training the local decision-making model; during training, prioritize or enforce the satisfaction of anchor point constraints to ensure that decisions do not deviate from the safety boundaries and real-world scenario requirements set by experts, fundamentally correcting the deviation between simulation and reality, and improving the model's credibility; including the following steps:
[0080] The raw decision data stored locally is cleaned, normalized, and its features are extracted. Outliers are removed, and missing values are filled, transforming the raw data into feature data suitable for training the local decision model. Simulation and real-world datasets for the local scenario are constructed based on the characteristics of the local working conditions for training and verifying bias corrections in the local decision model. A multi-agent local decision model is initialized, including a state input layer, a feature fusion layer, a decision output layer, and a constraint verification layer. A loss function incorporating expert experience constraints is constructed, comprising decision error loss, behavioral constraint loss, and generalization error loss. The behavioral constraint loss is calculated from the deviation between the behavioral constraint anchor point and the model's decision result. When the model's decision result deviates from the behavioral constraint anchor point, a significant behavioral constraint loss is generated, guiding the model to adjust its decision strategy to conform to expert experience and real-world requirements.
[0081] Based on locally preprocessed feature data, simulation datasets, and real-world datasets, a multi-agent local decision-making model is trained. Through the local node's experience call interface, behavioral constraint anchors in the expert experience database are invoked in real time to verify and correct the model's decision results. An iterative training approach is adopted; after each training round, the model's decision results on the simulation dataset and the real-world dataset are compared to calculate the deviation between simulation and reality. Model parameters are adjusted based on the behavioral constraint anchors to gradually reduce the deviation until the model's decision results on the real-world dataset meet the expert experience requirements and the deviation is within a preset threshold range, thus completing the local model training.
[0082] S4. Each local node encrypts and uploads the trained local decision model parameters to the central aggregation node. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters of each local node in encrypted form to generate global decision model parameters.
[0083] S5. Combine the cross-scenario knowledge transmitted by each local node to optimize the parameters of the global decision model and extract the optimal decision strategy for each scenario. Encrypt the optimized global model parameters and distribute them to each local node. Each local node fine-tunes its local decision model based on the global model parameters, combined with local scenario data and expert experience, to achieve deep integration of cross-scenario knowledge and local scenario knowledge and improve the robustness and generalization ability of the model. Repeat S1-S5 until the performance of the global decision model and each local decision model reaches the preset threshold.
[0084] S6. Deploy the trained and optimized multi-agent decision-making model to real-world working scenarios. During the multi-agent decision-making process, call the behavioral constraint anchor points of the expert experience base in real time to verify and constrain the decision-making behavior of the agents, ensuring that the decision-making behavior conforms to the requirements of expert experience and real-world working conditions, and avoiding decision-making errors caused by simulation and reality deviations. Collect decision-making data and working condition data in real-time and feed them back to local nodes for online fine-tuning of the local decision-making model. Each local node regularly extracts new scenario knowledge and uploads it to the central aggregation node to realize dynamic updates of cross-scenario knowledge and continuous optimization of the model, further improving the security and robustness of multi-agent decision-making.
[0085] A multi-scenario performance verification platform was built, setting up comparison groups for simulated and real-world conditions, with and without expert guidance, and with and without federated collaboration to verify the performance of the multi-agent decision-making model. Based on the verification results, the behavioral constraint anchor points of the expert experience base, the privacy protection parameters of the federated learning framework, and the model training parameters were adjusted. At the same time, new expert experience and cross-scenario data were continuously collected to update the knowledge fusion content of the expert experience base and federated learning, realizing continuous iterative optimization of the model and ensuring the stability of the model's performance under complex real-world conditions.
[0086] Working principle: An expert experience database is built by collecting experience from experts in multiple fields. After quantification of working condition features, priority calibration, and threshold optimization, behavioral constraint anchor points are generated to correct the deviation between simulation and reality. A federated learning collaborative framework is built, where each local node embeds the anchor points into the model as hard constraints to complete local training. The model parameters are then encrypted and uploaded to the central aggregation node. After aggregation and optimization by the central node, global parameters are generated and distributed to each local node, and each node fine-tunes its local model. After iterative optimization until the performance meets the standards, the application is deployed. The decision-making behavior is verified in real time through anchor points, and parameters are adjusted by combining multiple sets of comparison verifications to form a closed-loop mechanism, thereby improving the decision-making performance of multi-agent systems and protecting data privacy.
[0087] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or high-voltage switchgear that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or high-voltage switchgear.
[0088] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A multi-agent decision enhancement method based on expert experience guidance and federated collaboration, characterized in that: Includes the following steps: S1. Collect multi-domain expert experience related to multi-agent decision-making target scenarios and build an expert experience database; Based on the expert experience base, key constraints for multi-agent decision-making are extracted, and behavioral constraint anchor points are established to correct the deviation between the simulation environment and the real working conditions. S2. Build a multi-agent decision-making collaboration framework based on federated learning, including a central aggregation node and multiple local nodes. Each local node is used to store multi-agent decision data and local decision models in this scenario. The central aggregation node is responsible for the aggregation of model parameters, knowledge fusion and collaborative scheduling of each local node. S3. Embed the constraint anchor points into the local decision model as hard constraints for training the local decision model. S4. Each local node encrypts and uploads the trained local decision model parameters to the central aggregation node. The central aggregation node uses a federated averaging algorithm to aggregate the model parameters of each local node in encrypted form to generate global decision model parameters. S5. Combine the cross-scenario knowledge transmitted by each local node to optimize the parameters of the global decision model and extract the optimal cross-scenario decision strategy. The optimized global model parameters are encrypted and distributed to each local node. Each local node then fine-tunes its local decision-making model based on the global model parameters, combined with local scenario data and expert experience. Repeat steps S1-S5 until the performance of the global decision model and each local decision model reaches the preset threshold.
2. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 1, characterized in that, It also includes the following steps: S6. Deploy the trained and optimized multi-agent decision-making model to a real-world working scenario, and call the behavioral constraint anchor points of the expert experience base in real time to verify and constrain the decision-making behavior of the agents. A multi-scenario performance verification platform was built, and comparison groups were set up for simulated and real-world conditions, with and without expert guidance, and with and without federated collaboration to verify the performance of the multi-agent decision-making model. Based on the verification results, the behavioral constraint anchor points of the expert experience base, the privacy protection parameters of the federated learning framework, and the model training parameters were adjusted.
3. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 1, characterized in that, In S1, the construction of an expert experience base includes the following steps: Collect expert experience from different fields in the target scenario, including demonstration data, safety rules or key constraints, including state safety boundaries, prohibited action areas, and key behavior sequences; The collected expert experience is deduplicated and noise-reduced to remove invalid and contradictory experience data; natural language processing technology is used to transform unstructured expert experience into structured data. Construct a layered architecture for the expert experience base, including an experience storage layer, an experience parsing layer, and an experience retrieval layer; Based on the decision-making rules and behavioral norms in the expert experience base, the core constraint dimensions of multi-agent decision-making are extracted, including security constraints, efficiency constraints, compliance constraints, and robustness constraints. For each constraint dimension, set a quantification threshold and constraint conditions to form behavioral constraint anchor points.
4. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 1, characterized in that, For each constraint dimension, set quantization thresholds and constraints, including the following steps: The core operating condition features of multi-agent decision-making scenarios are extracted, and the unstructured operating condition features are transformed into structured quantitative indicators using the analytic hierarchy process. Based on the decision rules in the expert experience base, priority experience of constraint dimensions under each working condition is extracted, and the fuzzy comprehensive evaluation method is used to assign initial priority weights to safety constraints, efficiency constraints, compliance constraints, and robustness constraints. A working condition feature priority linkage correction model is constructed, and an influence coefficient that adapts to various working condition features is introduced. The initial weight ratio is corrected in real time by combining the quantitative value of the real-time working condition features to obtain a dynamic priority weight that adapts to the current working condition.
5. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 4, characterized in that, For each constraint dimension, setting quantization thresholds and constraints also includes the following steps: Based on the safety rules and industry safety standards in the expert experience database, determine the minimum safety baseline for safety constraints; Based on core thresholds and expert experience, early warning boundaries are set. When a decision-making behavior approaches the core threshold, an early warning is triggered to guide the local decision-making model to adjust the decision-making strategy in advance. A dynamic adjustment mechanism for the fault tolerance threshold is introduced, which dynamically adjusts the fault tolerance boundary based on the safety risk level under real-time operating conditions.
6. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 5, characterized in that, For each constraint dimension, setting quantization thresholds and constraints also includes the following steps: Based on the efficiency target experience and historical best decision data in the expert experience base, statistical analysis is used to take the historical best efficiency index as the core threshold. Set an early warning threshold based on a specified multiple of the core threshold; when the actual efficiency indicator is between the core threshold and the early warning threshold, trigger an efficiency early warning and adjust the decision-making strategy.
7. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 6, characterized in that, For each constraint dimension, setting quantization thresholds and constraints also includes the following steps: A coupled fault tolerance mechanism is introduced, which links the fault tolerance threshold with the priority weights of security constraints and compliance constraints. The fault tolerance threshold is set by combining the weights of efficiency constraints, security constraints, and compliance constraints. When the priority of security constraints and compliance constraints is increased, the sum of the weights of security constraints and compliance constraints increases, and the fault tolerance threshold decreases accordingly. Under normal operating conditions, as the weight of efficiency constraints increases, the fault tolerance threshold also increases.
8. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 4, characterized in that, After obtaining the dynamic priority weights adapted to the current working conditions, the following steps are included: Based on dynamic priority weights, calculate the conflict coefficient between thresholds of each constraint dimension; When the conflict coefficient exceeds the conflict threshold value calibrated by expert experience, a constraint conflict is determined to exist. Construct a coupled optimization objective and set the optimization direction; The particle swarm optimization algorithm is used to solve the coupled optimization objective, and the optimized three-layer thresholds for each constraint dimension are obtained. The thresholds are integrated into a standardized vector form to form behavioral constraint anchors, and the specific values of the three-layer thresholds for safety constraints, efficiency constraints, compliance constraints, and robustness constraints are defined. Real-time collection of multi-agent decision-making results and real-world working condition data; calculation of the deviation between decision-making results and behavioral constraint anchor points. When the deviation continues to exceed the preset range, the threshold re-optimization process is triggered.
9. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 1, characterized in that, In S3, embedding constraint anchors into the local decision model includes the following steps: The raw decision data stored locally is cleaned, normalized, and its features are extracted. Abnormal data is removed and missing values are filled in, transforming the raw data into feature data suitable for training the local decision model. Based on the characteristics of local working conditions, a simulation dataset and a real-world dataset for local scenarios are constructed for training local decision-making models and verifying bias corrections. Initialize the multi-agent local decision-making model, which includes a state input layer, a feature fusion layer, a decision output layer, and a constraint verification layer. Construct a loss function that integrates expert experience constraints. The loss function consists of three parts: decision error loss, behavioral constraint loss, and generalization error loss.
10. The multi-agent decision enhancement method based on expert experience guidance and federated collaboration according to claim 9, characterized in that, In S3, embedding constraint anchors into the local decision model also includes the following steps: The multi-agent local decision-making model is trained based on the locally preprocessed feature data, simulation dataset, and real dataset. By using the experience call interface of the local node, the behavioral constraint anchor points in the expert experience base are called in real time to verify and correct the model decision results. An iterative training method is used to compare the model's decision results on simulation datasets and real datasets, and to calculate the deviation between simulation and reality. The model parameters are adjusted based on behavioral constraint anchors to gradually reduce the bias until the model's decision results on the real dataset meet the requirements of expert experience and the bias is within the preset threshold range.