Method and system for event-driven multi-agent collaborative decision making

Abstract

Disclosed are a method and system for event-driven multi-agent collaborative decision-making. The method comprises: acquiring situation data of a plurality of agents and their environment at a current time; generating a situation feature vector at the current time based on the situation data at the current time; generating situation change metric data with a plurality of event semantic dimensions based on the situation feature vector at the current time and situation feature vectors at a plurality of previous times; determining whether a decision for controlling the plurality of agents needs to be executed based on the situation change metric data; in response to determining that the decision needs to be executed, selecting at least one decision execution layer from a plurality of decision execution layers based on the situation feature vector at the current time and the situation change metric data; and executing the decision using the selected at least one decision execution layer based on the situation data at the current time. Thus, the overall decision cost can be reduced, the overall decision efficiency can be improved, and the overall decision robustness and interpretability can be improved.

Description

Technical Field

[0001] This disclosure relates to control methods and systems, and more specifically, to methods and systems for event-driven multi-agent collaborative decision-making. Background Technology

[0002] In large-scale adversarial game and multi-agent collaborative decision-making systems, decisions are typically organized into a hierarchical structure: the upper layer is responsible for overall planning, the middle layer for task planning, and the lower layer for rapid on-the-spot response. The inter-layer collaboration mechanism in traditional hierarchical decision-making frameworks relies on manual settings, thus lacking adaptive capabilities (e.g., generating numerous redundant decisions or missing crucial ones). Furthermore, the computations between layers are fragmented, resulting in high overall decision-making costs, low efficiency, poor robustness, and poor interpretability. Summary of the Invention

[0003] The purpose of this disclosure is to provide a method and system for event-driven multi-agent collaborative decision-making that can reduce overall decision-making costs, improve overall decision-making efficiency, and enhance overall decision-making robustness.

[0004] According to one aspect of the embodiments of this disclosure, a method for event-driven multi-agent collaborative decision-making is provided. The method includes: acquiring situational data of multiple agents and their environment at a current moment; generating a situational feature vector at the current moment based on the situational data; generating situational change measurement data having multiple event semantic dimensions based on the situational feature vector at the current moment and situational feature vectors from multiple previous moments, wherein the situational change measurement data includes multiple measurement values, each of the multiple measurement values ​​corresponding to the multiple event semantic dimensions, each of the multiple event semantic dimensions corresponding to an event semantic, and each of the multiple measurement values ​​representing the degree of change of an event associated with the corresponding event semantic; determining, based on the situational change measurement data, whether a decision for controlling the multiple agents needs to be executed; in response to determining that the decision needs to be executed, selecting at least one decision execution layer from multiple decision execution layers based on the situational feature vector at the current moment and the situational change measurement data, wherein the multiple decision execution layers correspond to multiple event importance granularities; and executing the decision using the selected at least one decision execution layer based on the situational data at the current moment.

[0005] Optionally, the step of generating situation change measurement data with multiple event semantic dimensions based on the situation feature vector at the current moment and the situation feature vectors at multiple previous moments includes: generating multiple situation change data corresponding to multiple time spans based on the situation feature vector at the current moment and the situation feature vectors at multiple previous moments; generating situation change measurement data by mapping the multiple situation change data to the multiple event semantic dimensions, wherein the multiple time spans are associated with the multiple event importance granularities, and each of the multiple event semantic dimensions is associated with at least one of the multiple event importance granularities.

[0006] Optionally, the step of determining whether to execute a decision for controlling the plurality of agents based on situation change measurement data includes: mapping the situation change measurement data of the plurality of event semantic dimensions to a global event change value through a nonlinear transformation, and determining that the decision needs to be executed when the global event change value is greater than a global event change threshold, or determining that the decision needs to be executed when any of the plurality of measurement values ​​of the situation change measurement data is greater than the corresponding event change degree threshold.

[0007] Optionally, in response to determining that the decision needs to be executed, the step of selecting at least one decision execution layer from multiple decision execution layers based on the current situation feature vector and situation change measurement data includes: determining multiple activation values ​​corresponding to the multiple decision execution layers respectively based on the current situation feature vector and situation change measurement data; in response to a competitive selection mode, selecting the decision execution layer corresponding to the largest activation value among the multiple activation values; and in response to a multi-level joint triggering mode, selecting the at least one decision execution layer corresponding to at least one activation value among the multiple activation values ​​that exceeds the activation threshold of the corresponding decision execution layer.

[0008] Optionally, the steps of determining whether to execute a decision for controlling the plurality of agents and selecting at least one decision execution layer from a plurality of decision execution layers are performed based on an event-triggered neural network model. The event-triggered neural network model is trained to maximize the cumulative value of a composite reward function, which is composed of a weighted difference between a reward term and a penalty term. The reward term corresponds to the difference between the contribution made and the loss suffered by the target agent. The penalty term corresponds to a weighted sum of at least one activation value of the at least one decision execution layer, such that activation values ​​of decision execution layers with higher event importance granularity have higher penalty weights, and activation values ​​of decision execution layers with lower event importance granularity have lower penalty weights. Within a predetermined evaluation period, when the sum of the total number of times each of the plurality of decision execution layers is selected is lower than a first threshold, a first penalty value is applied to the cumulative value of the composite reward function. Within the predetermined evaluation period, when the total number of times any of the plurality of decision execution layers is selected is higher than a corresponding second threshold, a second penalty value is applied to the cumulative value of the composite reward function, wherein the second penalty value is greater than the first penalty value.

[0009] Optionally, the steps of determining whether to execute a decision for controlling the plurality of agents and selecting at least one decision execution layer from a plurality of decision execution layers are performed based on an event-triggered neural network model. The event-triggered neural network model is trained to minimize the value of a loss function, wherein the loss function is a weighted sum of a selection effect improvement term, a selection frequency anomaly term, and a selection accuracy term. The selection effect improvement term reflects whether the execution of the event-triggered neural network model improves the decision effect; the selection frequency anomaly term reflects whether the execution of the event-triggered neural network model leads to an abnormal frequency of selection of a specific decision execution layer; and the selection accuracy term reflects whether the selection of the at least one decision execution layer is accurate.

[0010] Optionally, the step of executing the decision using the selected at least one decision execution layer based on the situational data at the current moment includes: executing the decision using the at least one decision execution layer sequentially in descending order of event importance granularity, wherein the step of executing the decision using the selected at least one decision execution layer based on the situational data at the current moment further includes: suppressing the selection of the specific decision execution layer and other decision execution layers with event importance granularity lower than that of the specific decision execution layer for a predetermined period of time after executing the decision using the specific decision execution layer.

[0011] According to one aspect of the embodiments of this disclosure, a system for event-driven multi-agent collaborative decision-making is provided. The system includes: a data acquisition module configured to acquire situational data of multiple agents and their environment at a current moment; a layer selection module configured to: generate a situational feature vector at the current moment based on the situational data at the current moment; generate situational change measurement data having multiple event semantic dimensions based on the situational feature vector at the current moment and situational feature vectors from multiple previous moments; determine whether a decision for controlling the multiple agents needs to be executed based on the situational change measurement data; and, in response to determining that the decision needs to be executed, select at least one decision execution layer from multiple decision execution layers based on the situational feature vector at the current moment and the situational change measurement data, wherein the situational change measurement data includes multiple measurement values, each of which corresponds to one of the multiple event semantic dimensions, each of the multiple event semantic dimensions corresponding to an event semantic, each of the multiple measurement values ​​representing the degree of change of an event associated with the corresponding event semantic, and the multiple decision execution layers corresponding to multiple event importance granularities; and a decision execution module including the multiple decision execution layers and configured to: execute the decision using the selected at least one decision execution layer based on the situational data at the current moment.

[0012] According to one aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the above-described method for event-driven multi-agent cooperative decision-making.

[0013] According to one aspect of an embodiment of the present disclosure, a computer program product is provided. The computer program product includes computer instructions that, when executed by at least one processor, implement the above-described method for event-driven multi-agent cooperative decision-making.

[0014] The method and system for event-driven multi-agent collaborative decision-making according to embodiments of this disclosure can reduce the total number of decisions at the decision layer, lower the overall decision-making cost, improve overall decision-making efficiency, enhance overall decision-making robustness, and improve interpretability. By selecting decision execution layers of appropriate granularity based on situational change measurement data, events of different importance are handled by matching decision execution layers, thereby reducing redundant decisions and unnecessary computational overhead, and improving the efficiency and robustness of multi-agent collaborative decision-making. Attached Figure Description

[0015] The above and / or other aspects of this disclosure will become clearer and more readily understood from the following detailed description taken in conjunction with the accompanying drawings.

[0016] Figure 1This is a flowchart illustrating a method for event-driven multi-agent collaborative decision-making according to embodiments of the present disclosure.

[0017] Figure 2 This is a flowchart illustrating a method for generating situational change measurement data with multiple event semantic dimensions according to embodiments of the present disclosure.

[0018] Figure 3 This is a flowchart illustrating a method for selecting at least one of a plurality of decision execution layers according to an embodiment of the present disclosure.

[0019] Figure 4 This is a block diagram illustrating a system for event-driven multi-agent collaborative decision-making according to an embodiment of the present disclosure.

[0020] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0021] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be appropriately altered upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and brevity, descriptions of features known upon understanding this disclosure may be omitted.

[0022] The features described herein may be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein are provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be clear upon understanding the disclosure of this application.

[0023] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Therefore, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0024] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the features, quantities, operations, components, elements, and / or combinations thereof stated therein, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0025] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as understood based on the disclosure of this application and as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having the same meaning as they have in the context of the relevant art and in the disclosure of this application, and shall not be interpreted ideally or overly formally. The use of the term “may” herein with respect to examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, while not all examples are limited thereto.

[0026] Unless otherwise specified, the same reference numerals generally refer to the same elements (e.g., components, steps, and methods). Reference numerals described in previous embodiments that reappear in later embodiments may be omitted. Furthermore, technical features described in different or the same embodiments can be combined in any way, as long as the combined embodiment or technical solution is complete and can solve the technical problems of this application or achieve the technical effects described or not described in this disclosure but which can be determined based on the complete technical solution described above. The terminology used in this disclosure is explained below.

[0027] It should be noted that, where there is no conflict between the various embodiments, these embodiments and their features can be combined with each other.

[0028] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0029] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart illustrating a method for event-driven multi-agent collaborative decision-making according to embodiments of the present disclosure. Figure 2 This is a flowchart illustrating a method for generating situational change measurement data with multiple event semantic dimensions according to embodiments of the present disclosure. Figure 3This is a flowchart illustrating a method for selecting at least one of a plurality of decision execution layers according to an embodiment of the present disclosure.

[0031] Reference Figure 1 The method for event-driven multi-agent collaborative decision-making according to embodiments of the present disclosure includes steps S100 to S600.

[0032] In step S100, situational data of multiple intelligent agents and their environment at the current moment is acquired. According to embodiments of this disclosure, the multiple intelligent agents can be unmanned devices in an autonomous unmanned system, adversarial units in intelligent game-based combat, or combat units in wargaming. The situational data is not the raw, unprocessed observation state, but rather task-level information formed by fusing, correlating, or calculating the states of multiple intelligent agents, environmental states, and task data. In the field of autonomous unmanned systems, situational data can include formation maintenance level, track conflict risk, communication connectivity status, task area coverage, search area, and remaining task execution capability. For example, track conflict risk can be determined based on the UAV's position, speed, and heading, and search coverage can be determined based on the searched area and sensor range. In the field of intelligent game-based combat, situational data can include local enemy-friendly force comparison, target threat level, fire coverage relationship, encirclement formation degree, breakout direction, and current combat stage. For example, the encirclement degree and encirclement gap direction can be determined based on the relative positions of the pursuing unit and the escaping unit. In the field of wargaming, situational data can include local troop strength advantages, firepower coverage, battlefield control range, supply and support capabilities, risks associated with maneuver routes, and the current stage of combat. For example, the comparison of enemy and friendly combat capabilities can be determined based on troop numbers, equipment status, and spatial distribution, while the risks associated with maneuver routes can be determined based on roads, terrain, and enemy firepower coverage.

[0033] In step S200, a situation feature vector for the current moment is generated based on the situation data at the current moment. In one example, this can be achieved by using a state encoding network to encode the situation data at the current moment into a fixed-dimensional situation feature vector. In one example, the state encoding network can be one of a multilayer perceptron, convolutional network, recurrent network, or attention network. In one example, the situation feature vector for the current moment can be stored in a historical situation cache. In one example, the historical situation cache can employ a fixed-length sliding window structure and retain situation feature vectors from the most recent N moments, providing a temporal baseline for subsequent change measurements.

[0034] In step S300, based on the current situation feature vector and multiple previous situation feature vectors, situation change measurement data with multiple event semantic dimensions is generated. According to embodiments of this disclosure, the situation change measurement data includes multiple measurement values, each corresponding to a multiple event semantic dimension. Each of the multiple event semantic dimensions corresponds to an event semantic, and each of the multiple measurement values ​​represents the degree of change of the event associated with the corresponding event semantic. In one example, event semantics may include at least changes in the global situational pattern, changes in the state of local areas, changes in resource distribution, changes in threat situation, and changes in time urgency. The following will refer to… Figure 2 Detailed description of step S300.

[0035] Reference Figure 2 Step S300 may include steps S310 and S320.

[0036] In step S310, multiple situation change data corresponding to multiple time spans can be generated based on the situation feature vector at the current moment and the situation feature vectors at multiple previous moments. According to embodiments of this disclosure, multiple time spans can be associated with multiple event importance granularities. In one example, multiple time spans may include three time spans (e.g., a short-term time span, a medium-term time span, and a long-term time span), multiple situation change data may include three corresponding situation change data (e.g., short-term (or sudden) situation change data obtained by calculating the situation difference between the current moment and the previous moment, medium-term (or trend) situation change data obtained by calculating the difference between the current moment's situation and the average situation of several recent moments, and long-term (or long-term cumulative drift) situation change data obtained by calculating the degree of deviation of the current moment from the long-term baseline), and multiple event importance granularities may include three corresponding event importance granularities (e.g., high (or coarse, strategic, planning) event importance granularity used for global goal setting and phase division with the lowest decision frequency and the highest level of abstraction, medium (or tactical, planning) event importance granularity used for task allocation and resource scheduling with medium decision frequency and medium level of abstraction, and low (or fine, ad hoc) event importance granularity used for rapid ad hoc response with the highest decision frequency and the lowest level of abstraction). However, it should be understood that this disclosure is not limited thereto, and the number of time spans, the number of situation change data, and the number of event importance granularities may be more or less than three.

[0037] In step S320, situation change measurement data can be generated by mapping multiple situation change data to multiple event semantic dimensions. According to embodiments of this disclosure, each of the multiple event semantic dimensions can be associated with at least one of multiple event importance granularities. In one embodiment, after performing step S310 and before performing step S320, the multiple situation change data can be fused by a fusion layer (e.g., multi-head attention or temporal convolution).

[0038] Return to reference Figure 1 In step S400, based on the situation change measurement data, it is determined whether a decision needs to be made to control multiple agents. In one example, a dual-judgment mechanism (global judgment and dimension-specific judgment) can be used to perform step S400.

[0039] For example, in global judgment, situational change measurement data of multiple event semantic dimensions can be mapped to global event change values ​​through nonlinear transformation. When the global event change value is greater than the global event change threshold, it can be determined that a decision needs to be executed. In this example, the global event change value can be a scalar and can represent the probability of determining whether a decision needs to be executed. For example, in dimensional judgment, when any one of the multiple measurement values ​​of situational change measurement data is greater than the corresponding event change degree threshold, it can be determined that a decision needs to be executed. In one embodiment, both the global event change threshold and the event change degree thresholds of each event semantic dimension can be learnable parameters. For example, if the execution of the judgment in step S400 in a certain scenario brings about an improvement in decision-making effect, the corresponding threshold is lowered to encourage similar judgment execution; if the execution of the judgment in step S400 does not bring any gain or causes redundant computational overhead, the corresponding threshold is raised to suppress similar judgment execution.

[0040] In one embodiment, when the global event change value is less than the global event change threshold and multiple measure values ​​of the situation change measurement data are all less than the corresponding event change degree threshold, it can be determined that no decision needs to be made.

[0041] Return to reference Figure 1In step S500, in response to determining that a decision needs to be executed, at least one decision execution layer is selected from a plurality of decision execution layers based on the current situation feature vector and situation change measurement data. According to embodiments of this disclosure, the plurality of decision execution layers may each correspond to a plurality of event importance granularities. For example, when multiple event importance granularities include three event importance granularities (e.g., a high (or coarse, strategic, planning) event importance granularity used for global goal setting and phase division with the lowest decision frequency and the highest level of abstraction; a medium (or tactical, planning) event importance granularity used for task allocation and resource scheduling with a medium decision frequency and a medium level of abstraction; and a low (or fine, ad hoc) event importance granularity used for rapid ad hoc response with the highest decision frequency and the lowest level of abstraction), multiple decision execution layers may include three corresponding decision execution layers (an upper (or strategic, planning) decision layer used for global goal setting and phase division with the lowest decision frequency and the highest level of abstraction; a middle (or tactical, planning) decision layer used for task allocation and resource scheduling with a medium decision frequency and a medium level of abstraction; and a lower (or ad hoc) decision layer used for rapid ad hoc response with the highest decision frequency and the lowest level of abstraction).

[0042] In one embodiment, a hierarchical constraint propagation mechanism can be used between multiple decision execution layers to ensure decision consistency: the decision output of the upper-level decision layer serves as the constraint input of the lower-level decision layer, and the lower-level decision layer can perform reasoning within the feasible domain of the constraints of the upper-level decision layer; when the upper-level decision layer is not triggered, the lower-level decision layer can continue execution using the most recent valid decision output of the upper-level decision layer; when the execution result of the lower-level decision layer deviates from the target of the upper-level decision layer by more than a preset deviation threshold, a replanning request signal is sent to the upper-level decision layer.

[0043] In one embodiment, at least one of the multiple decision execution layers can integrate a large language model as a decision model: the current situation information and the constraints of the higher-level decision layer are constructed into a structured prompt text input large model, its output is parsed into structured decision instructions, and it is adapted to the output format and decision quality requirements of a specific task through fine-tuning or context learning. The large model can generate more diverse strategies, verifying its feasibility as a knowledge computing carrier.

[0044] Below, we will refer to Figure 3 Detailed description of step S500.

[0045] Reference Figure 3 Step S500 may include steps S510 to S530.

[0046] In step S510, multiple activation values ​​corresponding to multiple decision execution layers can be determined based on the current situation feature vector and situation change measurement data. In step S520, in response to the competitive selection mode, the decision execution layer corresponding to the largest activation value among the multiple activation values ​​can be selected. In step S530, in response to the multi-level joint triggering mode, at least one decision execution layer corresponding to at least one activation value among the multiple activation values ​​that exceeds the activation threshold of the corresponding decision execution layer can be selected.

[0047] According to embodiments of this disclosure, steps S400 and S500 can be performed based on an event-triggered neural network model.

[0048] In one embodiment, the event-triggered neural network model can use maximizing the cumulative value of the composite reward function as its training objective. In this case, the event-triggered neural network model and multiple decision execution layers can be jointly trained under a unified reward signal. Parameters can be updated using policy gradients or proximal policy optimization (PPO) methods. Multiple decision execution layers can achieve joint optimization through gradient sharing or alternating training. This transforms the model from a simple concatenation of the event-triggered neural network model and multiple decision execution layers into a unified computational loop formed through an event-triggered mechanism. The composite reward function is the core that guides the event-triggered neural network model to trigger (e.g., select) the appropriate level (e.g., the appropriate decision execution layer) at the right time, and its design considers both decision performance and decision frequency constraints. The composite reward function can be composed of a weighted difference between a reward term and a penalty term. In one example, the composite reward function can be represented as shown in Equation 1 below.

[0049]

[0050] in, reward Represents the composite reward function. reward dam Indicates a reward item (e.g., battle damage reward). penalty freq This indicates a penalty term (decision frequency penalty), and A and B This represents the balance coefficient between reward and penalty items.

[0051] The reward reflects actual effectiveness in multi-agent adversarial scenarios and corresponds to the difference between the contributions made and the losses suffered by the target agent. In one example, the reward... reward dam It can be represented by Equation 2 below.

[0052]

[0053] in, damage blueThis represents the contribution made by agents of the target faction (e.g., our faction). damage red This represents the first type of loss suffered by the target faction (e.g., the loss of agents within the target faction), and damage facility This refers to the second type of loss suffered by the target faction (e.g., the loss of facilities belonging to the target faction).

[0054] The penalty term reflects the trigger (e.g., selection) cost at different levels (e.g., different decision execution layers) and can correspond to a weighted sum of at least one activation value from at least one decision execution layer, such that activation values ​​from decision execution layers with higher event importance granularity have higher penalty weights, and activation values ​​from decision execution layers with lower event importance granularity have lower penalty weights. In one example, the penalty term... penalty freq It can be represented by Equation 3 below.

[0055]

[0056] in, plan 1 This indicates the trigger indication value (0 or 1) for the planning layer at the current time step. program 1 The trigger indicator value (0 or 1) of the planning layer at the current time step. linji 1 This indicates the trigger indication value (0 or 1) for the ad hoc decision-making level at the current time step. w 1 , w 2 and w 3 This represents the frequency penalty weight corresponding to each level, and satisfies... w 1 > w 2 > w 3 >0 ensures that the cost of each trigger at the planning layer is the highest and the cost at the ad hoc decision layer is the lowest, guiding the model to learn the hierarchical decision-making rhythm of high-level low-frequency triggers and low-level high-frequency triggers.

[0057] In one example, an additional penalty can be imposed on the composite reward function. For instance, if the sum of the total number of selections for each of the multiple decision execution layers falls below a first threshold within a predetermined evaluation period, a first penalty value can be imposed on the cumulative value of the composite reward function, as expressed in Equation 4 below.

[0058]

[0059] in, mIndicates the first threshold. penalty 1 This represents the first penalty value, and decision count This represents the total number of times each of the multiple decision-making layers is selected. In one example... decision count It can be represented by Equation 5 below.

[0060]

[0061] in, plan count This indicates the number of decisions made by the planning team within an evaluation period. program count This indicates the number of decisions made by the planning level within an evaluation period, and linji count This indicates the number of decisions made by the ad hoc decision-making team within an evaluation cycle.

[0062] Therefore, it can prevent the model from excessively suppressing triggers, leading to decision gaps. For example, within a predetermined evaluation period, when the total number of times any of the multiple decision execution layers is selected exceeds the corresponding second threshold, a second penalty value can be applied to the cumulative value of the composite reward function, as shown in Equation 6 below.

[0063]

[0064] in, threshold plan Indicates and plan count The corresponding second threshold, threshold program Indicates and program count The corresponding second threshold, threshold linji Indicates and linji count The corresponding second threshold, the thresholds at each level satisfy threshold plan < threshold program < threshold linji ,and penalty 2 This represents the second penalty value.

[0065] Therefore, over-triggering at any level can prevent the waste of computational resources. In one example, the second penalty value can be greater than the first penalty value. For example, the first penalty value could be 0.2, and the second penalty value could be 0.5. This demonstrates that the computational waste caused by over-triggering needs to be suppressed more than occasional poor decision-making.

[0066] In another embodiment, the event-triggered neural network model can use minimizing the value of the loss function as its training objective. In this case, the loss function can be a weighted sum of selecting performance enhancement terms, selecting frequency outliers, and selecting accuracy terms. In one example, the loss function can be represented as shown in Equation 7 below.

[0067]

[0068] in, L Represents the loss function. L trigger It is a parameter that measures whether triggering improves decision-making effectiveness (trigger utility). L efficiency It is a parameter that determines whether the triggering frequency deviates from the target frequency (leading to the need for regularization constraints) (triggering efficiency). L routing It is a parameter that measures whether the hierarchy selection is correct (hierarchical accuracy), and λ 1 , λ 2 and λ 3 This indicates adjustable weights.

[0069] Selecting the "Effect Improvement" option reflects whether the execution of the event-triggered neural network model improves the decision-making effect. Selecting the "Frequency Anomaly" option reflects whether the execution of the event-triggered neural network model leads to abnormally high selection frequencies for a specific decision-making execution layer. Selecting the "Accuracy" option reflects whether the selection of at least one decision-making execution layer is accurate. The selection of the "Effect Improvement" option, the "Frequency Anomaly" option, and the "Accuracy" option can be obtained through preset methods.

[0070] In one embodiment, the training of the event-triggered neural network model can adopt a hybrid approach combining offline pre-training and online fine-tuning: first, extract situational change samples from pre-collected historical interaction data, and pre-train the event-triggered model using supervised learning to obtain a reasonable initial triggering strategy; then, collect triggering decision-making experience in real-time interaction with the environment, and fine-tune it online using the policy gradient method to make the triggering strategy continuously evolve.

[0071] Return to reference Figure 1 In one embodiment, in response to determining that no decision needs to be made, steps S500 and S600 may not be performed, and step S100 may be re-performed.

[0072] In step S600, based on the situational data at the current moment, a decision is executed using at least one selected decision execution layer. According to embodiments of this disclosure, at least one decision execution layer can be used sequentially in descending order of event importance granularity. In one embodiment, the selection of other decision execution layers with event importance granularity lower than that of the specific decision execution layer can be suppressed for a predetermined period after a decision is executed using that specific decision execution layer.

[0073] Therefore, the method for event-driven multi-agent collaborative decision-making according to embodiments of this disclosure can reduce the total number of decisions at the decision layer, lower the overall decision-making cost, improve overall decision-making efficiency, enhance overall decision-making robustness, and improve interpretability. By selecting decision execution layers of appropriate granularity based on situational change measurement data, events of different importance are handled by matching decision execution layers, thereby reducing redundant decisions and unnecessary computational overhead, and improving the efficiency and robustness of multi-agent collaborative decision-making.

[0074] Figure 4 This is a block diagram illustrating a system for event-driven multi-agent collaborative decision-making according to an embodiment of the present disclosure.

[0075] Reference Figure 4 A system 100 for event-driven multi-agent collaborative decision-making according to an embodiment of the present disclosure includes a data acquisition module 110, a layer selection module 120, and a decision execution module 130.

[0076] According to embodiments of this disclosure, the data acquisition module 110 is configured to acquire situational data of multiple agents and their environment at the current moment. Therefore, the data acquisition module 110 can perform operations corresponding to step S100 described above.

[0077] According to embodiments of this disclosure, the layer selection module 120 is configured to: generate a current-time situation feature vector based on current-time situation data; generate situation change measurement data with multiple event semantic dimensions based on the current-time situation feature vector and multiple previous-time situation feature vectors; determine whether a decision for controlling multiple agents needs to be executed based on the situation change measurement data; and in response to determining that a decision needs to be executed, select at least one decision execution layer from multiple decision execution layers based on the current-time situation feature vector and the situation change measurement data. The situation change measurement data includes multiple measurement values, each corresponding to a multiple event semantic dimension. Each of the multiple event semantic dimensions corresponds to an event semantic, and each of the multiple measurement values ​​represents the degree of change of an event associated with its corresponding event semantic. The multiple decision execution layers correspond to multiple event importance granularities. Therefore, the layer selection module 120 can perform operations corresponding to steps S200 to S500 described above.

[0078] According to embodiments of this disclosure, the decision execution module 130 includes multiple decision execution layers and is configured to execute decisions using at least one selected decision execution layer based on the current situational data. Therefore, the decision execution module 130 can perform the operations corresponding to step S600 described above.

[0079] Therefore, the system for event-driven multi-agent collaborative decision-making according to embodiments of this disclosure can reduce the total number of decisions at the decision-making layer, lower the overall decision-making cost, improve overall decision-making efficiency, enhance overall decision-making robustness, and improve interpretability. By selecting the appropriate granularity of the decision execution layer based on situational change measurement data, events of different importance are handled by matching decision execution layers, thereby reducing redundant decisions and unnecessary computational overhead, and improving the efficiency and robustness of multi-agent collaborative decision-making.

[0080] According to embodiments of this disclosure, a computer-readable storage medium may also be provided, wherein when instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the processor to perform the above-described method for event-driven multi-agent cooperative decision-making according to embodiments of this disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0081] According to embodiments of the present disclosure, a computer program product may also be provided, including computer instructions that, when executed by at least one processor, implement the above-described method for event-driven multi-agent cooperative decision-making according to embodiments of the present disclosure.

[0082] The method and system for event-driven multi-agent collaborative decision-making according to embodiments of this disclosure can reduce the total number of decisions at the decision layer, lower the overall decision-making cost, improve overall decision-making efficiency, enhance overall decision-making robustness, and improve interpretability. By selecting decision execution layers of appropriate granularity based on situational change measurement data, events of different importance are handled by matching decision execution layers, thereby reducing redundant decisions and unnecessary computational overhead, and improving the efficiency and robustness of multi-agent collaborative decision-making.

[0083] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0084] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for event-driven multi-agent cooperative decision-making, characterized in that, The method includes: Acquire situational data of multiple agents and their environment at the current moment; Based on the situation data at the current moment, generate the situation feature vector at the current moment; Based on the current situation feature vector and the situation feature vectors from multiple previous times, situation change measurement data with multiple event semantic dimensions is generated. The situation change measurement data includes multiple measurement values, each of which corresponds to one of the multiple event semantic dimensions. Each of the multiple event semantic dimensions corresponds to one event semantic, and each of the multiple measurement values ​​represents the degree of change of the event associated with the corresponding event semantic. Based on situational change measurement data, determine whether a decision needs to be made to control the multiple agents; In response to determining that the decision needs to be executed, at least one decision execution layer is selected from multiple decision execution layers based on the current situation feature vector and situation change measurement data, wherein the multiple decision execution layers correspond to multiple event importance granularities respectively; Based on the situational data at the current moment, the decision is executed using the selected at least one decision execution layer.

2. The method according to claim 1, characterized in that, The steps for generating situation change measurement data with multiple event semantic dimensions based on the situation feature vector at the current moment and situation feature vectors from multiple previous moments include: Based on the current situation feature vector and the situation feature vectors from multiple previous times, multiple situation change data corresponding to multiple time spans are generated. By mapping the multiple situation change data to the multiple event semantic dimensions, situation change measurement data is generated. The plurality of time spans are associated with the plurality of event importance granularities, and each of the plurality of event semantic dimensions is associated with at least one of the plurality of event importance granularities.

3. The method according to claim 1, characterized in that, The steps for determining whether a decision needs to be made to control the multiple agents based on situational change measurement data include: The situational change measurement data of the multiple event semantic dimensions are mapped to a global event change value through nonlinear transformation. When the global event change value is greater than the global event change threshold, it is determined that the decision needs to be executed, or When any one of the multiple metrics in the situation change measurement data is greater than the corresponding event change degree threshold, it is determined that the decision needs to be executed.

4. The method according to claim 1, characterized in that, In response to determining that the decision needs to be executed, the step of selecting at least one decision execution layer from a plurality of decision execution layers based on the current situation feature vector and situation change measurement data includes: Based on the current situation feature vector and situation change measurement data, determine multiple activation values ​​corresponding to the multiple decision execution layers respectively; In response to the competitive selection mode, the decision execution layer corresponding to the largest activation value among the plurality of activation values ​​is selected; In response to a multi-level joint triggering mode, at least one decision execution layer is selected, corresponding to at least one activation value among the plurality of activation values ​​that exceeds the activation threshold of the corresponding decision execution layer.

5. The method according to claim 4, characterized in that, The process utilizes an event-triggered neural network model to determine whether a decision for controlling the multiple agents needs to be made, and to select at least one decision execution layer from a plurality of decision execution layers. In this model, the event-triggered neural network is trained with the goal of maximizing the cumulative value of the composite reward function. The composite reward function consists of the weighted difference between the reward and penalty terms. The reward corresponds to the difference between the contribution made by the target faction's agents and the losses they suffer. The penalty term corresponds to a weighted sum of at least one activation value from the at least one decision execution layer, such that activation values ​​from decision execution layers with higher event importance granularity have higher penalty weights, and activation values ​​from decision execution layers with lower event importance granularity have lower penalty weights. Specifically, within a predetermined evaluation period, when the sum of the total number of times each of the plurality of decision execution layers is selected is lower than a first threshold, a first penalty value is applied to the cumulative value of the composite reward function. Specifically, within a predetermined evaluation period, when the total number of times any of the plurality of decision execution layers is selected exceeds the corresponding second threshold, a second penalty value is applied to the cumulative value of the composite reward function. The second penalty value is greater than the first penalty value.

6. The method according to claim 4, characterized in that, The process utilizes an event-triggered neural network model to determine whether a decision for controlling the multiple agents needs to be made, and to select at least one decision execution layer from a plurality of decision execution layers. In this context, the event-triggered neural network model uses minimizing the value of the loss function as its training objective. The loss function is composed of a weighted sum of the selection effect enhancement term, the selection frequency outlier term, and the selection accuracy term. Among them, the selection of the effect improvement item reflects whether the execution of the event-triggered neural network model improves the decision-making effect. Among them, the selection of frequency anomalies reflects whether the execution of the event-triggered neural network model leads to the selection of abnormal frequencies for a specific decision-making execution layer. Among them, the accuracy selection item reflects whether the selection of the at least one decision execution layer is accurate.

7. The method according to claim 1, characterized in that, Based on the current situational data, the steps of executing the decision using the selected at least one decision execution layer include: The decision is executed sequentially using at least one decision execution layer, in descending order of event importance. The step of making a decision using the selected at least one decision execution layer based on the current situation data further includes: For a predetermined period of time after the decision is executed using a specific decision execution layer, the selection of the specific decision execution layer and other decision execution layers with event importance granularity lower than that of the specific decision execution layer is suppressed.

8. A system for event-driven multi-agent cooperative decision-making, characterized in that, The system includes: The data acquisition module is configured to acquire situational data of multiple agents and their environment at the current moment; The layer selection module is configured to: generate a current-time situation feature vector based on the current-time situation data; generate situation change measurement data with multiple event semantic dimensions based on the current-time situation feature vector and situation feature vectors from multiple previous times; determine whether a decision for controlling the multiple agents needs to be executed based on the situation change measurement data; and in response to determining that the decision needs to be executed, select at least one decision execution layer from multiple decision execution layers based on the current-time situation feature vector and the situation change measurement data, wherein the situation change measurement data includes multiple measurement values, each of which corresponds to one of the multiple event semantic dimensions, each of the multiple event semantic dimensions corresponds to one event semantic, each of the multiple measurement values ​​represents the degree of change of the event associated with the corresponding event semantic, and the multiple decision execution layers correspond to multiple event importance granularities. The decision execution module includes the plurality of decision execution layers and is configured to execute the decision using the selected at least one decision execution layer based on the situation data at the current moment.

9. A computer-readable storage medium, characterized in that, When the instructions stored in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the method for event-driven multi-agent cooperative decision-making according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by at least one processor, implement the method for event-driven multi-agent cooperative decision-making according to any one of claims 1 to 7.

Images