Multi-agent decision-making interpretable analysis method and device and electronic equipment

Abstract

The invention provides an interpretable analysis method and device for multi-agent decision and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring real-time state data and real-time decision data of a plurality of agents in a task execution process; inputting the real-time state data into a pre-trained time sequence encoder, and converting the real-time state data into a causal graph network through the pre-trained time sequence encoder; the causal graph network comprises causal relationships between the intelligent agents and between the intelligent agents and task targets; according to the real-time state data and the real-time decision data, determining contribution of each state feature of each agent to a decision; and generating a multi-granularity visual decision path according to the causal graph network and the contribution of each state feature of each agent to the decision.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an explainable analysis method, device and electronic equipment for multi-agent decision-making. Background Art

[0002] Multi-agent decision making studies the behavior of groups of multiple agents that interact with each other in a system. In collaborative multi-agent reinforcement learning, a group of agents learns strategies through training to solve tasks that require working as a team to achieve a common goal. In recent years, people have paid more and more attention to the various possibilities brought to machine learning by better understanding causal relationships. Explicit causal modeling in artificial intelligence is essential for achieving general intelligence. For example, in medical diagnosis and treatment plan recommendations, doctors and patients need to understand and trust the decision basis of the system; in financial risk assessment and investment decision-making, financial institutions need to clarify the judgment logic of the system to meet regulatory requirements and internal audit standards; in the optimal decision-making of unmanned driving, algorithm designers need to clarify the decision logic of the driving strategy to ensure that the driving strategy is safe and reliable.

[0003] The input information that can be obtained in the multi-agent decision-making process includes the state characteristics of the agent to itself, the surrounding environment and the task goal, as well as the decision-making behavior of the agent. Explainable causal reasoning is expected to establish a causal relationship between the agent's behavior and the task goal. There are currently some methods of causal explanation, such as: clustering the decision-making behaviors of different agents using clustering methods, analyzing different categories of behavior, and giving the causal relationship of decisions. However, although different categories of decision-making behaviors can assist in analyzing the role of agents in the decision-making process, this method lacks clear causal relationships, resulting in unclear explainable causal relationships. Summary of the invention

[0004] The present invention provides an explainable analysis method, device, electronic device, non-transitory computer-readable storage medium and computer program product for multi-agent decision-making, so as to solve the defect of unclear causal relationship of decision-making in the prior art and achieve the effect of improving the interpretability of decision causal relationship.

[0005] The present invention provides an explainable analysis method for multi-agent decision making, comprising the following steps.

[0006] Obtain real-time status data and real-time decision data of multiple agents during task execution; Inputting the real-time state data into a pre-trained time series encoder, and converting the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network includes the causal relationship between each of the intelligent agents and between each of the intelligent agents and the task target; Determine, based on the real-time state data and the real-time decision data, the contribution of each state feature of each of the intelligent agents to the decision; A multi-granularity visualized decision path is generated according to the causal graph network and the contribution of each state feature of each intelligent agent to the decision.

[0007] According to an interpretable analysis method for multi-agent decision making provided by the present invention, the training steps of the pre-trained time series encoder include: Acquire multiple segments of sample state data; each segment of the sample state data contains state data of multiple sample agents in a time sequence; Dividing each segment of the sample state data into sequence segments including a plurality of preset time steps; In each round of iteration, a plurality of the sequence segments are selected and input into the time series encoder to be trained, a predicted causal graph network is output, the predicted causal graph network and the selected sequence segments are input into the decoder to be trained, and the predicted state data of the next moment corresponding to each of the selected sequence segments is output; According to the difference between the predicted state data at the next moment and the actual state data at the next moment in the sample state data, the parameters of the time series encoder to be trained and the decoder to be trained are adjusted to enter the next round of iteration until the iteration is stopped to obtain the trained time series encoder.

[0008] According to an interpretable analysis method for multi-agent decision making provided by the present invention, the obtaining of multiple segments of sample state data comprises: Perform multiple rounds of dynamic decision making on the multiple sample agents through a pre-trained multi-agent reinforcement learning model; Based on the state data of the multiple sample agents in each round of dynamic decision-making process, multiple sections of sample state data are obtained.

[0009] According to an interpretable analysis method for multi-agent decision-making provided by the present invention, the contribution of each state feature of each agent to the decision is determined based on the real-time state data and the real-time decision data, including: Taking each state feature in the real-time state data as a target state feature, and determining a state feature subset that can be composed of each state feature except the target state feature; Determine, according to the real-time decision data, a first decision corresponding to each of the state feature subsets, and a second decision corresponding to each of the state feature subsets plus the target state feature; According to the difference between the first decision and the second decision corresponding to each of the state feature subsets, the contribution of the target state feature to the decision is determined, and the contribution of each state feature of each of the intelligent agents to the decision is obtained.

[0010] According to an interpretable analysis method for multi-agent decision-making provided by the present invention, the contribution of the target state feature to the decision is determined based on the difference between the first decision and the second decision corresponding to each of the state feature subsets, including: For each of the state feature subsets, respectively, according to the difference between the corresponding first decision and the second decision, calculating the marginal contribution of the target state feature to the state feature subset; The weight of each of the state feature subsets is calculated, and according to the weight of each of the state feature subsets, the corresponding marginal contributions are weighted summed to obtain the contribution of the target state feature to the decision.

[0011] According to an interpretable analysis method for multi-agent decision-making provided by the present invention, generating a multi-granularity visualized decision path according to the contribution of each state feature of each agent to the decision-making, includes: According to the causal graph network, determining important intelligent agents; Determine the important state features corresponding to each of the intelligent agents according to the contribution of each state feature of each of the intelligent agents to the decision; A multi-granularity visual decision path is generated according to the important intelligent agents and the important state features.

[0012] The present invention also provides an interpretable analysis device for multi-agent decision making, comprising the following modules: The data acquisition module is used to obtain the real-time status data and real-time decision data of multiple agents during the execution of tasks; A coarse-grained causal discovery module is used to input the real-time state data into a pre-trained time series encoder, and convert the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each of the intelligent agents and between each of the intelligent agents and the task objectives; A fine-grained feature importance analysis module, used to determine the contribution of each state feature of each of the agents to the decision-making according to the real-time state data and the real-time decision data; A decision path visualization module is used to generate a multi-granularity visualization decision path according to the causal graph network and the contribution.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements an interpretable analysis method for multi-agent decision-making as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an explainable analysis method for multi-agent decision-making as described in any one of the above.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the explainable analysis method for multi-agent decision-making as described in any one of the above-mentioned methods.

[0016] The interpretable analysis method, device, electronic device, non-transitory computer-readable storage medium and computer program product of multi-agent decision-making provided by the present invention input the real-time state data of multiple agents in the process of executing tasks into a pre-trained time series encoder, and convert the real-time state data into a causal graph network through the pre-trained time series encoder. The causal graph network contains the causal relationship between each agent and between each agent and the task goal, so that it can reflect the influence of the agent on the decision in a coarse-grained manner, and then determine the contribution of each state feature of each agent to the decision based on the real-time state data and the real-time decision data. According to the causal graph network and the contribution of each state feature of each agent to the decision, a multi-granular visual decision path is generated, which can reflect the influence of the state characteristics of the agent on the decision in a fine-grained manner, realize multi-granular interpretable analysis of agent decision causality, and improve the interpretability of decision causality. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is one of the flow charts of the explainable analysis method for multi-agent decision-making provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the self-encoding process in the interpretable analysis method of multi-agent decision-making provided by the present invention.

[0020] Figure 3 This is the second flowchart of the explainable analysis method for multi-agent decision-making provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the structure of the interpretable analysis device for multi-agent decision-making provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Combine the following Figure 1-Figure 5 The present invention describes an interpretable analysis method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for multi-agent decision making.

[0025] Figure 1 This is one of the flow charts of the interpretable analysis method for multi-agent decision making provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 102, obtaining real-time status data and real-time decision data of multiple agents during task execution.

[0026] Among them, real-time state data refers to the real-time state data of the agent during the execution of the task. The state data may include the state of the agent itself, the surrounding environment, and the task objectives. Real-time decision data refers to the real-time decision data of the agent during the execution of the task. Decision data refers to the actions taken by the agent according to the state through the reinforcement learning algorithm.

[0027] Specifically, multiple intelligent agents perform tasks through reinforcement learning algorithms and obtain real-time status data and real-time decision data during the task execution process.

[0028] Step 104, input the real-time state data into a pre-trained time series encoder, and convert the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each intelligent agent and between each intelligent agent and the task goal.

[0029] Among them, the task objective can be one or more.

[0030] In one embodiment, the causal graph network can be dimensional matrix. N represents the number of agents. M represents the number of task objectives.

[0031] In one embodiment, the pre-trained time series encoder may be composed of a multi-layer attention Transformer network.

[0032] In one embodiment, during the multi-agent task execution process, real-time status data sequence fragments with a preset number of time steps can be input into a pre-trained time series encoder in real time, and the real-time status data sequence fragments can be converted into a causal graph network through the pre-trained time series encoder to obtain a real-time causal graph network that changes over time.

[0033] For example, if the preset number is 4, then each time, the real-time state data sequence fragments of 4 preset time steps are input into the pre-trained time series encoder. That is, the real-time state data of the current moment and the previous 3 moments are input into the pre-trained time series encoder, and the time interval between adjacent moments is equal to the preset time step.

[0034] In one embodiment, the pre-trained time series encoder can extract key features from the real-time status data and convert the real-time status data into a causal graph network according to the key features.

[0035] Step 106, based on the real-time state data and the real-time decision data, determine the contribution of each state feature of each agent to the decision.

[0036] Step 108, generating a multi-granularity visualized decision path according to the causal graph network and the contribution of each state feature of each intelligent agent to the decision.

[0037] In one embodiment, important intelligent agents can be determined based on the causal graph network, and the important state characteristics corresponding to each intelligent agent can be determined based on the contribution of each state characteristic of each intelligent agent to the decision. Based on the important intelligent agents and important state characteristics, a multi-granularity visual decision path can be generated.

[0038] The above-mentioned interpretable analysis method of multi-agent decision-making inputs the real-time state data of multiple agents in the process of executing tasks into a pre-trained time series encoder, and converts the real-time state data into a causal graph network through the pre-trained time series encoder. The causal graph network contains the causal relationship between each agent and between each agent and the task goal, so that the influence of the agent on the decision can be reflected in a coarse-grained manner. Then, according to the real-time state data and real-time decision data, the contribution of each state feature of each agent to the decision is determined. According to the causal graph network and the contribution of each state feature of each agent to the decision, a multi-granular visual decision path is generated, which can reflect the influence of the state feature of the agent on the decision in a fine-grained manner, realize the multi-granular interpretable analysis of the causal relationship of the agent decision, and improve the interpretability of the decision causality. In addition, the transparency of the decision-making process is also improved: by introducing the time dynamic causal graph network, the system can provide comprehensive agent causal relationships and clear decision paths, so that users can understand and verify the basis of each decision and enhance trust in the system. It can also enhance the system's explanatory power: using the time-dynamic causal graph network to extract important agents, display the decision path and basis, so that each decision process can be clearly explained and understood, thus providing a detailed explanation of the complex decision process. It can also enhance the decision-making ability of multi-agents: when the credit of agents is not correctly distributed and some agents in the team become lazy, the lazy agents learn suboptimal strategies instead of cooperating to achieve the common goals of the team. Multi-agent causal relationship estimation can be used to detect and punish lazy agents, thereby improving the overall effectiveness of multi-agents.

[0039] In one embodiment, the training steps of a pre-trained time series encoder include: obtaining multiple segments of sample state data; each segment of sample state data contains state data of multiple sample agents in a time series; each segment of sample state data is divided into sequence segments containing multiple preset time steps; in each round of iteration, multiple sequence segments are selected and input into the time series encoder to be trained, and a predicted causal graph network is output, and the predicted causal graph network and the selected sequence segments are input into the decoder to be trained, and the predicted state data of the next moment corresponding to each selected sequence segment is output; according to the difference between the predicted state data at the next moment and the actual state data at the next moment in the sample state data, the parameters of the time series encoder to be trained and the decoder to be trained are adjusted to enter the next round of iteration until the iteration is stopped to obtain a trained time series encoder.

[0040] The next moment refers to the moment next to the end moment of the time period corresponding to the input sequence segment.

[0041] In one embodiment, the time series encoder to be trained can be composed of a multi-layer attention Transformer network.

[0042] In one embodiment, the decoder to be trained may be composed of a recurrent neural network (RNN).

[0043] In one embodiment, the decoder may decode the state data in the sequence segment according to the context information of the sequence segment and in combination with the causal graph network to obtain the predicted state data at the next moment.

[0044] In one embodiment, S segments of sample state data are obtained, each segment of sample state data contains state data of N sample agents in a time series of T steps. Each segment of sample state data is divided into sequence segments containing multiple preset time steps. For example, if the preset time step t' is 4, the sample state data corresponding to time 1 to time 4, time 2 to time 5, and time 3 to time 6 in each segment of sample state data can be used as sequence segments. Time series encoder to be trained It can convert the input sequence fragment into a causal graph network . The predicted causal graph network The selected sequence fragment from time t to time t+t' is input into the decoder to be trained, and the predicted state data at time t+t' is output ,in Indicates the current interference factor.

[0045] In one embodiment, the total loss function for each round of iteration is:

[0046] Wherein, L represents the total loss function. s represents the order of the sample state data. S represents the number of segments of the sample state data. t represents the starting time of the selected sequence segment. T represents the total time length of each segment of the sample state data (i.e., the number of time steps included). t' represents the preset time step. Represents the loss function. Represents the actual state data at the next moment. A causal graph network representing the output of a time series encoder. Represents the selected sequence fragment. Represents the predicted state data of the next moment output by the decoder. Represents the sparse penalty, which is used to make the causal graph network as sparse as possible. Represents the parameters of the decoder to be trained. Represents the parameters of the time series encoder to be trained.

[0047] like Figure 2 As shown, it is a schematic diagram of the entire self-encoding process. Input to the time series encoder to be trained In the process, the causal graph network G is output, and then the causal graph network G and the selected sequence fragment are combined. Input to the decoder to be trained Output the predicted state data for the next moment .

[0048] In the above embodiment, the causal graph network is predicted by inputting the sequence fragments in the sample state data into the time series encoder to be trained, and then the predicted causal graph network and the sequence fragments are decoded by the decoder to obtain the predicted state data at the next moment, and then the time series encoder and decoder are iteratively trained based on the difference between the predicted state data at the next moment and the actual state data, and finally a time series encoder for generating a causal graph network can be obtained, which can achieve a coarse-grained reflection of the influence of the intelligent agent on the decision.

[0049] In one embodiment, obtaining multiple segments of sample state data includes: performing multiple rounds of dynamic decision-making on multiple sample agents through a pre-trained multi-agent reinforcement learning model; and obtaining multiple segments of sample state data based on the state data of the multiple sample agents in each round of dynamic decision-making.

[0050] Among them, the pre-trained multi-agent reinforcement learning model can use reinforcement learning algorithms such as MAPPO. The system gradually optimizes its decision-making process by continuously interacting with the environment to achieve the desired goal. The state data in each round of dynamic decision-making process corresponds to a piece of sample state data.

[0051] In the above embodiment, multiple rounds of dynamic decision-making are performed on multiple sample agents through a pre-trained multi-agent reinforcement learning model. Based on the state data of the multiple sample agents in each round of dynamic decision-making, multiple segments of sample state data can be obtained for training the time series encoder.

[0052] In one embodiment, based on real-time state data and real-time decision data, the contribution of each state feature of each intelligent agent to the decision is determined, including: taking each state feature in the real-time state data as a target state feature, and determining a state feature subset that can be composed of each state feature other than the target state feature; based on the real-time decision data, determining the first decision corresponding to each state feature subset, and the second decision corresponding to each state feature subset plus the target state feature; based on the difference between the first decision and the second decision corresponding to each state feature subset, determining the contribution of the target state feature to the decision, and obtaining the contribution of each state feature of each intelligent agent to the decision.

[0053] For example, the state features of the agent include A, B, C, and D. A is used as the target state feature. The state feature subsets that can be composed of the state features other than A include {}, {B}, {C}, {D}, {B,C}, {C,D}, {B,D}, and {B,C,D}. For the state feature subset {BC}, the corresponding first decision is f({B,C}), and the corresponding second decision after adding the target state feature A is f({A,B,C}).

[0054] In one embodiment, if there is no decision corresponding to the state feature subset in the real-time decision data, the state data corresponding to the state feature subset is input into the multi-agent reinforcement learning model to obtain the corresponding decision.

[0055] In the above embodiment, each state feature in the real-time state data is used as the target state feature, and a state feature subset that can be composed of each state feature except the target state feature is determined. According to the real-time decision data, the first decision corresponding to each state feature subset and the second decision corresponding to each state feature subset plus the target state feature are determined. According to the difference between the first decision and the second decision corresponding to each state feature subset, the contribution of the target state feature to the decision is determined, and the contribution of each state feature of each intelligent agent to the decision can be accurately obtained, thereby reflecting the influence of the state characteristics of the intelligent agent on the decision in a fine-grained manner.

[0056] In one embodiment, the contribution of the target state feature to the decision is determined based on the difference between the first decision and the second decision corresponding to each state feature subset, including: for each state feature subset, according to the difference between the corresponding first decision and the second decision, calculating the marginal contribution of the target state feature to the state feature subset; calculating the weight of each state feature subset, and according to the weight of each state feature subset, performing weighted summation on the corresponding marginal contributions to obtain the contribution of the target state feature to the decision.

[0057] In one embodiment, the weight of the state feature subset may be calculated according to the total number of features and the number of features in the state feature subset.

[0058] In the above embodiment, for each state feature subset, the marginal contribution of the target state feature to the state feature subset is calculated according to the difference between the corresponding first decision and the second decision, and the weight of each state feature subset is calculated. According to the weight of each state feature subset, the corresponding marginal contributions are weighted summed to obtain the contribution of the target state feature to the decision, which can accurately obtain the contribution of each state feature of each intelligent agent to the decision, thereby reflecting the influence of the state characteristics of the intelligent agent on the decision in a fine-grained manner.

[0059] In one embodiment, a multi-granularity visual decision path is generated based on the causal graph network and the contribution of each state feature of each agent to the decision, including: determining important agents based on the causal graph network; determining the important state features corresponding to each agent based on the contribution of each state feature of each agent to the decision; and generating a multi-granularity visual decision path based on the important agents and the important state features.

[0060] Among them, important agents refer to agents that have a significant impact on achieving task goals. Important state features refer to state features that have a significant impact on the agent's achievement of task goals.

[0061] In one embodiment, important agents may be highlighted in a multi-granularity visualization of decision paths.

[0062] In one embodiment, a decision tree corresponding to each agent can be generated according to the important state features corresponding to each agent, and the decision tree corresponding to each agent can be displayed in a multi-granularity visual decision path.

[0063] For example, if the important state feature of an intelligent agent is speed, the corresponding decision tree can be: if the speed is greater than the preset threshold, go left; if the speed is less than or equal to the preset threshold, go right.

[0064] In the above embodiment, important intelligent agents are determined based on the causal graph network, and the important state characteristics corresponding to each intelligent agent are determined based on the contribution of each state characteristic of each intelligent agent to the decision; multi-granularity visual decision paths are generated based on the important intelligent agents and important state characteristics, which realizes multi-granularity interpretable analysis of intelligent agent decision causality and improves the interpretability of decision causality.

[0065] like Figure 3 FIG. 2 is a flow chart of the interpretable analysis method for multi-agent decision-making provided by the present invention. The method includes the following steps: Step 302, multi-agent reinforcement learning decision generation.

[0066] Step 304, self-encoded agent causal graph network generation.

[0067] Step 306: Extract fine-grained key features for feature contribution analysis.

[0068] Step 308: Multi-agent decision-making multi-granularity visualization.

[0069] The interpretable analysis device for multi-agent decision-making provided by the present invention is described below. The interpretable analysis device for multi-agent decision-making described below and the interpretable analysis method for multi-agent decision-making described above can be referenced to each other.

[0070] like Figure 4 As shown, a multi-agent decision-making interpretable analysis device 400 is provided, comprising the following modules: The data acquisition module 402 is used to acquire real-time status data and real-time decision data of multiple agents during the execution of tasks.

[0071] The coarse-grained causal discovery module 404 is used to input the real-time state data into a pre-trained time series encoder, and convert the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each intelligent agent and between each intelligent agent and the task goal.

[0072] The fine-grained feature importance analysis module 406 is used to determine the contribution of each state feature of each agent to the decision based on the real-time state data and the real-time decision data.

[0073] The decision path visualization module 408 is used to generate a multi-granularity visualization decision path based on the causal graph network and contributions.

[0074] In one embodiment, the data acquisition module 402 is also used to acquire multiple segments of sample state data; each segment of sample state data contains state data of multiple sample agents in a time sequence; and each segment of sample state data is divided into sequence segments containing multiple preset time steps.

[0075] The coarse-grained causal discovery module 404 is also used to select multiple sequence segments in each round of iteration and input them into the time series encoder to be trained, output a predicted causal graph network, input the predicted causal graph network and the selected sequence segments into the decoder to be trained, and output the predicted state data of the next moment corresponding to each selected sequence segment; according to the difference between the predicted state data at the next moment and the actual state data at the next moment in the sample state data, adjust the parameters of the time series encoder to be trained and the decoder to be trained, enter the next round of iteration, and stop the iteration to obtain a trained time series encoder.

[0076] In one embodiment, the data acquisition module 402 is also used to perform multiple rounds of dynamic decision-making on multiple sample agents through a pre-trained multi-agent reinforcement learning model; and obtain multiple segments of sample state data based on the state data of the multiple sample agents in each round of dynamic decision-making.

[0077] In one embodiment, the fine-grained feature importance analysis module 406 is also used to take each state feature in the real-time state data as the target state feature, and determine the state feature subsets that can be composed of each state feature except the target state feature; according to the real-time decision data, determine the first decision corresponding to each state feature subset, and the second decision corresponding to each state feature subset plus the target state feature; according to the difference between the first decision and the second decision corresponding to each state feature subset, determine the contribution of the target state feature to the decision, and obtain the contribution of each state feature of each intelligent agent to the decision.

[0078] In one embodiment, the fine-grained feature importance analysis module 406 is also used to calculate the marginal contribution of the target state feature to the state feature subset according to the difference between the corresponding first decision and the second decision for each state feature subset respectively; calculate the weight of each state feature subset, and perform weighted summation on the corresponding marginal contributions according to the weights of each state feature subset to obtain the contribution of the target state feature to the decision.

[0079] In one embodiment, the decision path visualization module 408 is also used to determine important intelligent agents based on the causal graph network; determine the important state characteristics corresponding to each intelligent agent based on the contribution of each state characteristic of each intelligent agent to the decision; and generate a multi-granularity visualized decision path based on the important intelligent agents and important state characteristics.

[0080] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the interpretable analysis method of multi-agent decision-making, which includes: obtaining real-time state data and real-time decision data of multiple agents in the process of executing tasks; inputting the real-time state data into a pre-trained time series encoder, and converting the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network includes the causal relationship between each agent and between each agent and the task target; according to the real-time state data and the real-time decision data, determining the contribution of each state feature of each agent to the decision; generating a multi-granularity visual decision path according to the contribution of the causal graph network and each state feature of each agent to the decision.

[0081] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the interpretable analysis method of multi-agent decision-making provided by the above-mentioned methods, which includes: obtaining real-time status data and real-time decision data of multiple agents in the process of executing tasks; inputting the real-time status data into a pre-trained time series encoder, and converting the real-time status data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each agent and between each agent and the task goal; based on the real-time status data and the real-time decision data, determining the contribution of each state feature of each agent to the decision; generating a multi-granularity visual decision path based on the causal graph network and the contribution of each state feature of each agent to the decision.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the explainable analysis method of multi-agent decision-making provided by the above-mentioned methods, the method comprising: obtaining real-time status data and real-time decision data of multiple agents in the process of executing tasks; inputting the real-time status data into a pre-trained time series encoder, and converting the real-time status data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each agent and between each agent and the task goal; based on the real-time status data and the real-time decision data, determining the contribution of each state feature of each agent to the decision; generating a multi-granularity visual decision path based on the causal graph network and the contribution of each state feature of each agent to the decision.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for interpretable analysis of multi-agent decision making, characterized in that: include: Obtain real-time status data and real-time decision data of multiple agents during task execution; Inputting the real-time state data into a pre-trained time series encoder, and converting the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network includes the causal relationship between each of the intelligent agents and between each of the intelligent agents and the task target; Determine, based on the real-time state data and the real-time decision data, the contribution of each state feature of each of the intelligent agents to the decision; A multi-granularity visual decision path is generated according to the causal graph network and the contribution of each state feature of each intelligent agent to the decision.

2. The interpretable analysis method for multi-agent decision making according to claim 1 is characterized in that: The training steps of the pre-trained time series encoder include: Acquire multiple segments of sample state data; each segment of the sample state data contains state data of multiple sample agents in a time sequence; Dividing each segment of the sample state data into sequence segments including a plurality of preset time steps; In each round of iteration, a plurality of the sequence segments are selected and input into the time series encoder to be trained, a predicted causal graph network is output, the predicted causal graph network and the selected sequence segments are input into the decoder to be trained, and the predicted state data at the next moment corresponding to each of the selected sequence segments is output; According to the difference between the predicted state data at the next moment and the actual state data at the next moment in the sample state data, the parameters of the time series encoder to be trained and the decoder to be trained are adjusted to enter the next round of iteration until the iteration is stopped to obtain the trained time series encoder.

3. The interpretable analysis method for multi-agent decision making according to claim 2 is characterized in that: The obtaining of multiple sections of sample status data comprises: Perform multiple rounds of dynamic decision making on the multiple sample agents through a pre-trained multi-agent reinforcement learning model; Based on the state data of the multiple sample agents in each round of dynamic decision-making process, multiple sections of sample state data are obtained.

4. The interpretable analysis method for multi-agent decision making according to claim 1, characterized in that: Determining the contribution of each state feature of each of the agents to the decision according to the real-time state data and the real-time decision data includes: Taking each state feature in the real-time state data as a target state feature, and determining a state feature subset that can be composed of each state feature except the target state feature; Determine, according to the real-time decision data, a first decision corresponding to each of the state feature subsets, and a second decision corresponding to each of the state feature subsets plus the target state feature; According to the difference between the first decision and the second decision corresponding to each of the state feature subsets, the contribution of the target state feature to the decision is determined, and the contribution of each state feature of each of the intelligent agents to the decision is obtained.

5. The interpretable analysis method for multi-agent decision making according to claim 4 is characterized in that: Determining the contribution of the target state feature to the decision according to the difference between the first decision and the second decision respectively corresponding to each of the state feature subsets includes: For each of the state feature subsets, respectively, according to the difference between the corresponding first decision and the second decision, calculating the marginal contribution of the target state feature to the state feature subset; The weight of each of the state feature subsets is calculated, and according to the weight of each of the state feature subsets, the corresponding marginal contributions are weighted summed to obtain the contribution of the target state feature to the decision.

6. The interpretable analysis method for multi-agent decision making according to any one of claims 1 to 5, characterized in that: The step of generating a multi-granularity visualized decision path according to the causal graph network and the contribution of each state feature of each intelligent agent to the decision making comprises: According to the causal graph network, determining important intelligent agents; Determine the important state features corresponding to each of the intelligent agents according to the contribution of each state feature of each of the intelligent agents to the decision; A multi-granularity visual decision path is generated according to the important intelligent agents and the important state features.

7. An interpretable analysis device for multi-agent decision making, characterized in that: include: The data acquisition module is used to obtain the real-time status data and real-time decision data of multiple agents during the execution of tasks; A coarse-grained causal discovery module is used to input the real-time state data into a pre-trained time series encoder, and convert the real-time state data into a causal graph network through the pre-trained time series encoder; the causal graph network contains the causal relationship between each of the intelligent agents and between each of the intelligent agents and the task objectives; A fine-grained feature importance analysis module, used to determine the contribution of each state feature of each of the agents to the decision-making according to the real-time state data and the real-time decision data; A decision path visualization module is used to generate a multi-granularity visualization decision path according to the causal graph network and the contribution.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the explainable analysis method for multi-agent decision-making as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the explainable analysis method for multi-agent decision-making as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the explainable analysis method for multi-agent decision-making as described in any one of claims 1 to 6.

Images