Weight distribution method and system based on Shapley value under multi-source perception

Abstract

The invention discloses a weight distribution method and system based on a Shapley value under multi-source perception, and the method comprises the following steps: S1, integrating an image, voice, information data and a radar signal through a fusion function, and generating a comprehensive state representation; s2, the intelligent agent selects and executes an action based on a current strategy and exploration noise to obtain a global reward and updated multi-modal data; the experience tuple is stored in a playback buffer area, and slow updating is carried out through target network parameters; s3, defining a Shapril Q value for each agent, and realizing global revenue maximization by maximizing the Shapril Q value of each agent; adopting a depth deterministic strategy gradient (DDPG) to optimize a strategy network, calculating a modal contribution degree in combination with random sampling approximation, and updating Critic network parameters; s4, updating Critic network parameters and synchronizing target network parameters by adopting a soft update mechanism; according to the invention, the performance and reliability of the multi-source sensing system are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a weight distribution method and system based on Shapley values ​​under multi-source perception. Background Art

[0002] In modern environments, multi-source perception systems are widely used for target detection, recognition, and tracking. These different sensors can provide different types of data, forming a multi-level and multi-dimensional information network. However, how to effectively integrate these heterogeneous data and assign weights to optimize the decision-making process is a complex and challenging problem. In a multi-source perception environment, existing methods have many limitations in weight assignment, such as fixed weights, single modality dependence, and lack of interpretability. Summary of the invention

[0003] In order to solve the above problems in the prior art, the present invention proposes a weight allocation method and system based on Shapley value under multi-source perception, which takes Shapley value as the basis of weight allocation and utilizes its advantages in feature importance evaluation and multi-agent system to realize a more flexible, adaptive and transparent weight allocation method.

[0004] The weight allocation method based on Shapley value under multi-source perception described in the present invention comprises the following steps: S1: Multimodal feature fusion and initialization: Integrate images, voice, information data, and radar signals through fusion functions to generate comprehensive state representations; at the beginning of each round of training, initialize the action exploration mechanism through a random process and receive initial multimodal data; S2: Dynamic strategy execution and experience storage: The agent selects and executes actions based on the current strategy and exploration noise, obtains global rewards and updated multimodal data; stores experience tuples in the playback buffer, and achieves strategy stability through slow updates of target network parameters; S3: Shapley value driven policy optimization: define the Shapley Q value for each agent, and maximize the global benefit by maximizing the Shapley Q value of each agent; use deep deterministic policy gradient DDPG to optimize the policy network, combine random sampling to approximate the modal contribution, and update the critic network parameters; S4: Global-local reward collaborative update: Update the critic network parameters by minimizing the difference between the global reward and the individual Shapley Q value, and use a soft update mechanism to synchronize the target network parameters.

[0005] Further, step S1 includes the following steps: S1-1: Initialize the random process N and configure the noise distribution for action exploration; S1-2: Receive initial multimodal data, including images ,voice , Zhixin Data , radar signal ; S1-3: Through the fusion function Generate an initial composite state.

[0006] Further, step S2 includes the following steps: S2-1: Agent i performs actions ,in, , get a new state and reward r; S2-2: Storage Experience to playback buffer B, where X and They are the current and updated comprehensive status respectively; S2-3: Sample M samples from B , calculate the target Q value , and update the Critic network by gradient descent.

[0007] Further, step S3 includes the following steps: S3-1: Initialize Actor network parameters , Critic network parameters , set the target network parameters , and learning rate ; S3-2: Select actions based on current strategy , observe the global reward after execution and transfer status , store experience in the buffer; S3-3: For each sample , calculate the action set and target action set ; S3-4: Shield irrelevant modal actions by random sampling to generate a set of shielded actions , ; S3-5: Approximate calculation of Shapley's Q value , and use deterministic policy gradients to update the Actor network parameters .

[0008] Further, step S4 includes the following steps: S4-1: Calculate target value ; S4-2: By minimizing the loss function Update Critic parameters; S4-3: Using soft update rules , Synchronize target network parameters.

[0009] The present invention provides a weight allocation system based on Shapley value under multi-source perception, comprising: Multimodal module: used to integrate images, voice, information data, and radar signals through fusion functions to generate comprehensive state representations; at the beginning of each round of training, the action exploration mechanism is initialized through a random process and the initial multimodal data is received; Strategy module: used by the agent to select and execute actions based on the current strategy and exploration noise, obtain global rewards and updated multimodal data; store experience tuples in the playback buffer, and achieve strategy stability through slow updates of target network parameters; Shapley value module: used to define the Shapley Q value for each agent, and maximize the global benefit by maximizing the Shapley Q value of each agent; use deep deterministic policy gradient DDPG to optimize the policy network, combine random sampling to approximate the modal contribution, and update the Critic network parameters; Update module: used to update the Critic network parameters by minimizing the difference between the global reward and the individual Shapley Q value, and synchronize the target network parameters using a soft update mechanism and .

[0010] Furthermore, in the multimodal module, the following is done: Initialize the random process N and configure the noise distribution for action exploration; Receive the initial multimodal data, including images ,voice , Zhixin Data , radar signal ; Through the fusion function Generate an initial composite state.

[0011] Furthermore, the strategy module includes: Agent i performs actions ,in, , get a new state and reward r; storage experience to playback buffer B, where X and are the current and updated comprehensive states respectively; sample M samples from B and calculate the target Q value , and update the Critic network by gradient descent.

[0012] Furthermore, the Shapley value module includes: Initializing Actor network parameters , Critic network parameters , set the target network parameters , and learning rate ; Select actions based on the current strategy , observe the global reward after execution and transfer status , store the experience in the buffer; for each sample , calculate the action set and target action set ; Shield irrelevant modal actions by random sampling and generate a set of shielded actions , ; Approximate calculation of Shapley Q value , and use deterministic policy gradients to update the Actor network parameters .

[0013] Furthermore, the update module includes: calculating the target value ; By minimizing the loss function Update Critic parameters; use soft update rules , Synchronize target network parameters.

[0014] Beneficial effects: Compared with the prior art, the present invention has the following advantages: through dynamic adaptation, comprehensive performance improvement, transparency and explainability, multi-modal synergy, flexibility and scalability, and efficient computing, the performance and reliability of the multi-source perception system are significantly improved. These beneficial effects make this method an important technical means for future battlefield decision support and have broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the intelligent agent training process of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further explained and illustrated below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the embodiments are only used to illustrate and explain the present invention, and do not impose any limitation on the scope of implementation of the present invention.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a weight allocation method based on Shapley value under multi-source perception, comprising the following steps: Step 1: Feature extraction, identify and extract key information through images ,voice , Zhixin Data , radar signal The fusion function f is used to construct a comprehensive state representation to enhance the agent's environmental perception ability. At the beginning of each episode, the action exploration mechanism is initialized through a random process to ensure the exploratory nature of the strategy. The following steps are included: S1-1: Initialize a random process N, initialize a random process For motion exploration, accepting initial multimodal data; S1-2: Receive initial multimodal data, including images ,voice , Zhixin Data , radar signal ; S1-3: Through the fusion function Generate an initial composite state.

[0018] Step 2: At each time step t, agent i calculates the current strategy based on the exploration noise. Select Action , after execution, the reward r and new multimodal data are obtained, and the state s is further updated. Stored in the revisit buffer B, providing samples for the learning process, the target network parameters By slowly tracking the current network parameters Update. This includes the following steps: S2-1: Agent i chooses ,in, , obtain reward r and new multimodal data image ,voice , Zhixin , radar signal , use the fusion function f to update the state ; S2-2: Set the new comprehensive perception state as the current state , storage experience to playback buffer B, where X and They are the current and updated comprehensive status respectively; S2-3: Sample M samples from B , calculate the target Q value , and update the Critic network by gradient descent.

[0019] Step 3: Combined with the approximate method of Shapley Q value, each agent in the multi-agent system needs to maximize its own Shapley value , the global benefit can be maximized Now the problem is transformed into solving the problem for each agent. Deep deterministic policy gradient provides a powerful tool for optimizing individual strategies in multi-agent systems. Traditional discrete methods are difficult to apply. Therefore, by optimizing the Shapley Q value, deterministic policy gradient can guide each agent to adjust its strategy and maximize its expected return. It includes the following steps: Step 3-1. Initialize actor parameters , critic parameter , for each modal agent , initialize the sample size M of the approximate Shapley Q value, the learning rate , discount factor , target actor parameters , target critic parameter .

[0020] Step 3-2: Observe the initial state from the environment , select actions based on agent i’s current strategy and exploration noise , perform the action And observe the global rewards and the next state ,Will Stored in revisit buffer B.

[0021] Step 3-3: For each sample Available and .

[0022] Step 3-4: Sample M multimodal data, according to For each , , Sort, shield the actions of irrelevant agents, and store them in , , .

[0023] Steps 3-5: For each sample , can be obtained , updated with deterministic policy gradient .

[0024] Step 4: The Shapley Q value of each agent is related to the local reward, and each agent cannot be updated directly through the global reward. , then a minimization problem is used to combine the global reward and individual contribution. It includes the following steps: S4-1: For each sample , calculate the target value ; S4-2: By minimizing the loss function Update Critic parameters; S4-3: Using soft update rules , Synchronize target network parameters.

Claims

1. A weight allocation method based on Shapley value under multi-source perception, characterized in that: The following steps are involved: S1: Multimodal feature fusion and initialization: Integrate images, voice, information data, and radar signals through fusion functions to generate comprehensive state representations; at the beginning of each round of training, initialize the action exploration mechanism through a random process and receive initial multimodal data; S2: Dynamic strategy execution and experience storage: The agent selects and executes actions based on the current strategy and exploration noise, obtains global rewards and updated multimodal data; stores experience tuples in the playback buffer and slowly updates them through the target network parameters; S3: Shapley value driven policy optimization: define the Shapley Q value for each agent, and maximize the global benefit by maximizing the Shapley Q value of each agent; use deep deterministic policy gradient DDPG to optimize the policy network, combine random sampling to approximate the modal contribution, and update the critic network parameters; S4: Global-local reward collaborative update: Update the critic network parameters by minimizing the difference between the global reward and the individual Shapley Q value, and use a soft update mechanism to synchronize the target network parameters.

2. The weight allocation method based on Shapley value under multi-source perception according to claim 1 is characterized in that: Step S1 includes the following steps: S1-1: Initialize the random process N and configure the noise distribution for action exploration; S1-2: Receive initial multimodal data, including images ,voice , Zhixin Data , radar signal ; S1-3: Through the fusion function Generate an initial composite state.

3. The weight allocation method based on Shapley value in multi-source perception according to claim 1 is characterized in that: Step S2 includes the following steps: S2-1: Agent i performs actions ,in, , get a new state And reward r; Among them, the updated multimodal data is image ,voice , Zhixin , radar signal ; S2-2: Store experience To playback buffer B, where X and They are the current and updated comprehensive status respectively; S2-3: Sample M samples from B , calculate the target Q value , and update the Critic network by gradient descent.

4. The weight allocation method based on Shapley value in multi-source perception according to claim 1 is characterized in that: Step S3 includes the following steps: S3-1: Initialize Actor network parameters , Critic network parameters , set the target network parameters , , and learning rate ; S3-2: Select actions based on current strategy , observe the global reward after execution and transfer status , store the experience in buffer B; S3-3: For each sample , calculate the action set and target action set ; S3-4: Shield irrelevant modal actions by random sampling to generate a set of shielded actions , ; S3-5: Approximate calculation of Shapley's Q value , and use deterministic policy gradients to update the Actor network parameters .

5. The weight allocation method based on Shapley value in multi-source perception according to claim 1 is characterized in that: Step S4 includes the following steps: S4-1: Calculate target value ; S4-2: By minimizing the loss function Update Critic parameters; S4-3: Using soft update rules , Synchronize target network parameters.

6. A weight allocation system based on Shapley value under multi-source perception, characterized in that: include: Multimodal module: used to integrate images, voice, information data, and radar signals through fusion functions to generate comprehensive state representations; at the beginning of each round of training, the action exploration mechanism is initialized through a random process and the initial multimodal data is received; Strategy module: used by the agent to select and execute actions based on the current strategy and exploration noise, and obtain global rewards and updated multimodal data; The experience tuple is stored in the replay buffer and slowly updated by the target network parameters; Shapley value module: used to define the Shapley Q value for each agent, and maximize the global benefit by maximizing the Shapley Q value of each agent; use deep deterministic policy gradient DDPG to optimize the policy network, combine random sampling to approximate the modal contribution, and update the Critic network parameters; Update module: used to update the Critic network parameters by minimizing the difference between the global reward and the individual Shapley Q value, and synchronize the target network parameters using a soft update mechanism.

7. The weight allocation system based on Shapley value under multi-source perception according to claim 6, characterized in that: In the multimodal module, the following is done: Initialize the random process N and configure the noise distribution for action exploration; Receive the initial multimodal data, including images ,voice , Zhixin Data , radar signal ; Through the fusion function Generate an initial composite state.

8. The weight allocation system based on Shapley value under multi-source perception according to claim 6, characterized in that: The strategy module includes: Agent i performs actions ,in, , get a new state and reward r; where, the updated multimodal data is image ,voice , Zhixin , radar signal ; Storage experience To playback buffer B, where X and are the current and updated comprehensive states respectively; sample M samples from B and calculate the target Q value , and update the Critic network by gradient descent.

9. The weight allocation system based on Shapley value under multi-source perception according to claim 6, characterized in that: The Shapley value module includes: Initializing Actor network parameters , Critic network parameters , set the target network parameters , and learning rate ; Select actions based on the current strategy , observe the global reward after execution and transfer status , store the experience in the buffer; for each sample , calculate the action set and target action set ; Shield irrelevant modal actions by random sampling and generate a set of shielded actions , ; Approximate calculation of Shapley Q value , and use deterministic policy gradients to update the Actor network parameters .

10. The weight allocation system based on Shapley value under multi-source perception according to claim 6, characterized in that: The update module includes: calculating the target value ; By minimizing the loss function Update Critic parameters; use soft update rules , Synchronize target network parameters.

Images