Multi-agent decision-making method and system based on dynamic scene perception

Abstract

The application provides a multi-agent decision-making method and system based on dynamic scene perception, and relates to the technical field of agent decision-making.The application regards each tenant as an agent, obtains time sequence data of each agent, and analyzes the time sequence data to obtain hidden states and communication characteristics.The communication characteristics and the hidden states of each agent are input into a pre-trained intention reasoning network to obtain a predicted governance action of each agent relative to any agent.The predicted governance action of each agent on itself constitutes an initial governance scheme, which is input into an action value network to obtain a global joint value of the initial governance scheme.The type of the agent is determined.Based on the global joint value and the type of the agent, the governance action of each agent in the initial governance scheme is adjusted, and the adjusted initial governance scheme is taken as a decision-making scheme at a current time.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent decision-making technology, specifically to a multi-agent decision-making method and system based on dynamic scene perception. Background Technology

[0002] With the continuous development of cloud computing and multi-tenant information systems, multiple tenants often coexist on the same cloud platform. These tenants exhibit significant differences in access behavior, data usage, computing resource consumption, and security risks, and these states continuously change as business operations progress. To ensure the stable and secure operation of the cloud platform, it is necessary to dynamically determine corresponding governance actions based on the operational status of each tenant at different times. Existing governance methods typically process different tenants separately based on currently collected single state information or preset rules. This makes it difficult to fully consider the tenant's historical state and the impact of previous governance actions on current state changes. Consequently, in dynamic scenarios where tenant states are constantly changing, the ability to represent the true governance needs of tenants is insufficient.

[0003] Furthermore, in a multi-tenant environment, tenants are not entirely independent; their resource consumption, security risks, and governance behaviors can collectively affect the overall operational status of the cloud platform. Existing technologies, when determining governance schemes for each tenant, often lack effective analysis of the governance actions and interrelationships between different tenants. This can easily lead to inconsistencies between the governance actions of a single tenant and the state changes of other tenants or the overall governance goals. In particular, when a tenant's governance action deviates abruptly from its historical actions and other tenants' assessments of its governance trends, directly adopting that action may reduce the overall coordination of the governance scheme. Therefore, how to comprehensively utilize the temporal states of each tenant, historical governance actions, and interaction information between agents in dynamic multi-tenant scenarios to collaboratively predict and comprehensively evaluate governance actions, and adaptively adjust inconsistent governance actions, is a technical problem that existing technologies urgently need to solve. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-agent decision-making method and system based on dynamic scene perception to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A multi-agent decision-making method based on dynamic scene perception, comprising the following steps: S1: Treat each tenant as an intelligent agent, obtain the time-series data of each intelligent agent's tenant behavior, data access, resource operation and security status to form the local state of each intelligent agent, and obtain the hidden state and communication characteristics of the current intelligent agent based on the local state at the current moment and the governance action at the previous moment. S2: Input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. S3: For any agent, the predicted governance actions of each agent to itself constitute the initial governance scheme. The hidden state, communication characteristics and governance actions of each agent in the initial governance scheme are input into the action value network to obtain the global joint value of the initial governance scheme. S4: Based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, and combined with the predicted governance actions of other agents on the current agent, the agents are divided into regular agents, uncoordinated mutation agents, or general agents. S5: Keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the types of intelligent agents, adjust the governance actions of each incoordination mutation intelligent agent in the initial governance plan, and use the adjusted initial governance plan as the decision plan for the current moment.

[0006] Furthermore, the time-series data of tenant behavior includes access frequency, operation anomaly rate, and number of permission requests; The time-series data on data access includes the percentage of access to sensitive data; Time-series data on resource operation includes load computing power and storage capacity; The time-series data on security status includes the access rate of high-risk IPs and the number of intrusions.

[0007] The time-series data of each dimension are associated and aggregated according to the tenant identifier to form the local state of the corresponding tenant agent at the current decision moment.

[0008] Furthermore, the governance actions include no processing, risk processing, permission processing, and agent processing; Risk handling includes account freezing and data blocking; permission handling includes raising and lowering permission levels; and agent handling includes increasing and decreasing the computing power by a preset amount.

[0009] The logic for obtaining the hidden state and communication characteristics of the current agent is as follows: For any tenant agent at any decision time, the local state of the tenant agent and the governance action executed by the tenant agent at the previous time are input into the multi-layer fully connected network to obtain the intermediate features at the current time. The intermediate features and the hidden state at the previous time are input into a preset gated loop unit to output the hidden state. Specifically, at the initial decision time, the hidden state of the previous time is set to the preset initial hidden state, and the governance action of the previous time is set to the preset initial governance action, wherein the initial governance action is to do nothing; at each subsequent decision time, the hidden state actually obtained and the governance action actually executed at the previous decision time are used to participate in the state update of the current time. The local state at the current moment and the governance action at the previous moment are input into the pre-trained supernetwork to obtain communication features.

[0010] Furthermore, the logic for obtaining the global joint value is as follows: The hidden state, communication characteristics, and predicted governance actions of each agent are input into a multi-layer fully connected network to generate the individual action value of each agent. The individual action value of each agent is calculated through a nonlinear mixing function in the hybrid network to obtain the global joint value. The hybrid network is trained using a multi-agent reinforcement learning algorithm that minimizes temporal difference loss.

[0011] Furthermore, the types of intelligent agents include conventional intelligent agents, uncoordinated mutant intelligent agents, and general intelligent agents.

[0012] The logic for determining the type of intelligent agent is as follows: For each predictive governance action of each agent, the number of times other agents infer that the agent will execute the predictive governance action is counted, and the predictive governance action with the most occurrences is taken as the baseline predictive governance action. The difference in the number of each of the remaining predictive governance actions relative to the baseline predictive governance action is calculated. If the difference is less than a preset number threshold, the predictive governance action and the baseline predictive governance action are taken together as candidate predictive governance actions. The preset number threshold is no greater than 3% of the total number of agents. For any agent at the current moment: If there is an intersection between the candidate predicted governance action and the governance action at the current moment, and the intersection is the governance action of the agent in the initial governance plan, then the agent is called a regular agent. If the candidate predicted governance action does not intersect with the governance action at the current moment, and the governance action of the agent in the initial governance scheme also does not intersect with the candidate predicted governance action, then the agent is called an incoherent mutation agent. If an agent is neither a conventional agent nor an uncoordinated mutant agent, then the agent is a general agent.

[0013] Furthermore, the logic of S5 is as follows: For any incoherent mutant agent, the number of predicted governance actions performed by the incoherent mutant agent is inferred from the statistics of the regular agent, and the predicted governance action with the most executions is taken as the regular baseline predicted governance action. The difference in the number of the remaining predicted governance actions relative to the regular baseline predicted governance action is calculated. If the difference is less than a preset number threshold, the predicted governance action and the regular baseline predicted governance action are taken together as candidate calibration predicted governance actions. For any incoherent mutant agent, count the predicted governance actions of all regular agents for that incoherent mutant agent to form a set of regular predicted governance actions. If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the incoherent mutant agent in the governance scheme do not intersect with the candidate calibration predicted governance actions, then the governance actions in the initial scheme will be used as the predicted governance actions to be adjusted for the agent. If the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutant agent in the governance scheme intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection are taken as the predicted governance actions to be adjusted. If the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutation agent in the governance scheme also do not intersect with the candidate calibration predicted governance actions, then all predicted governance actions of the agent are taken as predicted governance actions to be adjusted. If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the agent in the governance scheme also intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection of the two are taken as the predicted governance actions to be adjusted. Keep the governance actions of regular agents and general agents unchanged in the initial governance scheme; for each incoordination mutation agent, select an action from its predicted governance actions to be adjusted, and replace the original governance action of the agent in the initial governance scheme to obtain the updated governance scheme; calculate the global joint value of each updated governance scheme, and select the governance scheme with the highest global joint value as the decision scheme at the current moment.

[0014] The present invention further provides a multi-agent decision-making system based on dynamic scene perception, the system being used to implement the above-mentioned multi-agent decision-making method based on dynamic scene perception, specifically including: The state construction module is used to treat each tenant as an agent and obtain time-series data on the tenant behavior, data access, resource operation and security status of each agent to form the local state of each agent. Based on the local state at the current moment and the governance action at the previous moment, the hidden state and communication characteristics of the current agent are obtained. The action prediction module is used to input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. The value analysis module is used to construct an initial governance scheme from the predicted governance actions of each agent for any given agent. The hidden states, communication characteristics, and governance actions of each agent in the initial governance scheme are input into the action value network to obtain the global joint value of the initial governance scheme. The category determination module is used to classify agents into regular agents, uncoordinated mutation agents, or general agents based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, combined with the predicted governance actions of other agents on the current agent. The scheme adjustment module is used to keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the type of intelligent agent, it adjusts the governance actions of each uncoordinated and mutated intelligent agent in the initial governance scheme and uses the adjusted initial governance scheme as the decision scheme at the current moment.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention treats each tenant as an intelligent agent, constructing the local state of each agent by acquiring time-series data across four dimensions: tenant behavior, data access, resource operation, and security status. It then combines this with the governance actions from the previous moment to obtain the hidden state and communication characteristics of the current agent. This allows for the simultaneous representation of dynamic scenarios using the current operating state, historical states, and historical governance behaviors. Compared to governance decisions based solely on the current moment or a single dimension of information, this invention more fully reflects the characteristics of each tenant's state changing over time, providing a more complete foundation of state information for determining governance actions in dynamic multi-tenant environments.

[0016] This invention uses the communication characteristics and hidden states of each agent to infer governance actions, obtaining mutual predictions of governance actions among agents. It then calculates the global joint value based on the initial governance scheme formed by these predictions. Furthermore, it determines the agent type by combining the current predicted governance action, the previous time step's governance action, and the general agent's prediction of the current agent's governance action. Based on the agent type and the global joint value, it adjusts the governance actions in the initial governance scheme. This approach allows for consideration of the governance needs of individual tenants while simultaneously ensuring governance consistency and overall governance effectiveness across multiple tenants, reducing the possibility of decision-making biases caused by sudden changes in local governance actions or incoordination between different agents' governance actions. Attached Figure Description

[0017] Figure 1This is a schematic diagram of the overall method flow of the present invention.

[0018] Figure 2 This is a schematic diagram of the overall system structure of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0020] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] Example: Please see Figure 1 The present invention provides a technical solution: A multi-agent decision-making method based on dynamic scene perception, comprising the following steps: S1: Treat each tenant as an intelligent agent, obtain the time-series data of each intelligent agent's tenant behavior, data access, resource operation and security status to form the local state of each intelligent agent, and obtain the hidden state and communication characteristics of the current intelligent agent based on the local state at the current moment and the governance action at the previous moment. Furthermore, the time-series data of tenant behavior includes access frequency, operation anomaly rate, and number of permission requests; The time-series data on data access includes the percentage of access to sensitive data; Time-series data on resource operation includes load computing power and storage capacity; The time-series data on security status includes the access rate of high-risk IPs and the number of intrusions.

[0022] The time-series data of each dimension are associated and aggregated according to the tenant identifier to form the local state of the corresponding tenant agent at the current decision moment.

[0023] Specifically, time-series data for each dimension are collected and statistically analyzed according to preset time windows. Specifically, access frequency is determined by the number of times the corresponding tenant accesses data within a unit of time in the access log; operation anomaly rate is determined by the ratio of the number of times anomaly operations are identified in the audit log to the total number of operations; permission request count is determined by the number of permission request records initiated by the corresponding tenant in the permission management log; sensitive data access ratio is determined by the ratio of the number of times sensitive data is accessed in the data access log to the total number of data accesses; load computing power is determined by the computing resource usage of the corresponding tenant collected by the resource monitoring module; storage capacity is determined by the difference between the storage capacity allocated to the corresponding tenant and the used storage capacity; high-risk IP access rate is determined by the ratio of the number of accesses from a preset high-risk IP set in the access log to the total number of accesses; and the number of intrusions is determined statistically based on intrusion alarm records generated by the security detection module. All data carries a tenant identifier and timestamp, and is associated and aggregated within the same time window according to the tenant identifier to form the local state of the corresponding tenant's intelligent agent at the current decision-making moment.

[0024] Furthermore, the governance actions include no processing, risk processing, permission processing, and agent processing; Risk handling includes account freezing and data blocking; permission handling includes raising and lowering the permission level by one level; and agent handling includes increasing and decreasing the load computing power by a preset amount. The preset range is the average of the historical adjustment ranges.

[0025] The logic for obtaining the hidden state and communication characteristics of the current agent is as follows: For any tenant agent at any decision time, the local state of the tenant agent and the governance action performed by the tenant agent in the previous time are input into a multi-layer fully connected network. Feature mapping and fusion are performed on the local observation information and the governance action in the previous time to obtain the intermediate features at the current time. The intermediate features and the hidden state in the previous time are input into a preset gated recurrent unit (GRU). The GRU recursively updates the historical hidden state to obtain the hidden state at the current time. Specifically, at the initial decision time, the hidden state of the previous time is set to the preset initial hidden state, and the governance action of the previous time is set to the preset initial governance action, wherein the initial governance action is to do nothing; at each subsequent decision time, the hidden state actually obtained and the governance action actually executed at the previous decision time are used to participate in the state update of the current time. The local state at the current moment and the governance action at the previous moment are input into the pre-trained supernetwork to obtain communication features.

[0026] The multilayer fully connected network consists of an input layer, at least one hidden layer, and an output layer connected sequentially. Features are transferred between adjacent network layers using a fully connected approach. Each hidden layer performs a linear transformation on the output features of the previous layer and performs nonlinear mapping through a preset nonlinear activation function to extract combined features from the input data layer by layer. When processing tenant state features, the local state of the tenant agent is concatenated with the encoding vector corresponding to the governance action at the previous time step and input into the multilayer fully connected network. After feature mapping and fusion by each hidden layer, the output layer outputs intermediate features of a preset dimension. When calculating action value, the hidden state, communication features, and predicted governance action of the corresponding agent are concatenated and input into the multilayer fully connected network. The individual action value of the corresponding predicted governance action is output through layer-by-layer feature mapping. The weight parameters and bias parameters of each network layer are updated through backpropagation during the training process.

[0027] The gated recurrent unit (GRU) is used to recursively encode the state information of the tenant agent at consecutive decision moments. At any current decision moment, the intermediate features obtained by the multilayer perceptron and the hidden state of the tenant agent at the previous moment are input into the GRU. The reset gate in the GRU determines the proportion of historical information participating in the calculation of the current candidate state based on the current intermediate features and the hidden state at the previous moment. The update gate determines the retention ratio of the hidden state at the previous moment and the current candidate state in the current hidden state. Based on the calculation results of the reset gate and the update gate, the hidden state at the current moment is generated, so that the obtained current hidden state simultaneously represents the local operating characteristics of the current tenant, the impact of the governance action at the previous moment on the current state, and the historical state information accumulated at multiple previous decision moments. At the initial decision moment, the hidden state at the previous moment is set as the preset initial hidden state. Thereafter, at each decision moment, the hidden state actually obtained at the previous decision moment is used as the historical input for the current GRU recursively update.

[0028] The hypernetwork comprises an input layer, a first fully connected hidden layer, a second fully connected hidden layer, and a parameter output layer connected in sequence. The first and second fully connected hidden layers perform linear transformation and nonlinear activation on the input features, respectively. The parameter output layer outputs a parameter vector used to generate communication features. Each tenant agent uses a parameter-sharing hypernetwork, and the dynamic mapping parameters corresponding to different tenant agents are determined by their current local state and the governance action of the previous time step.

[0029] The input to the hypernetwork is the concatenation of the normalized local state vector at the current moment and the one-hot encoded vector of the governance action at the previous moment. The hidden states of each tenant agent at the same moment are concatenated according to a preset agent index order to form a hidden global state. This hidden global state is then input into at least one fully connected reconstruction layer, mapping it to a reconstructed global state of a preset fixed dimension. This reconstructed global state serves as the mapped object in the communication feature generation process.

[0030] The length of the parameter vector output by the parameter output layer is determined based on the dimension of the reconstructed global state and the preset dimension of the communication features. It is then split into a dynamic weight matrix and a dynamic bias vector in a preset order. The number of columns in the dynamic weight matrix equals the dimension of the reconstructed global state, and the number of rows equals the preset dimension of the communication features. A linear mapping is performed on the reconstructed global state using the dynamic weight matrix and dynamic bias vector, and the communication features of the corresponding tenant agent are obtained through a nonlinear activation function. Therefore, the hypernetwork directly outputs the dynamic weight matrix and dynamic bias vector, and the communication features are jointly determined by the dynamic weight matrix and dynamic bias vector and the reconstructed global state.

[0031] The training samples of the supernetwork consist of historical continuous decision data. Each training sample includes at least the current local state, the previous governance action, the hidden states of each agent, the current actual governance action, the global feedback after executing the governance action, and the local state at the next moment. In the first training phase, the supernetwork generates communication features and inputs them into the intent reasoning network. The supernetwork and intent reasoning network are updated based on the classification loss between the predicted governance action and the current actual governance action, as well as the regularization loss of the network parameters, to obtain the pre-trained parameters of the supernetwork. In the second training phase, the communication features are further input into the action value network and the hybrid network. The temporal difference loss of the global joint value is added to the original loss, and the supernetwork is jointly fine-tuned with a learning rate lower than that of the first training phase. The updates are repeated in small batches until the validation set loss no longer decreases within a preset number of rounds or reaches the preset number of training rounds. In the actual decision-making phase, the supernetwork parameters are fixed, and dynamic weight matrices, dynamic bias vectors, and communication features are generated only based on the current input.

[0032] S2: Input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. The intent reasoning network takes the communication characteristics and hidden state of an agent as input and outputs the predicted governance action of any agent, including itself. The intent reasoning network employs a multivariate Gaussian distribution model. Specifically, the communication characteristics, hidden states, and predicted governance actions of each agent at several historical moments are obtained, and 80% is used as the training set and 20% as the validation set. For any agent, the communication features and hidden state are concatenated and input into the encoder to generate... A multivariate Gaussian distribution ,…, ( ), … ( ),in, ( ) for intelligent agents Inferential agent Gaussian distribution, and respectively intelligent agents Inferential agent The mean and variance of Gaussian, and For the index of the agent, It can be equal to ,and , , This represents the number of agents, and the agents themselves. Inferential agent The predicted governance actions are sampled from the corresponding multivariate Gaussian distribution. Since the sampling operation is non-differentiable, this paper uses a reparameterization method to obtain the agent's parameters. Inferential agent Predictive intent parameters; Specifically, the formula for reparameterization is: in, For the Hadamard product operator, For intelligent agents Inferential agent The noise variables, specifically, are noise variables randomly sampled from a standard normal distribution and determined through training. Considering that the intention representation of each agent is difficult to learn automatically, intention inference maximizes the relationship between the intention representation and the agent's... With intelligent agents Mutual information between them is used to ensure the amount of information in the representation of intent parameters, where mutual information is the energy of the entity. With intelligent agents The difference between information entropy and conditional entropy; It should be noted that intelligent agents Inferential agent Predictive intent parameters These are just parameters; they need to be mapped to specific governance actions through mapping relationships. These mapping relationships are obtained directly through training. Information entropy and conditional entropy are technical terms in this field and are calculated using existing formulas, which will not be listed here. The training data of the intent inference network is sorted according to the decision-making time sequence. Each data point uses the communication features and hidden state at the corresponding time as input, and the governance action actually performed by the agent at that time as the supervision label. The first 80% of the data is used as the training set, and the last 20% as the validation set. Mini-batch forward computation is performed on the training set, using the cross-entropy between the predicted governance action probability and the actual governance action as the action classification loss, and adding a regularization loss of a multivariate Gaussian distribution. At the same time, an auxiliary decoder is set to reconstruct the actual governance action of the agent based on the predicted intent parameters. By maximizing the lower bound of the mutual information between the predicted intent parameters and the actual governance actions, the action information contained in the predicted intent parameters is improved. The action classification loss, distribution regularization loss, and mutual information loss are summed according to preset weights, and the parameters of the shared encoder, the distribution parameter output branch, the action mapping branch, and the auxiliary decoder are updated through backpropagation. When the validation set loss no longer decreases within a preset number of rounds, training stops and the network parameters with the lowest validation set loss are saved. During actual decision-making, the network parameters are fixed, and the noise variable is set to zero. The deterministic predicted governance action is output using the predicted intent parameters corresponding to the mean vector.

[0033] S3: For any agent, the predicted governance actions of each agent to itself constitute the initial governance scheme. The hidden state, communication characteristics and governance actions of each agent in the initial governance scheme are input into the action value network to obtain the global joint value of the initial governance scheme. The hybrid network is trained using a multi-agent reinforcement learning algorithm that minimizes temporal difference loss.

[0034] Furthermore, the logic for obtaining the global joint value is as follows: For any agent in the governance scheme to be evaluated, the agent's hidden state, communication characteristics, and governance actions to be evaluated are input into the action value network to obtain the individual action value of the agent under the governance actions to be evaluated; the individual action values ​​of all agents and the current reconstructed global state are input into the hybrid network to obtain the global joint value of the governance scheme to be evaluated.

[0035] The action value network comprises an input layer, at least two fully connected hidden layers, and a scalar output layer, with a nonlinear activation unit following each fully connected hidden layer. The input to the action value network is a concatenation of the current agent's hidden state, communication characteristics, and the one-hot encoded vector of the governance action to be evaluated. The scalar output layer outputs an individual action value. The above calculation is performed on each agent within the same governance scheme to obtain individual action value vectors arranged according to a preset agent index order. Each agent can share the action value network parameters and can be distinguished by appending agent identifier codes to the input.

[0036] The action value network does not assign manual labels to individual action values; instead, it is jointly trained end-to-end with a hybrid network. Training data uses state transitions between two consecutive decision points as basic samples. Each state transition includes at least the current local and hidden states of each agent, the current joint governance action, the global reward obtained after executing the joint governance action, the local states of each agent at the next time step, and a termination flag. The global reward is determined according to preset weights based on changes in security risk after the governance action is executed, the stability of resource operation, and the cost of executing the governance action.

[0037] The hybrid network comprises a state condition parameter generation branch, a first hybrid layer, a nonlinear activation layer, and a second hybrid layer. The hybrid network uses the individual action value vector as the direct input to the first hybrid layer and the reconstructed global state as the input to the state condition parameter generation branch. The state condition parameter generation branch generates the weights and biases of the first and second hybrid layers based on the reconstructed global state. The first hybrid layer performs a nonlinear combination of the individual action value vectors, and the second hybrid layer outputs a scalar as the global joint value. The state condition parameter generation branch and the aforementioned supernetwork used to generate communication features employ independent network parameters.

[0038] The weights of the hybrid layer output from the state condition parameter generation branch are processed using an absolute value function or a positive activation function to ensure that all hybrid layer weights are non-negative, thus maintaining a monotonically constant global joint value relative to the value of any individual action. Therefore, when the individual action values ​​of other agents remain constant, increasing the individual action value of any agent will not decrease the global joint value, and it ensures that the governance actions chosen by each agent based on its individual action value align with the optimization direction of the global joint value. The input to the hybrid network is the vector sum of the individual action values ​​of all agents, reconstructing the global state; the output is the single global joint value corresponding to the joint governance action.

[0039] Both the action value network and the hybrid network employ online and target networks with identical parameter structures. During training, mini-batch state transitions are extracted from the experience replay data. The online action value network and the online hybrid network calculate the global joint value of the current joint governance action. Then, the hidden states and communication characteristics of each agent at the next time step determine the individual action values ​​of each optional governance action, selecting candidate joint governance actions for the next time step. The target action value network and the target hybrid network then calculate the corresponding target global joint value. For non-termination samples, the sum of the current global reward and the discounted target global joint value is used as the temporal difference objective; for termination samples, the current global reward is used as the temporal difference objective.

[0040] Calculate the mean squared error or smoothed absolute error between the global joint value output of the online hybrid network and the temporal difference objective, and use this error as the temporal difference loss. Update the parameters of both the online action value network and the online hybrid network simultaneously via backpropagation, and copy the online network parameters to the target network according to a preset update cycle, or update the target network parameters using a soft update method. Repeat training until the average temporal difference loss on the validation set converges or reaches a preset number of training epochs, and save the action value network and hybrid network parameters with the lowest validation set loss.

[0041] The overall training process for the four types of networks includes: handling missing values, normalizing, and encoding governance actions in historical time-series data, and constructing training samples according to continuous decision-making moments; firstly, pre-training the supernetwork and intent reasoning network using actual governance actions to ensure that communication features support governance intent recognition; then loading pre-trained parameters, jointly training the action value network and hybrid network using state transition data and global rewards, retaining the action classification loss as an auxiliary loss, and jointly fine-tuning the supernetwork and intent reasoning network with a small learning rate; finally, selecting network parameters based on the action prediction accuracy and average temporal difference loss on the independent validation set. The training gradients of each network are propagated only during the training phase, and the parameters of the supernetwork, intent reasoning network, action value network, and hybrid network are fixed after training is completed.

[0042] In actual decision-making, a hypernetwork with fixed parameters first generates the communication characteristics of each agent, then an intent reasoning network with fixed parameters generates predicted governance actions between agents and forms an initial governance scheme. Subsequently, an action value network with fixed parameters calculates the individual action value of each agent under the initial governance action or candidate alternative governance actions, and a hybrid network with fixed parameters calculates the global joint value of each governance scheme. Gradient updates are not performed during the actual decision-making process. When the number of new historical samples reaches a preset amount or the model evaluation index falls below a preset threshold, the above training process is re-executed in the offline training environment, and the validated network parameters are replaced.

[0043] S4: Based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, and combined with the predicted governance actions of other agents on the current agent, the agents are divided into regular agents, uncoordinated mutation agents, or general agents. Furthermore, the types of intelligent agents include conventional intelligent agents, uncoordinated mutant intelligent agents, and general intelligent agents.

[0044] The logic for determining the type of intelligent agent is as follows: For each predictive governance action of each agent, count the number of times other agents infer that the agent will execute the predictive governance action, and take the predictive governance action with the most occurrences as the benchmark predictive governance action; calculate the difference in the number of each of the remaining predictive governance actions relative to the benchmark predictive governance action. If the difference is less than a preset number threshold, then the predictive governance action and the benchmark predictive governance action are taken together as candidate predictive governance actions. The preset number threshold is not greater than 3% of the total number of agents. If 3% of the total number of agents is not an integer, then it is rounded up. For different predictive governance actions of the same agent, the predictive governance action with the most predicted support from the general agent can reflect the main judgment of most agents on the governance trend of the agent. Therefore, the predictive governance action with the most predictions is first determined as the benchmark predictive governance action. However, when the predictions of other predictive governance actions are close to the benchmark predictive governance action, the predictive governance action with the most predictions does not have a significant advantage over other predictive governance actions. For example, if the two differ by only one prediction, determining one as the sole benchmark based solely on the magnitude of the predictions may lead to the artificial differentiation of nearly parallel prediction results. Therefore, the difference in the number of predictions between other predictive governance actions and the benchmark predictive governance action is further calculated. When the difference in the number of predictions is less than a preset threshold, the corresponding predictive governance action and the benchmark predictive governance action are jointly determined as candidate predictive governance actions. This way, the main judgment is determined by the majority of prediction results, and the benchmark result, which may also be equally representative, is retained when the number of predicted governance actions with similar support is close.

[0045] Since the preset quantity threshold is intended to address the fact that the most numerous predictive governance actions do not have a significant advantage over other predictive governance actions, the quantity threshold is set to a small value, usually no more than 3% of the total number of agents.

[0046] Furthermore, for any agent at the current moment: If there is an intersection between the candidate predicted governance action and the governance action at the current moment, and the intersection is the governance action of the agent in the initial governance plan, then the agent is called a regular agent. If the candidate predicted governance action does not intersect with the governance action at the current moment, and the governance action of the agent in the initial governance scheme also does not intersect with the candidate predicted governance action, then the agent is called an incoherent mutation agent. If an agent is neither a conventional agent nor an uncoordinated mutant agent, then the agent is a general agent.

[0047] The rationality of the governance action determined for a specific agent in the initial governance plan cannot be judged solely based on the action itself. It also requires considering the continuity of governance reflected in the agent's current and predicted actions, as well as the collective predictions of other agents regarding its governance trends. Candidate predicted actions reflect the main judgments of other agents regarding the agent's potential current governance actions, while the governance actions in the initial plan reflect the historical continuity of the agent's governance behavior. When the governance actions in the initial plan, the current governance action, and the candidate predicted actions are consistent, it indicates that the current governance action is in line with the development trend and the multi-agent prediction results; therefore, the corresponding agent is identified as a regular agent. Conversely, when the governance action in the current plan differs from both the initial and candidate predicted actions, and there is no consistency between the candidate predicted actions and the previous action, it indicates that the agent's current governance action has changed relative to the historical trend, and this change is not supported by the general agent prediction results; therefore, it is identified as an uncoordinated, mutated agent. This process allows for the screening of governance actions in the initial governance plan that simultaneously exhibit historical mutability and group prediction incoordination.

[0048] S5: Keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the types of intelligent agents, adjust the governance actions of each incoordination mutation intelligent agent in the initial governance plan, and use the adjusted initial governance plan as the decision plan for the current moment.

[0049] The logic of S5 is as follows: For any incoherent mutant agent, the number of predicted governance actions performed by the incoherent mutant agent is inferred from the statistics of the regular agent, and the predicted governance action with the most executions is taken as the regular baseline predicted governance action. The difference in the number of the remaining predicted governance actions relative to the regular baseline predicted governance action is calculated. If the difference is less than a preset number threshold, the predicted governance action and the regular baseline predicted governance action are taken together as candidate calibration predicted governance actions. Since the function is the same, the quantity threshold here is equal to the aforementioned quantity threshold.

[0050] The current governance actions of the inconsistent mutant agent have shown inconsistencies with historical governance trends and overall prediction results. Therefore, when calibrating its governance actions, it is necessary to further extract governance judgments for the inconsistent mutant agent from regular agents with higher credibility and relatively stable behavior to reduce the interference of abnormal agent prediction results on the calibration process. Among them, the governance action with the most predictions from regular agents is identified as the regular baseline predicted governance action, which can reflect the main judgment of regular agents on the governance trend of the inconsistent mutant agent. When the difference in the number of other predicted governance actions with the regular baseline predicted governance action is small, it indicates that the support from regular agents for the two is relatively similar, and the governance action with the most predictions does not have a significant advantage. Therefore, it is included in the candidate calibration predicted governance actions. This approach can not only utilize the group prediction results of regular agents to form a relatively reliable calibration basis, but also avoid missing equally representative governance actions due to a small difference in the number of predictions. Furthermore, for any incoherent mutant agent, the predicted governance actions of all regular agents for that incoherent mutant agent are counted to form a set of regular predicted governance actions; If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the incoherent mutant agent in the governance scheme do not intersect with the candidate calibration predicted governance actions, then the governance actions in the initial scheme will be used as the predicted governance actions to be adjusted for the agent. For incoherent mutant agents, the governance actions in their initial governance scheme have shown inconsistencies with the candidate calibration prediction governance actions. Therefore, it is necessary to determine whether there are alternative governance actions that can be supported by the prediction results of regular agents. When there is an intersection between the set of regular predicted governance actions and the initial governance scheme, it indicates that there are at least some governance actions in the initial governance scheme that can be supported by the predictions of regular agents. However, the governance actions currently determined for the incoherent mutant agent are not candidate calibration prediction governance actions, indicating that the current governance action lacks the support of the group judgment of regular agents. Therefore, the governance actions provided by the initial governance scheme and supported by the predictions of regular agents in the intersection are taken as the predicted governance actions to be adjusted for the agent. It is possible to give priority to using the governance actions that are already included in the initial governance scheme and can be recognized by the predictions of regular agents to calibrate the incoherent mutant agent, avoiding the direct retention of mutant governance actions that lack the support of group predictions.

[0051] Furthermore, if the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutant agent in the governance scheme intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection are taken as the predicted governance actions to be adjusted. When there is no intersection between the set of conventional predicted governance actions and the initial governance scheme, it indicates that the original governance actions in the initial governance scheme have not received direct support from the prediction results of conventional agents. Therefore, it is difficult to continue to determine the adjustment direction of the incoherent mutant agent based on the initial governance scheme. However, when there is an intersection between the governance actions of the incoherent mutant agent in the initial governance scheme and the candidate calibration predicted governance actions, it indicates that although the governance action has not formed a consistent relationship with the initial governance scheme, it has at least received support from the relatively concentrated prediction results of conventional agents and has a certain degree of group consistency and acceptability. Therefore, the predicted governance actions in the intersection are taken as the predicted governance actions to be adjusted. In the case that the initial governance scheme lacks the support of conventional agents, the more representative adjustment direction is determined based on the candidate calibration predicted governance actions, so that the governance actions of the incoherent mutant agent move closer to the group judgment of conventional agents.

[0052] If the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutation agent in the governance scheme also do not intersect with the candidate calibration predicted governance actions, then all predicted governance actions of the agent are taken as predicted governance actions to be adjusted. If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the agent in the governance scheme also intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection of the two are taken as the predicted governance actions to be adjusted. When there is no overlap between the set of conventional predicted governance actions and the initial governance scheme, and the governance actions of the incoordinating mutant agent in the governance scheme also do not overlap with the candidate calibration predicted governance actions, it indicates that neither the adjustment basis supported by the conventional agent's predictions can be obtained from the initial governance scheme, nor can a clear adjustment direction coordinated with the current governance action be determined from the candidate calibration predicted governance actions. In this case, continuing to limit the predicted governance actions to a certain subset may exclude actually better governance actions due to a lack of sufficient evidence. Therefore, all predicted governance actions of the agent are treated as predicted governance actions to be adjusted to expand the range of governance actions that the agent can adjust. Subsequently, each predicted governance action to be adjusted is substituted into the governance scheme, and the final governance action is determined by calculating the global joint value of the updated governance scheme. This avoids prematurely limiting the adjustment direction in the absence of clear calibration basis and uses the global joint value to screen governance actions that are more coordinated with the overall governance goal from all possible predicted governance actions.

[0053] If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the agent in the governance scheme also intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection of the two are taken as the predicted governance actions to be adjusted. In this scenario, the set of routine predicted governance actions overlaps with the governance actions at the current moment, indicating that some predicted governance actions have already been reflected in the current actual governance process. Simultaneously, the governance actions of this agent in the governance scheme overlap with the candidate calibration predicted governance actions, indicating that some of these predicted governance actions correspond to the governance requirements determined by the governance scheme. Therefore, identifying the predicted governance actions at the intersection of these two sets as the predicted governance actions to be adjusted can further filter out predicted governance actions that are both relevant to the current actual governance state and meet the calibration requirements of the governance scheme, avoiding adjustments to irrelevant predicted governance actions and thus improving the targeting and accuracy of predicted governance action adjustments.

[0054] Furthermore, the governance actions of the regular agents and general agents in the initial governance scheme remain unchanged; for each incoordination mutation agent, select an action from its predicted governance actions to be adjusted, and replace the original governance action of the agent in the initial governance scheme to obtain the updated governance scheme; calculate the global joint value of each updated governance scheme, and select the governance scheme with the highest global joint value as the decision scheme at the current moment.

[0055] The governance actions of regular and general agents did not exhibit any inconsistent mutation characteristics requiring focused calibration. Therefore, maintaining their governance actions in the initial governance scheme avoids additional interference to the already relatively stable or temporarily unadjustable governance results. For agents with identified inconsistent mutations, alternative actions are selected only within the range of their corresponding predicted governance actions to be adjusted, forming different updated governance schemes. This concentrates the adjustment scope of the governance scheme on agents with obvious anomalies, reducing the overall search and adjustment complexity. Based on this, the overall governance effect formed by different alternative actions is uniformly evaluated by calculating the global joint value corresponding to each updated governance scheme. The governance scheme with the highest global joint value is selected as the decision scheme at the current moment. This allows the local calibration of agents with inconsistent mutations to consider not only the rationality of their own actions but also the collaborative relationships between multiple agents and the overall governance goals of the platform. This enables targeted correction of abnormal governance actions and improves the overall coordination and global benefits of the final decision scheme.

[0056] Please see Figure 2 The present invention further provides a multi-agent decision-making system based on dynamic scene perception, the system being used to implement the above-mentioned multi-agent decision-making method based on dynamic scene perception, specifically including: The state construction module is used to treat each tenant as an agent and obtain time-series data on the tenant behavior, data access, resource operation and security status of each agent to form the local state of each agent. Based on the local state at the current moment and the governance action at the previous moment, the hidden state and communication characteristics of the current agent are obtained. The action prediction module is used to input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. The value analysis module is used to construct an initial governance scheme from the predicted governance actions of each agent for any given agent. The hidden states, communication characteristics, and governance actions of each agent in the initial governance scheme are input into the action value network to obtain the global joint value of the initial governance scheme. The category determination module is used to classify agents into regular agents, uncoordinated mutation agents, or general agents based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, combined with the predicted governance actions of other agents on the current agent. The scheme adjustment module is used to keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the type of intelligent agent, it adjusts the governance actions of each uncoordinated and mutated intelligent agent in the initial governance scheme and uses the adjusted initial governance scheme as the decision scheme at the current moment.

[0057] The above formula uses dimensionless values ​​for calculation, and the preset parameters are set by those skilled in the art according to the actual scenario.

[0058] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0059] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that cannot be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A multi-agent decision-making method based on dynamic scene perception, characterized in that, The specific steps include: S1: Treat each tenant as an intelligent agent, obtain the time-series data of each intelligent agent's tenant behavior, data access, resource operation and security status to form the local state of each intelligent agent, and obtain the hidden state and communication characteristics of the current intelligent agent based on the local state at the current moment and the governance action at the previous moment. S2: Input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. S3: For any agent, the predicted governance actions of each agent to itself constitute the initial governance scheme. The hidden state, communication characteristics and governance actions of each agent in the initial governance scheme are input into the action value network to obtain the global joint value of the initial governance scheme. S4: Based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, and combined with the predicted governance actions of other agents on the current agent, the agents are divided into regular agents, uncoordinated mutation agents, or general agents. S5: Keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the types of intelligent agents, adjust the governance actions of each incoordination mutation intelligent agent in the initial governance plan, and use the adjusted initial governance plan as the decision plan for the current moment.

2. The multi-agent decision-making method based on dynamic scene perception according to claim 1, characterized in that: The time-series data of tenant behavior includes access frequency, operation anomaly rate, and number of permission requests; The time-series data on data access includes the percentage of access to sensitive data; Time-series data on resource operation includes load computing power and storage capacity; The time-series data on security status includes the access rate of high-risk IPs and the number of intrusions. The time-series data of each dimension are associated and aggregated according to the tenant identifier to form the local state of the corresponding tenant agent at the current decision moment.

3. The multi-agent decision-making method based on dynamic scene perception according to claim 1, characterized in that: The governance actions include no action, risk handling, permission handling, and agent handling; Risk handling includes account freezing and data blocking; permission handling includes raising and lowering permission levels; and agent handling includes increasing and decreasing the computing power by a preset amount.

4. The multi-agent decision-making method based on dynamic scene perception according to claim 1, characterized in that: The logic for obtaining the hidden state and communication characteristics of the current agent is as follows: For any tenant agent at any decision time, the local state of the tenant agent and the governance action executed by the tenant agent at the previous time are input into the multi-layer fully connected network to obtain the intermediate features at the current time. The intermediate features and the hidden state at the previous time are input into a preset gated loop unit to output the hidden state. Specifically, at the initial decision time, the hidden state of the previous time is set to the preset initial hidden state, and the governance action of the previous time is set to the preset initial governance action, wherein the initial governance action is to do nothing; at each subsequent decision time, the hidden state actually obtained and the governance action actually executed at the previous decision time are used to participate in the state update of the current time. The local state at the current moment and the governance action at the previous moment are input into the pre-trained supernetwork to obtain communication features.

5. A multi-agent decision-making method based on dynamic scene perception according to claim 1, characterized in that: The logic for obtaining the global joint value is as follows: The hidden state, communication characteristics, and predicted governance actions of each agent are input into a multi-layer fully connected network to generate the individual action value of each agent. The individual action value of each agent is calculated through a nonlinear mixing function in the hybrid network to obtain the global joint value. The hybrid network is trained using a multi-agent reinforcement learning algorithm that minimizes temporal difference loss.

6. The multi-agent decision-making method based on dynamic scene perception according to claim 1, characterized in that: The types of intelligent agents include conventional intelligent agents, uncoordinated mutant intelligent agents, and general intelligent agents; The logic for determining the type of intelligent agent is as follows: For each predictive governance action of each agent, the number of times other agents infer that the agent will execute the predictive governance action is counted, and the predictive governance action with the most occurrences is taken as the baseline predictive governance action. The difference in the number of each of the remaining predictive governance actions relative to the baseline predictive governance action is calculated. If the difference is less than a preset number threshold, the predictive governance action and the baseline predictive governance action are taken together as candidate predictive governance actions. The preset number threshold is no greater than 3% of the total number of agents. For any agent at the current moment: If there is an intersection between the candidate predicted governance action and the governance action at the current moment, and the intersection is the governance action of the agent in the initial governance plan, then the agent is called a regular agent. If the candidate predicted governance action does not intersect with the governance action at the current moment, and the governance action of the agent in the initial governance scheme also does not intersect with the candidate predicted governance action, then the agent is called an incoherent mutation agent. If an agent is neither a conventional agent nor an uncoordinated mutant agent, then the agent is a general agent.

7. A multi-agent decision-making method based on dynamic scene perception according to claim 6, characterized in that: The logic of S5 is as follows: For any incoherent mutant agent, the number of predicted governance actions performed by the incoherent mutant agent is inferred from the statistics of the regular agent, and the predicted governance action with the most executions is taken as the regular baseline predicted governance action. The difference in the number of the remaining predicted governance actions relative to the regular baseline predicted governance action is calculated. If the difference is less than a preset number threshold, the predicted governance action and the regular baseline predicted governance action are taken together as candidate calibration predicted governance actions. For any incoherent mutant agent, count the predicted governance actions of all regular agents for that incoherent mutant agent to form a set of regular predicted governance actions. If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the incoherent mutant agent in the governance scheme do not intersect with the candidate calibration predicted governance actions, then the governance actions in the initial scheme will be used as the predicted governance actions to be adjusted for the agent. If the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutant agent in the governance scheme intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection are taken as the predicted governance actions to be adjusted. If the set of conventional predicted governance actions does not intersect with the governance actions at the current moment, and the governance actions of the incoordination mutation agent in the governance scheme also do not intersect with the candidate calibration predicted governance actions, then all predicted governance actions of the agent are taken as predicted governance actions to be adjusted. If the set of conventional predicted governance actions intersects with the governance actions at the current moment, and the governance actions of the agent in the governance scheme also intersect with the candidate calibration predicted governance actions, then the predicted governance actions in the intersection of the two are taken as the predicted governance actions to be adjusted. Keep the governance actions of regular agents and general agents unchanged in the initial governance scheme; for each incoordination mutation agent, select an action from its predicted governance actions to be adjusted, and replace the original governance action of the agent in the initial governance scheme to obtain the updated governance scheme; calculate the global joint value of each updated governance scheme, and select the governance scheme with the highest global joint value as the decision scheme at the current moment.

8. A multi-agent decision-making system based on dynamic scene perception, characterized in that: The system is used to implement the multi-agent decision-making method based on dynamic scene perception as described in any one of claims 1-7, specifically including: The state construction module is used to treat each tenant as an agent and obtain time-series data on the tenant behavior, data access, resource operation and security status of each agent to form the local state of each agent. Based on the local state at the current moment and the governance action at the previous moment, the hidden state and communication characteristics of the current agent are obtained. The action prediction module is used to input the communication features and hidden states of each agent into a pre-trained intent reasoning network to obtain the predicted governance action of each agent relative to any other agent, wherein any other agent includes the agent itself. The value analysis module is used to construct an initial governance plan from the predicted governance actions of each agent for any given agent, and to extract the hidden states of each agent in the initial governance plan. The governance actions of the initial governance scheme are input into the action value network based on the state, communication characteristics, and the initial governance scheme, to obtain the global joint value of the initial governance scheme; The category determination module is used to classify agents into regular agents, uncoordinated mutation agents, or general agents based on the governance actions and predicted governance actions of each agent in the current initial governance scheme, combined with the predicted governance actions of other agents on the current agent. The scheme adjustment module is used to keep the governance actions of regular and general intelligent agents unchanged. Based on the global joint value and the type of intelligent agent, it adjusts the governance actions of each uncoordinated and mutated intelligent agent in the initial governance scheme and uses the adjusted initial governance scheme as the decision scheme at the current moment.

Images