Calculation network ecological digital simulator based on multi-agent self-organizing interaction
Through the computing network ecological digital simulator with multi-agent self-organizing interaction, the dynamic and security problems of computing power network in resource allocation, business scheduling and risk management are solved, the efficient operation and security guarantee of computing network platform is achieved, and the resource utilization rate and risk control capabilities are improved.
Patent Information
- Application Number
- CN202510458395.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-11
- Publication Date
- 2025-07-25
AI Technical Summary
The computing power network (computer network) has problems of insufficient efficiency and security in resource allocation, business scheduling and risk management. The existing technology is difficult to cope with dynamic changes and the coordinated optimization of multi-dimensional factors, resulting in low resource utilization, inefficient service quality, and lack of effective dynamic risk control capabilities.
The computing network ecological digital simulator is adopted based on multi-agent self-organizing interaction, including multi-agent self-organizing interaction module, resource modeling and dynamic allocation module, business scheduling and collaborative optimization module, risk management and emergency response module, and strategy evaluation and optimization regulation module. Through multi-agent reinforcement learning and game theory optimization, dynamic allocation of resources, efficient task scheduling and intelligent risk control are achieved.
It realizes accurate resource allocation and task collaborative optimization of the computing network platform, improves task scheduling efficiency and overall performance, flexibly responds to new security threats, enhances the comprehensiveness and adaptability of risk control, and ensures the safe operation of the platform in complex and changing environments.
Smart Images

Figure CN120378314A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computing network services, and in particular to a computing network ecosystem digital simulator that simulates computing network resource allocation, business scheduling, and risk management to ensure the safe operation of a computing network platform and provide efficient services. Background Art
[0002] As a new infrastructure architecture, computing network (computing network) provides efficient and reliable service support for intelligent business through the deep integration and dynamic coordination of various heterogeneous resources such as computing and network, combined with the integrated coordination and deployment of new production factors such as computing power, algorithms and data. However, how to achieve efficient allocation of platform resources, business scheduling optimization and dynamic risk management in this architecture, and then ensure the safe operation of the platform and provide efficient services, is the core problem that needs to be solved urgently.
[0003] Computing networks present the dual characteristics of high-order dynamics and deep coupling. On the one hand, the dynamic changes in computing environments, demand types, and resource collections make it difficult to predict the relationship between resource supply and demand. Existing methods generally rely on static rules and preset strategies, such as priority-based static allocation models or heuristic algorithms, which are difficult to achieve real-time optimization and flexible allocation of global resources by pre-setting task priorities and local optimization. On the other hand, the computing elements, optimal allocation processes, and security dimensions in the computing network platform are highly coupled. Traditional separate management methods, such as traditional centralized scheduling algorithms (such as polling, shortest job first, etc.) and hierarchical scheduling strategies, although they can optimize task scheduling in a single dimension, cannot effectively respond to the needs of collaborative optimization of multi-dimensional factors.
[0004] The resource scheduling method based on reinforcement learning can effectively solve the dynamic resource allocation and task scheduling problems in specific scenarios such as cloud computing and edge computing by adaptively adjusting strategies through environmental interactions. Among them, the deep Q network (DQN) and multi-agent deep deterministic policy gradient (MADDPG) are two typical reinforcement learning scheduling methods.
[0005] DQN uses a deep neural network to approximate the Q-value function, enabling the agent to make efficient decisions in a large-scale state space. Its core idea is to store and reuse historical data through an experience replay mechanism, combined with a target network to stabilize the training process, thereby optimizing the scheduling strategy of computing tasks on different computing nodes. DQN can adapt to the dynamically changing resource status in the cloud computing environment, improve system throughput, and reduce task execution latency. However, this method has obvious limitations. Due to the discretization limitation of Q-values, DQN is difficult to scale in high-dimensional state and action spaces. In a complex heterogeneous computing environment where the resource requirements of tasks are variable, the training efficiency of DQN is low, and it is prone to falling into local optima. In addition, in a multi-agent environment, the single-agent training method of DQN cannot fully consider the cooperation between different computing nodes, resulting in insufficient global scheduling optimization ability.
[0006] Compared with DQN, MADDPG adopts a multi-agent reinforcement learning framework. By independently training the policies of each computing node and sharing policy gradient information globally, it can achieve more efficient resource allocation and task scheduling. Based on the policy gradient method, MADDPG can handle continuous action spaces, making the scheduling strategy more flexible and suitable for complex scenarios of cloud-edge-end collaborative computing. In addition, MADDPG supports collaborative optimization between different computing nodes. Through information sharing among agents, it improves the global optimization ability of task scheduling. However, the main drawbacks of MADDPG are the large computational overhead. A large amount of computing resources are required during the multi-agent training process, and the convergence speed is slow, making it difficult to meet the requirements of real-time and low latency in the edge computing environment. In addition, the communication and synchronization costs between multi-agents are high, and information lag or inconsistency may occur in a dynamic environment, affecting the robustness of scheduling.
[0007] Although both DQN and MADDPG have made important progress in the research of reinforcement learning scheduling, they still face challenges such as scalability, convergence, and computational cost.
[0008] In summary, the main drawbacks of the existing technologies are as follows: As a new type of infrastructure, the high-order dynamicity and deep coupling characteristics of the computing power network (computing-network) significantly increase the complexity of resource allocation and security management. The existing technologies rely on static rule algorithms based on historical data, making it difficult to efficiently handle the dynamic changes in the resource supply-demand relationship in the computing-network environment, and difficult to adapt to dynamic resource allocation and task overall scheduling, resulting in low resource utilization and low service quality. The traditional separate management method cannot effectively solve the deep coupling problem of computing elements, deployment processes, and security requirements in the computing-network platform.
[0009] In addition, the risk management ability brought by dynamic interaction is weak, and there is a lack of effective countermeasures for privacy leakage and security threats in the high-frequency interaction scenarios of multiple users and multiple tasks. The deep coupling of multiple elements and the high-frequency dynamic interaction of multiple users and multiple tasks in the computing and networking platform exacerbate security risks. Existing technologies mainly rely on static and single defense mechanisms, making it difficult to effectively cope with new security threats during dynamic interaction, resulting in limitations in the flexibility and comprehensiveness of the system's risk management.
[0010] Based on the above analysis, the computing and networking platform urgently needs a digital simulator that can comprehensively simulate the dynamic allocation of computing and networking resources, business scheduling optimization, and risk management. Through the self-organizing interaction of multiple agents, it can build a refined and virtualized computing and networking model, dynamically reproduce the resource interaction behavior in a complex environment, evaluate the effectiveness of scheduling strategies, and optimize the system's adaptability in a highly dynamic environment, thereby providing a scientific basis for the intelligent decision-making of the computing and networking ecosystem. This simulation technology helps to break through the limitations of existing methods in resource integration and interaction security protection, and provides technical support for the efficient operation and ecological stable development of the computing and networking platform. Summary of the Invention
[0011] Aiming at the shortcomings of existing technologies in computing and networking resource allocation, task scheduling, and security protection, the present invention proposes a digital simulator for computing and networking services, which dynamically models and simulates the computing resources, task requirements, and security risks in the computing and networking environment through the self-organizing interaction of multiple agents, aiming to build a refined and virtualized computing and networking model to support the dynamic simulation and optimization decision-making of resources, tasks, and risks in a complex computing and networking environment. The following objectives are achieved: 1) Provide dynamic resource allocation to solve the problem of insufficient adaptability caused by static rules and preset strategies.
[0012] 2) Provide real-time collaborative optimization of multiple tasks, and improve the overall performance of the system by simulating the task interaction process in the computing and networking.
[0013] 3) Provide dynamic risk management capabilities, simulate potential risk scenarios in the interaction of multiple users and multiple tasks, and provide a scientific basis for the security protection and efficient operation of the computing and networking ecosystem.
[0014] Technical Solution A computing and networking ecological digital simulator based on multi-agent self-organizing interaction, comprising: a multi-agent self-organizing interaction module, a resource modeling and dynamic allocation module, a service scheduling and collaborative optimization module, a risk management and emergency response module, and a policy evaluation and optimization control module. Among them, the multi-agent self-organizing interaction module, as the bottom layer module, is used to simulate the interaction and simulation of each link in the computing and networking system, including the real-time collaborative work among computing nodes, storage nodes, and network nodes; the resource modeling and dynamic allocation module, based on the multi-agent interaction module, formulates a preliminary resource allocation plan according to the current task requirements and resource status, providing a basis for task scheduling; the service scheduling and collaborative optimization module is mapped and associated with the resource allocation module, plans the execution order of tasks according to the resource allocation plan, and optimizes the overall performance indicators of the task chain, including execution delay and throughput, etc.; the risk management and emergency response module monitors and processes potential risks in the computing and networking system in real time by analyzing possible faults and bottlenecks in the process of resource allocation and task scheduling, ensuring the security and reliability of the system; the policy evaluation and optimization control module comprehensively evaluates the operation effects of the above modules, generates optimization feedback, and transmits it to the multi-agent interaction module to adjust the system simulation model, improving the efficiency and accuracy of the next round of resource allocation and scheduling. Through the collaborative work of the above modules, the system realizes the organic combination of multi-agent simulation, dynamic resource allocation, efficient task scheduling, and intelligent risk control.
[0015] The processing flow of the digital simulator is as follows: Step 1: The multi-agent self-organizing interaction module coordinates the information sharing and task collaboration of each agent, determines the resource requirements, and transmits them to the resource modeling and dynamic allocation module.
[0016] Step 2: The resource modeling and dynamic allocation module models the resources according to the current system state, optimizes the dynamic resource allocation plan, and after ensuring reasonable utilization, provides the results to the service scheduling and collaborative optimization module.
[0017] Step 3: The service scheduling and collaborative optimization module formulates a task execution strategy according to the allocated resources and optimizes the collaborative scheduling among tasks.
[0018] Step 4: The operation status and key decisions of all modules are monitored by the risk management and emergency response module to ensure the overall stability and security of the platform. When resource bottlenecks, privacy leaks, abnormal behaviors, or emergency events are detected, this module timely evaluates the risks, adjusts the resource allocation, scheduling strategy, or triggers the emergency response mechanism to reduce system risks.
[0019] Step 5, the strategy evaluation and optimization control module continuously analyzes the execution effects of each module, optimizes the agent interaction mode, resource management strategy, and scheduling logic based on historical data, and feeds back optimization suggestions to the multi-agent self-organization interaction module to enhance the intelligent adaptability of the system and form a closed-loop optimization.
[0020] Beneficial effects 1) Precise simulation and efficient resource allocation: Through the digital simulator based on multi-agent self-organization interaction, the present invention can accurately simulate and emulate the resource, service, risk, and network changes of the computing and networking platform, integrating multi-dimensional information (such as computing elements, allocation processes, and security requirements). This method is based on multi-agent theory and realizes effective response and optimization to the dynamic environment through the adaptive interaction between agents, thus providing comprehensive support for resource allocation and task scheduling. Compared with traditional static rules and preset strategies, the present invention can dynamically adjust resource allocation, achieve task collaborative optimization, effectively improve the task scheduling efficiency and overall performance of the computing and networking platform, and break through the limitations of static rules in the face of complex heterogeneous tasks.
[0021] 2) Flexible response to new security threats: The present invention uses the simulator to real-time simulate the potential risks in multi-user and multi-task interaction scenarios. Based on game theory and dynamic system theory, it can effectively identify and respond to new threats emerging in dynamic interactions. Compared with traditional static defense mechanisms, the present invention uses the self-adaptability and real-time feedback mechanism of the model to flexibly respond to the changing privacy and security risks in the computing and networking environment. Through dynamic simulation, the risk evolution process under high-frequency interaction can be simulated, significantly improving the comprehensiveness and adaptability of risk control, making up for the limitations of static security protection, and providing theoretical support and scientific basis for the secure operation of the computing and networking platform. This theoretical support ensures that the platform can effectively respond to complex and changing attacks and security threats through real-time feedback and optimization adjustment. Description of the drawings
[0022] Figure 1 Shows the overall architecture diagram of the system of the present invention; Figure 2 Shows the schematic flow diagram of dynamic resource allocation based on multi-objective optimization in an embodiment of the present invention; Figure 3 Shows the schematic diagram of the business collaborative scheduling process based on multi-task chains in an embodiment of the present invention; Figure 4 Shows the schematic diagram of the strategy generation and control process based on loop feedback in an embodiment of the present invention. Detailed implementation manners
[0023] The technical solutions provided by the present application will be further described below in conjunction with specific embodiments and their accompanying drawings. In combination with the following description, the advantages and features of the present application will become clearer.
[0024] In view of the main problems existing in the resource allocation, service scheduling and dynamic risk management of the computing power network (CPN), a resource scheduling and risk management system and method based on a dynamic simulator are proposed. Through the dynamic modeling and simulation of computing resources, task requirements and security risks in the CPN environment, combined with a multi-dimensional optimization algorithm, the coordinated optimization of resource allocation, task scheduling and risk control is realized, providing support for the efficient operation and security guarantee of the CPN system.
[0025] Example 1: A CPN ecological digital simulator based on multi-agent self-organizing interaction includes: a multi-agent self-organizing interaction module, a resource modeling and dynamic allocation module, a service scheduling and coordinated optimization module, a risk management and emergency response module, and a policy evaluation and optimization control module. (As Figure 1 ) Among them, the multi-agent self-organizing interaction module, as the bottom layer module, is used to simulate the interaction and simulation of each link in the CPN system, including the real-time collaborative work among computing nodes, storage nodes and network nodes; the resource modeling and dynamic allocation module, based on the multi-agent interaction module, formulates a preliminary resource allocation plan according to the current task requirements and resource status, providing a basis for task scheduling; the service scheduling and coordinated optimization module is mapped and associated with the resource allocation module, plans the execution order of tasks according to the resource allocation plan, and optimizes the overall performance indicators of the task chain, including execution delay and throughput, etc. The risk management and emergency response module monitors and processes potential risks in the CPN system in real time by analyzing possible faults and bottlenecks in the process of resource allocation and task scheduling, ensuring the security and reliability of the system. The policy evaluation and optimization control module comprehensively evaluates the operation effects of the above modules, generates optimization feedback, and transmits it to the multi-agent interaction module to adjust the system simulation model, improving the efficiency and accuracy of the next round of resource allocation and scheduling. Through the collaborative work of the above modules, the system realizes the organic combination of multi-agent simulation, dynamic resource allocation, efficient task scheduling and intelligent risk control.
[0026] The functions of each module are as follows: Multi-agent self-organizing interaction module: This module introduces an agent model to simulate the interaction and cooperation among multiple roles in the CPN ecosystem. Through multi-agent reinforcement learning (MARL) technology, the adaptive cooperation among different agents is realized. The multi-agents include resource agents, service agents and risk agents. Each agent continuously optimizes its strategy through independent learning to adapt to environmental changes. The interaction among agents includes dynamic allocation of resources, scheduling of service tasks, early warning and response to risks, etc., ensuring the stability and efficiency of the system.
[0027] Resource Modeling and Dynamic Allocation Module: This module is responsible for simulating and managing the dynamic characteristics of computing and networking resources, which include data, computing, storage, and networking. Based on the real-time resource status in the computing and networking environment collected (such as computing node load, network bandwidth), task requirements (such as task type, priority), and security information (such as user permissions, interaction behavior), etc., it establishes a detailed resource model, tracks the resource status in real-time, and dynamically allocates resources according to system requirements. This module can simulate the topological structure of resources, define the characteristics of different resources (such as bandwidth, latency, capacity, etc.), and optimize the resource allocation strategy through intelligent algorithms to cope with changes in the network environment.
[0028] Service Scheduling and Cooperative Optimization Module: This module realizes the optimized scheduling of tasks to ensure that each task can be reasonably allocated according to factors such as priority, resource requirements, and latency sensitivity. By constructing an efficient task scheduling algorithm, it optimizes the execution efficiency of tasks in a heterogeneous resource environment and ensures that tasks are completed under the premise of meeting constraints. This module also takes into account the optimization of multi-task chains to ensure the efficient operation of complex business processes.
[0029] Risk Management and Emergency Response Module: The risk management module monitors potential risks in the computing and networking environment and, through simulating potential privacy leaks and security threats in interactions, constructs a risk assessment model based on historical data and simulation results to provide a basis for system improvement and optimization. By dynamically adjusting scheduling strategies and access permissions, it monitors the load, performance, and fault status of computing and networking resources in real-time. It constructs a dynamic risk map to predict possible risk points (such as node failures, network congestion, etc.) and provides timely risk warnings. In addition, by constructing a risk protection system integrating multiple privacy technologies, it actively defends against malicious attacks on platform data security, etc. When a risk occurs, through agent collaboration, it adjusts resource allocation and task scheduling strategies to ensure that the system responds quickly and takes emergency measures such as fault isolation and task reconstruction. The system is also equipped with a redundant resource pool and a dynamic fault tolerance mechanism to enhance the reliability and fault tolerance of the system and reduce the negative impact of risks.
[0030] Policy Evaluation and Optimization Regulation Module: The simulation evaluation module is responsible for outputting the performance indicators of the system according to the simulation results, such as resource utilization rate, task completion rate, average latency, risk detection rate, and recovery time, etc., to evaluate the overall operation status of the system. According to these indicators, it generates optimization suggestions, proposes improvement plans, and further optimizes resource scheduling algorithms, task allocation strategies, or risk management methods to ensure that the system performance can reach the best state in different scenarios.
[0031] The processing flow of the digital simulator is as follows: Step 1: The multi-agent self-organizing interaction module coordinates the information sharing and task collaboration of each agent, determines the resource requirements, and transmits them to the resource modeling and dynamic allocation module.
[0032] Step 2: Based on the current system state, the Resource Modeling and Dynamic Allocation Module models the resources and optimizes the dynamic resource allocation plan. After ensuring rational utilization, it provides the results to the Business Scheduling and Cooperative Optimization Module.
[0033] Step 3: The Business Scheduling and Cooperative Optimization Module formulates task execution strategies based on the allocated resources and optimizes the cooperative scheduling among tasks.
[0034] Step 4: The operating status and key decisions of all modules are monitored by the Risk Management and Emergency Response Module to ensure the overall stability and security of the platform. When resource bottlenecks, privacy leaks, abnormal behaviors, or emergencies are detected, this module timely assesses the risks, adjusts resource allocation, scheduling strategies, or triggers emergency response mechanisms to reduce system risks.
[0035] Step 5: The Policy Evaluation and Optimization Regulation Module continuously analyzes the execution effects of each module, optimizes the intelligent agent interaction methods, resource management strategies, and scheduling logics based on historical data, and feeds back optimization suggestions to the Multi-Agent Self-Organizing Interaction Module to enhance the intelligent adaptability of the system and form a closed-loop optimization.
[0036] Step 1 includes the following steps: Step 101: Agent Modeling 1) Agent Definition: In this step, for various roles in the system, independent reinforcement learning agent models are successively constructed, including resource agents, business agents, and risk agents. Each agent will select corresponding actions according to the real-time obtained environmental state and continuously learn and optimize decision-making strategies.
[0037] Specifically, Resource Agent: Extract the real-time status information of resource allocation as the state space , and perform resource scheduling operations according to the current state as the action space ; Business Agent: Capture the dynamic status of task execution to form the state space , and generate decision-making actions for task allocation based on this to form the action space ; Risk Agent: Update the state space based on the latest information of risk assessment , and take risk response measures according to this state to define the action space . 2) Decision strategy optimization: During the process of various agents executing actions, the system adopts a decision-making mechanism based on Q-learning to continuously adjust and optimize their behavioral strategies. Each agent evaluates possible actions in different environmental states and selects the action that maximizes the expected return. Specifically, the system selects actions according to the following decision rules: , where is the policy function, is the state-action value function. During the policy execution process, the agent dynamically perceives environmental changes, updates experience data, and continuously improves the effectiveness of the policy through repeated iterations to ensure that various agents can adaptively adjust decisions and complete interactions and collaborations in complex environments.
[0038] Step 102, perform reinforcement learning using the multi-agent reinforcement learning (MARL) framework 1) Multi-agent reinforcement learning (MARL) mechanism: In the computing and networking environment, the availability of computing, storage, and network resources changes over time, and there are significant differences in the performance, bandwidth, and latency of heterogeneous resources, which increases the complexity of simulation and scheduling. How to construct a dynamic resource model and accurately capture environmental changes is a challenging problem. In addition, each agent (resource agent, task agent, risk agent) has different goals, and there are coexisting resource competition and collaboration requirements, which are prone to conflicts, leading to a decline in system performance. How to design an information sharing and decision-making mechanism between agents to avoid local optima and conflicts is an urgent problem to be solved.
[0039] The present invention adopts a multi-agent reinforcement learning framework, utilizes the self-organizing interaction and collaboration between multi-agents, and continuously optimizes the overall performance. Each agent real-time perceives the environmental state, executes actions, and updates the policy according to the environmental feedback, gradually improving the global benefit of the system. For agents with different roles, the system dynamically adjusts the policy through multiple rounds of reinforcement learning training to ensure that each agent achieves efficient collaboration in a complex environment.
[0040] Specifically, each agent first real-time monitors the dynamic changes of computing and networking resources and extracts environmental information related to its own tasks: Resource agent: Perceives resource utilization rate, resource idle amount, and computing node load, and forms a resource status vector ; Business agent: Monitors the task execution status, including task arrival rate, execution time, and resource consumption, and constructs a task status vector ; Risk agent: Based on historical data, identifies resource bottlenecks and security hazards, and forms a risk assessment vector .
[0041] These state vectors are updated at each time step, and the update strategy is as follows: , where is the feature extraction function, represents the current observation, is the historical interaction record, is the time step.
[0042] Then, each agent executes a reinforcement learning decision based on the current state space, generates and executes corresponding resource scheduling, task allocation, or risk response actions. The resource agent selects the optimal resource allocation plan in the resource scheduling space to ensure the efficient use of computing resources; the business agent adjusts task scheduling according to the resource situation within the task allocation space to ensure the timeliness and balance of task execution; the risk agent generates risk warnings based on the prediction model in the risk response space to assist other agents in avoiding potential threats. Action selection follows the Q-learning decision strategy, and the reward for each possible action is calculated through the following formula: where represents the state-action value function, is the time when the agent selects an action.
[0043] 2) Agent collaboration: After the action is executed, the agents share key environmental information in real time and adjust their respective behavior strategies: the resource agent provides the business agent with the latest resource utilization rate, and the business agent adjusts the task scheduling plan accordingly; the risk agent predicts potential risks through a classification model and sends risk warnings to other agents to assist in resource and task adjustment; each agent updates its own experience pool according to environmental changes to enhance decision-making accuracy.
[0044] Information sharing is balanced using a local game model to ensure the effectiveness and optimality of information during the collaboration process, described as: , where is the utility function of agent , represents the reward, represents the execution cost of the action.
[0045] Each agent updates the Q value according to the environmental feedback after the action is executed and continuously optimizes the strategy. The system uses the temporal difference (TD) update method: , Among them, is the learning rate, γ is the discount factor, is the immediate reward. As the training iterates, the agent continuously improves the accuracy of decision-making and the global benefit of the system, and finally realizes the dynamic optimization and collaborative scheduling of computing and network resources.
[0046] 3) Game theory optimization: Design a collaboration mechanism among agents through a game theory model to encourage agents to adjust their behaviors in a non-equilibrium state and improve the overall benefit.
[0047] The non-equilibrium state refers to the imbalance of global resource allocation caused by each agent's local optimality, manifested as the exhaustion of resources of some computing nodes or task delays.
[0048] Each agent adjusts its behavior based on the global benefit function with the goal of maximizing the system benefit: . During the game process, the system continuously balances resource consumption and task execution to ensure that each agent reaches the Nash Equilibrium, achieving efficient collaboration and global optimality of computing and network resources.
[0049] Step 2 includes the following steps: Step 201, resource modeling: The system first uses a graph theory model to map various resources (data, computing power, storage, network, etc.) in the computing and network platform into a graph structure for dynamic modeling and monitoring of the resource topology. Each resource type is represented by different types of graph nodes, and the node attributes include resource capabilities (such as computing power, storage capacity, bandwidth, etc.), and the edges represent the network connections between resources, such as: Among them, each node has different attributes, data , bandwidth , storage capacity , computing power etc. The resource status is updated through real-time monitoring, and the topological structure of the resources is represented by the following mathematical model: , Among them, represents the set of resource nodes , represents the connection set between resource nodes.
[0050] After completing the resource modeling in Step 202, the system enters the resource allocation stage, and optimizes the dynamic scheduling of resources based on reinforcement learning (RL). This process involves the following steps: 1) State Representation: Set the environmental state , which is defined as the current state of resources. The state vector contains information such as available bandwidth, storage space, and computing resource occupancy rate.
[0051] 2) Action Selection: The agent selects a resource allocation plan according to the current state , and the action space includes different resource allocation strategies, such as task migration, bandwidth adjustment, storage allocation, etc.
[0052] 3) Execution of the Plan: After selection , the system immediately performs the corresponding resource scheduling operations; tasks are migrated between different computing nodes, and storage and bandwidth resources are reallocated.
[0053] 4) Reward Calculation and Policy Update: After the task is completed, the system calculates the reward function , which measures the task completion rate and resource utilization rate; it is designed based on factors such as task completion and resource utilization rate. The allocation policy is optimized by maximizing the cumulative reward: , where is the discount factor, representing the attenuation of future rewards.
[0054] Step 203, intelligent resource allocation further performs the dual-objective optimization of resource utilization rate and task completion time on the basis of reinforcement learning optimization by designing a multi-objective optimization algorithm. The specific process is as follows: Combining the resource requirements such as computing, storage, and bandwidth, with the goal of minimizing the system latency and maximizing the resource utilization rate : , where is the latency, represents the function of task execution latency, indicating the relationship between latency and the system execution time.
[0055] Specifically, describes the impact of resource allocation on latency, and the optimization goal is to minimize this latency. is a mapping function, which depends on the specific implementation of the system and the resource allocation policy; represents the total number of resource nodes in the system, is the utilization rate of each resource node.
[0056] The described resource allocation policy determines how to allocate resources to different tasks or applications in the system according to different resource requirements (such as computing, storage, bandwidth, etc.). This policy needs to be dynamically adjusted according to the system state, task requirements, and resource availability.
[0057] System latency : The resource allocation strategy directly affects the execution time of tasks and the system response speed. Poor resource allocation may lead to tasks being executed in a high-latency environment. Therefore, factors for reducing latency need to be considered in resource allocation, such as by reasonably allocating bandwidth, storage, and computing resources to avoid overload and resource competition.
[0058] Resource utilization : Resource utilization is an indicator to measure the effective usage degree of each resource node in the system. The resource allocation strategy realizes the maximization of resource utilization by reasonably arranging resources to avoid resource idleness or overuse.
[0059] Reasonable resource allocation can effectively improve the utilization rate of each node, making the use of resources such as computing, storage, and bandwidth more efficient. The resource allocation strategy achieves two goals by optimizing resource scheduling in the system: 1) Minimize system latency, that is, by reducing the idle time of resources or avoiding latency caused by task competition; 2) Maximize resource utilization, that is, by balancing the use of each resource node to avoid waste of resources.
[0060] Specifically, the resource allocation strategy is implemented through particle swarm optimization (PSO). The particles represent the resource allocation strategy, and the particle positions are adjusted according to the objective function. The position vector of each particle represents a resource allocation decision, while the velocity of the particle represents the adjustment speed of the resource allocation strategy. In each iteration, the particle updates its position and velocity according to the value of the objective function, thereby gradually optimizing the resource allocation strategy. The update of the particle follows the classical PSO formula, depending on the current position and velocity of the particle, as well as the historical optimal position and global optimal position of the particle.
[0061] , Among them, represents the position of the particle (i.e., the resource allocation strategy), represents the velocity of the particle, is the historical optimal position of the particle, is the global optimal position, is the inertia weight, , is the learning factor, , is a random number.
[0062] Step 3 includes the following steps: Step 301, Task Scheduling Algorithm: Design a task scheduling strategy based on priorities and constraints (such as latency sensitivity, resource requirements, etc.). First, establish a task dependency graph (DAG) to represent the execution order and dependency relationships of tasks. The task scheduling objective function is designed as: , where, is the start time of task , is the execution time of the task.
[0063] Step 302, Multi-Task Chain Optimization: Adopt the Particle Swarm Optimization (PSO) method to search for the optimal solution of task scheduling through the particle swarm, ensuring the execution efficiency of the multi-task chain and minimizing the overall latency: . Step 303, Resource Constraint and Scheduling Optimization: During the scheduling process, ensure the efficient utilization of resources by combining resource allocation and scheduling tasks. The resource constraint formula is: , where, is the resource allocated to task , is the maximum resource capacity of the system.
[0064] For example, Figure 3 is a schematic diagram of the service collaborative scheduling process based on the multi-task chain.
[0065] Step 4 is specifically as follows: Step 401, Risk Assessment: In the computing and network environment, risk assessment is the monitoring of potential threats such as privacy leakage, network attacks, node failures, resource congestion, etc. To conduct dynamic risk management, the present invention introduces a risk assessment model based on historical data and real-time resource status, and uses machine learning models (such as support vector machines, decision trees, etc.) for risk prediction to evaluate the occurrence probability and impact of risks.
[0066] The risk assessment includes the following execution processes: Data Collection: The system collects real-time resource status (such as computing load, network latency, task scheduling information, privacy access records, etc.) and historical risk events to construct a multi-dimensional risk feature data set.
[0067] Risk Modeling: Perform feature engineering on the collected data, use machine learning models to classify and predict risk factors, and output the occurrence probability and impact degree of each type of risk.
[0068] Dynamic update: The system dynamically updates the risk model according to real-time data to ensure the accuracy and timeliness of risk prediction.
[0069] Step 402, Risk indicators: For each resource node and task , evaluate its risk status .
[0070] Specifically, it includes the following risk indicators: Resource load risk: Based on the current load of the resource and the maximum capacity of the resource, calculate the load ratio as the risk indicator of the resource load: . Network latency risk: Network latency is an important factor affecting the task completion time, and it will increase the risk of task scheduling when the latency is relatively high. The latency risk can be expressed as the ratio between the network latency at time and the maximum tolerable latency of the resource.
[0071] Task scheduling risk: The risk of task scheduling can be expressed as the ratio between the task execution delay and the task priority.
[0072] Privacy and security risk: There may be a risk of security leakage during the task execution. Specifically, the calculation of privacy and security risks can be based on the historical data of the system and the task interaction behavior prediction. By analyzing the correlation between task execution and privacy leakage in the historical data, evaluate the risk probability of the task in the current state.
[0073] Step 403, Risk comprehensive assessment: The comprehensive risk assessment of all resources and tasks can be expressed as: , where represents the total risk type, represents the risk of the th type of risk at time. is used to weigh the influence of different risk factors. Through the comprehensive analysis of various risks, construct a risk map to represent and evaluate the potential risks in the system, model the relationships and interactions between various risk factors in the system in the form of a graph. Display various risk nodes, risk propagation paths, influencing factors and risk intensities that may appear in the system, so as to help identify and respond to potential threats. Use graph analysis technology to evaluate the risk propagation path to judge the current risk status of the system.
[0074] Step 404, Risk Simulation: Based on the above risk assessment model, conduct risk simulation and dynamic adjustment.
[0075] The present invention designs a multi-stage simulation process for predicting and responding to different types of risks. By simulating different network topologies and resource load changes, the system can predict potential risk points. Suppose at time the system is in state , and the multi-stage simulation process includes the following stages: 1) Resource Change Simulation Stage: In this stage, the system simulates the load changes of resources by setting different load change rates to simulate the load changes of resources at future times: , where is the load change rate, and is the time step.
[0076] 2) Task Scheduling Change Simulation Stage: This stage simulates the impact of different task scheduling strategies on the task completion time, with a focus on evaluating the risks of task scheduling, especially the interaction between scheduling strategies and resource states. According to the task scheduling strategy, simulate the impact of different scheduling methods on the task completion time: , where is the mapping function of the task completion time to the current resource state and scheduling strategy.
[0077] 3) Privacy Leakage Risk Simulation Stage: In this stage, the system predicts the possibility of privacy leakage by analyzing the privacy leakage risks during task execution, considering factors such as the interaction between tasks and resource access patterns. Through continuous monitoring of the system operation state, resource usage, and simulation results, construct a dynamic risk map, including resource nodes, task nodes, risk factors, risk propagation paths, risk intensities, and the occurrence probability and impact of risks, to help the system conduct a comprehensive risk assessment and dynamic adjustment.
[0078] The dynamic risk map is a dynamic extension of the risk map. It not only shows the static risk state of the system but also includes how risks change over time and how to perform real-time assessment and adjustment under changing system states. The dynamic risk map is updated in real time as the system state changes to reflect new risk predictions and emergency responses. The risk map correlates each node, resource, and computing task in the computing and networking environment to form risk hotspots, visually showing possible risk points and dynamically adjusting according to the current system state. Key risk nodes in the computing and networking environment are identified through graph theory and big data analysis techniques. For example, node failures, network congestion, resource bottlenecks, etc. These factors may lead to privacy leakage or system crashes. The system dynamically adjusts task allocation, scheduling strategies, and access permissions based on real-time resource status and risk point predictions. For example, when a privacy leakage risk is detected in a certain computing node, the system automatically migrates the task to a secure node to avoid risk spread.
[0079] Specifically, the process of simulating the privacy leakage risk is as follows: 1. Task execution path modeling: The execution process of a computing task can be represented as a directed graph , where represents computing nodes, storage resources, and network devices; represents the execution path of the task, that is, the data transmission or computing dependency relationship. Given a task executing on the path , the privacy leakage probability of the task can be modeled as: , where is the privacy leakage probability of the edge on the path. Assuming that the leakage events are independent, the total leakage probability is the product of the complements of the leakage probabilities of each edge on the path.
[0080] 2. Task interaction pattern analysis: Interaction between tasks leads to information sharing, thus increasing the privacy leakage risk. The interaction between tasks can be modeled as a task interaction matrix . If tasks and share data, then the matrix element , otherwise . The privacy leakage contribution of a task can be expressed as: , where represents the entropy of the data shared by tasks and , and represent the data entropy of each task, Privacy leakage gain caused by quantization task interaction. The global privacy leakage contribution of the task is: . 3. Resource access pattern modeling: The privacy leakage risk is also affected by resource access behaviors such as computing, storage, and network. We define the access frequency matrix , and its element is the proportion of the number of times each task accesses resource to the total number of accesses. Based on this, the leakage risk of a resource is equal to the weighted sum of the risks of its accessing tasks: .
[0081] 4. Dynamic risk score calculation: To quantify the overall privacy leakage risk of the system, we define the risk score of a task: , where is an adjustable weight to balance different factors. The overall risk score of the system is the sum of the task scores.
[0082] 5. Privacy leakage propagation modeling: The propagation of privacy leakage can be modeled as a Markov process, and the propagation of leakage between nodes can be represented by the transition matrix : , where is the data sharing intensity between tasks and , and the hyperparameter controls the propagation attenuation. The propagation of privacy leakage between tasks is: . 6. Dynamic task scheduling optimization: To reduce the privacy leakage risk, the scheduling strategy of tasks is dynamically adjusted with the goal of minimizing the privacy leakage risk while ensuring the computing performance of the system. If the privacy leakage risk at a certain starvation point exceeds the threshold, the task is migrated to a low-risk node.
[0083] Step 405, Risk protection: To address privacy leakage and security threats in the computing and networking platform, the present invention proposes a protection system that integrates multiple privacy technologies for risk protection.
[0084] The protection system that integrates multiple privacy technologies specifically includes: 1) Multi - technology integration: Based on technologies such as homomorphic encryption, differential privacy, privacy computing, and SMPC, a privacy protection mechanism suitable for different risk scenarios is constructed. When the system detects a privacy leakage risk, it can dynamically switch to a more stringent privacy protection technology, that is, adopt an automated privacy policy selection algorithm. According to the risk level, computing requirements, and privacy sensitivity of the task, a suitable privacy protection scheme is selected.
[0085] 2) Multi - layer protection adaptation: Through a hierarchical protection mechanism, privacy protection is organically combined with data security protection. Differential privacy is used for data processing in low - risk tasks, while homomorphic encryption or SMPC is used for encryption processing in high - risk tasks to ensure that the platform security is not threatened.
[0086] Step 406, Risk response: When the system detects that the potential risk exceeds the set threshold, an emergency response mechanism will be activated to adjust resource allocation and task scheduling. The specific adjustment process is as follows: 1) Resource allocation adjustment: When the load risk is high, the system will schedule tasks to resource nodes with lighter loads.
[0087] 2) Task scheduling adjustment: According to the real - time task execution situation and network latency, the system will dynamically adjust the scheduling order of tasks based on the migration learning and task reconstruction optimization algorithm to minimize the task completion time and network latency. For example, when a node failure occurs, the system performs task reconstruction through a redundant resource pool and a fault - tolerance mechanism to avoid the expansion of system failures.
[0088] 3) Risk emergency response: When the system detects security risks (such as privacy leakage, malicious attacks, etc.), it closes the access rights in a timely manner. The system quickly transfers tasks to safe and healthy nodes through the task migration mechanism to ensure that the computing tasks are not interrupted. At the same time, it activates the secondary identity authentication mechanism to upgrade the defense of the leaked sensitive information, quickly conducts an audit review, cuts off the attack path in a timely manner, and improves the defense system.
[0089] Step 5 comprehensively analyzes and optimizes the system's resource scheduling, task allocation, and risk management strategies through real - time simulation and quantitative evaluation of performance indicators. The strategy evaluation and optimization control module integrates functions such as simulation evaluation, generation of optimization suggestions, and iterative optimization of strategies to ensure the system's dynamic adaptability and operation efficiency in a complex environment.
[0090] The specific execution process is as follows: Step 501, Simulation evaluation: Execute the simulation: Calculate multiple performance indicators of the system based on the simulation data, including resource utilization rate, task completion rate, average latency, risk detection rate, and recovery time, providing a quantitative basis for optimization.
[0091] Real-time monitoring: Continuously monitor the ratio of the load to the maximum capacity of each resource node to evaluate the overall utilization of resources. Meanwhile, track the completion of tasks within a specific time period, record the ratio of successful tasks to the total number of tasks, and judge the effectiveness of the task scheduling strategy.
[0092] Performance analysis: Calculate the average time required for a task from assignment to completion to evaluate the efficiency of task execution and the timeliness of system response. Based on real-time data, analyze the detection rate of the system for potential risks such as privacy leakage and resource overload, and evaluate by comparing simulation predictions and actual risk occurrences.
[0093] Abnormal recovery detection: Statistically calculate the average time required for the system to recover from an abnormal state to normal operation, measure the system's fault tolerance ability, and provide a quantitative basis for subsequent optimization.
[0094] Result summary: Integrate the above-mentioned indicator data to generate a comprehensive performance evaluation report, and identify key issues affecting system performance based on the bottleneck analysis method, such as low resource utilization, excessive task latency, or false detection of risks.
[0095] Step 502, Policy optimization: Construct the objective function: According to the indicators in the performance evaluation report, set corresponding weight values to construct a multi-objective optimization function that reflects the optimization requirements in multiple dimensions such as resource utilization, task completion, and risk prevention. Adopt a dynamic optimization and feedback mechanism to construct a multi-objective optimization function based on the performance indicator results: , where represents the weight of the performance indicator, which is set according to the actual situation.
[0096] Solve the optimization algorithm: According to the set objective function, calculate the optimal solution through a dynamic optimization mechanism to generate policy optimization suggestions. These suggestions include adjusting resource allocation, optimizing the scheduling algorithm, and strengthening risk prevention strategies.
[0097] Policy adjustment: According to the optimization results, adjust the weights and priorities of each node in the resource model in real time to ensure the balance of resource usage, reduce resource waste, and optimize the task scheduling strategy to reduce latency and increase the task completion rate.
[0098] Risk detection optimization: According to the optimization suggestions, adjust the parameters and response strategies of the risk detection model to improve the accuracy of risk identification and enhance the system's recovery ability to ensure the effectiveness of dealing with future potential risks.
[0099] Step 503, Loop feedback: (such as Figure 4 ). Real-time monitoring and feedback: By monitoring the running status of the system in real time, dynamically adjust the resource allocation and task scheduling strategies, and feedback the new adjustment plan to the system.
[0100] Execute simulation and prediction: Based on historical data and real-time collected information, execute simulation to predict the future running status of the system, and generate corresponding improvement suggestions for resource allocation and scheduling.
[0101] Continuous optimization: Based on the new strategy adjustment and simulation prediction, dynamically monitor the running changes of the system, continuously evaluate and optimize the performance, and continuously approach the global optimal goal.
[0102] Strategy iteration: According to the real-time feedback and performance evaluation results, continuously iterate and optimize the strategy to ensure that the system can flexibly adapt to the changing environment and continuously improve the running efficiency.
[0103] Embodiment 2: As Figure 2 shown, the process of the resource modeling and dynamic allocation module of the present invention includes the following steps: Step 1. Resource modeling: Perform real-time characterization modeling on the computing resources, storage resources, and network resources in the computing and network system, and generate a resource status table using a graph theory model.
[0104] Step 2. Task requirement analysis: The resource allocation module receives the task chain requirement information, including task priority, execution time limit, and dependency relationship, and analyzes the resource requirement characteristics of the task.
[0105] Step 3. Preliminary resource allocation: According to the global resource status table, task requirements, and environmental risks, adopt a resource dynamic scheduling strategy based on reinforcement learning to formulate a preliminary resource allocation plan, and ensure the balance and feasibility of resource allocation.
[0106] Step 4. Optimize the allocation plan: Use a multi-objective optimization algorithm to interactively iterate and optimize the preliminary plan, further perform dual-objective optimization of resource utilization rate and task completion time, and implement the resource allocation strategy through particle swarm optimization to balance multiple objectives such as resource utilization rate, task execution efficiency, and energy consumption.
[0107] Step 5. Generate the allocation result: Output the finally optimized resource allocation plan to the requester, and transfer the result to the service scheduling module as the basis for subsequent task scheduling.
[0108] Through the above process, the resource modeling and dynamic allocation module can efficiently match resources with task requirements and achieve the optimization of resource utilization.
[0109] Key points of the present invention: 1. Digital Simulator for Computing and Network Ecosystem Based on Multi-Agent Self-Organizing Interaction: The present invention proposes a digital simulator for computing and network ecosystem based on multi-agent self-organizing mechanism, which can simultaneously simulate the dynamic interaction of heterogeneous computing resources, network environment, risk changes and task requirements. The innovation lies in that the simulator captures and responds to environmental changes and task execution status in real time through the autonomous interaction and cooperation among multi-agents, and then provides efficient resource scheduling decisions. Different from traditional static simulation, the simulator of the present invention can dynamically adjust the model according to the feedback information in the actual task execution process, so as to accurately predict the performance of computing and network coordination in different scenarios, improve the simulation accuracy and adaptability, and is especially suitable for complex and changeable computing and network environments.
[0110] 2. Dynamic Resource Allocation Mechanism Based on Multi-Objective Optimization: The present invention proposes a dynamic resource allocation mechanism based on intelligent optimization algorithm. By real-time monitoring of computing task requirements, network status and resource bottlenecks, a multi-objective optimization framework is adopted for resource scheduling. Different from traditional static allocation strategies, this mechanism can adjust the allocation of computing power and network resources in real time to ensure that the system can still optimize the utilization of each resource when resources are tense or requirements change. The innovation lies in that this mechanism not only optimizes a single performance index (such as latency or throughput), but also comprehensively considers multiple performance objectives (such as task completion time, task priority, etc.) for dynamic adjustment, so as to achieve optimal resource allocation for cross-domain tasks.
[0111] 3. Business Collaboration Scheduling Strategy Based on Multi-Task Chains: The present invention adopts the particle swarm optimization algorithm and proposes a collaborative scheduling strategy for multi-task chains. The innovation point is that this strategy can utilize the particle swarm algorithm to optimize the scheduling among multi-task chains according to the priorities, task characteristics and resource status of different services. Different from the prior art, this strategy realizes efficient scheduling of cross-domain tasks by dynamically adjusting the task execution order and resource allocation. Specifically, during the operation of the system, the particle swarm algorithm continuously optimizes the scheduling paths of each task in the task chain, reduces resource conflicts, and improves the task response speed and the overall execution efficiency of the system.
[0112] 4. Integrated Risk Management and Dynamic Regulation Mechanism: The present invention designs an integrated risk management system combined with a multi-dimensional evaluation model, which can predict and control various risks faced by the computing and networking platform in real time (such as resource bottlenecks, network delay fluctuations, privacy leaks, etc.). This system analyzes risk sources through a risk assessment model, and on this basis, adopts real-time monitoring, privacy protection technologies, and risk simulation means to effectively identify potential risks and automatically take measures for intervention. The innovation lies in that risk management is not only a post-event response, but also can prevent potential risks through a dynamic regulation mechanism based on historical data and real-time monitoring information, ensuring the stability and security of the system in a complex multi-task environment, achieving active defense, timely response, and rapid recovery of the risks of the computing and networking platform, and enhancing the privacy protection and security protection capabilities of the platform in complex computing tasks and multi-domain environments.
[0113] 5. Strategy Generation and Regulation Technology Based on Multi-Agent Self-Organizing Interaction: The present invention proposes a strategy generation and regulation technology for multi-agent self-organizing interaction based on reinforcement learning. The innovation lies in that through the interactive learning among agents, the agents continuously adjust their behavior strategies according to local task requirements and environmental changes, so as to achieve adaptive optimization of task scheduling, resource allocation, and risk management. Different from traditional centralized scheduling methods, this mechanism improves the adaptability of the system to complex tasks and dynamic changing environments through the distributed interaction and collaborative work of multi-agents, and significantly improves the overall efficiency and flexibility of the computing and networking system. Through the mechanism of reinforcement learning, the agents can dynamically adjust strategies according to the priorities of tasks and the idle conditions of resources, ensuring the reasonable allocation of resources among different tasks and maximizing the system performance to cope with complex computing and networking collaborative tasks, and enhancing the overall efficiency and flexibility of the system.
[0114] The above description is only a description of the preferred embodiments of the present application, and does not limit the scope of the present application in any way. Any change or modification made by any person skilled in the art according to the technical content disclosed above shall be regarded as an equivalent effective embodiment, and all belong to the scope protected by the technical solution of the present application.
Claims
1. A computing and networking ecological digital simulator based on multi-agent self-organizing interaction, characterized in that including: a multi-agent self-organizing interaction module, a resource modeling and dynamic allocation module, a service scheduling and collaborative optimization module, a risk management and emergency response module, and a policy evaluation and optimization control module; Among them, the multi-agent self-organizing interaction module, as the bottom layer module, is used to simulate the interaction and simulation of each link in the computing and networking system, including the real-time collaborative work among computing nodes, storage nodes, and network nodes; Based on the multi-agent interaction module, the resource modeling and dynamic allocation module formulates a preliminary resource allocation plan according to the current task requirements and resource status, providing a basis for task scheduling; The service scheduling and collaborative optimization module is mapped and associated with the resource allocation module, plans the execution order of tasks according to the resource allocation plan, and optimizes the overall performance indicators of the task chain, including execution delay and throughput, etc.; The risk management and emergency response module monitors and processes potential risks in the computing and networking system in real time by analyzing possible faults and bottlenecks in the process of resource allocation and task scheduling, ensuring the security and reliability of the system; The policy evaluation and optimization control module comprehensively evaluates the operation effects of the above modules, generates optimization feedback, and transmits it to the multi-agent interaction module to adjust the system simulation model, improving the efficiency and accuracy of the next round of resource allocation and scheduling; Through the collaborative work of the above modules, the system realizes the organic combination of multi-agent simulation, dynamic resource allocation, efficient task scheduling, and intelligent risk control.
2. The digital simulator for computing network ecosystem based on multi-agent self-organizing interaction according to claim 1, characterized in that, The processing flow of the digital simulator is as follows: Step 1: The multi-agent self-organizing interaction module coordinates the information sharing and task collaboration of each agent, determines the resource requirements, and transmits them to the resource modeling and dynamic allocation module; Step 2, the resource modeling and dynamic allocation module models the resources based on the current system state, optimizes the dynamic resource allocation plan, and provides the results to the service scheduling and collaborative optimization module after ensuring reasonable utilization; Step 3, the service scheduling and collaborative optimization module formulates a task execution strategy according to the allocated resources and optimizes the collaborative scheduling among tasks; Step 4, the operation status and key decisions of all modules are monitored by the risk management and emergency response module to ensure the overall stability and security of the platform; when resource bottlenecks, privacy leaks, abnormal behaviors, or emergencies are detected, this module timely evaluates the risks, adjusts the resource allocation, scheduling strategy, or triggers the emergency response mechanism to reduce system risks; Step 5, the policy evaluation and optimization control module continuously analyzes the execution effects of each module, optimizes the agent interaction mode, resource management strategy, and scheduling logic based on historical data, and feeds back optimization suggestions to the multi-agent self-organizing interaction module to enhance the intelligent adaptability of the system and form a closed-loop optimization.
3. The digital simulator for computing network ecosystem based on multi-agent self-organizing interaction according to claim 2, wherein, Step 1 includes the following steps: Step 101, agent modeling 1) Agent definition: For various roles in the system, independent reinforcement learning agent models are sequentially constructed, including resource agents, service agents, and risk agents; each agent will select corresponding actions according to the real-time obtained environmental state and continuously learn and optimize the decision-making strategy; Specifically, Resource agent: Extract real-time status information of resource allocation as the state space , and perform resource scheduling operations according to the current state as the action space ; Business agent: captures the dynamic state of task execution to form the state space and generates decision-making actions for task allocation based on this to form the action space ; Risk agent: Update the state space based on the latest information of risk assessment , and take risk response measures according to this state to define the action space ; 2) Decision strategy optimization: During the process of various agents executing actions, the system adopts a decision-making mechanism based on Q-learning to continuously adjust and optimize their behavioral strategies; each agent evaluates possible actions under different environmental states and selects the action that can maximize the expected return; Step 102, perform reinforcement learning using the multi-agent reinforcement learning (MARL) framework 1) Multi-agent reinforcement learning mechanism: Adopt the multi-agent reinforcement learning (MARL) framework, utilize the self-organization interaction and collaboration among multiple agents to continuously optimize the overall performance; each agent perceives the environmental state in real time, executes actions and updates the strategy according to the environmental feedback, gradually improving the global benefit of the system; for agents with different roles, the system dynamically adjusts the strategy through multiple rounds of reinforcement learning training to ensure that each agent achieves efficient collaboration in a complex environment; Specifically, each agent first monitors the dynamic changes of computing and network resources in real time and extracts the environmental information related to its own tasks: Resource Agent: Sense resource utilization rate, resource idle amount, and computing node load to form a resource status vector ; Business agent: Monitor the execution status of tasks, including task arrival rate, execution time, and resource consumption, and construct a task status vector ; Risk Agent: Based on historical data, identify resource bottlenecks and security risks, and form a risk assessment vector ; These state vectors are updated at each time step, and the update strategy is: , Among them, is a feature extraction function, represents the current observation, is the historical interaction record, is the time step; Then, each agent performs reinforcement learning decisions based on the current state space, generates and executes corresponding resource scheduling, task allocation, or risk response actions. The resource agent selects the optimal resource allocation plan in the resource scheduling space to ensure the efficient utilization of computing resources; the business agent adjusts task scheduling according to the resource situation in the task allocation space to ensure the timeliness and balance of task execution; the risk agent generates risk warnings based on the prediction model in the risk response space to assist other agents in avoiding potential threats; the action selection follows the Q-learning decision strategy, and the rewards of each possible action are calculated through the following formula: Among them, represents the state-action value function, is the time when the agent selects an action; 2) Agent collaboration: After the action is executed, the agents share key environmental information in real time and adjust their respective behavioral strategies: the resource agent provides the latest resource utilization rate to the service agent, and the service agent adjusts the task scheduling plan accordingly; the risk agent predicts potential risks through a classification model and sends risk warnings to other agents to assist in resource and task adjustment; each agent updates its own experience pool according to environmental changes to enhance the decision-making accuracy; Information sharing is balanced using a local game model to ensure the effectiveness and optimality of information during the collaboration process, described as: , Among them, is the utility function of the agent , represents the reward and represents the execution cost of the action; Each agent updates the Q value according to the environmental feedback after the action is executed and continuously optimizes the strategy; the system adopts the temporal difference update method: , where, is the learning rate, γ is the discount factor, is the immediate reward. As the training iterates, the agent continuously improves the accuracy of decision-making and the global benefit of the system, and finally realizes the dynamic optimization and collaborative scheduling of computing and network resources; 3) Game theory optimization: Design a collaboration mechanism among agents through a game theory model to encourage agents to adjust their behaviors in a non-equilibrium state and improve the overall return. The non-equilibrium state refers to the imbalance of global resource allocation caused by each agent's pursuit of local optimality, manifested as the exhaustion of resources on some computing nodes or task delays; Each agent adjusts its behavior based on the global reward function with the goal of maximizing the system benefit: . During the game process, the system continuously balances resource consumption and task execution to ensure that each agent reaches a Nash equilibrium, achieving efficient collaboration and global optimality of computing and network resources.
4. The digital simulator for computing and network ecosystem based on multi-agent self-organizing interaction according to claim 2, wherein Step 1 includes the following steps: Step 201, resource modeling: The system first uses a graph theory model to map various resources in the computing and network platform into a graph structure for dynamic modeling and monitoring of the resource topology; each resource type is represented by different types of graph nodes, and the node attributes include resource capabilities, and the edges represent the network connections between resources, such as: Among them, each node has different attributes, data , bandwidth , storage capacity , computing power ; The resource status is updated through real-time monitoring, and the topological structure of the resources is represented by the following mathematical model: , Among them, represents a set of resource nodes , represents the connection set between resource nodes; After completing the resource modeling in Step 201, the system enters the resource allocation stage, and optimizes the dynamic scheduling of resources based on reinforcement learning, including the following steps: 1) Status Representation: Set the environmental status , defined as the current status of resources. The status vector contains information on available bandwidth, storage space, and computing resource occupancy rate; 2) Action selection: The agent selects a resource allocation plan according to the current state and selects a resource allocation plan . The action space includes different resource allocation strategies, such as task migration, bandwidth adjustment, and storage allocation; 3) Implementation plan: After selecting the system immediately performs the corresponding resource scheduling operations; tasks are migrated between different computing nodes, and storage and bandwidth resources are reallocated; 4) Reward calculation and policy update: After the task is completed, the system calculates the reward function , which measures the task completion rate and resource utilization rate; designed based on the task completion situation and resource utilization rate factors; optimize the allocation policy by maximizing the cumulative reward: , Among them, is the discount factor, representing the attenuation of future rewards; Step 203, intelligent resource allocation designs a multi-objective optimization algorithm. On the basis of reinforcement learning optimization, it further performs the dual-objective optimization of resource utilization rate and task completion time. The specific process is as follows: Combining resource requirements such as computing, storage, and bandwidth, to minimize system latency and maximize resource utilization rate as the objectives: , Among them, is the delay, a function representing the task execution delay, representing the delay and the relationship with the system execution time; Specifically, describes the impact of resource allocation on latency, and the optimization goal is to minimize this latency; is a mapping function, which depends on the specific implementation of the system and the resource allocation strategy; represents the total number of resource nodes in the system, is the utilization rate of each resource node; The resource allocation strategy determines how to allocate resources to different tasks or applications in the system according to different resource requirements; the resource allocation strategy dynamically adjusts according to the system state, task requirements, and resource availability; The resource allocation strategy achieves two goals by optimizing resource scheduling in the system: 1) Minimize system latency, that is, by reducing the idle time of resources or avoiding the latency caused by task competition; 2) Maximize resource utilization, that is, by balancing the use of each resource node and avoiding waste of resources.
5. The digital simulator for computing network ecosystem based on multi-agent self-organizing interaction according to claim 4, characterized in that, The resource allocation strategy is implemented through particle swarm optimization. The particles represent the resource allocation strategy, and the particle positions are adjusted according to the objective function; the position vector of each particle represents a resource allocation decision, while the velocity of the particle represents the adjustment speed of the resource allocation strategy; in each iteration, the particle updates its position and velocity according to the value of the objective function, thereby gradually optimizing the resource allocation strategy; The update of the particle follows the classical PSO formula, depending on the current position and velocity of the particle, as well as the historical best position and the global best position of the particle. , Among them, represents the position of the particle, that is, the resource allocation strategy, represents the velocity of the particle, is the historical optimal position of the particle, is the global optimal position, is the inertia weight, , is the learning factor, , is a random number.
6. The digital simulator for computing and network ecosystem based on multi-agent self-organizing interaction according to claim 2, wherein Step 3 includes the following steps: Step 301, Task scheduling algorithm: Design a task scheduling strategy based on priorities and constraints; First, establish a task dependency graph to represent the execution order and dependency relationship of tasks; The task scheduling objective function is designed as: , Among them, is the start time of the task , and is the execution time of the task. Step 302, Multi-task chain optimization: Adopt the particle swarm optimization method to search for the optimal solution of task scheduling through the particle swarm, ensuring the execution efficiency of the multi-task chain and minimizing the overall time delay: . Step 303, Resource constraint and scheduling optimization: In the scheduling process, by combining resource allocation and scheduling tasks, ensure the efficient use of resources; The resource constraint formula is: , Among them, is the task allocated resources, is the maximum resource capacity of the system.
7. The digital simulator for computing and network ecosystem based on multi-agent self-organizing interaction according to claim 2, wherein Step 4 includes the following steps: Step 401, Risk assessment: Introduce a risk assessment model based on historical data and real-time resource status, and use a machine learning model for risk prediction to evaluate the occurrence probability and impact of risks; The risk assessment includes the following execution processes: Data collection: The system collects resource status and historical risk events in real time to construct a multi-dimensional risk feature data set; Risk Modeling: Perform feature engineering on the collected data, use machine learning models to classify and predict risk factors, and output the occurrence probability of each type of risk and the degree of impact ; Dynamic update: The system dynamically updates the risk model according to real-time data to ensure the accuracy and timeliness of risk prediction; Step 402, risk metrics: For each resource node and task , evaluate its risk status ; Specifically, it includes the following risk indicators: Resource load risk: Based on resources Current load and the maximum resource capacity , calculate the load ratio as the risk indicator of resource load: . Network latency risk: Network latency is an important factor affecting the task completion time. When the latency is relatively high, it will increase the risk of task scheduling. The latency risk can be expressed as the ratio between the network latency at time and the maximum tolerable latency of resources; Task scheduling risk: The risk of task scheduling can adopt the ratio between task execution time delay and task priority; Privacy and security risk: There may be a risk of security leakage during task execution. Specifically, the calculation of privacy and security risks can be predicted based on the historical data of the system and task interaction behaviors. By analyzing the correlation between task execution and privacy leakage in historical data, evaluate the risk probability of tasks in the current state; Step 403, Comprehensive Risk Assessment: All resources and tasks The comprehensive risk assessment can be expressed as: , Among them, represents the total risk type, represents the th type of risk at the moment, which is used to weigh the impacts of different risk factors; through comprehensive analysis of various risks, a risk map is constructed to represent and evaluate the potential risks in the system, and the relationships and interactions among various risk factors in the system are modeled in the form of a graph; various risk nodes, risk propagation paths, influencing factors, and risk intensities that may occur in the system are shown, so as to help identify and respond to potential threats; graph analysis technology is used to evaluate the risk propagation path to judge the current risk state of the system; Step 404, Risk simulation: Based on the above risk assessment model, conduct risk simulation and dynamic adjustment; Adopt a multi-stage simulation process to predict and respond to different types of risks; predict potential risk points by simulating different network topologies and resource load changes; assume that at time the system is in the state , the multi-stage simulation process includes the following stages: 1) Simulation of resource change stage: In this stage, the system simulates the load change of resources by setting different load change rates to simulate the resources load change at future times: , Among them, is the load change rate, is the time step; 2) The stage of simulating task scheduling changes: This stage simulates the impact of different task scheduling strategies on task completion time, focusing on evaluating the risk of task scheduling, especially the mutual relationship between scheduling strategies and resource status; According to the task scheduling strategy, simulate the impact of different scheduling methods on task completion time: , Among them, is a mapping function of the task completion time with the current resource status and scheduling policy; 3) Stage of simulating privacy leakage risk: In this stage, the system predicts the possibility of privacy leakage by analyzing the privacy leakage risk during the task execution process, considering factors such as the interaction between tasks and the resource access pattern; through continuous monitoring of the system operation state, resource usage, and simulation results, a dynamic risk map is constructed, including resource nodes, task nodes, risk factors, risk propagation paths, risk intensity, and the occurrence probability and impact of risks, to help the system conduct a comprehensive risk assessment and dynamic adjustment; Step 405, risk protection: To address privacy leakage and security threats in the computing and networking platform, a protection system integrating multiple privacy technologies is adopted for risk protection, specifically including: 1) Integration of multiple technologies: Based on homomorphic encryption, differential privacy, privacy computing, and SMPC technologies, a privacy protection mechanism adapted to different risk scenarios is constructed; when the system detects a privacy leakage risk, it can dynamically switch to a more stringent privacy protection technology, that is, an automated privacy policy selection algorithm is used to select an appropriate privacy protection scheme according to the risk level, computing requirements, and privacy sensitivity of the task; 2) Multi-layer protection adaptation: Through a hierarchical protection mechanism, privacy protection is organically combined with data security protection; differential privacy is used for data processing of low-risk tasks, while homomorphic encryption or SMPC is used for encryption processing of high-risk tasks to ensure that the platform security is not threatened; Step 406, risk response: When the system detects that the potential risk exceeds the set threshold, an emergency response mechanism will be activated to adjust resource allocation and task scheduling; the specific adjustment process is as follows: 1) Adjustment of resource allocation: When the load risk is high, the system will schedule tasks to resource nodes with lighter loads; 2) Adjustment of task scheduling: Based on the real-time task execution situation and network latency, the scheduling order of tasks is dynamically adjusted based on the transfer learning and task reconstruction optimization algorithm to minimize the task completion time and network latency; 3) Risk emergency response: When the system detects a security risk, the access permission is closed in a timely manner, and the system quickly transfers the task to a safe and healthy node through the task migration mechanism to ensure that the computing task is not interrupted; at the same time, the identity secondary authentication mechanism is activated to upgrade the defense of the leaked sensitive information, quickly conduct an audit review, cut off the attack path in a timely manner, and improve the defense system.
8. The digital simulator for computing network ecosystem based on multi-agent self-organizing interaction according to claim 7, wherein, The process of simulating privacy leakage risk is as follows: 1) Task execution path modeling: The execution process of a computing task can be represented as a directed graph , where represents computing nodes, storage resources, and network devices; represents the execution path of the task, that is, the data transmission or computing dependency; Given a task executing on the path , the privacy leakage probability of the task can be modeled as: , Among them, is the privacy leakage probability of the edge on the path. Assuming that the leakage events are independent, the total leakage probability is the product of the complements of the leakage probabilities of each edge on the path; 2) Task interaction pattern analysis: Interaction between tasks leads to information sharing, thus increasing the risk of privacy leakage; the interaction between tasks can be modeled as a task interaction matrix , if task and share data, then the matrix element , otherwise ; the contribution degree of task privacy leakage can be expressed as: , Among them, represents the task and the entropy of the shared data, and represents the data entropy of each task, quantifying the privacy leakage gain caused by task interaction; the global privacy leakage contribution degree of the task is: . 3) Resource access pattern modeling: The risk of privacy leakage is also affected by resource access behaviors such as computing, storage, and networking. We define the access frequency matrix , whose elements are the proportions of the number of times each task accesses the resource to the total number of accesses; based on this, the leakage risk of a resource is equal to the weighted sum of the risks of its accessing tasks: ; 4) Dynamic risk score calculation: To quantify the overall privacy leakage risk of the system, we define the risk score of the task : , Among them, is an adjustable weight to balance different factors. The overall system risk score is the sum of the task scores; 5) Modeling the Propagation of Privacy Leakage: The propagation of privacy leakage can be modeled as a Markov process, and the propagation of the leakage between nodes can be represented by the transition matrix as follows: , Among them, is the data sharing intensity between tasks, and the hyperparameter controls the propagation attenuation; the propagation of privacy leakage between tasks is as follows: . 6) Optimization of dynamic task scheduling: To reduce the privacy leakage risk, the scheduling strategy of tasks is dynamically adjusted with the goal of minimizing the privacy leakage risk while ensuring the computing performance of the system. If the privacy leakage risk at a certain starvation point exceeds the threshold, the task is migrated to a low-risk node.
9. The digital simulator for computing and network ecosystem based on multi-agent self-organizing interaction according to claim 2, characterized in that Step 5 includes the following steps: Step 501, simulation evaluation: Execute simulation: Calculate multiple performance indicators of the system based on the simulation data, including resource utilization rate, task completion rate, average latency, risk detection rate, and recovery time, to provide a quantitative basis for optimization; Real-time monitoring: Continuously monitor the ratio of the load to the maximum capacity of each resource node to evaluate the overall utilization of resources; meanwhile, track the task completion situation within a specific time period, record the ratio of successful tasks to the total number of tasks, and judge the effectiveness of the task scheduling strategy; Performance analysis: Calculate the average time required for a task to be allocated and completed to evaluate the efficiency of task execution and the timeliness of system response. Based on real-time data, analyze the detection rate of the system for potential risks such as privacy leakage and resource overload, and evaluate by comparing simulation predictions with actual risk occurrences; Abnormal recovery detection: Statistically calculate the average time required for the system to recover from an abnormal state to normal operation, measure the system's fault tolerance ability, and provide a quantitative basis for subsequent optimization; Result summary: Integrate the above-mentioned indicator data, generate a comprehensive performance evaluation report, and identify key issues affecting system performance based on the bottleneck analysis method, such as low resource utilization, excessive task latency, or false risk detection; Step 502, Policy optimization: Construct an objective function: According to the indicators in the performance evaluation report, set corresponding weight values to construct a multi-objective optimization function that reflects the optimization requirements in multiple dimensions of resource utilization, task completion, and risk prevention; Adopt a dynamic optimization and feedback mechanism to construct a multi-objective optimization function based on the performance indicator results; , Among them, represents the weight of the performance index, which is set according to the actual situation; Solve the optimization algorithm: According to the set objective function, calculate the optimal solution through the dynamic optimization mechanism to generate policy optimization suggestions; these suggestions include adjusting resource allocation, optimizing the scheduling algorithm, and strengthening the risk prevention strategy; Policy adjustment: According to the optimization results, adjust the weights and priorities of each node in the resource model in real time to ensure the balance of resource use, reduce resource waste, and optimize the task scheduling strategy to reduce latency and increase the task completion rate; Risk detection optimization: According to the optimization suggestions, adjust the parameters and response strategies of the risk detection model to improve the accuracy of risk identification and enhance the system's recovery ability to ensure the effectiveness of dealing with future potential risks; Step 503, Loop feedback: Real-time monitoring and feedback: Dynamically adjust the resource allocation and task scheduling strategy by monitoring the system operation status in real time, and feedback the new adjustment plan to the system; Execute simulation and prediction: Based on historical data and real-time collected information, execute simulation to predict the future system operation status and generate corresponding resource allocation and scheduling improvement suggestions; Continuous optimization: Based on the new policy adjustment and simulation prediction, dynamically monitor the operation changes of the system, continuously evaluate and optimize the performance, and continuously approach the global optimal goal; Policy iteration: Continuously iterate and optimize the policy according to the real-time feedback and performance evaluation results to ensure that the system can flexibly adapt to the changing environment and continuously improve the operation efficiency.
Citation Information
Cited By
Multi-agent collaborative operation control method and system based on wireless deterministic network
CN120603038A
Multi-agent traffic incident real-time response system and method
CN120612843A
Intelligent motor remote monitoring method based on industrial bus
CN120915212A
Intelligent motor remote monitoring method based on industrial bus
CN120915212B
Data processing system for land space planning based on big data
CN120950517A