Multi-level distributed collaborative optimization method and system based on agent internet

Abstract

The application discloses a multi-level distributed collaborative optimization method and system based on an agent internet. The method comprises the following steps: distributed access and task distribution of an agent gateway; incomplete information environment sensing based on a free energy principle; dynamic election of a broadcast agent, fusion of local neighbor deviation and global broadcast deviation to generate a weighted control instruction, and realization of fast collaborative convergence; positioning of a key failure step and a responsible agent by using a process reward model, generation of a correction action by means of counterfactual reasoning, and directional reflection and strategy self-evolution by combining proximal policy optimization. The system comprises a reasoning platform layer, an agent gateway layer, an agent cluster layer and a self-evolution monitoring module. The application reduces communication overhead, improves sensing efficiency and collaborative convergence speed under incomplete information, and realizes accurate error correction and efficient resource utilization.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, distributed computing and network communication technology, and in particular to a multi-level distributed collaborative optimization method and system based on the Internet of Intelligent Agents. Background Technology

[0002] In existing multi-agent systems, information interaction and collaboration between agents are mainly achieved through centralized architecture or single-hop / multi-hop local routing protocols. When facing complex and ever-changing task environments, the following problems exist: (1) The system architecture has single-point bottlenecks and high communication overhead: Traditional agent inference systems mostly adopt centralized architecture, which not only has the risk of single-point failure, but also lacks an efficient task distribution mechanism, resulting in high cross-cluster communication overhead and low inference efficiency. (2) Lack of global collaboration leads to slow collaborative convergence: Current multi-agent control research mostly focuses on local information interaction between neighboring agents, lacking effective guidance of the global state. This results in a long time required for multi-agents to complete system tasks in complex environments, and poor group collaboration performance. (3) The error correction mechanism is crude and the resource utilization rate is low: When collaboration fails, existing systems usually assume that each agent is equally responsible for the task failure and perform indiscriminate reflection on each agent. This homogenization ignores the actual situation that usually only the behavior of individual agents leads to global failure, thus introducing ineffective reflection, wasting computing resources and interfering with collaborative optimization. (4) Low efficiency of perception and inference under incomplete information environment: In real-world scenarios such as decision-making in complex environments, intelligent agents often cannot obtain complete environmental information and rely on long-term exploration to obtain effective feedback, resulting in a slow training process and easy to get trapped in local optima.

[0003] To address the aforementioned issues, a technical solution is needed that integrates global and local state deviations for collaborative control based on distributed access of intelligent agent gateways, combines counterfactual reasoning for targeted and precise error correction, and introduces the free energy principle to optimize environmental perception. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-level distributed collaborative optimization method and system based on the Internet of Intelligent Agents, aiming to solve the problems of high single-point failure risk, large cross-cluster communication overhead, slow collaborative convergence in dynamic environments, and difficulty in accurately attributing errors in existing centralized multi-agent systems.

[0005] To achieve the objectives of this invention, the technical solution is as follows:

[0006] A multi-level distributed cooperative optimization method based on the intelligent agent Internet includes the following steps:

[0007] The inference platform generates inference tasks, selects a cluster of capable intelligent agents as the target intelligent agent cluster based on the task type, and distributes the inference tasks.

[0008] After receiving the inference task, the target intelligent agent cluster transforms the inference task into the expected task state of each intelligent agent. Each intelligent agent executes the task according to the expected task state. During the task execution, each intelligent agent completes its own true state inference.

[0009] Simultaneously, a broadcast agent is dynamically elected within the target agent cluster; the remaining ordinary agents use the difference between their actual state and the desired task state of the broadcast agent as the state deviation, calculate the relative state deviation with the state deviations of neighboring agents and the broadcast agent, and then weight and fuse them to form the final control command of the agent. The agent executes the task according to the final control command.

[0010] Furthermore, the optimization method further includes distributed access based on an agent gateway before the inference platform generates inference tasks: using the agent gateway as a virtual gateway, when multiple agents belong to the same virtual local area network, the multiple agents connect to the same agent gateway through their respective access ports to form an agent cluster. The agent gateway then connects to the inference platform through at least one relay port to receive tasks issued by the inference platform; the inference tasks generated by the inference platform are issued through the relay ports of the agent gateway.

[0011] Furthermore, each intelligent agent completes its own true state inference by incorporating the free energy principle into the generative model.

[0012] Furthermore, in the distributed access based on the agent gateway, one agent gateway corresponds to one agent cluster.

[0013] Furthermore, the expected task state includes the physical expected task state and the logical expected task state; the actual state itself includes the physical state and the external logical state; the state deviation includes the physical position deviation and the task semantic deviation; and the control instructions include the physical control input and the logical task control input.

[0014] Furthermore, the agent incorporates the free energy principle into the generative model to infer its true state, including the following steps:

[0015] The agent constrains its actions by adjusting the generative model to minimize the surprise factor S(o). However, since minimizing the surprise factor is unmeasurable, it minimizes the free energy to minimize the surprise factor. The calculation formula is:

[0016] Where s is the environmental state and C is the prior state set. For the agent's action strategy, Let Q be a variational distribution, p be the prior state distribution, EQ be the mathematical expectation operator derived from the variational distribution Q, o be the information observed by the agent, and D be the expected value. KL The KL divergence between the variational distribution and the prior state distribution is used to iteratively update the agent's variational distribution, state estimation results, and action policy by minimizing the free energy. This enables the agent to obtain a state estimate that is closer to the real state in an environment with incomplete information, and to generate an action policy that matches the desired task state.

[0017] Furthermore, the internal dynamic election of a broadcast agent is achieved by each agent calculating the maximum number of hops from itself to all other agents, and selecting the agent with the smallest maximum number of hops as the broadcast agent.

[0018] Furthermore, the remaining ordinary agents use the difference between their actual state and the desired task state as the state deviation, calculate the relative state deviation by comparing it with the state deviations of neighboring agents and the broadcast agents, and then weighted and fused these deviations to form the final control command for that agent. Each agent executes the task according to the final control command, specifically including the following steps:

[0019] S1: Local control input generation: Current local control input The calculation formula is:

[0020] ,in For intelligent agents , The current state of the agent's neighbors. For intelligent agents , For the neighbors , , The neighbor connection status is 1 if connected and 0 otherwise. N is the total number of agents in the agent cluster.

[0021] S2: Global control input generation: Current global control input The calculation formula is:

[0022] in, For broadcast intelligent agents in The relative state deviation at any given moment. Let be the relative state deviation of agent i at time t. For intelligent agents The time delay for receiving the relative state deviation transmitted by the broadcast agent. Weighting coefficients;

[0023] S3: Cooperative Strategy Fusion and Execution: Final Update Control Command for the Agent : .

[0024] Furthermore, the collaborative optimization method also includes: collecting the agent's action trajectory in real time during task execution; if the task fails, performing self-evolutionary error correction based on counterfactual reasoning, including the following steps:

[0025] S1 Collaborative Trajectory Monitoring and Process Reward Scoring: If the task fails, the correctness of the collaborative action trajectory before the task failure is scored step by step. The collaborative action trajectory contains multiple steps. These multiple steps are input into the process reward model, which assigns a correctness score to each step.

[0026] S2 Critical Failure Step and Responsible Agent Diagnosis: Analyze the scoring results and identify the first step among multiple steps whose correctness score is less than a predetermined scoring threshold as a "critical failure step" and find the corresponding "responsible agent".

[0027] S3 Counterfactual Correction and Policy Generation: For the responsible agent and key failure steps, input the collaborative trajectory before the key failure step and the responsible agent's identifier into the counterfactual reasoning model; generate hypothetical corrective actions through counterfactual intervention, while keeping the steps before the key failure step unchanged;

[0028] S4 Correction Verification and Dynamic Reflection Issuance: The correction step is re-input into the process reward model for verification. If the correction score is greater than the original failure score, it is determined as the target correction step. Subsequently, based on the failure diagnosis results and the target correction step, the reflector generates targeted reflection suggestions only for the responsible agent, and the reflector policy is dynamically adjusted using the proximal policy optimization algorithm. The correction instruction is then issued to the responsible agent for execution.

[0029] Furthermore, the reflector policy is dynamically adjusted using a proximal policy optimization algorithm, specifically including:

[0030]

[0031] in, The loss function represents the near-end policy optimization, used to optimize the policy parameters of the reflexor. , The reflective quality score is used to evaluate the feedback from the pre-trained reward model. For the current strategy of the reflector, For reference strategy, Used to constrain the magnitude of policy updates. For reflector Input, For the reflector based on input The generated output results Output the reflector for a specific responsible intelligent agent; by adjusting the reflector strategy, the reflector is made to tend to generate high-quality targeted error correction strategies in subsequent similar scenarios.

[0032] Accordingly, the present invention also provides a multi-level distributed collaborative optimization system based on the Internet of Intelligent Agents, implemented through the above method, including:

[0033] The inference platform layer serves as a computing power and control center for centralized or distributed deployment. It is responsible for receiving user instructions, generating inference tasks based on the data volume and complexity of the task, and allocating computing resources across multiple intelligent agent clusters globally.

[0034] The intelligent agent gateway layer is a virtual gateway deployed on the edge or cloud side, including relay ports and access ports. The relay ports are responsible for interfacing with the inference platform and performing cross-domain routing based on the "cluster identifier list". The access ports are responsible for accurately distributing tasks to specific intelligent agent clusters and supporting cross-cluster information synchronization.

[0035] The agent cluster layer contains several agent nodes belonging to the same virtual local area network. The agents are connected through mobile ad hoc networks or wireless networks to perform specific physical actions or logical inference tasks.

[0036] The self-evolution and monitoring module, which includes a process reward model and a counterfactual reasoning engine, is responsible for collecting the agent's collaborative trajectory, actions and status information in real time, and performing fault diagnosis and strategy optimization in the business execution process.

[0037] In the agent cluster layer, each agent embeds a collaborative control and state fusion module for state interaction and collaborative execution. Within the target agent cluster, a broadcast agent is dynamically elected; state deviations are calculated and the final control command is output.

[0038] In the agent cluster layer, each agent embeds a task planning module, which is used to transform the inference task into the agent's desired task state.

[0039] Compared with existing technologies, the significant advantages of this invention are as follows: Firstly, this method routes and distributes distributed tasks based on an agent gateway. Simultaneously, it introduces the free energy principle during the state awareness phase to ensure efficient inference by agents in environments with incomplete information. By dynamically sensing the local relative state deviations between neighboring agents and the global relative state deviations transmitted by the broadcast agent, and combining this with a cooperative control algorithm that uses a weighted fusion of dual deviations, the broadcast agent automatically becomes the leader, driving the other agents to synchronize, significantly improving the system's cooperative convergence speed. Furthermore, this method supports self-evolutionary error correction based on process rewards and counterfactual reasoning, achieving efficient error correction and policy self-evolution in agent networks. The intelligent agent gateway centrally manages task routing, resulting in low and transparent controllable cluster network communication overhead. It integrates global and local dual state deviation control, significantly improving the exponential convergence speed of multi-agent systems in complex environments. Through a counterfactual reasoning-based directional error correction mechanism, it accurately identifies the responsible intelligent agent, avoiding the waste of computing resources caused by indiscriminate reflection across the entire network in traditional solutions. It supports high-accuracy inference in incomplete information environments, improving resource utilization and business reliability in large-scale intelligent agent applications (such as drone swarms and smart factory collaboration). Attached Figure Description

[0040] Figure 1 This is a diagram of the distributed reasoning system architecture of the intelligent agent of the present invention.

[0041] Figure 2 This is a diagram of the distributed access and perception architecture of the present invention.

[0042] Figure 3 This is a schematic diagram of the dynamic reflection mechanism of the intelligent agent in this invention. Detailed Implementation

[0043] The following detailed description of the multi-level distributed collaborative optimization method and system based on the Internet of Intelligent Agents provided by the present invention, with reference to the accompanying drawings and embodiments, is provided in detail. However, the implementation of the present invention is not limited thereto.

[0044] A multi-level distributed cooperative optimization method based on the intelligent agent Internet includes the following steps:

[0045] Step 1: Distributed Access Based on Agent Gateway: An agent gateway is used as a virtual gateway, connecting to the inference platform through at least one relay port. It receives inference tasks and dynamically adjusts communication links based on the agent's business needs and communication status. There is a one-to-one correspondence between the agent gateway and the agent cluster. The agent cluster belongs to the same virtual local area network (VLAN) (and connects to the same gateway). Multiple agents belonging to the same VLAN connect to the same agent gateway through access ports to form a dedicated agent cluster. The agent gateway enables cross-cluster information exchange, allowing agents in different domains to discover and collaborate with each other. Multiple agents connect to the agent gateway through access ports; the agent gateway dynamically allocates computing resources based on task characteristics and forwards tasks to the agent cluster.

[0046] Step 2 Initialization and Task Distribution Phase: (e.g.) Figure 2 As shown, after receiving and parsing instructions, the inference platform generates inference tasks. Based on the type of instructions required for the task, the inference platform selects one or more agent clusters with corresponding execution capabilities as the "target agent cluster" and distributes the task. The agent gateway identifies the agent cluster transmitting the data based on the configured cluster identifier list and forwards the task to the target agent cluster under the corresponding access port. After receiving the task, the target agent cluster internally transforms the inference task into the desired task state for each agent. Each agent then executes the task according to the desired task state. Furthermore, the agent gateway dynamically allocates computing resources to the target agent cluster based on task characteristics. A task planning module is embedded in each agent to transform the agent's desired task state.

[0047] Step 3: Environmental Perception and Inference: During task execution, due to the agent's incomplete information environment, it needs to infer its true state. This is achieved by incorporating the free energy principle into the generative model. Each agent incorporates the free energy principle to construct a generative model, which infers its true state and outputs a priori state distribution to determine its true state and external logical state. The true state includes both physical and external logical states. The agent incorporates the free energy principle into the environment to construct a generative model, unifying the agent's perceptual state space, action space, and internal state space. The agent infers its true state through the generative model and outputs a priori state distribution. By calculating the KL divergence between the variational distribution and the priori state distribution, the surprise factor (i.e., the difference between the prior environmental model and actual perception) is minimized to ensure efficient inference in an incomplete information environment.

[0048] The agent introduces the free energy principle into the environment to construct a generative model, infers the true state and outputs the prior state distribution, and minimizes the surprise factor S(o) by adjusting the generative model, where the surprise factor is calculated using the following formula: ,in P(o) represents the environmental information observed by the agent, and P(o) represents the probability of the event occurring.

[0049] Since the degree of surprise cannot be measured, it is minimized by minimizing free energy. Free energy is essentially a variational upper bound on the degree of surprise. In the fields of biological intelligence and algorithms, an intelligent agent possesses a priori world model. When events in the real world differ from this prior model, this difference is called the degree of surprise. Because directly calculating or minimizing the degree of surprise in complex environments is usually extremely difficult, algorithms introduce free energy as a substitute. By minimizing free energy, the goal of minimizing the degree of surprise can be mathematically achieved.

[0050] Free energy The calculation formula is: , where D KL Let be the KL divergence of the agent based on the variational distribution and the prior state distribution, where s is the environment state and C is the prior state set. For the agent's action strategy, Let p be a variational distribution, and E be a prior state distribution. Q Let be the mathematical expectation operator derived from the variational distribution Q, where o represents the information observed by the agent. The free energy is calculated by adding the expected value of the logarithm of the quotient of the variational probability and the conditional probability, based on the KL divergence between the variational distribution and the prior state distribution.

[0051] Free energy is a measure describing a future trajectory. The prior state set C in the formula essentially represents a certain "expectation" of the agent. By continuously minimizing the free energy, the agent's action policy will tend to make the actual environmental state develop as close as possible to its expected direction (i.e., the direction of C). The calculated free energy (and its gradient) is used to iteratively update the "prior state set C" and "action policy π" in the model.

[0052] This allows each agent to determine both its exact physical location (its own state) and the approximate location of unknown environmental targets (the external logical state). Only after determining these two states can the agent calculate the first state deviation between its current state and the desired state in the subsequent "state interaction and cooperative execution" stages.

[0053] Step 4: State Interaction and Cooperative Execution: Within the target agent cluster, a broadcast agent is dynamically elected. The remaining ordinary agents use the difference between their actual state and the broadcast agent's desired state as their state deviation. This deviation is then compared with the state deviations of neighboring agents and the broadcast agent to calculate the relative state deviation. These relative deviations are then weighted and fused to form the final control command for the agent, guiding it to adjust its state. Here, the neighboring agents are referred to as "single-hop neighbors" on the network.

[0054] The dynamic election of broadcast agents is achieved by each agent calculating the maximum number of hops from itself to all other agents, and selecting the agent with the smallest maximum number of hops as the broadcast agent.

[0055] The broadcast agent distributes the third state deviation (i.e., the broadcast agent's own state deviation) across the entire network within the agent cluster. Simultaneously, each ordinary agent collects the second state deviation from its neighboring nodes. The agent fuses the weighted global control input with the local control input to automatically adjust its own state. This process includes real-time state deviation perception and information interaction, as well as weighted fusion and collaborative control of the dual deviations. Real-time state deviation perception and information interaction includes the following steps:

[0056] S1: Local state deviation information collection. The first state deviation is the difference between the agent's current state and the system's expected state; at the same time, the second state deviation of each of the agent's neighbors (i.e., the difference between the neighbors' current state and the system's expected state) is received through the communication network.

[0057] S2: Local relative deviation calculation. Based on the current first state deviation and the current second state deviation of the agent, the local relative state deviation between the agent and each neighboring agent is calculated.

[0058] S3: Broadcasting and sensing of global state deviations within the agent cluster. The broadcast agent obtains its own third state deviation at this moment (considering broadcast latency) and broadcasts this current third state deviation globally within the agent cluster to all other agents except the broadcast agent.

[0059] S4: Global relative deviation calculation. Each follower agent receives the third state deviation sent by the broadcast agent, and calculates its current global relative state deviation relative to the broadcast agent based on the third state deviation and its own first state deviation.

[0060] S5: Deviation Information Topology Maintenance. Based on the aforementioned local and global relative state deviation data, the system constructs a dynamic state topology model for multiple agents, providing a unified data reference and management foundation for subsequent collaborative control and policy issuance.

[0061] The agent-based dual-bias weighted fusion and cooperative control includes the following steps:

[0062] S1: Local control input generation. The agent accumulates the collected local relative state deviations and uses a first weighting coefficient to weight the accumulated local relative state deviations to generate the current local control input for the agent.

[0063] Agent i acquires its own current first state deviation and receives the current second state deviation of its neighbor node j; the agent accumulates the collected local relative state deviations and uses a first weighting coefficient. Perform weighted processing to obtain the current local control input. :

[0064] in, and These represent the current states of the agent and its neighboring nodes, respectively. and This represents the desired state for the task. The neighbor connection status is 1 (connected is 1, otherwise is 0), and N is the total number of agents.

[0065] S2: Global Control Input Generation. The agent calculates the current global relative state deviation and weights it using a second weighting coefficient to generate the current global control input. This step ensures that the broadcast agent can act as a leader, driving the other agents to synchronize.

[0066] The broadcast agent broadcasts its current third state deviation to the other agents. Received with broadcast delay taken into account After obtaining the global information, the second weighting coefficient is used. Calculate the current global control input:

[0067] in, For broadcast intelligent agents in The relative state deviation at any given moment.

[0068] S3: Cooperative Strategy Fusion. The agent accumulates and fuses the weighted current local control input with the weighted current global control input, using this as the agent's final cooperative control input.

[0069] The agent accumulates and fuses the weighted local control input with the global control input to obtain the final update control command, as shown in the formula: .

[0070] This represents the derivative (rate of change) of the state with respect to time. Taking spatial movement (state p representing position) as an example, its derivative is the velocity. A first-order single integrator model is used. The calculated "control input" "Directly equal to the agent's rate of change (speed / step size)." The process of converting deviation into control commands is essentially a multivariable proportional controller (P-Controller) logic: calculating direction and distance. The calculated result is a vector representing "how far I am from the desired state and which direction I should move in order to align with my neighbors." This is then weighted and converted into an execution command: the weighting coefficient α is the "gain" for proportional control. Multiplying the "distance deviation" by α proportionally converts it into the underlying "drive speed command." The larger the deviation, the greater the generated correction speed (control input); when there is no deviation, the calculation result is 0, and the agent no longer generates additional speed compensation for that neighbor.

[0071] S4: Policy issuance and dynamic execution. The agent automatically updates its current state (such as physical location or logical state) based on the fused control input commands.

[0072] Step 5: Track monitoring and fault diagnosis: such as Figure 3 As shown, the collaborative trajectory is collected in real time during execution. If the task fails, the process reward model scores the collaborative trajectory step by step. The "responsible agent" is assigned to the first step whose correctness score is less than the predetermined score threshold.

[0073] Step 6: Counterfactual Error Correction and Optimization: (e.g.) Figure 3 As shown, for the responsible agent, corrective actions are generated through counterfactual reasoning. After verifying their effectiveness, targeted reflection suggestions are generated. The PPO algorithm is used to update the underlying reflector policy, and the corrective instructions are issued to the responsible agent for execution. Specifically, this includes:

[0074] (1) Counterfactual correction and strategy generation: For the locked “responsible agent” and its key failure steps, input the collaborative trajectory and the responsible agent identifier before the key failure steps into the counterfactual reasoning model (which can be implemented by a pre-trained large language model, causal reasoning model or reasoning model fine-tuned by task data); generate hypothetical correction actions through counterfactual intervention, while keeping the previous steps unchanged.

[0075] (2) Correction Verification and Dynamic Reflection Issuance: The correction step is re-input into the process reward model for verification. If the correction score is greater than the original failure score, it is determined as the target correction step. Subsequently, based on the failure diagnosis results and the target correction step, a targeted reflection suggestion is generated using the reflector, which is only for the responsible agent. The reflector strategy is dynamically adjusted using the proximal policy optimization algorithm, and the correction instruction is issued to the responsible agent for execution. The entire process ensures that network resources are not wasted on ineffective global reflection, and realizes efficient error correction and self-evolution of the agent network.

[0076] The correctness score of the verification correction action is calculated. If the score is higher than the score of the original critical failure step, reflection prompts are generated based on the critical failure step, the target correction step, and the collaborative failure trajectory to guide the reflector in generating targeted reflections. Based on the reward score corresponding to the reflection, the reflector policy is updated using the Proximal Policy Optimization (PPO) algorithm.

[0077] in, This represents the loss function for Proximal Policy Optimization (PPO), used to optimize the policy parameters of the reflexor. . The reflective quality score is used to evaluate the feedback from the pre-trained reward model. For the current strategy of the reflector, For reference strategy, Used to constrain the magnitude of policy updates. Reflector Input. The reflector is based on the input. The generated output results This represents the output of the reflector for a specific responsible agent. By adjusting the reflector policy, the reflector is made to tend to generate high-quality, targeted error correction policies in subsequent similar scenarios.

[0078] Steps 2–6 are executed continuously in a loop to form a closed loop of perception, collaboration, diagnosis, and evolution, enabling dynamic, real-time agent collaboration and cross-domain network optimization.

[0079] As shown in Figure 1, a multi-level distributed collaborative optimization system based on the Agent Internet is constructed using the four-layer architecture model of the Agent Internet, including:

[0080] Inference platform layer: The computing power and control center, which is deployed in a centralized or distributed manner, is responsible for receiving user instructions, generating inference tasks based on the data volume and complexity of the task, and allocating computing resources in a global cluster of multiple intelligent agents.

[0081] Agent Gateway Layer: A virtual gateway deployed at the edge or in the cloud, comprising relay ports and access ports. The relay ports are responsible for interfacing with the inference platform and performing cross-domain routing based on the "cluster identifier list"; the access ports are responsible for accurately distributing tasks to specific agent clusters and supporting cross-cluster information synchronization.

[0082] Agent cluster layer: contains several agent nodes (Node / Agent) belonging to the same virtual local area network. Agents are connected through mobile ad hoc networks (MANET) or wireless networks to perform specific physical actions or logical inference tasks.

[0083] The self-evolution and monitoring module includes a process reward model (PRM) and a counterfactual reasoning engine. It is responsible for collecting real-time information on the agents' collaborative trajectories, actions, and states, and for fault diagnosis and strategy optimization during business execution. Through this architecture, the system can achieve efficient distribution, unified control, and dynamic collaboration of heterogeneous agents within the agent network.

[0084] In the agent cluster layer, each agent embeds a collaborative control and state fusion module for state interaction and collaborative execution. Within the target agent cluster, a broadcast agent is dynamically elected; state deviations are calculated and the final control command is output.

[0085] In the agent cluster layer, each agent embeds a task planning module, which is used to transform the agent's desired task state.

[0086] The election of broadcast agents is carried out autonomously within the agent cluster.

[0087] Furthermore, each agent cluster has a broadcast agent.

[0088] The following section illustrates the implementation of this invention by demonstrating how a drone swarm performs an inspection of abnormal heat sources in a park.

[0089] Suppose a suspected equipment overheating alarm occurs in an industrial park. The user inputs the task into the inference platform: "Conduct a low-altitude inspection of area A, identify suspected high-temperature anomalies, and have a drone swarm maintain a triangular formation to cover three sub-areas, A1, A2, and A3, and finally output the coordinates of the anomalies and the inspection report."

[0090] The system already has multiple agent clusters, as shown in the table below:

[0091] Cluster number Cluster type ability Cluster-UAV-01 Unmanned aerial vehicle (UAV) visual inspection cluster Low-altitude flight, thermal imaging, target recognition Cluster-AGV-02 Ground robot swarm Ground verification and close-up photography Cluster-LLM-03 Document reasoning agent cluster Report generation and anomaly explanation

[0092] Therefore, the following was selected: Target agent cluster: Cluster-UAV-01; Corresponding agent gateway: Agent-Gateway-UAV-01; Agents within the cluster: UAV-A, UAV-B, UAV-C, UAV-D, UAV-E. Agent-Gateway-UAV-01 and Cluster-UAV-01 have a one-to-one relationship. The five drones in this cluster belong to the same virtual local area network and are connected to the same agent gateway through their respective access ports.

[0093] After receiving a user-input industrial park heat source inspection task, the inference platform generates a high-level inference task. This high-level inference task includes the inspection area boundary, target identification type, inspection altitude, formation requirements, safety distance, no-fly zones, and task completion conditions. Based on the task type, the inference platform selects a cluster of UAV agents with low-altitude flight, thermal imaging acquisition, and target identification capabilities as the target agent cluster, and distributes the task to this target agent cluster (Cluster-UAV-01) through the relay port of the agent gateway. Instead of directly assigning specific waypoints to each UAV, the inference platform sends the high-level task and constraints to the target agent cluster.

[0094] After receiving the task, the target agent cluster's internal task planning module transforms the high-level inspection task into the desired task state for each drone based on the inspection area boundary, the number of drones, the current position of each drone, battery level, payload capacity, and communication topology. The desired task state includes several sub-desired task states, such as desired waypoint, desired altitude, desired inspection sub-area, desired formation role, desired coverage progress, and suspected target handling strategy, as shown in the table below:

[0095] intelligent agent Subtask UAV-A Inspect the left boundary of area A1 UAV-B Inspect the boundary area between A1 and A2 UAV-C As the core area of ​​the central node inspection UAV-D Inspect the boundary area between A2 and A3 UAV-E Inspect the right boundary of area A3

[0096] The UAV-D drone is assigned to sub-area A3. Its expected mission status will include several sub-expected mission statuses, such as flying to the center waypoint of sub-area A3, maintaining an altitude of 50 meters, completing thermal imaging coverage of sub-area A3, and requesting cross-verification from neighboring drones when a suspected heat source is detected.

[0097] Each agent executes a task based on the desired task state. During task execution, because the agent operates in an environment with incomplete information, it needs to infer its own true state in order to continuously update instructions to guide the agent in completing the desired task. This true state includes two aspects: first, its own physical state, including the UAV-D's current position, altitude, speed, and attitude; and second, the external logical state, such as whether there are abnormal heat sources, obstacles, no-fly zones, or smoke obstructions in the surrounding environment.

[0098] Suppose that during a routine inspection, the UAV-D detects a suspected heat source, but due to smoke obscuring the view and a poor camera angle, it only obtains the following observation information:

[0099] Drone D observed: the target's brightness intensity in the thermal imaging image is 0.78 (medium to high level); the confidence level for visible light recognition is only 0.42 (not very certain, barely recognizable); the target is 36 meters away; the smoke obscuration is quite severe, with an obscuration coefficient of 0.65 (not very clear); GPS coordinates: 120.1221 E, 30.2519 N; flight altitude: 50 meters.

[0100] UAV-D cannot directly confirm "this must be an abnormal device overheating," but can only infer from several possible states:

[0101] Status number Environmental state s meaning s1 normal_machine Normal equipment heating s2 abnormal_heat_source Abnormal high temperature point s3 sunlight_reflection Metallic reflections cause misjudgment s4 smoke_interference Smoke interference creates false heat sources

[0102] UAV-D has an internal generative model that estimates the probability of the current observation o occurring if the true states are s1, s2, s3, s4. For example, the generative model might provide:

[0103] state s P(o|s) S1 Normal equipment heats up 0.25 S2 Abnormal High Temperature Point 0.55 S3 Metallic Reflective 0.10 S4 Smoke Interference 0.10

[0104] Meanwhile, combining historical inspection data and the status of equipment in the park, the prior distribution of UAV-D is as follows:

[0105] state s P(s) S1 Normal equipment heats up 0.40 S2 Abnormal High Temperature Point 0.30 S3 Metallic Reflective 0.15 S4 Smoke Interference 0.15

[0106] Based on Bayesian principles, UAV-D calculates the overall probability of observation o occurring:

[0107] P(o) = Σ P(o|s)P(s)

[0108] = 0.25×0.40 + 0.55×0.30 + 0.10×0.15 + 0.10×0.15 = 0.100 + 0.165 + 0.015 + 0.015 = 0.295

[0109] The level of surprise can be understood as "how unexpected this observation was":

[0110] S(o) = -ln P(o)= -ln(0.295) ≈ 1.22

[0111] The greater the surprise factor, the more difficult it is for the current observation to be explained by existing models; the smaller the surprise factor, the more stable the model's explanation of the current environment.

[0112] Since the agent cannot directly know the true state, a variational distribution is used. When approximating the initial posterior distribution, UAV-D provides an internal conjecture:

[0113] state s initial S1 Normal equipment heats up 0.30 S2 Abnormal High Temperature Point 0.40 S3 Metallic Reflective 0.15 S4 Smoke Interference 0.15

[0114] Based on the updated observations, UAV-D needs to adjust its variational distribution. This makes it closer to the current observation. and prior state set The true posterior distribution under the given conditions. Combined with the aforementioned free energy formula:

[0115]

[0116] in, Indicates the prior state set Under constraints, when the environmental state is Current observation is generated at time The conditional probability. For the discrete set of environment states in this embodiment, the prior state distribution can be denoted as:

[0117]

[0118] The observed likelihood is denoted as:

[0119]

[0120] Therefore, current observations The marginal probability can be expressed as:

[0121]

[0122] Substitute the data from this embodiment:

[0123]

[0124] Within this free energy framework, the true posterior distribution can be expressed as:

[0125]

[0126] Therefore, we can conclude that:

[0127] state s S1 Normal equipment heats up 0.100 / 0.295 = 0.339 S2 Abnormal High Temperature Point 0.165 / 0.295 = 0.559 S3 Metallic Reflective 0.015 / 0.295 = 0.051 S4 Smoke Interference 0.015 / 0.295 = 0.051

[0128] At this point, UAV-D adjusts its variational distribution. To approximate the aforementioned posterior distribution, for example, by updating it to:

[0129] state s after update S1 Normal equipment heats up 0.34 S2 Abnormal High Temperature Point 0.56 S3 Metallic Reflective 0.05 S4 Smoke Interference 0.05

[0130] Therefore, in this embodiment, the posterior distribution is used to guide the process of minimizing free energy. Updated target distribution. When The closer The more the state estimate of the agent can explain the current observation, the lower the free energy becomes.

[0131] Furthermore, the free energy can also be written in the following equivalent decomposition form:

[0132]

[0133] in, Indicates the current observation Given a strategy and prior state set Theoretical surprise level under certain conditions

[0134] This represents the difference between the internal variational distribution of the agent and the true posterior distribution.

[0135] It should be noted that in real-world complex environments, due to Since it is necessary to perform edge computing on all possible environmental states, which is usually difficult to obtain directly, the agent does not directly use surprise as a computable optimization objective. Instead, it indirectly reduces surprise and updates the state estimate by minimizing the free energy, a computable variational upper bound.

[0136] When UAV-D is updated and When they approach each other, the KL divergence term approaches the mean. ,therefore:

[0137] Substituting the observation edge probabilities into this embodiment:

[0138]

[0139] state s Error S1 Normal equipment heats up 0.10 S2 Abnormal High Temperature Point 0.20 S3 Metallic Reflective 0.60 S4 Smoke Interference 0.10

[0140] this It differs greatly from the true posterior. The divergence increases, and so does the free energy. The system minimizes the free energy to achieve the variational distribution of UAV-D. Gradually approximating the true posterior distribution In both the posterior distribution and the updated variational distribution, the probability of state s2 corresponding to the "abnormal high temperature point" is the highest. However, since this probability does not reach the direct confirmation threshold, and the smoke obscuration is high and the visible light confidence is insufficient, the target is marked as a "suspected abnormal heat source pending verification" state.

[0141] Ultimately, UAV-D was determined to be located at 120.1221 degrees east longitude, 30.2519 degrees north latitude, and an altitude of 50 meters. There is an unusual heat source at this location, with a 56% probability of identification.

[0142] The UAV-D agent infers its current state using a generative model based on the free energy principle. The UAV then compares this inferred state with the expected task state generated by the swarm planning module to obtain the current state deviation. This deviation includes physical position deviation and task semantic deviation, such as position deviation, altitude deviation, coverage deviation, and target confirmation deviation. Specifically, if the UAV-D's current altitude is lower than the expected altitude, or its current position deviates from the expected waypoint in the A3 sub-region, a physical position deviation occurs; simultaneously, if an unverified suspected heat source exists, a task semantic deviation occurs.

[0143] During the state interaction and cooperative execution phase, the cluster dynamically elects a broadcast agent based on the communication topology. The broadcast agent broadcasts its own comprehensive task state deviation as a third state deviation to other UAVs within the cluster. Ordinary UAVs simultaneously receive the second state deviations of their neighboring agents and the third state deviation of the broadcast agent, and then weightedly fuse their current state deviation (derived from their own free energy) with the second state deviations of their neighbors and the broadcast global deviation to generate the final control input. This final control input includes physical control input and logical task control input. The physical control input is used to adjust the UAV's position, altitude, speed, and heading, while the logical task control input is used to adjust the target recognition state, coverage strategy, and cooperative verification strategy.

[0144] For example, UAV-D identifies a target as a suspected heat source in its free energy inference, but due to high smoke obscuration and insufficient visible light confidence, there is a "target confirmation bias" between its current mission state and its expected mission state. Meanwhile, its neighboring UAV-E does not observe the same heat source, and the third state bias issued by the broadcast agent UAV-C indicates that sub-region A3 still needs to maintain its coverage formation and complete cross-validation. After fusing the aforementioned self-bias, second state bias, and third state bias, UAV-D's final control input does not directly report the abnormal heat source, but instead adjusts its course to continue covering sub-region A3, marks the target as a suspected abnormal state, and requests UAV-C and UAV-E to perform cross-validation from different angles.

[0145] If the UAV-D fails to execute the fusion control results as described above and instead generates the "confirm abnormal heat source" task action directly when the free energy inference confidence is insufficient, this action will cause the broadcast agent to adjust the global formation, other UAVs to move closer to the wrong location for verification, and ultimately result in an error anomaly in the inspection report. The self-evolution and monitoring module records this execution process as a collaborative failure trajectory and inputs each step in the trajectory into the process reward model for step-by-step scoring. When the process reward model identifies the step where the UAV-D directly confirms the abnormal heat source under low confidence and high smoke obscuration conditions as the first step below the threshold, the system determines this step as a critical failure step and locks the UAV-D as the responsible agent.

[0146] Subsequently, the counterfactual reasoning module, while maintaining the preceding normal steps unchanged, intervenes in the critical failure steps of the UAV-D, generating a corrective action: marking the target as a suspected heat source and requesting cross-validation from nearby UAVs, instead of directly confirming it as an abnormal heat source. The system re-enters this corrective action into the process reward model for verification. If the corrected score is higher than the original failure action score, the corrective action is determined to be the target corrective action. The system further generates targeted reflection prompts for the UAV-D based on the critical failure steps, corrective actions, and failure trajectories, and updates the reflector policy using a proximal policy optimization algorithm, enabling the UAV-D to form a policy constraint of "cross-validation is mandatory when there is low visible light confidence and high smoke obscuration" in subsequent similar scenarios.

[0147] In the method of this invention, the state perception and access of the intelligent agent in an environment with incomplete information are performed, such as... Figure 2 As shown, distributed access and perception modeling are implemented, specifically as follows:

[0148] (1) Gateway traffic routing control: Inference tasks do not broadcast directly across the network, but instead perform identifier matching through the relay port of the agent gateway. When the identifier carried by the data matches the default cluster identifier, the gateway correctly guides the traffic to the target agent cluster, realizing efficient and isolated communication among a large number of agents.

[0149] (2) Free Energy Perception Modeling: After receiving the task, the target intelligent agent cluster infers the state based on the free energy principle. Specifically, a generative model is built inside the intelligent agent, abstracting the environmental object into a perception state space, an internal state space, and an action space. The intelligent agent does not simply rely on passive data collection, but actively calculates the KL divergence between the variational distribution and the prior distribution to minimize the "surprise" when interacting with the environment, thus efficiently inferring the current true state even when some environmental data is missing.

[0150] Within the agent cluster layer, multiple agents communicate via mobile ad hoc networks and other means. To achieve efficient collaborative control, the agent cluster dynamically perceives global and local states in the following ways to achieve collaborative control:

[0151] (1) Broadcast agent election: According to the network topology, each agent calculates the maximum number of hops from itself to other nodes, and selects the agent with the smallest maximum number of hops as the "broadcast agent" to minimize the routing delay of global information transmission.

[0152] (2) Local and global deviation mapping: The agent collects two types of deviation data in real time: one is the "second state deviation" between the current state and the expected state from the neighboring nodes; the other is the "third state deviation" from the broadcast agent.

[0153] (3) Dual-weighted cooperative strategy: The agent calculates its local relative deviation (first weight) relative to its neighbors and its global relative deviation (second weight) relative to the broadcast agent, and sums the two as the final control input command for the next moment. This mechanism ensures that the broadcast agent becomes the leader, driving the other follower agents to quickly synchronize.

[0154] like Figure 3 As shown, the method of the present invention dynamically performs precise error correction and strategy optimization based on the real-time monitored collaboration trajectory:

[0155] (1) Process reward scoring (PROFA): When a failure trajectory occurs in the multi-agent system, the self-evolution module scores each step of reasoning and action through the process reward model, identifies the first step below the threshold as the "critical failure step", and accurately locates the "responsible agent".

[0156] (2) Counterfactual correction generation: For the critical failure steps of the responsible agent, the counterfactual reasoning engine is used to generate hypothetical corrective actions (such as adjusting decision logic or communication content) while keeping other normal agent steps unchanged.

[0157] (3) Targeted Reflection and Policy Update: After verifying that the score of the corrected action is higher than that of the original failed action, a targeted reflection prompt is generated for the responsible agent. Further, combined with the Proximal Policy Optimization (PPO) algorithm, the reflector policy model is dynamically updated to achieve self-evolution of error correction capability.

[0158] Therefore, the free energy suggests that instruction fusion and post-error correction are not independent processes, but rather form a closed loop around the same comprehensive task state: the free energy is used to estimate the current comprehensive task state under incomplete information; instruction fusion is used to convert the deviation between the current comprehensive task state and the expected task state into physical control instructions and logical task instructions; and post-error correction is used to locate the responsible agent that caused the erroneous state transition or erroneous action when the fused instructions cause the task to fail, and to update the subsequent behavior strategy of the responsible agent through counterfactual correction and policy optimization.

[0159] Through the above implementation methods, the present invention can achieve the following advantages: the intelligent agent gateway uniformly manages task routing, and the cluster network communication overhead is low, transparent, and controllable; the integration of global and local dual state deviation control significantly improves the exponential convergence speed of multi-agent systems in complex environments; through a counterfactual reasoning-based directional error correction mechanism, the responsible intelligent agent is accurately located, avoiding the waste of computing resources caused by indiscriminate reflection across the entire network in traditional solutions; and it supports high-accuracy inference in incomplete information environments, improving the resource utilization and business reliability of large-scale intelligent agent applications (such as drone swarms and smart factory collaboration).

[0160] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the present invention without departing from its novel spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-level distributed collaborative optimization method based on the Internet of Intelligent Agents, characterized in that: Includes the following steps: The inference platform generates inference tasks, selects a cluster of capable intelligent agents as the target intelligent agent cluster based on the task type, and distributes the inference tasks. After receiving the inference task, the target intelligent agent cluster transforms the inference task into the expected task state of each intelligent agent. Each intelligent agent executes the task according to the expected task state. During the task execution, each intelligent agent completes its own true state inference. Simultaneously, a broadcast agent is dynamically elected within the target agent cluster; Other ordinary agents use the difference between their actual state and the desired task state as their state deviation. They calculate the relative state deviation by comparing it with the state deviations of neighboring agents and broadcast agents. After weighted fusion, they form the final control command for the agent. The agent then executes the task according to the final control command.

2. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 1, characterized in that: The optimization method further includes distributed access based on an agent gateway before the inference platform generates inference tasks: using the agent gateway as a virtual gateway, when multiple agents belong to the same virtual local area network, the multiple agents connect to the same agent gateway through their respective access ports to form an agent cluster. The agent gateway then connects to the inference platform through at least one relay port to receive tasks issued by the inference platform; the inference tasks generated by the inference platform are issued through the relay ports of the agent gateway.

3. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 2, characterized in that: In the distributed access based on the agent gateway, one agent gateway corresponds to one agent cluster.

4. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 1, characterized in that: The expected task state includes the physical expected task state and the logical expected task state; the actual state itself includes the physical state and the external logical state; the state deviation includes the physical location deviation and the task semantic deviation; the control instructions include the physical control input and the logical task control input.

5. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 1, characterized in that: Each agent completes its own true state inference by incorporating the free energy principle into the generative model, including the following steps: The agent constrains its actions by adjusting the generative model to minimize the surprise factor S(o), and minimizes the free energy to further minimize the surprise factor. The formula is: Where s is the environmental state and C is the prior state set. For the agent's action strategy, Let p be a variational distribution, and E be a prior state distribution. Q Let D be the mathematical expectation operator derived from the variational distribution Q, o be the information observed by the agent, and D be the expected value operator. KL The KL divergence between the variational distribution and the prior state distribution is used to minimize the free energy as the optimization objective, so that the calculated free energy can be used to continuously iteratively update the prior state set C and the action policy in the model. .

6. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 1, characterized in that: The internal dynamic election of a broadcast agent is achieved by each agent calculating the maximum number of hops from itself to all other agents, and selecting the agent with the smallest maximum hop count as the broadcast agent.

7. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 4, characterized in that: The remaining ordinary agents use the difference between their actual state and their desired task state as the state deviation. This state deviation is then calculated by combining it with the state deviations of neighboring agents and the broadcast agent to determine the relative state deviation. After weighted fusion, this relative state deviation is formed as the final control command for that agent. Each agent executes the task according to the final control command, specifically including the following steps: S1: Local control input generation: Current local control input The calculation formula is: ,in For intelligent agents , The current state of the agent's neighbors. For intelligent agents , For the neighbors , , The neighbor connection status is 1 if connected and 0 otherwise. N is the total number of agents in the agent cluster. S2: Global control input generation: Current global control input The calculation formula is: ,in, For broadcast intelligent agents in The relative state deviation at any given moment. Let be the relative state deviation of agent i at time t. For intelligent agents The time delay for receiving the relative state deviation transmitted by the broadcast agent. Weighting coefficients; S3: Cooperative Strategy Fusion and Execution: Final Update Control Command for the Agent :

8. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 1, characterized in that: The collaborative optimization method further includes: collecting the agent's action trajectory in real time during task execution; if the task fails, performing self-evolutionary error correction based on counterfactual reasoning, including the following steps: S1 Collaborative Trajectory Monitoring and Process Reward Scoring: If the task fails, the correctness of the collaborative action trajectory before the task failure is scored step by step. The collaborative action trajectory contains multiple steps. These multiple steps are input into the process reward model, which assigns a correctness score to each step. S2 Critical Failure Step and Responsible Agent Diagnosis: Analyze the scoring results and identify the first step among multiple steps whose correctness score is less than a predetermined scoring threshold as a "critical failure step" and find the corresponding "responsible agent". S3 Counterfactual Correction and Policy Generation: For the responsible agent and key failure steps, input the collaborative trajectory before the key failure step and the responsible agent's identifier into the counterfactual reasoning model; generate hypothetical corrective actions through counterfactual intervention, while keeping the steps before the key failure step unchanged; S4 Correction Verification and Dynamic Reflection Issuance: The correction step is re-input into the process reward model for verification. If the correction score is greater than the original failure score, it is determined as the target correction step. Subsequently, based on the failure diagnosis results and the target correction step, the reflector generates targeted reflection suggestions only for the responsible agent, and the reflector policy is dynamically adjusted using the proximal policy optimization algorithm. The correction instruction is then issued to the responsible agent for execution.

9. The multi-level distributed cooperative optimization method based on the intelligent agent Internet according to claim 8, characterized in that: The reflexor policy is dynamically adjusted using a proximal policy optimization algorithm, specifically including: ,in, The loss function represents the near-end policy optimization, used to optimize the policy parameters of the reflexor. , The reflective quality score is used to evaluate the feedback from the pre-trained reward model. For the current strategy of the reflector, For reference strategy, Used to constrain the magnitude of policy updates. For reflector Input, For the reflector based on input The generated output results Output for a specific responsible intelligent agent's reflector.

10. A multi-level distributed collaborative optimization system based on the Internet of Intelligent Agents, characterized in that: The method described in claims 1-9 is characterized by comprising: The inference platform layer serves as a computing power and control center for centralized or distributed deployment. It is responsible for receiving user instructions, generating inference tasks based on the data volume and complexity of the task, and allocating computing resources across multiple intelligent agent clusters globally. The intelligent agent gateway layer is a virtual gateway deployed on the edge or cloud side, including relay ports and access ports. The relay ports are responsible for interfacing with the inference platform and performing cross-domain routing based on the "cluster identifier list". The access ports are responsible for accurately distributing tasks to specific intelligent agent clusters and supporting cross-cluster information synchronization. The agent cluster layer contains several agent nodes belonging to the same virtual local area network. The agents are connected through mobile ad hoc networks or wireless networks to perform specific physical actions or logical inference tasks. The self-evolution and monitoring module, which includes a process reward model and a counterfactual reasoning engine, is responsible for collecting the agent's collaborative trajectory, actions and status information in real time, and performing fault diagnosis and strategy optimization in the business execution process. In the agent cluster layer, each agent embeds a collaborative control and state fusion module for state interaction and collaborative execution. Within the target agent cluster, a broadcast agent is dynamically elected; state deviations are calculated and the final control command is output. In the agent cluster layer, each agent embeds a task planning module, which is used to transform the inference task into the agent's desired task state.

Images