Decentralized multi-agent information interaction method and system

Abstract

The invention discloses a decentralized multi-agent information interaction method and system, and the method comprises the steps: generating a subtask carrying a unique identifier and metadata based on mixed dimension task decomposition, and embedding a global utility function; sub-task-agent mapping is dynamically adjusted through Kafka and Flink, and a utility function weight coefficient is broadcasted; the sub-agent determines an executor through contract network protocol bidding, and performs execution permission arbitration on key resources in combination with a token bucket algorithm; metadata are transmitted by adopting a Gossip protocol, and intermediate data are transmitted by P2P direct connection (abnormal degradation to Kafka); key metadata are synchronized through lightweight Raft consensus, and non-key metadata are merged by adopting CRDT; and after a final result is written in through Kafka, duplicate removal, sorting and aggregation are carried out through Flink, and key states are synchronized through Gossip. According to the method, the problem of message disorder under a hybrid protocol is solved; and high-reliability message transmission and state synchronization are realized by combining the strong consistency bottom of Kafka and the conflict resolution of lightweight Raft and CRDT.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of artificial intelligence and distributed computing, and relates to a decentralized multi-agent information interaction method and system. Background Technology

[0002] With the deep integration of artificial intelligence and distributed computing technologies, intelligent agent systems have shown great potential in key areas such as the Industrial Internet of Things, autonomous driving, and edge computing. However, existing collaborative architectures face multiple technical bottlenecks: at the communication level, mainstream solutions rely on centralized message queues (such as Kafka and RabbitMQ) or purely decentralized protocols (such as Gossip and P2P), both of which have significant drawbacks. While centralized architectures can guarantee strong message consistency and ordering, they are prone to single-point failures and experience a surge in latency in highly dynamic edge network environments; decentralized protocols, while offering advantages in elastic scalability and low latency, struggle to maintain global state consistency, often leading to out-of-order or duplicate message propagation. In terms of task scheduling and resource allocation, traditional static rules and simple load balancing strategies are ill-suited to heterogeneous intelligent agent environments. Homogeneous systems cannot dynamically allocate heterogeneous computing resources, and the performance of distributed locks deteriorates sharply under high-concurrency scenarios, making cross-protocol deadlocks more likely. Insufficient adaptability to dynamic environments further restricts system reliability—frequent node additions / exits require second-level load rebalancing, and the lack of a hybrid consistency model to balance availability and data correctness during network partitioning further complicates the system. These issues are summarized into four core challenges: cross-protocol messages suffer from causal consistency problems due to the lack of timing mechanisms; manual downgrading of communication protocols during network fluctuations leads to recovery delays; competition for heterogeneous resources lacks an efficient coordination mechanism; and slow convergence of the Gossip protocol in large-scale clusters affects real-time decision-making efficiency. Systemic innovation is urgently needed to overcome these technical barriers. Summary of the Invention

[0003] The purpose of this invention is to solve the technical problems in the prior art, such as the lack of a timing mechanism for cross-protocol messages causing causal consistency issues; the need to manually downgrade the communication protocol during network fluctuations leading to recovery delays; the lack of an efficient coordination mechanism for heterogeneous resource competition; and the slow convergence of the Gossip protocol in large-scale clusters. The invention provides a decentralized multi-agent information interaction method and system.

[0004] To achieve the above objectives, the present invention employs the following technical solution: The first aspect of this invention provides a decentralized multi-agent information interaction method, comprising the following steps: User tasks are decomposed into multiple subtasks by a hybrid dimension of task type, resource requirements, and dependencies, resulting in subtasks carrying unique identifiers and metadata. Simultaneously, a global utility function is loaded into the task description of each subtask. The metadata includes resource identifiers (resource IDs) and dependencies. Based on the resource ID, the subtask hash is mapped to different partitions of Kafka, and the mapping relationship between the subtask and the sub-agent is dynamically adjusted through the Flink engine. At the same time, the weight coefficient of the global utility function is broadcast to the sub-agent through Kafka's config Topic. Sub-agents bid against each other through a contract network protocol to find the sub-agent with the most suitable bid to serve as the task executor. Based on the priority of sub-agents in the global utility function, the execution permission arbitration control of critical resources is performed through the token bucket algorithm. Sub-agents use the Gossip protocol to transmit metadata; intermediate data is transmitted using P2P direct communication, and automatically degrades to Kafka relay in case of anomalies; the message header of the intermediate data carries the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter; at the same time, Flink is responsible for message integrity monitoring, out-of-order detection and breakpoint resumption. Sub-agents execute tasks, using the lightweight Raft consensus protocol to synchronize critical metadata and using CRDT increment / decrement counters to merge non-critical metadata. The execution results of the sub-agents are written to the Kafka result topic. The Flink engine consumes the Kafka result topic and performs deduplication, sorting and aggregation operations in sequence. Based on the aggregated results, the agent combines the knowledge graph to generate the final response and synchronizes the key states through the Gossip protocol.

[0005] Furthermore, the hybrid dimensionality decomposition includes: The first stage involves roughly breaking down tasks into logically independent modules based on logical units. The second stage involves breaking down resource requirements into smaller parts, labeling the types of resources required by each module and generating resource IDs. The third stage involves completing the dependency chain by adding the `depends on` field to the Kafka message to establish a DAG dependency chain.

[0006] Furthermore, the global utility function is described as follows: Global utility function = task priority × sub-agent priority; in: Task priority = Urgency level + • Resource requirements - • Estimated energy consumption; Sub-agent priority = Efficiency+ Energy consumption+ • Fairness; Weighting coefficient , , , , , Real-time dynamic broadcasting via topics in Kafka.

[0007] Furthermore, the contract network protocol is as follows: The sub-agent calculates the local quote based on the global utility function and dynamic weight coefficients; The main agent allocates tasks based on local quotes, task priorities, and sub-agent priorities.

[0008] Furthermore, the token bucket algorithm includes: Maintain an independent token bucket for each critical resource, with a fixed total number of tokens in each bucket; Allocate resources to sub-agents based on their priorities; High-priority sub-agents send borrowing requests via Kafka; after arbitration by the main agent, tokens from low-priority sub-agents are forcibly revoked and allocated to high-priority sub-agents; after borrowing is completed, token quotas are updated via Kafka broadcast, and tokens are automatically returned after the task is completed.

[0009] Furthermore, the transmission of metadata using the Gossip protocol includes: Each sub-agent sorts its neighbor list by network topology distance; based on the neighbor list, a multipath routing algorithm selects the optimal three paths according to path scores; the multipath routing algorithm uses the Dijkstra-KNN algorithm; the path scores are represented as:

[0010] in, Indicates real-time delay; Indicates reliability; Indicates load; , and All are weighting coefficients.

[0011] Furthermore, the method of using P2P direct communication to transmit intermediate data, and automatically downgrading to Kafka for relay in case of an anomaly, specifically involves: P2P direct communication uses the QUIC protocol. The message header embeds the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter. The receiver prioritizes sorting by Kafka Offset, and if the Kafka Offsets are the same, sorting is done by Local Counter. When P2P communication fails, it automatically degrades to Kafka relay mode. When the network recovers, it performs a gradual switchback. The gradual switchback specifically means that initially a portion of the traffic goes through P2P, and then gradually increases until a full switchback is finally performed. When Flink detects a break in Kafka Offset or Local Counter, it actively requests to resume transmission from the breakpoint.

[0012] Furthermore, the specific steps of performing deduplication, sorting, and aggregation operations sequentially on the Kafka result topic consumed by the Flink engine are as follows: Deduplication is achieved through a Bloom filter that performs parallel hash calculations on the FPGA, and the system automatically switches to software mode when the FPGA is unavailable. The deduplicated results are sorted by Kafka Offset. If the results are out of order, they are determined according to preset rules / knowledge graph. The sorted results are aggregated and submitted to the main agent.

[0013] Furthermore, the key metadata includes task allocation results, task progress markers, global configuration version numbers, and cross-task dependency status; The non-critical metadata includes intermediate cache results, statistical counts, log information, and temporary performance metrics.

[0014] A second aspect of the present invention provides a decentralized multi-agent information interaction system, comprising: The task decomposition module decomposes user tasks according to a mixed dimension of task type, resource requirements, and dependencies, resulting in multiple subtasks carrying unique identifiers and metadata. Simultaneously, a global utility function is loaded into the task description of each subtask. The metadata includes resource identifiers (resource IDs) and dependencies. The task mapping module maps subtask hashes to different partitions of Kafka based on the resource ID, and dynamically adjusts the mapping relationship between subtasks and sub-agents through the Flink engine. At the same time, it broadcasts the weight coefficients of the global utility function to the sub-agents through Kafka's config Topic. In the task allocation module, sub-agents compete for bids through the Contract Network Protocol to find the sub-agent with the most suitable bid as the task executor. The resource allocation module uses the token bucket algorithm to arbitrate execution permissions for critical resources based on the priority of sub-agents in the global utility function. The sub-agent collaboration module uses the Gossip protocol to transmit metadata between sub-agents; it uses P2P direct communication to transmit intermediate data, and automatically degrades to Kafka relay in case of anomalies; the message header of the intermediate data carries the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter; at the same time, Flink is responsible for message integrity monitoring, out-of-order detection and breakpoint resume. The task execution module consists of sub-agents that execute tasks. Key metadata is synchronized using a lightweight Raft consensus protocol, while non-key metadata is merged using CRDT increment / decrement counters. The task aggregation module writes the execution results of the sub-agents into the Kafka result topic, and consumes the Kafka result topic through the Flink engine to perform deduplication, sorting and aggregation operations in sequence; based on the aggregated results, the agent combines the knowledge graph to generate the final response and synchronizes the key states through the Gossip protocol.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention discloses a decentralized multi-agent information interaction method. At the task processing level, the system significantly improves resource utilization and execution efficiency through dynamic load balancing and fine-grained task decomposition. During the task distribution phase, Kafka partition hashing and Flink dynamic mapping mechanisms are employed, combined with a target function loaded from a global utility function. This enables sub-agents to evaluate task priorities based on a unified standard and achieve precise matching of tasks and resources through contract network protocol bidding. This design not only avoids resource idleness or overload caused by traditional static allocation but also ensures mutual exclusion and priority scheduling of critical resources (such as dedicated hardware and data shards) through the token bucket algorithm and partition serialization mechanism.

[0016] In terms of communication reliability, the hybrid protocol architecture (P2P+Gossip+Kafka) and cross-protocol sequence number alignment mechanism ensure low latency and strong consistency in message transmission, resolving message out-of-order issues under hybrid protocols. The QUIC protocol's 0-RTT handshake and header compression technology keep P2P direct communication latency at a low level, while the improved Gossip protocol achieves causal consistency in metadata synchronization through dynamic neighbor selection and vector clock mechanisms. When a P2P channel fails due to network jitter, the system automatically degrades to Kafka relay mode, and combined with the maximum value merging rule of CRDT increment / decrement counters, ensures eventual consistency of the distributed state. The cross-protocol message out-of-order rate is significantly reduced, the success rate of breakpoint resumption is extremely high, and the degradation switching process is completely transparent to upper-layer services. By combining Kafka's strong consistency fallback, lightweight Raft, and CRDT conflict resolution, highly reliable message transmission and state synchronization are achieved, significantly enhancing the system's fault tolerance.

[0017] In terms of system scalability and fault tolerance, dynamic resource allocation and progressive recovery mechanisms provide the system with elastic scaling capabilities. When nodes join or leave, an improved version of the Gossip protocol enables rapid topology discovery and fault detection. Combined with Flink's global state monitoring, task redistribution latency is extremely low. When network quality fluctuates, the system automatically adjusts transmission channels based on real-time network metrics and avoids oscillations after network recovery through a traffic gradient back-switch strategy. Furthermore, FPGA-accelerated Bloom filters significantly improve result deduplication throughput and maintain high throughput even during software mode switching, ensuring business continuity.

[0018] At the goal consistency level, the combination of a global utility function and a centralized coordination layer effectively solves the joint dilemma of distributed systems. Through dynamic broadcasting of weight coefficients in Kafka-specific topics, all sub-agents share a unified task priority and their own priority calculation model. Combined with a lightweight consensus protocol's snapshot mechanism for key metadata, the decision conflict rate is significantly reduced. In scenarios where multiple agents compete for the same resource, the system ensures a high completion rate for high-priority tasks and effectively controls the starvation time of low-priority tasks through dynamic token transfer and priority arbitration.

[0019] In summary, this invention, through the deep integration of technologies such as protocol hybridization, dynamic scheduling, and consistency assurance, effectively solves the problems of message synchronization, resource competition, and target consistency in distributed environments while ensuring the advantages of decentralization. It can be widely applied to scenarios with extremely high requirements for real-time performance and reliability, such as industrial IoT, autonomous driving fleet collaboration, distributed AI training, and edge computing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the decentralized multi-agent message passing process of this invention. Figure 2 This is a block diagram of the dynamic degradation mechanism in an embodiment of the present invention; Figure 3 This is a diagram of the decentralized multi-agent information interaction architecture of the present invention; Figure 4 This is a flowchart illustrating the metadata synchronization process for sub-intelligent agents according to an embodiment of the present invention. Figure 5This is a decentralized multi-agent messaging protocol stack according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the dynamic degradation mechanism of an embodiment of the present invention. Figure 7 This is a flowchart of the contract network protocol of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] The present invention will now be described in further detail with reference to the accompanying drawings: See Figures 1-6 This invention discloses a decentralized multi-agent information interaction method. Through a hierarchical design, it achieves a closed-loop process encompassing task distribution, inter-agent collaboration, and result aggregation. The core objective is to solve the challenges of message synchronization, resource contention, and target alignment in distributed scenarios while ensuring high reliability and low latency. The overall process can be divided into three core stages: task distribution, sub-agent collaboration and interaction, and result aggregation and feedback. Each stage achieves efficient and reliable information interaction through technological collaboration. The method includes the following steps: S1, Task Distribution Phase (Task-Sub-Agent): In this phase, the user task is first decomposed into multiple sub-tasks using a hybrid dimension decomposition mechanism. Each sub-task carries a unique ID, resource ID, and dependency information (depends on). Subsequently, the system writes the sub-tasks to the specified Kafka Topic and binds them to the corresponding partitions using resource ID hash modulo, achieving sequential mapping between tasks and critical resources and natural load balancing—each sub-agent subscribes to a specific partition, avoiding multiple agents consuming the same task repeatedly. To further optimize resource allocation, the Flink stream processing engine monitors the CPU, memory, and other load metrics of the sub-agents in real time and dynamically adjusts the mapping relationship between task partitions and sub-agents: high-priority tasks can temporarily adjust partition allocation through a preemption mechanism to ensure that urgent tasks are processed first. The key design of this phase lies in combining Kafka's partition ordering with Flink's global state monitoring capabilities, ensuring reliable message delivery (Kafka's persistent storage and retry mechanism) while achieving fine-grained dynamic load balancing. Tasks are written to the specified Kafka Topic, and tasks are distributed to different partitions according to the hash of the task ID. Sub-agents subscribe to their respective assigned partitions to achieve load balancing and prevent multiple sub-agents from consuming the same task repeatedly. Specifically, this includes the following steps: S101, the task issued by the user is first broken down into multiple subtasks (considering logical dependencies, size limits, resource matching, etc.); a unique ID and metadata (resource ID and dependency relationship) are generated for each subtask.

[0026] Specifically, a large task can be automatically broken down into several sub-tasks using the following method: The first stage involves coarsely breaking down tasks into logical units, dividing them into logically independent partitions. For example, image recognition tasks can be divided into two logical units: "object detection" and "classification." Data processing tasks can be split into fixed-size data blocks (e.g., each 64MB is a subtask). Computational workflow tasks can be divided according to processing stages (e.g., data preprocessing, model calculation, and result summarization). For tasks requiring cross-node communication, the system prioritizes assigning steps with high interaction frequency to the same partition to reduce communication latency.

[0027] The second phase involves further breaking down the resource requirements into smaller segments. For each segment, the resource requirements are labeled (e.g., object detection requires GPU, classification requires CPU). Specifically, each subtask is labeled with its required resource type (CPU / GPU / FPGA / storage bandwidth, etc.). For resources requiring mutually exclusive access, a resource ID (which can be a physical device ID or a logical resource ID) is generated. The resource ID is then hashed and moduloed to allocate the task to the corresponding Kafka segment. This ensures that tasks with the same resource are executed sequentially within the same segment, avoiding resource conflicts.

[0028] Phase 3: Dependency Completion. If there are data dependencies between subtasks, add the `depends on` field to the Kafka message to indicate the dependency relationship. For example, the classification subtask can depend on the output offset of the detection subtask as a dependency identifier to ensure the correct execution order. Specifically: Dependency-based splitting (DAG decomposition): To ensure that tasks with dependencies are executed sequentially, tasks that can be parallelized should be parallelized as much as possible. Specifically, tasks are abstracted into a Directed Acyclic Graph (DAG), where each node is a minimum computational unit. Nodes without dependencies can be split in parallel, with different nodes assigned to different agents for execution. For nodes with dependencies, the Kafka Offset of the upstream partition is first generated as the `depends on` field of the downstream, ensuring that the downstream only executes after the upstream has completed its task.

[0029] S102, when the user issues a task, the system loads the global utility function into the task description and broadcasts it to all sub-agents. The system first sets an "objective function" applicable to all sub-agents; this function acts like a unified benchmark to measure the quality of task completion. When a sub-agent receives a task, it uses this same standard to calculate its own gains or costs in completing the task.

[0030] The objective function, also known as the global utility function, is a calculation formula that all sub-agents follow when making decisions, used to measure the task priority. Weighting coefficients are used to adjust the importance of different factors. Global utility function = Task priority × Sub-agent priority; Task priority is determined by both the task's own attributes (such as deadline and manual marking) and global goals (such as energy saving). The expression is: Task Priority= w 1. Urgency Level+ w 2. Resource Requirements - w 3. Estimated energy consumption.

[0031] The priority of sub-agents is determined by hardware capabilities (such as CPU and network) and real-time performance (such as historical task execution time). The expression is: Agent Priority= α •Efficiency+ β •Energy consumption+ γ • Fairness.

[0032] Weight coefficients w1, w2, w3, α, β, and γ are dynamically broadcast via Kafka's config topic, and sub-agents update their local strategies in real time. These weight coefficients are periodically updated based on continuous system monitoring of the operating environment (latency, packet loss rate, task completion rate, node load, etc.). Efficiency: Measures the speed at which the sub-agent completes tasks; faster completion results in a higher score. Energy Consumption: Measures the resource consumption efficiency of the sub-agent while completing tasks; lower energy consumption results in a higher score. Fairness: Measures the stability and success rate of the sub-agent's task execution; greater stability results in a higher score.

[0033] The main agent sends the weight coefficients of the global utility function to all child agents in real time via a dedicated Kafka configuration topic. This ensures that regardless of where the child agents are located or when they receive a task, they immediately receive the latest weight values ​​and use the same parameters to calculate and optimize their execution strategies. This guarantees that all agents follow the same set of priorities and standards when making decisions, preventing inconsistent actions due to different parameters.

[0034] S103, Next, the sub-agents will communicate and bid using the Contract Network protocol, see [link / reference] Figure 7 : Sub-agents calculate execution parameters (required resources, estimated completion time, etc.) based on their local state, resource consumption, and global utility function, and submit these parameters to the bidding process. The system selects the most suitable executor based on the bids. In this way, all sub-agents' decisions are based on the same evaluation criteria, and the bidding process clearly defines task allocation, avoiding problems such as different agents acting independently, conflicting with each other, or duplicating work. Specifically: 1) Each sub-agent calculates a quote based on its local state, resource consumption, and objective function.

[0035] 2) The overall intelligent agent allocates tasks based on the quote and priority.

[0036] S104 employs a token bucket algorithm combined with Kafka partition serialization for critical resources (such as exclusive hardware and important data), using dynamic token transfer to preempt high-priority tasks. This approach ensures that execution permissions for subtasks are granted through a token bucket algorithm combined with Kafka partition serialization, preventing contention and state inconsistencies.

[0037] First, resources requiring mutually exclusive access (such as shared devices and data shards) are bound to specific Kafka partitions (with a fixed number of partitions) using a hash (modulo operation, based on the total number of Kafka partitions) of their resource IDs (divided by physical devices or logical resources). Leveraging the single-threaded consumption characteristic of partitions, resource requests are forced to be serialized, avoiding concurrency conflicts. In other words, each resource (such as a machine or a piece of data) is bound to a Kafka partition, and all requests are queued and processed sequentially within this channel, ensuring that only one sub-agent can operate on the resource at a time.

[0038] Next, each sub-agent needs to acquire the corresponding resource token (corresponding to the subtasks assigned to the task) before it can use the resource. The system maintains an independent token bucket for each critical resource (such as shared equipment), with a fixed total number of tokens in the bucket (e.g., 10). This limits the number of sub-agents that can use the resource simultaneously. Sub-agents belonging to high-priority tasks are allocated more initial tokens (e.g., 3 for high-priority tasks and 1 for normal-priority tasks), making it easier for them to gain resource usage rights. When multiple sub-agents compete, only those with enough tokens can execute operations. Sub-agent priorities are calculated by weighting dynamic metrics such as efficiency, energy consumption, and fairness, and the weight coefficients are synchronized in real-time via Kafka. The weights are dynamically broadcast through Kafka's config topics, and sub-agents update their local strategies in real-time.

[0039] A high-priority agent can borrow a token from a low-priority agent: Specifically, (1) the high-priority agent sends a Kafka borrowing request; (2) after receiving the request, the Kafka token management service checks whether the priority is high enough; it forcibly reclaims the token from the low-priority agent and allocates it to the high-priority agent; and it broadcasts the result through Kafka to inform all sub-agents. After the transfer is completed, the system automatically adjusts the subsequent token allocation ratio to avoid starvation. If a high-priority task urgently needs resources, it can temporarily "borrow" the token of a low-priority task and immediately gain the right to use it (after borrowing, everyone will be notified to update the quota). Once the urgent task is completed, the token will be automatically returned to ensure fairness.

[0040] In summary, the global utility function is essentially the weight of the utility function, which achieves dynamic and intelligent resource allocation by quantifying the priorities of tasks and sub-agents.

[0041] S2, Sub-agent Cooperation Phase (P2P+Gossip): The collaboration between sub-agents is achieved through a hybrid architecture of P2P direct communication (QUIC protocol) and Gossip protocol, which balances low latency and decentralized reliability. The specific interaction process is as follows: S201, Metadata Synchronization: Sub-agents periodically exchange metadata such as network topology and load status (e.g., CPU utilization, data fragment location) via an improved Gossip protocol (optimized based on the SWIM protocol). Neighbor selection employs a "dynamic neighbor list + logarithmic random push" strategy: each sub-agent maintains a neighbor list sorted by network topology distance, and randomly selects neighbors in each round. Messages are pushed to neighbors to avoid network flooding. When the receiver merges metadata, it follows the "latest timestamp first" rule and uses vector clocks to solve message out-of-order problems, ensuring causal consistency. In addition, a multi-path routing algorithm (Dijkstra-KNN) dynamically selects the optimal three paths based on real-time latency, reliability, and load to improve transmission efficiency.

[0042] Gossip metadata synchronization algorithm:

[0043] Neighbor selection strategy: Each sub-agent maintains a dynamic list of neighbors, sorted by network topology distance. Random selection is performed in each round. Push messages to neighbors to avoid network flooding.

[0044] Multi-path routing algorithms:

[0045] Dynamic weight adjustment: The coefficients are given an initial value and are adjusted every 5 seconds based on the real-time network status.

[0046] Path selection: The Dijkstra-KNN algorithm is used to select the three optimal paths.

[0047] In S202, sub-agents primarily communicate directly via the QUIC protocol to transmit intermediate computation results or state updates. QUIC's low-latency characteristics (0-RTT handshake, header compression) significantly reduce P2P communication latency. To address cross-protocol message order alignment issues, a Kafka global sequence number (Kafka Offset) and a local monotonically increasing counter (LocalCounter) are embedded in the message header: the receiver prioritizes sorting by Kafka Offset, and if Kafka Offsets are the same, sorting by the Local Counter; if a sequence number gap is detected, the message is buffered and a retransmission is actively requested. This mechanism ensures global order consistency between P2P and Kafka messages.

[0048] P2P sequence number alignment:

[0049] Sequence number generation rule: The Kafka Offset is obtained from the current Offset of the partition to which the task belongs, which ensures strong consistency. Simultaneously, a monotonically increasing counter maintained locally by the sub-agent, i.e., Local Counter, is used for generation, with each P2P connection operating independently.

[0050] In S203, when P2P communication fails due to network jitter (such as a sudden increase in 5G latency) or node failure, the system automatically degrades to Kafka relay mode: the sub-agent writes the message to the corresponding Kafka partition, and the consumer pulls it again. At this time, CRDT (Conflict-Free Data Type) resolves distributed conflicts through the "maximum value merging" rule of incrementing and decrementing counters, ensuring eventual consistency. Flink undertakes message integrity monitoring at this stage; if a message is missing, it triggers breakpoint resumption or data repair (e.g., fast deduplication via an FPGA-accelerated Bloom filter to avoid redundant processing). Specifically: The system selects the transmission channel based on the real-time network status (latency, packet loss rate, load): if the network is good, P2P / Gossip is selected; if the network deteriorates, it automatically switches to Kafka; when the network recovers, a gradual switchback is performed: initially 10% of the traffic goes through P2P, and then gradually increases until a full switch is finally performed.

[0051] When a sequence number gap is detected, a precise retransmission is triggered, filling in only the missing portion rather than retransmitting the entire packet. The receiver prioritizes sorting by Kafka Offset; if Kafka Offsets are the same, they are sorted by Local Counter; when a sequence number gap is detected, the receiver actively requests to resume transmission from the breakpoint.

[0052] S3, Local processing by sub-agents (ensuring state consistency), specifically: (1) The sub-agent receives the fragment and performs local processing.

[0053] (2) For critical metadata (such as task allocation results, task progress markers, global configuration version numbers, cross-task dependency states, and other core information that affects task correctness and consistency): use the lightweight Raft consensus protocol for synchronization, only synchronizing critical metadata, and generating a snapshot every 10 operations to reduce synchronization overhead. Conflicts in critical metadata will lead to task errors or inconsistent global states, therefore strong consistency must be achieved through Raft.

[0054] (3) For non-critical metadata (such as intermediate cache results, statistical counts, log information, temporary performance metrics, and other auxiliary information that will not affect global correctness): use CRDT increment / decrement counter merging, and resolve task conflicts according to the maximum value priority strategy (i.e., take the maximum value of the increment / decrement counters of each node during merging) to ensure eventual consistency. Specifically, the expression for merging non-critical data using CRDT increment / decrement counters is:

[0055] in, This indicates the value of the increment counter; This indicates the value of the decrement counter.

[0056] Even if non-critical metadata is temporarily inconsistent, it does not affect the correctness of the final calculation, but only affects performance or monitoring accuracy. Therefore, CRDT merging is adopted to reduce overhead.

[0057] S4. Result Aggregation and Feedback Phase (Sub-agent - Summarizing Agent) The sub-agent writes the task execution results to a Kafka results topic. The Flink engine consumes this topic in real time, performing deduplication (based on Kafka Offset and message digest), sorting (by task dependencies and global timestamps), and aggregation (such as summation and concatenation) operations to ensure the consistency and integrity of the results. The aggregated intermediate results are submitted to the main agent, which combines knowledge graph retrieval (such as associating task context and historical data) to generate the final response and synchronizes key state updates (such as task completion markers and global configuration versions) via the Gossip protocol, forming a closed loop of "distributed computing - centralized aggregation - decentralized state synchronization". Specifically: The sub-agents send their processing results back to the aggregation module via Kafka for merging (this can be used in conjunction with Flink real-time stream processing). During merging, topics are consumed. In the topic consumption phase, a Bloom filter using parallel hash computation on an FPGA is used to achieve real-time deduplication, capable of simultaneously detecting duplicates in tens of thousands of records; with extremely low latency, it is several times faster than pure software processing. If the FPGA fails, it automatically switches to software mode; although the speed decreases, it ensures continuous system operation. The aggregation module merges the deduplicated results, generates the final output, and feeds it back to the user.

[0058] The aggregation module is responsible for merging the execution results of each sub-agent to form a complete output result. The core principles of merging are: (1) Deduplication: Ensure that the same data is not calculated repeatedly (e.g., FPGA has filtered out duplicates). (2) Timing alignment: Organize the results according to the task order (Kafka Offset) to avoid disorder. (3) Conflict handling: If there are contradictions in the results of the sub-agents, they are adjudicated according to preset rules / knowledge graph.

[0059] To avoid congestion or data loss, Kafka is used to queue and transmit the results of all sub-agents sequentially. Each result message includes: task ID, sub-agent ID, and data content. If real-time statistics are required, Flink is used for statistics. Sub-agents write their results to a Kafka result topic, and the aggregation module (or Flink) consumes, organizes, and merges them in real time, forming a closed loop of "distributed computing—centralized aggregation—decentralized state synchronization".

[0060] One embodiment of the present invention provides a decentralized multi-agent information interaction system, comprising: The task decomposition module decomposes user tasks according to a mixed dimension of task type, resource requirements, and dependencies, resulting in multiple subtasks carrying unique identifiers and metadata. Simultaneously, a global utility function is loaded into the task description of each subtask. The metadata includes resource identifiers (resource IDs) and dependencies. The task mapping module maps subtask hashes to different partitions of Kafka based on the resource ID, and dynamically adjusts the mapping relationship between subtasks and sub-agents through the Flink engine. At the same time, it broadcasts the weight coefficients of the global utility function to the sub-agents through Kafka's config Topic. In the task allocation module, sub-agents compete for bids through the Contract Network Protocol to find the sub-agent with the most suitable bid as the task executor. The resource allocation module uses the token bucket algorithm to arbitrate execution permissions for critical resources based on the priority of sub-agents in the global utility function. The sub-agent collaboration module uses the Gossip protocol to transmit metadata between sub-agents; it uses P2P direct communication to transmit intermediate data, and automatically degrades to Kafka relay in case of anomalies; the message header of the intermediate data carries the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter; at the same time, Flink is responsible for message integrity monitoring, out-of-order detection and breakpoint resume. The task execution module consists of sub-agents that execute tasks. Key metadata is synchronized using a lightweight Raft consensus protocol, while non-key metadata is merged using CRDT increment / decrement counters. The task aggregation module writes the execution results of the sub-agents into the Kafka result topic, and consumes the Kafka result topic through the Flink engine to perform deduplication, sorting and aggregation operations in sequence; based on the aggregated results, the agent combines the knowledge graph to generate the final response and synchronizes the key states through the Gossip protocol.

[0061] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A decentralized multi-agent information interaction method, characterized in that, Includes the following steps: User tasks are decomposed into multiple subtasks by a hybrid dimension of task type, resource requirements, and dependencies, resulting in multiple subtasks carrying unique identifiers and metadata; at the same time, a global utility function is loaded into the task description of the subtasks. The metadata includes resource ID and dependencies; Based on the resource ID, the subtask hash is mapped to different partitions of Kafka, and the mapping relationship between the subtask and the sub-agent is dynamically adjusted through the Flink engine. At the same time, the weight coefficient of the global utility function is broadcast to the sub-agent through Kafka's config Topic. Sub-agents bid against each other through a contract network protocol to find the sub-agent with the most suitable bid to serve as the task executor. Based on the priority of sub-agents in the global utility function, the execution permission arbitration control of critical resources is performed through the token bucket algorithm. Sub-agents use the Gossip protocol to transmit metadata; intermediate data is transmitted using P2P direct communication, and automatically degrades to Kafka relay in case of anomalies; the message header of the intermediate data carries the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter; at the same time, Flink is responsible for message integrity monitoring, out-of-order detection and breakpoint resumption. Sub-agents execute tasks, using the lightweight Raft consensus protocol to synchronize critical metadata and using CRDT increment / decrement counters to merge non-critical metadata. The execution results of the sub-agents are written to the Kafka result topic. The Flink engine consumes the Kafka result topic and performs deduplication, sorting and aggregation operations in sequence. Based on the aggregated results, the agent combines the knowledge graph to generate the final response and synchronizes the key states through the Gossip protocol.

2. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The hybrid dimensionality decomposition includes: The first stage involves roughly breaking down tasks into logically independent modules based on logical units. The second stage involves breaking down resource requirements into smaller parts, labeling the types of resources required by each module and generating resource IDs. The third stage involves completing the dependency chain by adding the `depends on` field to the Kafka message to establish a DAG dependency chain.

3. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The global utility function is described as follows: Global utility function = task priority × sub-agent priority; in: Task priority = Urgency level + • Resource requirements - • Estimated energy consumption; Sub-agent priority = Efficiency+ Energy consumption+ • Fairness; Weighting coefficient , , , , , Real-time dynamic broadcasting via topics in Kafka.

4. The decentralized multi-agent information interaction method according to claim 3, characterized in that, The contract network protocol is as follows: The sub-agent calculates the local quote based on the global utility function and dynamic weight coefficients; The main agent allocates tasks based on local quotes, task priorities, and sub-agent priorities.

5. The decentralized multi-agent information interaction method according to claim 4, characterized in that, The token bucket algorithm includes: Maintain an independent token bucket for each critical resource, with a fixed total number of tokens in each bucket; Allocate resources to sub-agents based on their priorities; High-priority sub-agents send borrowing requests via Kafka; after arbitration by the main agent, tokens from low-priority sub-agents are forcibly revoked and allocated to high-priority sub-agents; after borrowing is completed, token quotas are updated via Kafka broadcast, and tokens are automatically returned after the task is completed.

6. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The use of the Gossip protocol to transmit metadata includes: Each sub-agent sorts its neighbor list by network topology distance; based on the neighbor list, a multipath routing algorithm selects the optimal three paths according to path scores; the multipath routing algorithm uses the Dijkstra-KNN algorithm; the path scores are represented as: in, Indicates real-time delay; Indicates reliability; Indicates load; , and All are weighting coefficients.

7. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The intermediate data is transmitted using P2P direct communication, and automatically downgrades to Kafka relay in case of an anomaly. Specifically: P2P direct communication uses the QUIC protocol. The message header embeds the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter. The receiver prioritizes sorting by Kafka Offset, and if the Kafka Offsets are the same, sorting is done by Local Counter. When P2P communication fails, it automatically degrades to Kafka relay mode. When the network recovers, it performs a gradual switchback. The gradual switchback specifically means that initially a portion of the traffic goes through P2P, and then gradually increases until a full switchback is finally performed. When Flink detects a break in Kafka Offset or Local Counter, it actively requests to resume transmission from the breakpoint.

8. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The specific steps for performing deduplication, sorting, and aggregation operations sequentially on the Kafka result topic consumed by the Flink engine are as follows: Deduplication is achieved through a Bloom filter that performs parallel hash calculations on the FPGA, and the system automatically switches to software mode when the FPGA is unavailable. The deduplicated results are sorted by Kafka Offset. If the results are out of order, they are determined according to preset rules / knowledge graph. The sorted results are aggregated and submitted to the main agent.

9. The decentralized multi-agent information interaction method according to claim 1, characterized in that, The key metadata includes task allocation results, task progress markers, global configuration version numbers, and cross-task dependency status. The non-critical metadata includes intermediate cache results, statistical counts, log information, and temporary performance metrics.

10. A decentralized multi-agent information interaction system, characterized in that, include: The task decomposition module decomposes user tasks according to a mixed dimension of task type, resource requirements, and dependencies, resulting in multiple subtasks carrying unique identifiers and metadata. At the same time, the global utility function is loaded into the task description of the subtask; The metadata includes resource ID and dependencies; The task mapping module maps subtask hashes to different partitions of Kafka based on the resource ID, and dynamically adjusts the mapping relationship between subtasks and sub-agents through the Flink engine. At the same time, it broadcasts the weight coefficients of the global utility function to the sub-agents through Kafka's config Topic. In the task allocation module, sub-agents compete for bids through the Contract Network Protocol to find the sub-agent with the most suitable bid as the task executor. The resource allocation module uses the token bucket algorithm to arbitrate execution permissions for critical resources based on the priority of sub-agents in the global utility function. The sub-agent collaboration module uses the Gossip protocol to transmit metadata between sub-agents; it uses P2P direct communication to transmit intermediate data, and automatically degrades to Kafka relay in case of anomalies; the message header of the intermediate data carries the Kafka global sequence number Kafka Offset and the local monotonically increasing counter Local Counter; at the same time, Flink is responsible for message integrity monitoring, out-of-order detection and breakpoint resume. The task execution module consists of sub-agents that execute tasks. Key metadata is synchronized using a lightweight Raft consensus protocol, while non-key metadata is merged using CRDT increment / decrement counters. The task aggregation module writes the execution results of the sub-agents into the Kafka result topic, and consumes the Kafka result topic through the Flink engine to perform deduplication, sorting and aggregation operations in sequence; based on the aggregated results, the agent combines the knowledge graph to generate the final response and synchronizes the key states through the Gossip protocol.

Images