Multi-agent interaction method and system based on hierarchical negotiation and semantic driving

Abstract

The present application belongs to the technical field of artificial intelligence, robots and network communication, and particularly relates to a multi-agent interaction method and system based on layered negotiation and semantic driving, which can realize robust, reliable and highly adaptive collective intelligence. The present application has the semantic richness of LLM protocol, the real-time reliability of IUS protocol, supports dynamic negotiation ability of decentralization, and contains a security mechanism to deal with semantic attacks. The three mechanisms of layered decoupling, dynamic negotiation and endogenous security are integrated into one, which can produce a synergistic effect of 1+1+1>3, and solves the problem that the prior art cannot simultaneously satisfy semantic richness, execution reliability, dynamic adaptability and communication security.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, robotics and network communication technology, and specifically relates to a multi-agent interaction method and system based on hierarchical negotiation and semantic driving. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially the powerful understanding and generative capabilities provided by large language models (LLMs), intelligent agents are evolving from single, closed systems to open, collaborative multi-agent systems. This transformation offers a new paradigm for solving complex real-world problems such as collaborative combat of unmanned systems, smart logistics, and disaster relief. However, achieving efficient and reliable multi-agent collaboration relies heavily on an interaction mechanism capable of carrying complex intentions, adapting to dynamic environments, and ensuring precise execution. Currently, the technological paths in this field are fragmented, and a unified framework that can systematically bridge the gap between high-level cognitive flexibility and low-level physical execution determinism has not yet been formed.

[0003] The existing mainstream technical approaches can be summarized into four categories. The first category is traditional agent communication languages, such as FIPA-ACL and KQML. Based on rigorous speech act theory, they define structured communication primitives. However, their pre-defined, limited set of speech acts proves inflexible when dealing with the ever-increasing number of new tasks and ambiguous instructions in open worlds, making it difficult to fully integrate and utilize the open semantic understanding and generation capabilities offered by LLM. The second category is domain-specific real-time communication protocols, such as V2X in vehicle-to-everything (V2X) and DDS in robotics. These protocols excel in ensuring real-time performance, reliability, and quality of service, but their communication content is highly formatted, with rigid semantic expression capabilities. They primarily transmit state data or predefined instructions, failing to support advanced cognitive activities requiring complex intent negotiation and joint planning, leading to a semantic gap between high-level decision-making and low-level control. The third category is emerging auxiliary protocols or frameworks centered on LLM, such as model-context protocols. These primarily serve as bridges between LLMs and tools, essentially acting as API wrappers rather than designs for dynamic, autonomous collaboration between peer agents. Solutions that directly use natural language communication face the problem of a mismatch between the semantic space and the model's internal representation, easily leading to information distortion and drift in collaborative goals. The fourth category comprises learning-based communication optimization strategies, such as using attention mechanisms or information bottleneck principles to decide on communication timing or compress messages. While these methods can improve communication efficiency, they mainly address the questions of "whether to communicate" and "how to compress," failing to address the fundamental issue of how to establish a trustworthy, interpretable, and semantically consistent interaction foundation between heterogeneous agents.

[0004] In summary, existing technical solutions have failed to organically combine "layered protocols," "dynamic negotiation," and "message frameworks with built-in trust." Therefore, when addressing the collaborative needs of next-generation LLM-driven heterogeneous multi-agent systems, several fundamental contradictions are exposed: First, the system architecture and communication semantics are poorly aligned, with an unbridgeable gap between the semantic flexibility of high-level LLMs and the deterministic requirements of low-level physical execution. Second, there is a contradiction between static interaction and dynamic requirements; predefined fixed interaction patterns cannot adapt to the continuous changes in tasks, members, and network conditions in open environments. Third, interoperability barriers exist within the system; semantic gaps between heterogeneous agents hinder large-scale cross-domain integration. Fourth, there is a lack of defense against new security risks; traditional security mechanisms are insufficient to address the "semantic attack surface" introduced by LLMs. These core issues collectively constitute the key bottleneck in the development of current multi-agent interaction technologies, urgently requiring a novel and systematic solution. Summary of the Invention

[0005] In view of this, the present invention provides a multi-agent interaction method and system based on hierarchical negotiation and semantic driving, which can simultaneously satisfy semantic richness, execution reliability, dynamic adaptability and communication security.

[0006] To achieve the objectives of this invention, the following technical solutions are provided.

[0007] A multi-agent interaction method based on hierarchical negotiation and semantic-driven approach includes: The first and second intelligent agents perform dynamic negotiation and binding handshake to establish a cooperative relationship and a secure communication channel. During task execution, the first agent and the second agent conduct high-level semantic negotiation and intent alignment through the cognitive layer, wherein the cognitive layer adopts a semantic message format based on a shared ontology. The first intelligent agent recognizes and executes a verifiable conversion mechanism to convert the negotiated high-level intent into secure and verifiable low-level execution instructions, wherein the conversion process is forcibly injected with the capability limitations and / or environmental constraints of the second intelligent agent; The first intelligent agent sends the underlying execution instructions to the second intelligent agent through the execution layer, wherein the execution layer adopts a low-latency real-time communication mechanism; In the process of high-level semantic negotiation, sending of low-level execution instructions and task status reporting, the interactive messages are all encapsulated and parsed according to a preset trust-centered message framework. The message framework is required to include semantic grounding metadata for eliminating semantic ambiguity, data tracing and taint marking metadata for tracing data sources and identifying trustworthiness, and execution constraint metadata for specifying the pre-execution security conditions of actions.

[0008] The dynamic negotiation and binding handshake includes: Discovery and verification phase: The first and second agents discover each other and exchange cryptographically signed capability description documents to verify their identities and capabilities; Negotiation and Alignment Phase: Verified agents engage in dialogue based on meta-protocols to reach a consensus on task objectives, roles, resource commitments, and the specific communication protocols and message patterns used in this interaction; Binding and Execution Phase: Based on the negotiation results, a dedicated secure communication channel for task execution is instantiated, and the agreed message parser is loaded.

[0009] The meta-protocol followed in the negotiation and alignment phase is modeled as a finite state machine, and state transitions are triggered by meta-protocol messages that conform to a predetermined pattern.

[0010] Specifically, the recognition of the verifiable conversion mechanism involves the following steps: Parse semantic messages from the cognitive layer to extract intent and goals; The intent is matched and verified against the declared capabilities of the target intelligent agent; Generate an initial execution plan; The physical capability limits of the target intelligent agent and the safety constraints of the task environment are forcibly injected into the initial execution plan, and the plan is then modified and verified for safety. The verified security plan is serialized into the format defined by the execution layer communication protocol.

[0011] The security policy conditions declared in the execution constraint metadata include: requiring actions to pass simulation verification in a digital twin environment, or requiring human in-loop confirmation.

[0012] This invention also proposes a multi-agent interaction system based on hierarchical negotiation and semantic-driven interaction, including a first agent and a second agent connected via a network. Both the first agent and the second agent are heterogeneous agents, and each includes: The negotiation mechanism is used to perform dynamic negotiation and binding handshake with other intelligent agents to establish cooperative relationships and secure communication channels for specific tasks. The hierarchical communication module includes a logically separated cognitive layer communication module and an execution layer communication module. The cognitive layer communication module is used for high-level semantic interaction based on a shared ontology, and the execution layer communication module is used for real-time control and state synchronization command interaction. A verifiable conversion device for cognition and execution is connected to the cognition layer communication module and the execution layer communication module, and is used to securely convert the semantic intent from the cognition layer into verifiable execution layer instructions that conform to the physical constraints and environmental rules of the target intelligent agent; The message processing module is used to encapsulate and parse sent and received messages according to a trust-centric message framework, which is required to include semantic grounding, data tracing and taint marking, and execution constraint metadata. A local knowledge base is used to store the agent's own capability descriptions, physical constraint parameters, and / or environmental rules.

[0013] The negotiation device is configured to perform a handshake process that includes the following phases: Discovery and verification phase: Discover potential collaborative agents and exchange cryptographically signed capability description documents to verify their identity and capabilities; Negotiation and Alignment Phase: Based on the meta-protocol, dialogue is conducted with the verified intelligent agent to reach a consensus on the task objectives, roles, resource commitments, and the specific communication protocol and message patterns used in this interaction. The meta-protocol is modeled as a finite state machine. Binding and Execution Phase: Based on the negotiation results, instantiate a dedicated secure communication channel for task execution.

[0014] The verifiable conversion device is configured to perform the following steps: Parse semantic messages from the cognitive layer to extract intent and goals; The intent is matched and verified against the declared capabilities of the target intelligent agent; Generate an initial execution plan; The physical capability limits of the target intelligent agent and the safety constraints of the task environment are forcibly injected into the initial execution plan, and the plan is then modified and verified for safety. The verified security plan is serialized into the format defined by the execution layer communication protocol.

[0015] In the message framework upon which the message processing module is based, the execution constraint metadata is used to declare the security policy conditions that the receiver must meet before processing the message or performing related actions. These conditions include requiring the action to pass simulation verification in a digital twin environment or requiring human in-loop confirmation.

[0016] Beneficial effects 1. This invention simultaneously possesses the semantic richness of the LLM protocol, the real-time reliability of the IUS protocol, and the ability to support decentralized dynamic negotiation. It also includes a security mechanism to deal with semantic attacks. By integrating the three major mechanisms of layered decoupling, dynamic negotiation, and intrinsic security, it can generate a synergistic effect of 1+1+1>3, solving the problem that existing technologies cannot simultaneously satisfy semantic richness, execution reliability, dynamic adaptability, and communication security.

[0017] 2. This invention resolves the fundamental contradiction between semantic richness and execution reliability, achieving a secure and controllable mapping from "intent" to "action." By establishing independent "cognitive layer" and "execution layer," this invention logically and channel-wise separates non-real-time, complex semantic negotiation from real-time, precise physical control, avoiding channel congestion and priority conflicts caused by mixed communication. The invention's unique CEVT algorithm acts as a "secure compiler" between the two layers, enabling format conversion and forcibly injecting the target agent's capability limitations (such as maximum speed) and environmental constraints (such as no-fly zones) into the generated execution instructions during the conversion process. This ensures that the abstract intent originating from the high-level LLM has undergone deterministic and verifiable security review before being transformed into physical actions. This mechanism significantly reduces the latency of critical control loops, solving the fundamental problem of "what can be understood cannot be precisely executed, and what can be precisely executed cannot be deeply understood" through architectural innovation, supporting the application of advanced AI in security-critical fields.

[0018] 3. This invention achieves a paradigm shift from "static preset" to "dynamic adaptation," endowing heterogeneous multi-agent systems with stronger interoperability and robustness. The three-phase dynamic negotiation handshake mechanism proposed in this invention enables agents from different manufacturers and following different internal standards to autonomously complete identity verification and capability exchange before the task begins, and reach a consensus on the goals, roles, and communication protocols (including message formats) to be followed in this task. This frees the system from dependence on a unified standard, achieving "plug-and-play" collaboration. This mechanism effectively tailors the scope of interaction through the "discovery and verification" phase, reduces message redundancy in the negotiation phase, improves connection establishment efficiency, and also endows the system with the ability to quickly reconstruct collaborative relationships when facing member changes or task changes. Its task allocation theoretical success rate is higher than traditional fixed protocol schemes, enhancing the robustness and scalability of the entire system.

[0019] 4. This invention establishes an intrinsic security and trust system at the protocol level, effectively resisting novel semantic attacks that may arise from LLM characteristics. This invention uses security attributes as mandatory metadata for messages. Data profiling and trust measurement models enable the receiver to quantitatively assess the "credibility" of each message, thereby filtering out data from unreliable sources at the source of information processing. An execution constraint mechanism, as an additional check, ensures that even if an instruction comes from a trusted agent, if the instruction itself contains high-risk operations (such as entering a dangerous area), the protocol stack will enforce a pre-defined security process (such as "must be simulated in a digital twin"), thereby preventing potential dangerous or unreasonable behaviors caused by model illusions or accidental errors. This mechanism extends security defense from the network layer to the protocol content layer, embodying intrinsic security (Security-by-Design) design. Its near-zero-trust content censorship capability is not possessed by existing communication protocols and is of high value for deploying multi-agent systems in adversarial environments.

[0020] 5. This invention integrates collaborative innovation across the architecture, protocol, and messaging layers into an organic whole. The dynamic negotiation mechanism is a prerequisite for establishing a layered architecture among heterogeneous intelligent agents, while the built-in trust framework is the cornerstone ensuring the secure operation of this layered architecture. Through the combined effect of these three elements, a multi-agent system is enabled for the first time to possess multiple key characteristics, including advanced cognitive intelligence, physical execution security, dynamic environmental adaptation, and cross-platform interoperability. This represents a significant technological breakthrough in the field, demonstrating outstanding substantive features and remarkable progress. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the hierarchical architecture of the heterogeneous multi-agent interaction system proposed in this invention.

[0022] Figure 2 The following is a detailed flowchart of the dynamic negotiation and binding handshake mechanism of the present invention, which clearly shows the detailed steps of the three-stage interaction.

[0023] Figure 3 This is a schematic diagram of the message framework structure of the present invention, showing that in addition to the payload, the message packet is also required to include metadata headers such as semantic grounding, data tracing, and execution constraints.

[0024] Figure 4 This is a schematic diagram of the meta-protocol negotiation state machine control process in this invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0026] The technical terms used in this invention are as follows: Heterogeneous intelligent agents: These are autonomous or semi-autonomous computing entities that differ in hardware platforms, software architectures, internal knowledge representations, core models (such as LLMs and rule engines), and their affiliated organizations or manufacturers. Each agent can own and maintain its own proprietary knowledge base, such as a vector database storing domain documents.

[0027] Cognitive Layer / Execution Layer: These are two logical layers in the protocol architecture of this invention. The cognitive layer handles high-level semantic negotiation of "why to do it" and "what to do"; the execution layer handles low-level real-time control of "how to do it precisely".

[0028] Meta-Protocol: Refers to a "protocol about protocols". In this invention, it is a dialogue framework used to negotiate the specific protocols, roles, and rules that should be followed in this task interaction before the formal task interaction begins.

[0029] Semantic grounding refers to the process of forcibly associating data fields in communication content with explicit concepts in a shared, machine-readable ontology or vocabulary to eliminate ambiguity.

[0030] Taint Marking: A security technique for controlling information flow. This invention applies it to a messaging framework to mark data originating from untrusted channels, thereby restricting its processing at the protocol level.

[0031] Execution Constraints: Executable policies carried in a message that specify the preconditions that the receiver must meet before processing the message or performing related actions, such as "must be simulated in a digital twin".

[0032] Digital Twin: A high-fidelity virtual copy of a physical entity, providing an ideal sandbox environment for security verification before performing high-risk physical operations.

[0033] This invention provides a multi-agent interaction method based on hierarchical negotiation and semantic-driven approach, comprising the following steps: Step 1: The agent interaction is logically divided into a Cognitive Layer and an Execution Layer. The Cognitive Layer is dedicated to handling high-level, non-real-time semantic interactions. Agents at this layer perform intent alignment, goal negotiation, role assignment, and collaborative planning. Communication at this layer employs a flexible, semantically rich message format (e.g., JSON-LD) based on a shared ontology, similar to a structured Natural Language Communication Protocol (NL-CP), fully leveraging the reasoning and understanding capabilities of LLM. The Execution Layer is dedicated to handling real-time, deterministic commands and control, as well as state synchronization. This layer employs an efficient, low-latency transmission mechanism to deliver compact and unambiguous state and control instructions for physical operations, and supports differentiated Quality of Service (QoS) guarantees.

[0034] Step 2 defines a verifiable translation mechanism from the cognitive layer to the execution layer to ensure that the abstract plan negotiated at the upper layer can be safely and reliably "compiled" into a precise sequence of physical actions to be executed at the lower layer, and to forcibly inject the target agent's capability limitations and environmental constraints during the translation process.

[0035] Specifically, the "Cognitive-to-Execution Verifiable Translation (CEVT)" mechanism acts as a secure compiler connecting high-level large-scale models and low-level control systems. The originality of the CEVT algorithm lies in its forward placement of "physical constraints" at the protocol translation layer, achieving command filtering based on physical laws. This ensures that regardless of any "illusions" or ambiguous instructions generated by the high-level AI, the instructions ultimately delivered to the low-level executor are physically safe and compliant. It includes the following execution steps: Step 21: Intent Resolution and Target Extraction. The system first receives semantic messages from the cognitive layer. The resolution module extracts core elements from unstructured or semi-structured natural language instructions, including "intent" (e.g., "reconnaissance"), "target" (e.g., "coordinate point P1"), and "priority".

[0036] Step 22: Capability Matching and Pre-Verification Before generating any plan, the mechanism first searches the target agent's capability knowledge base and maps the extracted intent against the "capability set" declared by the target agent. For example, if the intent is "reconnaissance," the system must verify whether the target agent possesses the specific capabilities of path planning and image acquisition. If the capabilities do not match, the process terminates and returns an error to prevent the issuance of unexecutable instructions.

[0037] Step 23: Initial Execution Path Generation. After verification, the system calls the planner to generate an initial, geometric path plan or action sequence by combining the system's current perceived environment map (such as occupied grid map) and the extracted target points. At this stage, the plan only focuses on task completion and does not yet include physical constraints.

[0038] Step 24: Safety Constraint Injection and Forced Verification (Core Innovation Step) The system traverses each action step in the initial plan and enforces the following dual constraint checks to achieve "intrinsic safety": Physical Capability Constraint Injection: The system reads the target agent's physical limit parameters (such as maximum flight speed, maximum load). If the speed set in a step of the initial plan exceeds the agent's maximum speed, the algorithm will automatically stop the parameter within the safety threshold, rather than directly reporting an error or risky execution; Environmental and Policy Constraint Verification: The system loads a set of environmental constraints (such as "no-fly zone" and "radio silence zone") and performs spatial and logical intersection checks on each action step. If a step is detected to violate hard safety constraints (e.g., the path crosses a no-fly zone), the system will trigger replanning logic or mark the instruction as unexecutable. Only actions that pass all the above checks and complete parameter corrections will be tagged with "verified" metadata and accompanied by a verification timestamp.

[0039] Step 25: The execution layer protocol compilation system serializes the action sequences that have undergone security cleaning and constraint injection into a compact binary format specific to the execution layer (such as a custom EBP format or ROS message), and officially sends it to the underlying control loop.

[0040] Figure 1 The example demonstrates two heterogeneous intelligent agents (Agent A and Agent B). Each agent contains a module that implements the communication logic between the cognitive and execution layers. The interaction process between the two begins with "Phase 1: Discovery and Proof," proceeds through "Phase 2: Negotiation and Alignment" to reach consensus, and finally concludes with "Phase 3: Binding and Execution," establishing a secure channel based on a layered protocol.

[0041] Step 3, establish a dynamic negotiation and binding handshake mechanism, such as Figure 2 As shown, this involves formalizing a standardized, multi-step interaction process that enables heterogeneous intelligent agents to dynamically establish secure collaborative relationships. This process includes: Phase 1: Discover & Attest. Agents discover each other through decentralized mechanisms and exchange capability description documents that are cryptographically signed (e.g., using decentralized identifiers, DIDs) to securely prove their identity and claimed capabilities.

[0042] Phase Two: Negotiation & Alignment. Verified agents enter a meta-protocol dialogue to reach consensus on the common goals of the current task, roles, resource commitments, and the specific protocol and message schema used in this interaction. This phase is crucial for resolving interoperability issues between heterogeneous systems. (This negotiation process follows the meta-protocol state machine detailed later.) Phase 3: Bind & Execute. Based on the negotiation results, a secure communication channel dedicated to this task is dynamically instantiated, and subsequent interactions are performed on it using the agreed-upon protocol.

[0043] Step 4: Establish a trust-centric messaging framework. The message structure defined in this invention includes three metadata headers and two payloads, such as... Figure 3 As shown, security and semantic information are mandatory components of the protocol to address emerging security threats. The metadata header includes: Semantic Grounding: Message fields must be linked to concepts in the shared ontology to ensure unambiguous and machine-interpretable meaning, fundamentally resolving semantic inconsistencies; Data Provenance & Taint: The message includes metadata to trace the source of information; data from untrusted sources will be tagged with "taint" by the protocol, restricting the agent's access to this data (e.g., it cannot be used as the basis for critical decisions); Execution Constraints: The message may include security policy constraints defined by the sender, such as "this action must first be simulated and verified in a digital twin," and the protocol requires the receiver to be able to parse and enforce these constraints.

[0044] The two payloads are the mission data payload and the status data payload.

[0045] Figure 4 This diagram illustrates the control flow of the meta-protocol negotiation state machine in this invention. It shows that the meta-protocol in the "negotiation and alignment" phase is modeled as a finite state machine (FSM), defining the negotiation process and state transition conditions. State transitions are triggered by specific types of meta-protocol messages (such as propose_task, accept_roles, confirm_bind), and the messages used are standardized using JSON Schema. Each transition represents an advancement in the negotiation process.

[0046] Taking a collaborative search and rescue scenario involving unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) as an example: Imagine that in this scenario, a UAV equipped with a wide-angle camera (agent A, driven by an LLM and adept at global planning) and a UGV equipped with high-precision sensors and a robotic arm (agent B, controlled by an embedded system and adept at precision operations) need to work collaboratively. The specific collaborative process is as follows: The dynamic negotiation and binding phase includes three parts: discovery and proof, negotiation and alignment, and binding and execution.

[0047] The discovery and verification process involves: A broadcasting a discovery request seeking a ground exploration partner. B responds to the request and provides a capability description document with a DID signature. This document declares capabilities such as "centimeter-level positioning" and "robotic arm operation (API: `manipulator.open(door_id)`), and specifies its associated private knowledge base as the "UGV model X maintenance manual vector database". A verifies B's signature, confirms the authenticity of its identity and capabilities, and performs a matching assessment with its own needs.

[0048] Negotiation and alignment specifically refer to: A and B entering into a meta-protocol dialogue.

[0049] A (Cognitive Layer Message): "I propose a 'Joint search mission in Area A.' I will serve as the 'Aerial Observer,' responsible for identifying potential targets and planning routes. You will serve as the 'Ground Executor,' responsible for approaching and verifying the targets. Simultaneously, I propose establishing a 'Safety Supervisor' role, to be held by one of my sub-processes, to review all high-risk operational recommendations. Agree?" B (Cognitive Layer Message): "Role assignment agreed. I propose that subsequent high-level intention interactions use the Structured Natural Language protocol `nl-cp.v2`, low-level path instructions use the efficient binary protocol `ebp.path.v1`, and target confirmation reports use the `report.schema.v2` schema. Here is my supported schema list [URI to schemas]." {"performative":"accept", "content": "Roles accepted. Propose high-level comms via 'nl-cp.v2', path instructions via 'ebp.path.v1', and reports via 'report.schema.v2'. My supported schemas: [URI_to_schemas]."} A (Cognitive Layer Message): "Confirmed communication mode. Negotiation complete." {"performative": "confirm", "content": "Schemas agreed. Negotiation complete."} The binding and execution process is as follows: Based on the negotiation results, A and B establish a communication channel using TLS encryption and load the agreed-upon `nl-cp.v2`, `ebp.path.v1`, and `report.schema.v2` message parsers. A internally instantiates a "security supervisor" agent.

[0050] The layered interaction and trust execution phase includes: High-level planning (cognitive layer): A discovers a collapsed building in the air that may harbor survivors and sends the command through the cognitive layer: "Plan a route to target point P1 for confirmation." {"intent": "investigate", "target": {"@id": "P1"}, "priority": "high"} Verifiable conversion: The conversion module inside A receives this high-level intent and combines the abstract instruction "Go to P1" with the 3D map and B's capability model (such as maximum speed and obstacle crossing ability), "compiling" it into a secure sequence of execution-level GPS waypoints with timestamps and speed limits. (For its specific implementation, please refer to the CEVT algorithm pseudocode below.) Low-level control (execution layer): A sends the first instruction to B through the execution layer: "GOTO_WAYPOINT: {lat:x, lon: y, speed_limit: 1.5m / s}".

[0051] Trust Mechanism Application: B arrives at P1 and finds the entrance blocked by an obstacle. B reports to A through the cognitive layer: "The entrance to P1 is blocked. I propose using a robotic arm to clear it." (The receiver will calculate the message's credibility based on the trust metric model defined later.) This message carries two key metadata elements: data provenance and execution constraints. The data provenance is defined as: "provenance": {"source": "self_observed", "confidence": 0.98}, indicating that this information was observed by B's own sensors and has high credibility. The execution constraints are: "constraints": [{"type": "simulation_required", "twin_id": "DT-UGV-B"}, {"type": "human_in_the_loop_approval", "level": "critical"}], requiring A to simulate the action in B's digital twin before approval and to obtain final confirmation from personnel at the command and control center.

[0052] Constraint Enforcement: Upon receiving the message, A's protocol stack first parses the metadata, identifying it as a high-confidence report originating from B's own sensors. Then, the protocol stack enforces constraints: submitting the request to an internal "safety supervisor" agent for preliminary risk assessment and invoking B's digital twin for simulation. After successful simulation, the results and on-site images are reported to a human commander for approval. Finally, A sends a cognitive-level approval instruction to B, at which point B executes the physical operation. This achieves a closed-loop process from high-level intent to secure physical execution.

[0053] Furthermore, regarding the capability description format: In the "discovery and proof" phase, in addition to using the W3C WoT Thing Description, other machine-readable capability description standards, such as the OpenAPI Specification (OAS), can also be used for extension, as long as the format can provide structured and verifiable agent capability information.

[0054] Negotiation Protocol: The meta-protocol in the "negotiation and alignment" phase can be simplified and modified based on existing negotiation protocols (such as some FIPA standards), or designed using an automated mechanism based on multi-agent reinforcement learning (MARL) to allow agents to learn the most efficient negotiation strategy on their own.

[0055] Execution layer transmission technology: The execution layer is not limited to DDS or V2X modes. Depending on the physical layer and data link layer characteristics of the specific application scenario, other real-time communication protocols can be selected, such as relevant standards for Time-Sensitive Networking (TSN) or a custom low-latency UDP protocol. Furthermore, an event-triggered communication mechanism can be integrated, enabling communication only when state changes exceed a preset threshold, thus saving energy and bandwidth.

[0056] Trust metadata implementation: The implementation of data traceability and taint marking can draw on lightweight blockchain technology (such as IOTA) to provide tamper-proof distributed ledger records for key information exchanges; the verification of execution constraints can be linked to formal verification engines or runtime monitoring modules in addition to digital twins.

[0057] Communication topology optimization: After the initial communication topology is determined by the negotiation mechanism of this invention, a "communication optimization agent" can be introduced to monitor communication traffic and information value during task execution and dynamically prune or enhance the communication link to minimize communication overhead while ensuring task performance.

[0058] This invention also provides a multi-agent interaction system based on hierarchical negotiation and semantic-driven mechanisms to implement the aforementioned method. Physically, this system is a distributed system composed of multiple heterogeneous agents (such as drones, unmanned vehicles, and software robots) connected via a network. Each agent contains software and / or hardware modules implementing the protocol of this invention. The system includes multiple heterogeneous agents, such as... Figure 1 The system includes agents A and B. Each agent has a built-in negotiation mechanism for performing dynamic negotiation and binding handshake mechanisms (such as...). Figure 2 The process shown includes a discovery and proof module, and a negotiation and alignment module (whose internal logic can be derived from...). Figure 4 The device comprises a state machine control module and a binding and execution module, enabling intelligent agents to autonomously establish secure collaborative relationships before a task begins; a hierarchical communication module, logically divided into a cognitive layer communication module and an execution layer communication module. The cognitive layer communication module handles semantically rich message interactions based on shared ontology (such as JSON-LD) for advanced cognitive activities such as intent alignment and planning negotiation. The execution layer communication module handles real-time control and state synchronization instructions based on efficient, low-latency protocols (such as custom binary formats, DDS messages, etc.); a Cognitive-to-Executable Verifiable Transformation (CEVT) device, acting as a "secure compiler" connecting the cognitive and execution layers, receives abstract instructions from the cognitive layer and generates secure, verifiable execution layer instructions through steps such as intent parsing, capability matching, security constraint injection (such as mandatory compliance with physical limits and environmental rules), and protocol compilation, ensuring that high-level intents can be reliably and securely mapped to low-level actions; and a trust center message processing module, responsible for processing messages according to... Figure 3The message framework structure shown encapsulates and parses messages. This module mandates the addition of metadata headers to each message, including semantic grounding (linking to the ontology), data profiling and taint marking (tracing the source and identifying credibility), and execution constraints (specifying pre-existing security conditions, such as digital twin simulation). It also encapsulates task data and state data as payloads, thereby embedding security and trust mechanisms at the protocol level. A local knowledge base and capability model store the agent's own capability descriptions, physical constraint parameters, environmental rules, and possible domain knowledge (such as a vector database). A secure communication channel is instantiated between agents for task execution based on the results of dynamic negotiation. This channel logically supports the separation or multiplexing of messages between the cognitive and execution layers. Optional external trust and verification services are also provided, such as identity services for DID verification, digital twin simulation environments for execution constraint verification, or formal verification engines.

[0059] System workflow with Figure 1 , 2 Taking the scenario shown as an example: Agent A (drone) and Agent B (unmanned vehicle) reach an agreement on the roles, objectives, and communication protocols for the search and rescue mission through three stages—"discovery and verification," "negotiation and alignment," and "binding and execution"—using their internal negotiation mechanisms, and establish a secure channel. During the mission, A's cognitive layer generates the intention to "reconnoiter a point," which is converted into a waypoint instruction with speed limits by its internal CEVT device and sent to B through the execution layer. After discovering an obstacle, B generates a request through its trust center message processing module, carrying traceability information of "high-confidence observation" and an execution constraint of "requiring digital twin simulation," and sends it to A. After A's message processing module parses the constraints, it automatically invokes the digital twin for security verification before finally approving B to execute the robotic arm operation. Through the collaborative work of the above modules, the entire system achieves a unity of semantic understanding, reliable execution, dynamic adaptation, and intrinsic security.

[0060] This invention proposes an innovative multi-agent interaction framework that systematically addresses the challenges of adaptability, interoperability, security, and verifiability in collaboration among heterogeneous agents in complex physical tasks through three core elements: layered architecture, dynamic negotiation, and built-in trust. Compared with traditional and emerging technologies, its advantages are mainly reflected in the following five aspects.

[0061] First, in terms of communication paradigm, this invention achieves a fundamental shift from "using fixed protocols" to "negotiating and generating protocols." Compared to the lack of semantic expression in the static speech act set or domain-specific protocols of traditional ACLs, this framework, through flexible semantic interaction and dynamic negotiation mechanisms at the cognitive layer, enables intelligent agents to negotiate communication languages ​​and patterns in real time for tasks, thereby greatly improving adaptability to unstructured environments.

[0062] Secondly, through a unique layered protocol architecture and a verifiable inter-layer conversion mechanism, this invention successfully bridges the gap between high-level semantics and low-level execution. It incorporates the advantages of domain-specific protocols in terms of transmission reliability into the execution layer, while making up for their semantic shortcomings at the cognitive layer, and ensuring that high-level intentions can be securely and reliably mapped into precise physical world actions, systematically resolving the contradiction between "semantic richness" and "execution reliability".

[0063] Third, the trust and security mechanisms built into this framework, such as semantic grounding, data traceability, and execution constraints, lay the foundation for reliable collaboration. Compared to methods that only focus on optimizing communication "efficiency" or black-box models such as "emergent languages," this invention prioritizes ensuring the interpretability, traceability, and security of communication content and actions, providing a trusted operating platform for various optimization algorithms, which is crucial for the deployment of security-critical physical systems.

[0064] Fourth, this invention is a general "meta-framework" decoupled from specific applications, possessing unparalleled flexibility. Unlike various domain-specific systems with pre-defined fixed roles and linear workflows, its core dynamic negotiation and handshake mechanism allows agents to autonomously construct the most suitable collaborative topology (such as linear, star, or mesh) at runtime, thereby flexibly adapting to various scenarios from simple processes to complex decentralized collaborations, avoiding single points of failure and improving scalability.

[0065] Finally, this invention complements rather than competes with many existing technologies. It provides the necessary underlying communication infrastructure regarding "efficiency, security, and interoperability" for learning-based communication optimization, game-theoretic decision-making algorithms, and efficiency optimization methods.

[0066] In summary, this framework, through systematic innovation, provides a solid interactive foundation for addressing complex physical tasks involving heterogeneous multi-agent systems, especially those involving mixed human-machine and unmanned systems.

[0067] The interaction method of this invention does not limit the specific implementation of the intelligent agent, but it is recommended that the intelligent agent follow the design pattern of "Belief-Desire-Intention (BDI)" or "Plan-Act-Reflect (PAR)". Taking the PAR pattern as an example, the intelligent agent first plans and breaks down the task (planning), then calls tools to execute sub-tasks (action), and finally summarizes and decides the next action based on the execution results and external feedback (reflection), thereby realizing the dynamic processing of complex tasks.

[0068] This invention includes, but is not limited to, the above embodiments. Any equivalent substitutions or partial improvements made under the spirit and principles of this invention shall be considered within the scope of protection of this invention.

Claims

1. A multi-agent interaction method based on hierarchical negotiation and semantic-driven approach, characterized in that, include: The first and second intelligent agents perform dynamic negotiation and binding handshake to establish a cooperative relationship and a secure communication channel. During task execution, the first agent and the second agent conduct high-level semantic negotiation and intent alignment through the cognitive layer, wherein the cognitive layer adopts a semantic message format based on a shared ontology. The first intelligent agent recognizes and executes a verifiable conversion mechanism to convert the negotiated high-level intent into secure and verifiable low-level execution instructions, wherein the conversion process is forcibly injected with the capability limitations and / or environmental constraints of the second intelligent agent; The first intelligent agent sends the underlying execution instructions to the second intelligent agent through the execution layer, wherein the execution layer adopts a low-latency real-time communication mechanism; In the process of high-level semantic negotiation, sending of low-level execution instructions and task status reporting, the interactive messages are all encapsulated and parsed according to a preset trust-centered message framework. The message framework is required to include semantic grounding metadata for eliminating semantic ambiguity, data tracing and taint marking metadata for tracing data sources and identifying trustworthiness, and execution constraint metadata for specifying the pre-execution security conditions of actions.

2. The method according to claim 1, characterized in that, The dynamic negotiation and binding handshake includes: Discovery and verification phase: The first and second agents discover each other and exchange cryptographically signed capability description documents to verify their identities and capabilities; Negotiation and Alignment Phase: Verified agents engage in dialogue based on meta-protocols to reach a consensus on task objectives, roles, resource commitments, and the specific communication protocols and message patterns used in this interaction; Binding and Execution Phase: Based on the negotiation results, a dedicated secure communication channel for task execution is instantiated, and the agreed message parser is loaded.

3. The method according to claim 2, characterized in that, The meta-protocol followed by the negotiation and alignment phase is modeled as a finite state machine, and state transitions are triggered by meta-protocol messages conforming to a predetermined pattern.

4. The method according to any one of claims 1-3, characterized in that, The recognition-to-execution verifiable conversion mechanism performs the following steps: Parse semantic messages from the cognitive layer to extract intent and goals; The intent is matched and verified against the declared capabilities of the target intelligent agent; Generate an initial execution plan; The physical capability limits of the target intelligent agent and the safety constraints of the task environment are forcibly injected into the initial execution plan, and the plan is then modified and verified for safety. The verified security plan is serialized into the format defined by the execution layer communication protocol.

5. The method according to claim 4, characterized in that, The security policy conditions declared in the execution constraint metadata include: requiring actions to pass simulation verification in a digital twin environment, or requiring human in-loop confirmation.

6. A multi-agent interaction system based on hierarchical negotiation and semantic-driven interaction, characterized in that, It includes a first intelligent agent and a second intelligent agent connected via a network. Both the first intelligent agent and the second intelligent agent are heterogeneous intelligent agents, and each includes: The negotiation mechanism is used to perform dynamic negotiation and binding handshake with other intelligent agents to establish cooperative relationships and secure communication channels for specific tasks. The hierarchical communication module includes a logically separated cognitive layer communication module and an execution layer communication module. The cognitive layer communication module is used for high-level semantic interaction based on a shared ontology, and the execution layer communication module is used for real-time control and state synchronization command interaction. A verifiable conversion device for cognition and execution is connected to the cognition layer communication module and the execution layer communication module, and is used to securely convert the semantic intent from the cognition layer into verifiable execution layer instructions that conform to the physical constraints and environmental rules of the target intelligent agent; The message processing module is used to encapsulate and parse sent and received messages according to a trust-centric message framework, which is required to include semantic grounding, data tracing and taint marking, and execution constraint metadata. A local knowledge base is used to store the agent's own capability descriptions, physical constraint parameters, and / or environmental rules.

7. The system according to claim 6, characterized in that, The negotiation device is configured to perform a handshake process that includes the following phases: Discovery and verification phase: Discover potential collaborative agents and exchange cryptographically signed capability description documents to verify their identity and capabilities; Negotiation and Alignment Phase: Based on the meta-protocol, dialogue is conducted with the verified intelligent agent to reach a consensus on the task objectives, roles, resource commitments, and the specific communication protocol and message patterns used in this interaction. The meta-protocol is modeled as a finite state machine. Binding and Execution Phase: Based on the negotiation results, instantiate a dedicated secure communication channel for task execution.

8. The system according to claim 6, characterized in that, The cognitive verifiable conversion device is configured to perform the following steps: Parse semantic messages from the cognitive layer to extract intent and goals; The intent is matched and verified against the declared capabilities of the target intelligent agent; Generate an initial execution plan; The physical capability limits of the target intelligent agent and the safety constraints of the task environment are forcibly injected into the initial execution plan, and the plan is then modified and verified for safety. The verified security plan is serialized into the format defined by the execution layer communication protocol.

9. The system according to any one of claims 6-8, characterized in that, In the message framework upon which the message processing module is based, the execution constraint metadata is used to declare the security policy conditions that the receiver must meet before processing the message or performing related actions. These conditions include requiring the action to pass simulation verification in a digital twin environment or requiring human in-loop confirmation.

Images