Multi-agent distributed situation interaction device and method based on multi-modal large model

Abstract

The application discloses a multi-agent distributed situation interaction device and method based on a multi-modal large model, and relates to the technical field of computer model application. The device comprises the following steps: constructing a double-layer distributed interaction network; setting an environment monitoring time window according to a preset feedback adjustment period, collecting the number of node survival, the available communication bandwidth value and the task urgency level as the environment state parameters; inputting the environment state parameters into a pre-constructed interaction strategy adjustment model to output the adaptive local node interaction strategy and the adaptive agent node interaction strategy in the next feedback adjustment period; and inputting the adaptive local node interaction strategy and the adaptive agent node interaction strategy into the double-layer distributed interaction network in the next feedback adjustment period to control the local consensus node and the agent node to respectively perform distributed situation interaction and output a unified task situation cognition benchmark. The application realizes high reliability, low delay and consistency guarantee of the distributed situation interaction of the unmanned cluster in a complex battlefield or dynamic environment.

Description

Technical Field

[0001] This invention relates to the field of computer model application technology, specifically to a multi-agent distributed situational interaction device and method based on a multimodal large model. Background Technology

[0002] With the rapid development of multi-agent cooperative control and distributed reinforcement learning technologies, autonomous cooperative methods based on hierarchical communication and policy iteration have attracted widespread attention in fields such as unmanned swarm systems and multi-robot collaboration. Existing patent technologies, such as CN113269329A, achieve distributed policy generation for multi-agents by constructing a leader-member hierarchical grouping network and combining it with ActorCritic reinforcement learning; CN110782011A uses integral reinforcement learning and non-policy iteration methods to solve the networked multi-agent consensus optimization control problem when the system model is completely unknown.

[0003] However, existing methods typically employ fixed single-layer or static layered consensus protocols, which struggle to ensure continuity when nodes are damaged, smooth operation when bandwidth is limited, and timeliness when tactics are urgent. Furthermore, they cannot simultaneously balance interaction efficiency, consistency, and environmental adaptability in complex battlefields or dynamic environments, thus failing to support reliable collaborative operations of unmanned swarms.

[0004] In summary, existing technologies have not yet solved the technical challenge of balancing reliability, real-time performance, and environmental adaptability in complex environments for fixed single-layer consensus protocols in distributed situational interaction. Summary of the Invention

[0005] This invention provides a multi-agent distributed situational interaction device and method based on a multimodal large model, aiming to solve the technical problem in the prior art that it is impossible to simultaneously take into account interaction efficiency, consistency and environmental adaptability in complex battlefields or dynamic environments.

[0006] In view of the above problems, the present invention provides a multi-agent distributed situational interaction device and method based on a multimodal large model.

[0007] In a first aspect, the present invention provides a multi-agent distributed situational interaction device based on a multimodal large model, comprising: A two-layer distributed interactive network construction module is used to construct a two-layer distributed interactive network, wherein the two-layer distributed interactive network includes N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. Furthermore, the local consensus nodes and agent nodes are deployed with multimodal large models. The environmental status parameter acquisition module is used to set the environmental monitoring time window according to the preset feedback adjustment cycle, and to collect the number of surviving nodes, the available communication bandwidth value, and the task urgency level as environmental status parameters. The adaptive interaction strategy generation module is used to input the environmental state parameters into a pre-built interaction strategy adjustment model, calculate and output the adaptive local node interaction strategy and the adaptive agent node interaction strategy in the next feedback adjustment cycle. The distributed situation interaction module is used to input the adapted local node interaction strategy and the adapted agent node interaction strategy into the two-layer distributed interaction network in the next feedback adjustment cycle, control the N local consensus nodes and M agent nodes to perform distributed situation interaction respectively, and output a unified task situation cognition benchmark.

[0008] Secondly, this invention provides a multi-agent distributed situational interaction method based on a multimodal large model, including: A two-layer distributed interaction network is constructed, wherein the two-layer distributed interaction network includes N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. Furthermore, the local consensus nodes and agent nodes are deployed with multimodal large models. The environmental monitoring time window is set according to the preset feedback adjustment cycle, and the number of surviving nodes, available communication bandwidth value and task urgency level are collected as environmental status parameters. The environmental state parameters are input into the pre-built interaction strategy adjustment model, and the adaptive local node interaction strategy and the adaptive agent node interaction strategy in the next feedback adjustment cycle are calculated and output. In the next feedback adjustment cycle, the adapted local node interaction strategy and the adapted agent node interaction strategy are input into the two-layer distributed interaction network, and the N local consensus nodes and M agent nodes are controlled to perform distributed situational interaction respectively, and output a unified task situational awareness benchmark.

[0009] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention provides a multi-agent distributed situational interaction device and method based on a multimodal large model. By constructing a two-layer heterogeneous interaction architecture and an adaptive strategy adjustment mechanism, it can achieve highly reliable, low-latency, and environmentally adaptive collaborative interaction of unmanned swarms in complex environments. Specifically, the two-layer heterogeneous architecture ensures global consistency through Byzantine fault-tolerant interaction between upper-layer local consensus nodes and supports large-scale scalability through lightweight semantic interaction between lower-layer agent nodes. The adaptive strategy adjustment mechanism dynamically switches interaction modes and parameters according to node survival status, communication bandwidth conditions, and tactical urgency, thereby ensuring interaction continuity when nodes are damaged, smooth interaction when bandwidth is limited, and timely interaction when tactics are urgent, ultimately providing reliable support for unmanned swarm collaborative operations in complex environments. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the structure of a multi-agent distributed situational interaction device based on a multimodal large model provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the multi-agent distributed situational interaction method based on a multimodal large model provided in an embodiment of the present invention. The components represented by each number in the attached diagram are explained below: The system includes a two-layer distributed interactive network construction module 11, an environmental state parameter acquisition module 12, an adaptive interactive strategy generation module 13, and a distributed situational interaction module 14. Detailed Implementation

[0011] This invention provides a multi-agent distributed situational interaction device and method based on a multimodal large model, which is used to address the technical problem in the prior art that it is impossible to simultaneously take into account interaction efficiency, consistency and environmental adaptability in complex battlefields or dynamic environments.

[0012] Example 1, as Figure 1 As shown, the present invention provides a multi-agent distributed situational interaction device based on a multimodal large model, the device comprising: The two-layer distributed interaction network construction module 11 is used to construct a two-layer distributed interaction network, wherein the two-layer distributed interaction network includes N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. Furthermore, the local consensus nodes and agent nodes are respectively deployed with multimodal large models.

[0013] In this embodiment of the invention, a two-layer distributed interaction network is constructed, comprising N local consensus nodes and M agent nodes. The local consensus nodes interact with each other using a Byzantine fault-tolerant consensus protocol, while the agent nodes interact with each other using a lightweight semantic consensus protocol. Furthermore, the local consensus nodes and agent nodes are each deployed with a multimodal large model. In complex battlefields or dynamic environments, unmanned swarms need to collaboratively complete situational awareness, target identification, and task planning. In existing technologies, multi-agent devices typically employ a single-layer communication architecture or a fixed layered protocol, which suffers from the following technical drawbacks: Firstly, single-layer broadcast or fully connected communication generates exponentially increasing communication overhead as the node scale expands, making it difficult to support real-time interaction in large-scale swarms; secondly, there is a lack of fault tolerance mechanisms for malicious or faulty nodes, and erroneous information from a single node may contaminate global cognition; furthermore, different agents carry sensors with varying modalities, lacking a unified semantic representation space, making it difficult to fuse heterogeneous information. Therefore, there is an urgent need for a distributed interaction network that can guarantee global consistency, support large-scale expansion, and possess heterogeneous semantic fusion capabilities.

[0014] In one embodiment, the two-layer distributed interactive network construction module 11 is further configured to: The multimodal large model deployed by the local consensus nodes and agent nodes includes a visual encoder, a text encoder, a signal encoder, and a geographic information encoder. The visual encoder is used to convert optical images or synthetic aperture radar images into visual semantic feature vectors. The text encoder is used to convert intelligence text or instruction text into text semantic feature vectors. The signal encoder is used to convert radar signals or communication signals into signal semantic feature vectors. The geographic information encoder is used to convert coordinate data or terrain data into geographic semantic feature vectors. The multimodal large model aligns the above-mentioned multiple semantic feature vectors to the same semantic space through a cross-modal attention mechanism, and outputs a semantic vector of a unified dimension.

[0015] First, a hierarchical node architecture for a two-layer distributed interactive network is constructed. This two-layer distributed interactive network consists of an upper-layer local consensus node layer and a lower-layer intelligent agent node layer. The upper layer is responsible for global situational awareness and fault-tolerant consensus, while the lower layer is responsible for lightweight local cluster interaction. The two layers collaborate to complete distributed situational awareness interaction. Specifically, the local consensus nodes are the upper-layer computing nodes of the network, responsible for global situational awareness consensus, semantic arbitration, and cluster management functions. There are N such nodes, deployed on devices with sufficient computing power, such as command vehicles, edge gateways, and airborne command platforms. The intelligent agent nodes are the lower-layer terminal sensing nodes of the network, corresponding to unmanned equipment such as drones, unmanned vehicles, and unmanned boats. There are M such nodes, each intelligent agent node belonging to a local consensus node.

[0016] Specifically, set the number of local consensus nodes N and complete node deployment; set the number of lower-level intelligent agent nodes M and divide the coverage area of ​​the corresponding local consensus nodes according to the cluster; establish stable communication links between upper-level nodes, between lower-level nodes in the same region, and between upper and lower-level nodes.

[0017] For example, a two-layer network is constructed: N=3 local consensus nodes are set up, numbered LC-01, LC-02, and LC-03, which are deployed in the forward command vehicle, edge computing gateway, and air command platform, respectively; M=6 intelligent agent nodes are set up, numbered AG-01 to AG-06, all of which are reconnaissance drones. Among them, AG-01 and AG-02 are under the jurisdiction of LC-01, AG-03 and AG-04 are under the jurisdiction of LC-02, and AG-05 and AG-06 are under the jurisdiction of LC-03, forming a standardized two-layer architecture of 3 local consensus nodes + 6 intelligent agent nodes.

[0018] Secondly, a multimodal large model is deployed and configured with modal encoding capabilities for the two-layer distributed interactive network nodes. The multimodal large model comprises four encoding units: a visual encoder, a text encoder, a signal encoder, and a geographic information encoder. Multimodal semantic alignment is achieved through a cross-modal attention mechanism. The encoder is a network unit that converts raw data from a single modality into standardized semantic feature vectors, realizing the semantic transformation of heterogeneous data. The cross-modal attention mechanism maps semantic feature vectors from different modalities to the same semantic space, outputting a unified-dimensional semantic vector, thus solving the problem of semantic incompatibility between multimodal data. A lightweight multimodal large model is obtained by performing at least one compression operation—model pruning, weight quantization, and knowledge distillation—on the full-version multimodal large model, adapting to the limited computing power of the agent nodes.

[0019] Specifically, local consensus nodes deploy the full version of the multimodal large model and complete the configuration of the four major encoders; agent nodes deploy a lightweight multimodal large model to ensure the real-time reasoning capability of the terminal; and a cross-modal attention mechanism is enabled to align the feature vectors of the four modalities into a unified dimension semantic vector.

[0020] For example, local consensus nodes LC-01, LC-02, and LC-03 deploy a complete multimodal large model, configured with four encoders: a visual encoder (processing UAV optical images and SAR synthetic aperture radar images); a text encoder (processing battlefield reconnaissance intelligence text and command instruction text); a signal encoder (processing airborne radar detection signals and battlefield communication signals); and a geographic information encoder (processing target latitude and longitude, terrain elevation, and landform type data). AG-01 to AG-06 compress the complete model through knowledge distillation to obtain a lightweight version adapted to the UAV's onboard computing power. A cross-modal attention mechanism aligns the four modal features into a 768-dimensional unified semantic vector, achieving semantic unification of multimodal data.

[0021] The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, including: Each of the N local consensus nodes broadcasts the locally generated structured semantic situation information to the other N minus 1 local consensus nodes. The structured semantic situation information includes a semantic vector generated by a multimodal large model and the corresponding confidence value. After receiving structured semantic situation information from other local consensus nodes, each local consensus node needs to verify the correctness of the information format, the freshness of the timestamp, and the validity of the source node's identity signature, and retain the verified structured semantic situation information as valid information. Each local consensus node executes a three-stage interaction process in sequence, consisting of a pre-preparation stage, a preparation stage, and a submission stage, based on all the valid information collected. In the pre-preparation stage, the status proposal to be reached is determined; in the preparation stage, preparation confirmation messages from other nodes are collected; and in the submission stage, submission confirmation messages from other nodes are collected. After the three-stage interaction process is completed, when at least two-thirds of the N local consensus nodes send a submission confirmation message for the same situational awareness result, it is determined that a consensus has been reached on the situational awareness result, and the situational awareness result is output as a globally consistent situational benchmark.

[0022] First, each of the N local consensus nodes broadcasts its locally generated structured semantic situational information to the remaining N-1 local consensus nodes. This structured semantic situational information includes a semantic vector generated by a multimodal large model and its corresponding confidence value. Structured semantic situational information refers to standardized situational data generated by the local consensus nodes based on the multimodal large model, containing a semantic vector and a confidence value. The semantic vector is a unified-dimensional multimodal fusion feature, and the confidence value characterizes the reliability of the situational awareness. Each of the N local consensus nodes then broadcasts its own generated structured semantic situational information point-to-point to the remaining N-1 local consensus nodes in the network, completing the initial distribution of the situational information.

[0023] For example, in this network, the number of local consensus nodes is N=3. Each node needs to broadcast information to the other two nodes: LC-01 broadcasts the locally generated 768-dimensional semantic vector + structured semantic situation information with a confidence level of 0.92 to LC-02 and LC-03; LC-02 broadcasts the locally generated 768-dimensional semantic vector + structured semantic situation information with a confidence level of 0.90 to LC-01 and LC-03; LC-03 broadcasts the locally generated 768-dimensional semantic vector + structured semantic situation information with a confidence level of 0.93 to LC-01 and LC-02.

[0024] Secondly, after receiving structured semantic situational information from other local consensus nodes, each local consensus node must verify the correctness of the information's format, the freshness of its timestamp, and the validity of the source node's identity signature. The verified structured semantic situational information is then retained as valid information. Format correctness means that the fields, data types, and dimensions of the situational information conform to preset standards, with no missing or garbled characters. Timestamp freshness means that the situational information was generated within the current period and is not expired or outdated historical data. The source node's identity signature is the node's digital signature on the information, used to verify the legitimacy of the information's source and that it has not been tampered with.

[0025] First, perform format correctness verification: check whether the structured semantic situation information contains complete required fields, including semantic vector, confidence value, timestamp, source node identifier and digital signature, and verify whether the data type of each field conforms to the preset specification, wherein the semantic vector is a fixed-dimensional floating-point array, the confidence value is a floating-point number between 0 and 1, and the timestamp is a time value that conforms to the preset format.

[0026] Secondly, timestamp freshness verification is performed: the timestamp is extracted from the structured semantic situation information, the timestamp is compared with the timestamp threshold obtained by subtracting the preset freshness window duration from the current time, and the local historical timestamp cache is queried to see if there is a larger timestamp from the same source node. When the timestamp is greater than the timestamp threshold and less than or equal to the current time, and there is no larger timestamp in the historical timestamp cache, the freshness verification is deemed to be successful.

[0027] Next, the validity of the source node identity signature is verified: the corresponding public key is found based on the source node identifier in the structured semantic situation information, and the digital signature is decrypted using the public key to obtain the first hash value. At the same time, the other fields except the digital signature are concatenated in a predetermined order and the second hash value is calculated. When the first hash value is equal to the second hash value, the signature verification is deemed to be successful.

[0028] Finally, after receiving the structured semantic situational information sent by other nodes, each local consensus node checks the format correctness, timestamp freshness, and source node identity signature validity in turn. If all three items pass, the information is deemed valid and retained; if any item fails, it is discarded.

[0029] For example, after receiving information from LC-02 and LC-03, LC-01 verifies that: the format all conforms to the 768-dimensional semantic vector specification, the timestamps are all in the current period, and the node signatures are all valid, so all of them are retained as valid information; LC-02 and LC-03 perform the same verification process, and finally all nodes retain the valid situation information of the other two nodes.

[0030] Furthermore, each local consensus node executes a three-stage interaction process—preparation, preparation, and submission—in sequence based on all the collected valid information. In the preparation stage, the status proposal to be agreed upon is determined; in the preparation stage, preparation confirmation messages from other nodes are collected; and in the submission stage, submission confirmation messages from other nodes are collected.

[0031] Specifically, in the pre-preparation phase, the master node selects one of the valid information as the situation proposal to be agreed upon, assigns a view number and sequence number to the proposal, generates a pre-preparation message, signs it, and broadcasts it to all slave nodes. After receiving the message, the slave nodes verify the validity of the view number, sequence number, and signature. If the verification is successful, the node enters the preparation phase. During the preparation phase, each slave node generates a preparation confirmation message based on the verified pre-preparation message, signs it, and broadcasts it to all other nodes. Each node continuously collects preparation confirmation messages from different nodes. When the number of collected preparation confirmation messages reaches a preset legal threshold, the preparation phase is determined to be complete and the submission phase begins. During the submission phase, each node generates its own submission confirmation message based on the preparation confirmation messages collected during the preparation phase, signs it, and broadcasts it to all other nodes. Each node continuously collects submission confirmation messages from different nodes. When the number of submission confirmation messages collected reaches a preset statutory threshold, the submission phase is determined to be complete, and the situation proposal is formally confirmed as a consensus result. If no sufficient valid messages are received within the preset timeout period in any of the above stages, the view switching process is triggered, and consensus is re-initiated after the master node is replaced.

[0032] For example, in the pre-preparation phase: based on valid information, all nodes determine the proposed situation for consensus in this round as: the enemy armored target is tactically maneuvering in the northwest direction; in the preparation phase: LC-01, LC-02, and LC-03 send preparation confirmation messages to the other two nodes respectively, all of which acknowledge the validity of the situation proposal; in the submission phase: LC-01, LC-02, and LC-03 send submission confirmation messages to the other two nodes respectively, all of which agree to accept the proposal as the consensus result.

[0033] Finally, after the three-stage interaction process is completed, when at least two-thirds of the N local consensus nodes have sent a commit confirmation message for the same situational awareness result, it is determined that a consensus has been reached on the situational awareness result, and the situational awareness result is output as the global consistent situational benchmark. At least two-thirds of the nodes is the statutory threshold for Byzantine fault-tolerant consensus; that is, if ≥2 / 3 of the total number of nodes reach a consensus, the consensus is considered effective and can withstand node failures and malicious attacks. After the three-stage interaction is completed, the number of nodes that sent a commit confirmation message for the same situational awareness result is counted; if this number is ≥2 / 3 of N, it is determined that a consensus has been reached on the situational awareness result, and this result is output as the global consistent situational benchmark.

[0034] For example, the total number of nodes N=3, and the 2 / 3 threshold is 2 nodes; in this round, 3 nodes LC-01, LC-02 and LC-03 all sent submission confirmation messages, which meets the condition of ≥2 / 3; therefore, it is determined that the enemy armored target has reached a consensus on tactical maneuvering in the northwest direction, and this result is output as the global consistency situation benchmark.

[0035] The intelligent agent nodes use a lightweight semantic consensus protocol for situational interaction, including: Each agent node inputs its local observation data into a lightweight multimodal large model, which outputs a compressed semantic vector and broadcasts the semantic vector as a cognitive summary to other agent nodes within the coverage area of ​​the same local consensus node. The lightweight multimodal large model is obtained by performing at least one compression operation on the multimodal large model deployed on the local consensus node, including model pruning, weight quantization, or knowledge distillation. After receiving cognitive summaries from other intelligent agent nodes, the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node is calculated, wherein the semantic similarity is measured using cosine similarity. Based on the calculated semantic similarity, the received multiple cognitive summaries are weighted and aggregated. The aggregated semantic vector obtained after weighted aggregation is used as a snapshot of the local situation consensus and reported to the local consensus node. The weight coefficient of the weighted aggregation is positively correlated with the confidence value of each cognitive summary and the semantic similarity.

[0036] First, each agent node inputs its local observation data into a lightweight multimodal large model. The lightweight multimodal large model outputs a compressed semantic vector and broadcasts the semantic vector as a cognitive summary to other agent nodes within the coverage area of ​​the same local consensus node. The lightweight multimodal large model is obtained by performing at least one compression operation on the multimodal large model deployed on the local consensus node, including model pruning, weight quantization, or knowledge distillation.

[0037] Local observation data refers to single-source or multi-source situational data collected by agent nodes through their own sensors, such as optical images, radar echoes, and positioning information. Lightweight multimodal large model refers to a lightweight model obtained by compressing the full version of the local consensus node's multimodal large model through at least one of the following methods: model pruning, weight quantization, and knowledge distillation. It has low computational cost and fast inference speed. Compressed semantic vectors refer to low-dimensional, compact semantic features obtained through inference using the lightweight model, used to reduce the amount of data transmitted.

[0038] Specifically, each agent node inputs its collected local observation data into the lightweight multimodal large model; the lightweight multimodal large model infers and outputs a compressed semantic vector, which is used as a cognitive summary; the cognitive summary is then broadcast to other agent nodes within the coverage area of ​​the same local consensus node.

[0039] For example, agent nodes AG-01 and AG-02, which belong to the coverage of the local consensus node LC-01: Both AG-01 and AG-02 are equipped with a lightweight multimodal large model that has been compressed by knowledge distillation; AG-01 inputs locally collected optical reconnaissance images into the lightweight model and outputs a 256-dimensional compressed semantic vector, which is broadcast to AG-02 as a cognitive summary; AG-02 inputs locally collected radar observation data into the lightweight model and outputs a 256-dimensional compressed semantic vector, which is broadcast to AG-01 as a cognitive summary.

[0040] Secondly, after receiving cognitive summaries from other agent nodes, the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node is calculated, where the semantic similarity is measured using cosine similarity. Semantic similarity measures the degree of closeness between two cognitive summaries in terms of situational awareness; a higher value indicates a more consistent situational judgment. After receiving cognitive summaries from other nodes, the agent node calculates the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node using cosine similarity.

[0041] For example, after AG-01 receives the cognitive summary from AG-02, it calculates that the cosine similarity between its local cognitive summary and AG-02's cognitive summary is 0.94; after AG-02 receives the cognitive summary from AG-01, it calculates that the cosine similarity between its local cognitive summary and AG-01's cognitive summary is 0.94.

[0042] Finally, based on the calculated semantic similarity, the received multiple cognitive summaries are weighted and aggregated. The aggregated semantic vector obtained after weighted aggregation is used as a local situation consensus snapshot and reported to the corresponding local consensus node. The weight coefficients of the weighted aggregation are positively correlated with the confidence value of each cognitive summary and the semantic similarity. Weighted aggregation refers to a calculation method that assigns weights to different cognitive summaries according to preset rules, fusing multiple semantic vectors into a single aggregated vector. The weight coefficient refers to the weight assigned to a single cognitive summary, which is positively correlated with the summary's confidence value and semantic similarity. The local situation consensus snapshot refers to the locally unified situational awareness result obtained by the agent node through weighted aggregation, used to report to the local consensus node.

[0043] Specifically, the weight coefficients for weighted aggregation are determined based on semantic similarity and the confidence values ​​of each cognitive summary; all received cognitive summaries are weighted and aggregated according to the weight coefficients to obtain an aggregated semantic vector; the aggregated semantic vector is used as a snapshot of local situational consensus and reported to the corresponding local consensus node.

[0044] For example, AG-01 has a confidence value of 0.90, AG-02 has a confidence value of 0.93, and their semantic similarity is 0.94. The weight coefficients are positively correlated with the two, so AG-01 is assigned a weight of 0.48 and AG-02 a weight of 0.52. The two 256-dimensional cognitive summaries are weighted and aggregated to obtain an aggregated semantic vector. This aggregated semantic vector is used as a snapshot of the local situation consensus and is reported by AG-01 and AG-02 to their respective local consensus nodes LC-01.

[0045] In this embodiment of the invention, a two-layer network architecture with upper-layer fault tolerance and lower-layer efficiency is constructed to separate global consensus and local interaction functions from the bottom layer, thus resolving the performance contradictions of single-layer architectures. A unified semantic representation of visual, textual, signal, and geographic information is achieved through a multimodal large model, bridging the semantic gap in cross-node situational awareness. Local consensus nodes employ a Byzantine fault-tolerant protocol to resist node damage and data tampering, ensuring the robustness and consistency of global situational awareness. Agent nodes adopt a lightweight semantic consensus protocol to reduce communication bandwidth and terminal computing power consumption, supporting scalable interaction in large-scale unmanned clusters. The two-layer architecture and differentiated protocols work together to support subsequent adaptive situational interaction and unified cognitive benchmark output.

[0046] The environmental status parameter acquisition module 12 is used to set the environmental monitoring time window according to the preset feedback adjustment cycle, and to collect the number of surviving nodes, the available communication bandwidth value, and the task urgency level as environmental status parameters.

[0047] In this embodiment of the invention, an environmental monitoring time window is set according to a preset feedback adjustment cycle, and the number of surviving nodes, available communication bandwidth, and task urgency level are collected as environmental status parameters. The network status, node status, and task scenario of distributed situational interaction change dynamically in real time. Fixed interaction strategies cannot adapt to complex scenarios involving node additions and subtractions, bandwidth fluctuations, and changes in task urgency. It is necessary to obtain the external environment and device operating status in real time to provide an objective basis for the dynamic adjustment of the interaction strategy.

[0048] Among them, the preset feedback adjustment period refers to a pre-set time interval used for periodically adjusting the interaction strategy. The environmental monitoring time window refers to a continuous time interval, with the feedback adjustment period as its length, used for collecting environmental data. The number of surviving nodes refers to the total number of local consensus nodes and agent nodes that are currently operating normally in the network and can participate in interaction. The available communication bandwidth value refers to the actual communication transmission rate between nodes that can be used for situational interaction. The mission urgency level is a level parameter characterizing the urgency of the current combat mission. The environmental state parameters are a set of features used to describe the operating environment of the device, composed of the number of surviving nodes, the available communication bandwidth value, and the mission urgency level.

[0049] Specifically, based on the preset feedback adjustment cycle, an environmental monitoring time window of corresponding length is set. Data is continuously collected within each time window, and three types of data—the number of surviving nodes, the available communication bandwidth value, and the task urgency level—are summarized as the environmental status parameters of the device.

[0050] For example, the preset feedback adjustment cycle is 1 second, and a corresponding 1-second environmental monitoring time window is set. Within the current monitoring window, it is collected that 3 local consensus nodes are alive and 5 agent nodes are alive, for a total of 8 nodes alive. The available communication bandwidth is 5Mbps, and the task urgency level is high. The above three sets of data are integrated and output as environmental status parameters to subsequent modules.

[0051] In this embodiment of the invention, by periodically collecting environmental status parameters, the status of network nodes, communication resources, and task urgency can be perceived in real time, providing accurate and real-time data support for the dynamic adaptation of interaction strategies.

[0052] The adaptation interaction strategy generation module 13 is used to input the environmental state parameters into the pre-built interaction strategy adjustment model, calculate and output the adaptation local node interaction strategy and the adaptation agent node interaction strategy in the next feedback adjustment cycle.

[0053] In this embodiment of the invention, the environmental state parameters are input into a pre-constructed interaction strategy adjustment model, and the adaptive local node interaction strategy and the adaptive agent node interaction strategy for the next feedback adjustment cycle are calculated and output. Existing multi-agent distributed situational interaction devices generally use fixed interaction parameter configurations, which cannot simultaneously balance interaction efficiency, global consistency, and environmental adaptability, making it difficult to support reliable collaboration of unmanned swarms in complex scenarios. Therefore, this invention constructs an adaptive strategy adjustment model through deep reinforcement learning, dynamically optimizes interaction parameters based on real-time environmental states, achieves optimal matching between environmental states and interaction strategies, and solves the performance bottleneck of fixed strategies.

[0054] In one embodiment, the adaptation interaction strategy generation module 13 is further configured to: A dynamic policy network model is constructed using deep reinforcement learning algorithms; Construct a state space, which includes the dimensions of the number of surviving nodes, the available communication bandwidth value, and the task urgency level. Construct an action space, which includes the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in the consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach the consensus. Design a reward function that calculates a weighted reward value based on three indicators: interaction completion time, consensus rate, and communication resource consumption. The shorter the interaction completion time, the greater the reward value; the higher the consensus rate, the greater the reward value; and the smaller the communication resource consumption, the greater the reward value. The dynamic policy network model is pre-trained using the state space, action space, and reward function, so that the dynamic policy network model learns the optimal mapping relationship from the environment state to the interaction policy until the weighted reward value of the reward function converges to a preset threshold range. The trained dynamic policy network model is then used as the interaction policy adjustment model. During real-time operation, the collected environmental state parameters are input into the interaction strategy adjustment model, and the corresponding actions are output through the forward calculation process. The interaction timeout value between the local consensus nodes and the minimum number of nodes required to participate in the consensus are used as the local node interaction strategy, and the semantic vector compression ratio between the agent nodes and the minimum number of participating nodes required to reach consensus are used as the agent node interaction strategy.

[0055] First, a dynamic policy network model is constructed using a deep reinforcement learning algorithm. Deep reinforcement learning (DRL) is an algorithm that integrates the perceptual capabilities of deep learning with the decision-making capabilities of reinforcement learning. It can learn the mapping relationship from the environmental state to the optimal action through trial and error between the agent and the environment.

[0056] Secondly, a state space is constructed, comprising the dimensions of node survival count, available communication bandwidth, and task urgency level. The state space refers to a set of features describing the device's operating environment and serves as the input dimension for the dynamic policy network model to perceive the environment. The dimensions of the state space are the node survival count, available communication bandwidth, and task urgency level. Specifically, the node survival count dimension represents the survival status of network nodes, the available communication bandwidth dimension represents the communication resource status between nodes, and the task urgency level dimension represents the urgency of the current task.

[0057] Specifically, the three environmental state parameters—the number of surviving nodes, the available communication bandwidth, and the task urgency level—are used as the three input dimensions of the state space to construct a three-dimensional state space S={S1,S2,S3}, where S1 is the number of surviving nodes, S2 is the available communication bandwidth, and S3 is the task urgency level.

[0058] For example, the state space is constructed in three dimensions: S1 = number of surviving nodes, ranging from 0 to 9, with a standard value of 9, meaning all 3 local consensus nodes and 6 agent nodes are alive; S2 = available communication bandwidth, ranging from 0 to 20 Mbps, with a standard value of 10 Mbps; S3 = task urgency level, ranging from 1 to 3, where 1 is low, 2 is medium, and 3 is high, with a standard value of 2. This forms a complete three-dimensional state space.

[0059] Next, an action space is constructed, which includes the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach consensus. The action space refers to the set of all actions that a reinforcement learning agent can execute, and is the output dimension of the dynamic policy network model.

[0060] The interaction timeout value refers to the message waiting timeout threshold between local consensus nodes for Byzantine fault-tolerant consensus; if the message is not received within the timeout period, the node is considered invalid. The minimum number of nodes required for consensus participation is the minimum number of effective nodes needed for local consensus nodes to reach Byzantine fault-tolerant consensus, ensuring the robustness of the consensus. The semantic vector compression ratio is the ratio of the compressed semantic vector dimension to the original dimension output by the lightweight multimodal large model of the agent nodes. It is used to control the amount of communication data; the smaller the ratio, the higher the compression degree and the smaller the amount of data transmitted. The minimum number of participating nodes required for consensus is the minimum number of participating nodes required for lightweight semantic consensus between agent nodes, ensuring the effectiveness of local consensus.

[0061] Specifically, the four types of interaction parameters are used as the four output dimensions of the action space to construct a four-dimensional action space A={A1,A2,A3,A4}, where A1 is the interaction timeout value of local consensus nodes, A2 is the threshold of the minimum number of nodes participating in the local consensus node consensus, A3 is the semantic vector compression ratio of the agent node, and A4 is the threshold of the minimum number of nodes participating in the consensus of the agent node.

[0062] For example, the action space is constructed as a four-dimensional space: A1 = interaction timeout value, ranging from 100ms to 1000ms, with a baseline value of 500ms; A2 = minimum threshold for the number of nodes participating in consensus, ranging from 1 to 3, with a baseline value of 2; A3 = semantic vector compression ratio, ranging from 0.2 to 0.8, with a baseline value of 0.33, corresponding to compression from the original 768 dimensions to 256 dimensions; A4 = minimum threshold for the number of nodes participating in agent consensus, ranging from 1 to 2, with a baseline value of 2. This forms a complete four-dimensional action space.

[0063] Furthermore, a reward function is designed, which calculates a weighted reward value based on three indicators: interaction completion time, consensus rate, and communication resource consumption. The shorter the interaction completion time, the larger the reward value; the higher the consensus rate, the larger the reward value; and the lower the communication resource consumption, the larger the reward value. The reward function is used to evaluate the quality of the agent's actions, guiding the model to learn the optimal strategy through weighted indicators.

[0064] Among them, the interaction completion time refers to the total time from the initiation of situational interaction to the achievement of global consensus; the shorter the time, the better the real-time performance. The consensus achievement rate refers to the proportion of nodes that achieve consensus out of the total number of valid nodes; the higher the rate, the better the consistency. Communication resource consumption refers to the total amount of data transmitted during situational interaction; the lower the consumption, the lower the bandwidth usage. The weighted reward value is the comprehensive reward obtained by weighting and summing the above three indicators, used to evaluate the merits of the strategy.

[0065] Specifically, design the reward function. ,in: Rewards for interaction completion time: the shorter the interaction completion time, The larger; Consensus rate reward: The higher the consensus rate, The larger; Rewards are given based on communication resource consumption: the lower the communication consumption, The larger; These are weighting coefficients, which can be adjusted as needed to ensure a balance among the three indicators.

[0066] For example, setting weights When the interaction completion time is 300ms, it is far lower than the benchmark of 500ms. =0.9; consensus rate is 100%. =1.0; Communication consumption is 60% of the baseline. =0.8; then the weighted reward value R = 0.3 × 0.9 + 0.4 × 1.0 + 0.3 × 0.8 = 0.91, which is a high reward value, corresponding to the optimal strategy.

[0067] Subsequently, the dynamic policy network model is pre-trained using the state space, action space, and reward function. This allows the dynamic policy network model to learn the optimal mapping relationship from environmental states to interaction policies, until the weighted reward value of the reward function converges to a preset threshold range. The trained dynamic policy network model is then used as the interaction policy adjustment model. Pre-training refers to training the model based on historical environmental data before its deployment, enabling the model to learn the optimal mapping relationship until reward convergence. The optimal mapping relationship refers to the correspondence between different environmental states and the optimal interaction policy learned by the model, maximizing the reward value. Reward convergence means that the fluctuation range of the model's weighted reward value stabilizes within the preset threshold, indicating that the model training is complete.

[0068] Specifically, the dynamic policy network model is pre-trained using the constructed state space, action space, and reward function: the model continuously adjusts the interaction parameters through trial and error, optimizes the mapping from state to action, and continuously improves the weighted reward value; when the reward value converges to a preset threshold range, such as reward fluctuation ≤0.02 for 100 consecutive training rounds, training is stopped, and the trained model is used as the interaction policy adjustment model.

[0069] For example, the model is pre-trained based on historical environmental data. When the weighted reward value of the model in various scenarios is stable above 0.9 and fluctuates ≤0.02, the training is considered complete, and an interactive strategy adjustment model that can be used for real-time adjustment is obtained.

[0070] The process of calculating dynamic configuration parameters by the dynamic policy network model includes: Set the interaction timeout benchmark value, the minimum number of nodes participating in consensus benchmark value, the semantic vector compression ratio benchmark value, and the minimum number of participating nodes benchmark value under the standard environment state. The standard environment state includes the preset standard node survival number, standard available communication bandwidth value, and standard task urgency level. Calculate the difference in the number of surviving nodes between the currently collected number of surviving nodes and the standard number of surviving nodes; calculate the difference in available communication bandwidth between the currently collected available communication bandwidth and the standard available communication bandwidth; calculate the difference in task urgency level between the currently collected task urgency level and the standard task urgency level. Based on the differences in the number of surviving nodes, the differences in available communication bandwidth, and the differences in task urgency levels, corresponding adjustment amounts are calculated and the corresponding baseline values ​​are compensated to obtain the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach consensus between the agent nodes.

[0071] Specifically, the interaction timeout value is negatively correlated with the difference in the number of surviving nodes and the difference in task urgency level, and positively correlated with the difference in available communication bandwidth; the minimum number of nodes required to participate in consensus is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level; the semantic vector compression ratio is negatively correlated with the difference in available communication bandwidth and positively correlated with the difference in task urgency level; and the minimum number of participating nodes threshold is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level.

[0072] First, baseline values ​​are set for the interaction timeout, minimum number of nodes required for consensus, semantic vector compression ratio, and minimum number of participating nodes under standard environment conditions. The standard environment conditions include preset standard node survival counts, standard available communication bandwidth, and standard task urgency levels. The standard environment conditions refer to a preset ideal operating environment, including the standard node survival count, standard available communication bandwidth, and standard task urgency level, used as a benchmark for parameter adjustment. The baseline values ​​refer to the initial values ​​of the interaction parameters under the standard environment, including the interaction timeout baseline value, minimum number of nodes required for consensus, semantic vector compression ratio baseline value, and minimum number of participating nodes baseline value.

[0073] For example, the standard environment is set as follows: standard node survival count: 3 local consensus nodes + 6 agent nodes, totaling 9; standard available communication bandwidth: 10Mbps; standard task urgency level: 2, corresponding to medium. Four benchmark values ​​are set accordingly: interaction timeout benchmark T0 = 500ms; minimum number of nodes participating in consensus benchmark N0 = 2; semantic vector compression ratio benchmark C0 = 0.33; minimum number of participating nodes benchmark M0 = 2. These serve as the initial benchmarks for parameter adjustments.

[0074] Secondly, calculate the difference in node survival count between the currently collected number of surviving nodes and the standard number of surviving nodes; calculate the difference in available communication bandwidth between the currently collected available communication bandwidth and the standard available communication bandwidth; and calculate the difference in task urgency level between the currently collected task urgency level and the standard task urgency level. The node survival count difference ΔS1 is the difference between the current number of surviving nodes S1 and the standard number of surviving nodes S. 10 The difference, ΔS1=S1-S 10The available communication bandwidth difference ΔS2 is the difference between the current available communication bandwidth value S2 and the standard available communication bandwidth value S. 20 The difference, ΔS2=S2-S 20 The task urgency level difference ΔS3 is the difference between the current task urgency level S3 and the standard task urgency level S. 30 The difference, ΔS3=S3-S 30 Based on the real-time collected environmental state parameters, three types of differences, ΔS1, ΔS2, and ΔS3, are calculated for subsequent adjustment calculations.

[0075] For example, the current collected environmental status is as follows: number of surviving nodes S1=7, available bandwidth S2=3Mbps, task level S3=3; calculate the difference: ΔS1=7-9=-2, a negative difference, indicating that the number of surviving nodes is lower than the standard; ΔS2=3-10=-7, a negative difference, indicating that the bandwidth is lower than the standard; ΔS3=3-2=+1, a positive difference, indicating that the task urgency is higher than the standard.

[0076] Next, based on the difference in the number of surviving nodes, the difference in available communication bandwidth, and the difference in task urgency level, the corresponding adjustment amounts are calculated and the corresponding baseline values ​​are compensated to obtain the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in the consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach consensus between the agent nodes.

[0077] Specifically, the interaction timeout value is negatively correlated with the difference in the number of surviving nodes and the difference in task urgency level, and positively correlated with the difference in available communication bandwidth; the minimum number of nodes required to participate in consensus is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level; the semantic vector compression ratio is negatively correlated with the difference in available communication bandwidth and positively correlated with the difference in task urgency level; and the minimum number of participating nodes threshold is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level.

[0078] Specifically, a preset fixed linear compensation correlation rule is used to achieve a unique mapping from environmental state differences to interaction parameters: for local consensus node interaction parameters, the interaction timeout value is calculated using a linear compensation formula. This parameter is related to , Negative correlation, with Positive correlation; the minimum number of nodes required to participate in consensus is calculated using the following formula: This parameter is related to Positive correlation with Negative correlation results are rounded up. For agent node interaction parameters, the semantic vector compression ratio is calculated using a linear compensation formula: This parameter is related to Negative correlation, with Positive correlation; the minimum number of participating nodes required to reach consensus is calculated using the following formula: This parameter is related to Positive correlation with Negative correlation, the calculation result is rounded up.

[0079] Among them, the interaction timeout baseline value This refers to the default maximum waiting time for local consensus nodes to complete their interaction under standard conditions; and the minimum number of nodes required to participate in consensus. This refers to the minimum number of nodes required for local consensus nodes to reach a consensus under standard conditions. (Semantic vector compression ratio benchmark value) This refers to the default compression ratio of the semantic vectors of agent nodes under standard conditions. Minimum number of participating nodes as a baseline. It refers to the minimum number of nodes required for intelligent agent nodes to reach consensus under standard conditions.

[0080] For example, the interaction timeout baseline value =10s, the minimum number of nodes required to participate in consensus. =2, semantic vector compression ratio benchmark value =10:1, minimum number of participating nodes baseline value =3; Current environment state differences: ΔS1=-2, ΔS2=-7, ΔS3=+1. Substitute the above differences into the corresponding formulas for calculation: Local consensus node interaction timeout value: T=10-(-2)+(-7)-1=4s; Minimum number of nodes required for local consensus node participation: K=2+(-2)-1=-1, rounded up to 1; Agent node semantic vector compression ratio: R=10-(-7)+1=18:1; Minimum number of participating nodes required for agent node consensus: L=3+(-2)-1=0, rounded up to 1.

[0081] Finally, during real-time operation, the collected environmental state parameters are input into the interaction strategy adjustment model. Through a forward computation process, corresponding actions are output. The interaction timeout value between local consensus nodes and the minimum number of nodes required for consensus are used as the adapted local node interaction strategy. Similarly, the semantic vector compression ratio between agent nodes and the minimum number of participating nodes required for consensus are used as the adapted agent node interaction strategy. The interaction strategy adjustment model is a pre-trained deep learning model used to map environmental state parameters to optimal interaction parameters, enabling rapid output of environmental adaptive strategies.

[0082] The forward computation process refers to the calculation flow where the model receives input environmental state parameters, performs inference calculations through its internal neural network, and outputs corresponding interaction parameters. The local node interaction strategy refers to a dynamic set of interaction parameters for local consensus nodes, including interaction timeout values ​​and the minimum threshold number of nodes required for consensus. The agent node interaction strategy refers to a dynamic set of interaction parameters for agent nodes, including semantic vector compression ratio and the minimum threshold number of participating nodes required to reach consensus.

[0083] Specifically, during real-time operation, the number of surviving nodes, available communication bandwidth, and task urgency level obtained by the environmental state parameter acquisition module are input into the trained interaction strategy adjustment model. The model completes inference through forward computation and outputs the corresponding interaction parameter actions. The calculated local consensus node interaction timeout value and the minimum number of nodes required to participate in consensus are integrated into an adaptive local node interaction strategy. The agent node semantic vector compression ratio and the minimum number of participating nodes required to reach consensus are integrated into an adaptive agent node interaction strategy.

[0084] For example, the real-time collected environmental state parameters are input into the interaction strategy adjustment model: 7 surviving nodes, 3Mbps available communication bandwidth, and 3 task urgency levels. The model outputs corresponding interaction parameters after forward computation: 4s interaction timeout, 1 minimum number of nodes required for consensus, 18:1 semantic vector compression ratio, and 1 minimum number of participating nodes required for consensus. The 4s interaction timeout and 1 minimum number of participating nodes are used as the adaptation strategy for local node interaction, and the 18:1 semantic vector compression ratio and 1 minimum number of participating nodes required for consensus are used as the adaptation strategy for agent node interaction, thus completing the generation of the adaptation interaction strategy.

[0085] In this embodiment of the invention, by combining fixed linear compensation rules with an interaction strategy adjustment model, real-time dynamic adaptive adjustment of interaction parameters according to environmental conditions is achieved. This allows for automatic adaptation of interaction parameters between local consensus nodes and agent nodes based on node survival status, communication bandwidth conditions, and task urgency. It can reduce the consensus node threshold and ensure interaction continuity and timeliness when nodes are damaged or tasks are urgent, while also improving semantic vector compression ratio and ensuring smooth communication when bandwidth is limited. Simultaneously, it maintains the robustness of global consensus and the efficiency of local interaction. This effectively solves the technical problem that fixed interaction parameters cannot adapt to complex and dynamic battlefield environments, improving the environmental adaptability, operational stability, and real-time response capability of the entire unmanned swarm situational interaction device.

[0086] The distributed situation interaction module 14 is used to input the adapted local node interaction strategy and the adapted agent node interaction strategy into the two-layer distributed interaction network in the next feedback adjustment cycle, control the N local consensus nodes and M agent nodes to perform distributed situation interaction respectively, and output a unified task situation cognition benchmark.

[0087] In this embodiment of the invention, during the next feedback adjustment cycle, the adapted local node interaction strategy and the adapted agent node interaction strategy are input into the two-layer distributed interaction network. This controls the N local consensus nodes and M agent nodes to perform distributed situational awareness interactions, outputting a unified task situational awareness benchmark. Complex battlefield environments present problems such as communication interference, single-node perception anomalies, and sudden noise interference, which can easily lead to sudden abnormal jumps in single-cycle situational awareness results. Relying solely on the current consensus output will result in discontinuous and unreliable situational awareness. Simultaneously, the adaptively generated interaction strategy needs to be practically applied to the two-layer network to ensure that distributed interaction is executed with optimal parameters. Therefore, based on the completion of distributed situational awareness interaction, a sliding window is used to perform awareness consistency detection and correct abnormal results, ensuring that the output task situational awareness benchmark is continuous, stable, and reliable.

[0088] In one embodiment, the distributed situational interaction module 14 is further configured to: After outputting a unified task situation awareness benchmark, the process also includes a sliding window-based detection and correction step for cognitive consistency anomalies. Construct a sliding window of length L, and store the unified task situation awareness benchmarks output by the most recent L feedback adjustment cycles into the sliding window in sequence, where L is an integer greater than or equal to 3; Calculate the semantic vector cosine similarity between the unified task situation cognition benchmarks output in every two adjacent cycles within the sliding window to obtain the similarity value for L minus 1 cycles. Calculate the mean and standard deviation of the similarity values ​​for L minus 1 period, and set a dynamic anomaly threshold based on the mean and standard deviation, wherein the dynamic anomaly threshold is equal to the product of the mean minus a preset first multiple and the standard deviation; If the semantic vector cosine similarity between the current period and the previous period is lower than the dynamic anomaly threshold, the situational awareness result of the current period is determined to be an abnormal jump, and the cross-period awareness correction process is triggered.

[0089] The cross-cycle cognitive correction process includes: selecting the unified task situation cognitive benchmarks corresponding to the two consecutive cycles with the highest similarity values ​​from the sliding window, calculating the mean as a reference benchmark, weighting and fusing the reference benchmark with the cognitive benchmark of the current cycle, wherein the weight coefficient of the cognitive benchmark of the current cycle is positively correlated with the average confidence of the participating nodes when consensus is reached in the current cycle, and outputting the result of the weighted fusion as the corrected unified task situation cognitive benchmark of the current cycle.

[0090] First, within the next feedback adjustment cycle, the adapted local node interaction strategy and the adapted agent node interaction strategy are input into the two-layer distributed interaction network. This controls the N local consensus nodes and M agent nodes to perform distributed situational interaction, outputting a unified task situational awareness benchmark. The adapted local node interaction strategy refers to the operating parameters of the local consensus nodes, consisting of the interaction timeout value and the minimum threshold for the number of nodes participating in consensus. The adapted agent node interaction strategy refers to the operating parameters of the agent nodes, consisting of the semantic vector compression ratio and the minimum threshold for the number of nodes participating in consensus. The unified task situational awareness benchmark refers to the globally consistent and standardized multimodal semantic situational result after the two-layer distributed interaction.

[0091] Specifically, upon entering the next feedback adjustment cycle, the local node interaction strategy and the intelligent agent node interaction strategy will be loaded into the two-layer distributed interaction network; N local consensus nodes will perform Byzantine fault-tolerant interaction according to the set interaction timeout and minimum number of consensus nodes, and M intelligent agent nodes will perform lightweight semantic interaction according to the set compression ratio and minimum number of participating nodes; the entire network will collaboratively output a unified task situation awareness benchmark.

[0092] For example, the loading strategy for the next feedback cycle is as follows: local consensus node interaction timeout is 4 seconds, minimum number of participating nodes is 1; agent node semantic vector compression ratio is 18:1, minimum number of participating nodes is 1; 3 local consensus nodes and 7 agent nodes complete distributed interaction according to these parameters, outputting a 768-dimensional unified task situation awareness baseline vector V. t .

[0093] Secondly, a sliding window of length L is constructed. The unified task situation awareness benchmarks output from the most recent L feedback adjustment cycles are sequentially stored in the sliding window, where L is an integer greater than or equal to 3. The sliding window is a time-series data storage structure of fixed length L, retaining only the data from the most recent L cycles; as new data enters, older data is removed. The window length L is a preset positive integer not less than 3 to ensure the reliability of the time-series statistics. By setting a fixed window length L, the unified task situation awareness benchmarks output from the most recent L feedback adjustment cycles are sequentially stored in the sliding window in chronological order, maintaining the temporal arrangement.

[0094] Exemplarily, set L = 3, and the sliding window stores the most recent three - period cognitive benchmarks in sequence: V t-2 , V t-1 , V t , which are the unified situation awareness benchmarks for the pre - previous period, the previous period, and the current period respectively.

[0095] Next, calculate the cosine similarity of semantic vectors between the unified task situation awareness benchmarks output in every two adjacent periods within the sliding window, and obtain L - 1 period similarity values. The cosine similarity of semantic vectors is used to measure the cognitive consistency of two situation semantic vectors, with a value range of [0, 1]. The higher the value, the more continuous and consistent the cognition. Calculate the cosine similarity for the unified task situation awareness benchmarks of every two adjacent periods within the sliding window; a total of L - 1 period similarity values are obtained.

[0096] Exemplarily, for window L = 3, two similarity values are obtained: sim1 = cos(V t-2 , V t-1 ) = 0.94, sim2 = cos(V t-1 , V t ) = 0.78.

[0097] Furthermore, calculate the mean and standard deviation of the L - 1 period similarity values, and set a dynamic anomaly threshold according to the mean and standard deviation. Among them, the dynamic anomaly threshold is equal to the mean minus the product of a preset first multiple and the standard deviation. The mean μ is the average of L - 1 similarities, representing the historical cognitive stability. The standard deviation σ represents the similarity fluctuation degree. The dynamic anomaly threshold Th is an anomaly determination boundary adaptively generated based on historical data. The first multiple k1 is a preset fixed coefficient used to adjust the threshold sensitivity. Calculate the mean μ and standard deviation σ of L - 1 similarities; calculate the dynamic anomaly threshold according to the formula [[ID=2Z]]

[0098] Exemplarily, for the similarity sequence: [0.94, 0.78], the mean μ = 0.86, the standard deviation σ≈0.11; take the first multiple k1 = 2; the dynamic anomaly threshold: Th = 0.86 - 2×0.11 = 0.64.

[0099] Further, if the cosine similarity of semantic vectors between the current period and the previous period is lower than the dynamic anomaly threshold, determine that the situation awareness result of the current period is an abnormal jump, and trigger a cross - period cognitive correction process. An abnormal jump means that the situation awareness in the current period is significantly deviated from the historical trend, manifested as a sudden drop in the adjacent similarity below the anomaly threshold. Compare the similarity sim2 between the current period and the previous period with the dynamic anomaly threshold Th; if sim2 < Th, then determine that the situation awareness result of the current period is an abnormal jump, and start the cross - period cognitive correction process.

[0100] For example, if the current similarity sim2=0.78 and Th=0.64, and 0.78>0.64, it is considered normal for this period. If sim2=0.58<0.64, it is considered an abnormal jump, and the cross-period cognitive correction process is initiated.

[0101] The cross-cycle cognitive correction process includes: selecting the unified task situation cognitive benchmarks corresponding to the two consecutive cycles with the highest similarity values ​​from the sliding window, calculating the mean as a reference benchmark, weighting and fusing the reference benchmark with the cognitive benchmark of the current cycle, wherein the weight coefficient of the cognitive benchmark of the current cycle is positively correlated with the average confidence of the participating nodes when consensus is reached in the current cycle, and outputting the result of the weighted fusion as the corrected unified task situation cognitive benchmark of the current cycle.

[0102] Specifically, the group of consecutive periodic cognitive benchmarks with the highest similarity is selected from the sliding window, and their arithmetic mean is calculated as the reference benchmark V_ref; the average confidence score conf of the consensus nodes in the current period is obtained, and this is used as the weight of the current benchmark. The reference benchmark weight is Weighted fusion: V_corrected will be output as the corrected unified mission situation awareness benchmark.

[0103] For example, the highest similarity within the window is sim1=0.94; reference benchmark: V_ref=(V t-2 +V t-1 ) / 2; Current period average confidence level conf=0.85, weight , Corrected baseline: V_corrected = 0.85 × V t +0.15×V_ref is used as the final unified situational awareness benchmark output.

[0104] In this embodiment of the invention, an adaptive interaction strategy is loaded to drive a two-layer network to complete distributed situational interaction, output a unified situational cognition benchmark, and realize the detection of cognitive consistency anomalies based on sliding window, semantic cosine similarity and dynamic threshold. Anomaly correction is completed by weighted fusion of high-confidence historical benchmark and current cognitive results, which effectively avoids sudden changes in situational cognition caused by battlefield interference, ensures the continuity, stability and reliability of global mission situational cognition, and improves the robustness and cognitive accuracy of the distributed situational interaction device.

[0105] Through the specific implementation methods described above, the embodiments of the present invention achieve the following technical effects: This invention provides a multi-agent distributed situational awareness interaction device and method based on a multimodal large model. By constructing a two-layer distributed interaction network with upper-layer fault tolerance and lower-layer efficiency, it separates global consensus and local interaction functions. A multimodal large model is used to achieve unified semantic representation of multi-source information. This approach relies on Byzantine fault tolerance protocols to ensure the robustness and consistency of global situational awareness, while lightweight semantic consensus reduces the computational power and bandwidth consumption at the agent level. This addresses the contradiction between fault tolerance and real-time performance that is difficult to achieve in a single-layer network at the architectural level. By periodically collecting data on the number of surviving nodes, available communication bandwidth, and task urgency level, the device's operating status and changes in the battlefield environment are perceived in real time, providing data for interaction strategies. The dynamic adjustment provides objective and accurate data support, ensuring that subsequent parameter adaptation can fit the actual operating state; based on the environmental state difference and fixed linear compensation rules, it adaptively adjusts key parameters such as interaction timeout, consensus node threshold, and semantic compression ratio, so that the interaction strategy is dynamically matched in real time with node damage, bandwidth fluctuations, and task urgency, improving transmission efficiency and response speed while ensuring the effectiveness of consensus; based on the adaptive strategy, it executes distributed situational interaction, and completes cognitive consistency detection and anomaly weighting correction through a sliding window, effectively suppressing situational perception jumps and distortions caused by battlefield interference, ensuring that the output unified task situational perception benchmark is continuous, stable, and reliable.

[0106] Example 2, as Figure 2 As shown, this invention provides a multi-agent distributed situational interaction method based on a multimodal large model, the method comprising: S100: Construct a two-layer distributed interaction network, wherein the two-layer distributed interaction network includes N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. Furthermore, the local consensus nodes and agent nodes are each deployed with a multimodal large model.

[0107] Step S100 in the method provided in this embodiment of the invention includes: The multimodal large model deployed by the local consensus nodes and agent nodes includes a visual encoder, a text encoder, a signal encoder, and a geographic information encoder. The visual encoder is used to convert optical images or synthetic aperture radar images into visual semantic feature vectors. The text encoder is used to convert intelligence text or instruction text into text semantic feature vectors. The signal encoder is used to convert radar signals or communication signals into signal semantic feature vectors. The geographic information encoder is used to convert coordinate data or terrain data into geographic semantic feature vectors. The multimodal large model aligns the above-mentioned multiple semantic feature vectors to the same semantic space through a cross-modal attention mechanism, and outputs a semantic vector of a unified dimension.

[0108] The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, including: Each of the N local consensus nodes broadcasts the locally generated structured semantic situation information to the other N minus 1 local consensus nodes. The structured semantic situation information includes a semantic vector generated by a multimodal large model and the corresponding confidence value. After receiving structured semantic situation information from other local consensus nodes, each local consensus node needs to verify the correctness of the information format, the freshness of the timestamp, and the validity of the source node's identity signature, and retain the verified structured semantic situation information as valid information. Each local consensus node executes a three-stage interaction process in sequence, consisting of a pre-preparation stage, a preparation stage, and a submission stage, based on all the valid information collected. In the pre-preparation stage, the status proposal to be reached is determined; in the preparation stage, preparation confirmation messages from other nodes are collected; and in the submission stage, submission confirmation messages from other nodes are collected. After the three-stage interaction process is completed, when at least two-thirds of the N local consensus nodes send a submission confirmation message for the same situational awareness result, it is determined that a consensus has been reached on the situational awareness result, and the situational awareness result is output as a globally consistent situational benchmark.

[0109] The intelligent agent nodes use a lightweight semantic consensus protocol for situational interaction, including: Each agent node inputs its local observation data into a lightweight multimodal large model, which outputs a compressed semantic vector and broadcasts the semantic vector as a cognitive summary to other agent nodes within the coverage area of ​​the same local consensus node. The lightweight multimodal large model is obtained by performing at least one compression operation on the multimodal large model deployed on the local consensus node, including model pruning, weight quantization, or knowledge distillation. After receiving cognitive summaries from other intelligent agent nodes, the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node is calculated, wherein the semantic similarity is measured using cosine similarity. Based on the calculated semantic similarity, the received multiple cognitive summaries are weighted and aggregated. The aggregated semantic vector obtained after weighted aggregation is used as a snapshot of the local situation consensus and reported to the local consensus node. The weight coefficient of the weighted aggregation is positively correlated with the confidence value of each cognitive summary and the semantic similarity.

[0110] S200: Set the environmental monitoring time window according to the preset feedback adjustment cycle, and collect the number of surviving nodes, available communication bandwidth value and task urgency level as environmental status parameters.

[0111] S300: Input the environmental state parameters into the pre-built interaction strategy adjustment model, calculate and output the adaptive local node interaction strategy and the adaptive agent node interaction strategy in the next feedback adjustment cycle.

[0112] Step S300 in the method provided in this embodiment of the invention includes: A dynamic policy network model is constructed using deep reinforcement learning algorithms; Construct a state space, which includes the dimensions of the number of surviving nodes, the available communication bandwidth value, and the task urgency level. Construct an action space, which includes the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in the consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach the consensus. Design a reward function that calculates a weighted reward value based on three indicators: interaction completion time, consensus rate, and communication resource consumption. The shorter the interaction completion time, the greater the reward value; the higher the consensus rate, the greater the reward value; and the smaller the communication resource consumption, the greater the reward value. The dynamic policy network model is pre-trained using the state space, action space, and reward function, so that the dynamic policy network model learns the optimal mapping relationship from the environment state to the interaction policy until the weighted reward value of the reward function converges to a preset threshold range. The trained dynamic policy network model is then used as the interaction policy adjustment model. During real-time operation, the collected environmental state parameters are input into the interaction strategy adjustment model, and the corresponding actions are output through the forward calculation process. The interaction timeout value between the local consensus nodes and the minimum number of nodes required to participate in the consensus are used as the local node interaction strategy, and the semantic vector compression ratio between the agent nodes and the minimum number of participating nodes required to reach consensus are used as the agent node interaction strategy.

[0113] The process of calculating dynamic configuration parameters by the dynamic policy network model includes: Set the interaction timeout benchmark value, the minimum number of nodes participating in consensus benchmark value, the semantic vector compression ratio benchmark value, and the minimum number of participating nodes benchmark value under the standard environment state. The standard environment state includes the preset standard node survival number, standard available communication bandwidth value, and standard task urgency level. Calculate the difference in the number of surviving nodes between the currently collected number of surviving nodes and the standard number of surviving nodes; calculate the difference in available communication bandwidth between the currently collected available communication bandwidth and the standard available communication bandwidth; calculate the difference in task urgency level between the currently collected task urgency level and the standard task urgency level. Based on the differences in the number of surviving nodes, the differences in available communication bandwidth, and the differences in task urgency levels, corresponding adjustment amounts are calculated and the corresponding baseline values ​​are compensated to obtain the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach consensus between the agent nodes.

[0114] Specifically, the interaction timeout value is negatively correlated with the difference in the number of surviving nodes and the difference in task urgency level, and positively correlated with the difference in available communication bandwidth; the minimum number of nodes required to participate in consensus is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level; the semantic vector compression ratio is negatively correlated with the difference in available communication bandwidth and positively correlated with the difference in task urgency level; and the minimum number of participating nodes threshold is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level.

[0115] S400: In the next feedback adjustment cycle, the adapted local node interaction strategy and the adapted agent node interaction strategy are input into the two-layer distributed interaction network, and the N local consensus nodes and M agent nodes are controlled to perform distributed situational interaction respectively, and a unified task situational awareness benchmark is output.

[0116] Step S400 in the method provided in this embodiment of the invention includes: After outputting a unified task situation awareness benchmark, the process also includes a sliding window-based detection and correction step for cognitive consistency anomalies. Construct a sliding window of length L, and store the unified task situation awareness benchmarks output by the most recent L feedback adjustment cycles into the sliding window in sequence, where L is an integer greater than or equal to 3; Calculate the semantic vector cosine similarity between the unified task situation cognition benchmarks output in every two adjacent cycles within the sliding window to obtain the similarity value for L minus 1 cycles. Calculate the mean and standard deviation of the similarity values ​​for L minus 1 period, and set a dynamic anomaly threshold based on the mean and standard deviation, wherein the dynamic anomaly threshold is equal to the product of the mean minus a preset first multiple and the standard deviation; If the semantic vector cosine similarity between the current period and the previous period is lower than the dynamic anomaly threshold, the situational awareness result of the current period is determined to be an abnormal jump, and the cross-period awareness correction process is triggered.

[0117] The cross-cycle cognitive correction process includes: selecting the unified task situation cognitive benchmarks corresponding to the two consecutive cycles with the highest similarity values ​​from the sliding window, calculating the mean as a reference benchmark, weighting and fusing the reference benchmark with the cognitive benchmark of the current cycle, wherein the weight coefficient of the cognitive benchmark of the current cycle is positively correlated with the average confidence of the participating nodes when consensus is reached in the current cycle, and outputting the result of the weighted fusion as the corrected unified task situation cognitive benchmark of the current cycle.

[0118] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-agent distributed situational interaction device based on a multimodal large model, characterized in that, include: A two-layer distributed interactive network construction module is used to construct a two-layer distributed interactive network, wherein the two-layer distributed interactive network includes N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. The local consensus nodes and agent nodes are respectively deployed with multimodal large models. The agent nodes are terminal perception nodes in the lower layer of the network. The terminal perception nodes are unmanned equipment, including at least one of UAVs, unmanned vehicles, and unmanned boats. The perception data types of the agent nodes include optical images, synthetic aperture radar images, intelligence text, command text, radar signals, communication signals, coordinate data, and terrain data. The environmental status parameter acquisition module is used to set the environmental monitoring time window according to the preset feedback adjustment cycle, and to collect the number of surviving nodes, the available communication bandwidth value, and the task urgency level as environmental status parameters. An adaptive interaction strategy generation module is used to input the environmental state parameters into a pre-built interaction strategy adjustment model, calculate and output the adaptive local node interaction strategy and the adaptive agent node interaction strategy for the next feedback adjustment cycle, including: A dynamic policy network model is constructed using deep reinforcement learning algorithms; Construct a state space, which includes the dimensions of the number of surviving nodes, the available communication bandwidth value, and the task urgency level. Construct an action space, which includes the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in the consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach the consensus. Design a reward function that calculates a weighted reward value based on three indicators: interaction completion time, consensus rate, and communication resource consumption. The shorter the interaction completion time, the greater the reward value; the higher the consensus rate, the greater the reward value; and the smaller the communication resource consumption, the greater the reward value. The dynamic policy network model is pre-trained using the state space, action space, and reward function, so that the dynamic policy network model learns the optimal mapping relationship from the environment state to the interaction policy until the weighted reward value of the reward function converges to a preset threshold range. The trained dynamic policy network model is then used as the interaction policy adjustment model. During real-time operation, the collected environmental state parameters are input into the interaction strategy adjustment model, and the corresponding actions are output through the forward calculation process. The interaction timeout value between the local consensus nodes and the minimum number of nodes required to participate in the consensus are used as the local node interaction strategy, and the semantic vector compression ratio between the agent nodes and the minimum number of participating nodes required to reach the consensus are used as the agent node interaction strategy. The distributed situation interaction module is used to input the adaptive local node interaction strategy and the adaptive agent node interaction strategy into the two-layer distributed interaction network in the next feedback adjustment cycle, control the N local consensus nodes and M agent nodes to perform distributed situation interaction respectively, and output a unified task situation cognition benchmark. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, including: Each of the N local consensus nodes broadcasts the locally generated structured semantic situation information to the other N minus 1 local consensus nodes. The structured semantic situation information includes a semantic vector generated by a multimodal large model and the corresponding confidence value. After receiving structured semantic situation information from other local consensus nodes, each local consensus node needs to verify the correctness of the information format, the freshness of the timestamp, and the validity of the source node's identity signature, and retain the verified structured semantic situation information as valid information. Each local consensus node executes a three-stage interaction process in sequence, consisting of a pre-preparation stage, a preparation stage, and a submission stage, based on all the valid information collected. In the pre-preparation stage, the status proposal to be reached is determined; in the preparation stage, preparation confirmation messages from other nodes are collected; and in the submission stage, submission confirmation messages from other nodes are collected. After the three-stage interaction process is completed, when at least two-thirds of the N local consensus nodes send a submission confirmation message for the same situational awareness result, it is determined that the situational awareness result has reached a consensus, and the situational awareness result is output as a globally consistent situational benchmark. The intelligent agent nodes use a lightweight semantic consensus protocol for situational interaction, including: Each agent node inputs its local observation data into a lightweight multimodal large model, which outputs a compressed semantic vector and broadcasts the semantic vector as a cognitive summary to other agent nodes within the coverage area of ​​the same local consensus node. The lightweight multimodal large model is obtained by performing at least one compression operation on the multimodal large model deployed on the local consensus node, namely model pruning, weight quantization, or knowledge distillation. The local observation data refers to single-source or multi-source situational data collected by the agent node through its own sensors. After receiving cognitive summaries from other intelligent agent nodes, the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node is calculated, wherein the semantic similarity is measured using cosine similarity. Based on the calculated semantic similarity, the received multiple cognitive summaries are weighted and aggregated. The aggregated semantic vector obtained after weighted aggregation is used as a local situation consensus snapshot and reported to the local consensus node. The weight coefficient of the weighted aggregation is positively correlated with the confidence value of each cognitive summary and the semantic similarity. The local situation consensus snapshot refers to the local unified situation cognition result obtained by the agent node through weighted aggregation.

2. The multi-agent distributed situational interaction device based on a multimodal large model according to claim 1, characterized in that, The multimodal large model deployed by the local consensus nodes and agent nodes includes a visual encoder, a text encoder, a signal encoder, and a geographic information encoder. The visual encoder is used to convert optical images or synthetic aperture radar images into visual semantic feature vectors. The text encoder is used to convert intelligence text or command text into text semantic feature vectors. The signal encoder is used to convert radar signals or communication signals into signal semantic feature vectors. The geographic information encoder is used to convert coordinate data or terrain data into geographic semantic feature vectors. The multimodal large model aligns the above-mentioned multiple semantic feature vectors to the same semantic space through a cross-modal attention mechanism, and outputs a semantic vector of a unified dimension.

3. The multi-agent distributed situational interaction device based on a multimodal large model according to claim 1, characterized in that, The process of calculating dynamic configuration parameters by the dynamic policy network model includes: Set the interaction timeout benchmark value, the minimum number of nodes participating in consensus benchmark value, the semantic vector compression ratio benchmark value, and the minimum number of participating nodes benchmark value under the standard environment state. The standard environment state includes the preset standard node survival number, standard available communication bandwidth value, and standard task urgency level. Calculate the difference in the number of surviving nodes between the currently collected number of surviving nodes and the standard number of surviving nodes; calculate the difference in available communication bandwidth between the currently collected available communication bandwidth and the standard available communication bandwidth; calculate the difference in task urgency level between the currently collected task urgency level and the standard task urgency level. Based on the differences in the number of surviving nodes, the differences in available communication bandwidth, and the differences in task urgency levels, corresponding adjustment amounts are calculated and the corresponding baseline values ​​are compensated to obtain the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach consensus between the agent nodes.

4. The multi-agent distributed situational interaction device based on a multimodal large model according to claim 3, characterized in that, The interaction timeout value is negatively correlated with the difference in the number of surviving nodes and the difference in task urgency level, and positively correlated with the difference in available communication bandwidth; the minimum number of nodes required to participate in consensus is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level. The semantic vector compression ratio is negatively correlated with the difference in available communication bandwidth and positively correlated with the difference in task urgency level; the minimum number of participating nodes threshold is positively correlated with the difference in the number of surviving nodes and negatively correlated with the difference in task urgency level.

5. The multi-agent distributed situational interaction device based on a multimodal large model according to claim 1, characterized in that, After outputting a unified task situation awareness benchmark, the process also includes a sliding window-based detection and correction step for cognitive consistency anomalies. Construct a sliding window of length L, and store the unified task situation awareness benchmarks output by the most recent L feedback adjustment cycles into the sliding window in sequence, where L is an integer greater than or equal to 3; Calculate the semantic vector cosine similarity between the unified task situation cognition benchmarks output in every two adjacent cycles within the sliding window to obtain the similarity value for L minus 1 cycles. Calculate the mean and standard deviation of the similarity values ​​for L minus 1 period, and set a dynamic anomaly threshold based on the mean and standard deviation, wherein the dynamic anomaly threshold is equal to the product of the mean minus a preset first multiple and the standard deviation; If the semantic vector cosine similarity between the current period and the previous period is lower than the dynamic anomaly threshold, the situational awareness result of the current period is determined to be an abnormal jump, and the cross-period awareness correction process is triggered.

6. The multi-agent distributed situational interaction device based on a multimodal large model according to claim 5, characterized in that, The cross-cycle cognitive correction process includes: selecting the unified task situation cognitive benchmarks corresponding to the two consecutive cycles with the highest similarity values ​​from the sliding window, calculating the mean as a reference benchmark, weighting and fusing the reference benchmark with the cognitive benchmark of the current cycle, wherein the weight coefficient of the cognitive benchmark of the current cycle is positively correlated with the average confidence of the participating nodes when consensus is reached in the current cycle, and outputting the result of the weighted fusion as the corrected unified task situation cognitive benchmark of the current cycle.

7. A multi-agent distributed situational interaction method based on a multimodal large model, characterized in that, The multi-agent distributed situational interaction device based on a multimodal large model, as described in any one of claims 1 to 6, comprises: A two-layer distributed interactive network is constructed, comprising N local consensus nodes and M agent nodes. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, and the agent nodes use a lightweight semantic consensus protocol for situational interaction. The local consensus nodes and agent nodes are each deployed with a multimodal large model. The agent nodes are terminal sensing nodes at the lower layer of the network, and these terminal sensing nodes are unmanned equipment, including at least one of unmanned aerial vehicles (UAVs), unmanned vehicles, and unmanned surface vessels (USVs). The sensing data types of the agent nodes include optical images, synthetic aperture radar images, intelligence text, command text, radar signals, communication signals, coordinate data, and terrain data. The environmental monitoring time window is set according to the preset feedback adjustment cycle, and the number of surviving nodes, available communication bandwidth value and task urgency level are collected as environmental status parameters. The environmental state parameters are input into a pre-built interaction strategy adjustment model, and the adaptive local node interaction strategy and adaptive agent node interaction strategy for the next feedback adjustment cycle are calculated and output, including: A dynamic policy network model is constructed using deep reinforcement learning algorithms; Construct a state space, which includes the dimensions of the number of surviving nodes, the available communication bandwidth value, and the task urgency level. Construct an action space, which includes the interaction timeout value between the local consensus nodes, the minimum number of nodes required to participate in the consensus, the semantic vector compression ratio between the agent nodes, and the minimum number of participating nodes required to reach the consensus. Design a reward function that calculates a weighted reward value based on three indicators: interaction completion time, consensus rate, and communication resource consumption. The shorter the interaction completion time, the greater the reward value; the higher the consensus rate, the greater the reward value; and the smaller the communication resource consumption, the greater the reward value. The dynamic policy network model is pre-trained using the state space, action space, and reward function, so that the dynamic policy network model learns the optimal mapping relationship from the environment state to the interaction policy until the weighted reward value of the reward function converges to a preset threshold range. The trained dynamic policy network model is then used as the interaction policy adjustment model. During real-time operation, the collected environmental state parameters are input into the interaction strategy adjustment model, and the corresponding actions are output through the forward calculation process. The interaction timeout value between the local consensus nodes and the minimum number of nodes required to participate in the consensus are used as the local node interaction strategy, and the semantic vector compression ratio between the agent nodes and the minimum number of participating nodes required to reach the consensus are used as the agent node interaction strategy. In the next feedback adjustment cycle, the adapted local node interaction strategy and the adapted agent node interaction strategy are input into the two-layer distributed interaction network to control the N local consensus nodes and M agent nodes to perform distributed situational interaction and output a unified task situational awareness benchmark. The local consensus nodes use a Byzantine fault-tolerant consensus protocol for situational interaction, including: Each of the N local consensus nodes broadcasts the locally generated structured semantic situation information to the other N minus 1 local consensus nodes. The structured semantic situation information includes a semantic vector generated by a multimodal large model and the corresponding confidence value. After receiving structured semantic situation information from other local consensus nodes, each local consensus node needs to verify the correctness of the information format, the freshness of the timestamp, and the validity of the source node's identity signature, and retain the verified structured semantic situation information as valid information. Each local consensus node executes a three-stage interaction process in sequence, consisting of a pre-preparation stage, a preparation stage, and a submission stage, based on all the valid information collected. In the pre-preparation stage, the status proposal to be reached is determined; in the preparation stage, preparation confirmation messages from other nodes are collected; and in the submission stage, submission confirmation messages from other nodes are collected. After the three-stage interaction process is completed, when at least two-thirds of the N local consensus nodes send a submission confirmation message for the same situational awareness result, it is determined that the situational awareness result has reached a consensus, and the situational awareness result is output as a globally consistent situational benchmark. The intelligent agent nodes use a lightweight semantic consensus protocol for situational interaction, including: Each agent node inputs its local observation data into a lightweight multimodal large model, which outputs a compressed semantic vector and broadcasts the semantic vector as a cognitive summary to other agent nodes within the coverage area of ​​the same local consensus node. The lightweight multimodal large model is obtained by performing at least one compression operation on the multimodal large model deployed on the local consensus node, namely model pruning, weight quantization, or knowledge distillation. The local observation data refers to single-source or multi-source situational data collected by the agent node through its own sensors. After receiving cognitive summaries from other intelligent agent nodes, the semantic similarity between the received cognitive summaries and the cognitive summaries generated by the current node is calculated, wherein the semantic similarity is measured using cosine similarity. Based on the calculated semantic similarity, the received multiple cognitive summaries are weighted and aggregated. The aggregated semantic vector obtained after weighted aggregation is used as a local situation consensus snapshot and reported to the local consensus node. The weight coefficient of the weighted aggregation is positively correlated with the confidence value of each cognitive summary and the semantic similarity. The local situation consensus snapshot refers to the local unified situation cognition result obtained by the agent node through weighted aggregation.

Images