Off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method

Abstract

The application provides an off-grid heterogeneous intelligent agent dynamic formation and crowd intelligence reconstruction method, and belongs to the technical field of intelligent device management; comprising: receiving a natural language instruction, generating a task constraint graph containing probability space-time constraints, time window constraints and cooperative action relationships through edge visual-language model analysis; each intelligent agent locally maintains a distributed probability state estimation about the global state and capability factors of the cluster; a random optimal control solution is performed in the probability state space to minimize the joint cost function, and a control sequence is generated; the joint cost function includes semantic alignment cost, capability reliability cost, cooperative alignment cost and information uncertainty cost; the application enables the cluster to generate a robust formation in an off-grid environment through only natural language instructions, realizes adaptive balance between time urgency and action robustness, and has capability degradation perception and cooperative cognitive alignment capability, and has important application value in military reconnaissance, field rescue and other scenes.

Description

Technical Field

[0001] This invention relates to the field of intelligent device management technology, and in particular to a method for dynamic formation and collective intelligence reconstruction of off-grid heterogeneous intelligent agents. Background Technology

[0002] Multi-agent swarm formation coordination refers to the technology of multiple unmanned systems (such as drones, unmanned vehicles, and unmanned surface vessels) coordinating with each other to form and maintain a specific spatial configuration under a specific mission objective. It is widely used in fields such as military reconnaissance, disaster relief, and environmental monitoring. As mission scenarios become increasingly complex and unpredictable, swarm formation technology faces increasingly higher requirements for autonomy. Existing swarm formation technologies are mainly based on the following types of solutions: The first type of approach is a formation method based on a pre-defined rule base. This method pre-stores several typical formation templates (such as wedges, rings, lines, diamonds, etc.) and corresponding switching conditions in the system. The cluster selects the appropriate formation from the rule base and executes it based on the task type or environmental trigger conditions. For example, when the system detects a threat direction, it automatically switches to a wedge formation facing the threat. The core drawback of this approach is that the types of formation modes and the logic for changing them are completely limited by the pre-written, finite rules. It cannot cope with unknown scenarios not covered by the rule base and lacks true on-the-spot adaptability.

[0003] The second type of approach is a leader-follower formation control method. This method designates one node in the cluster as the leader, and the remaining nodes act as followers, forming a formation by maintaining their relative pose (distance and angle) with the leader. The formation shape is defined by pre-setting the desired relative positions of the followers and the leader. The drawbacks of this approach are: the geometric configuration of the formation is determined offline by the designer's desired relative positions, making it impossible to dynamically change the configuration parameters based on task semantics or the environment; and when the leader fails, the formation often faces the risk of overall collapse.

[0004] The third type of approach is a formation method based on artificial potential fields or virtual forces. This method defines virtual attraction or repulsion between agents and between agents and obstacles, and uses force balance to cause the cluster to converge to a certain regular geometric distribution. The drawback of this approach is that the final formation shape is the result of force field parameter balance, making it difficult for designers to precisely control the macroscopic geometry of the formation; more importantly, the potential field function parameters are fixed once set, and cannot be adjusted at the semantic level according to higher-level task intentions.

[0005] The fourth category of solutions is based on swarm intelligence optimization for formation generation, such as using particle swarm optimization and genetic algorithms to optimize formation configurations offline or online. While these solutions can find relatively optimal formation configurations, their optimization objective functions still need to be predefined manually (e.g., minimizing energy consumption or maximizing coverage area). The optimization results are essentially still limited by the guidance of the preset optimization objectives and cannot be automatically extracted and dynamically adjusted from natural language instructions.

[0006] Furthermore, all of the aforementioned solutions face even more severe challenges in off-grid environments. In outdoor, underground, or strong electromagnetic interference environments without public network coverage, clusters rely on ad-hoc networks for communication, resulting in limited bandwidth, unpredictable latency, and packet loss and interruptions in inter-node information transmission. Existing solutions typically assume that each node can obtain accurate global status information in real time (such as precise pose, remaining battery power, and sensor operating status), but this assumption is difficult to hold under off-grid conditions. When status information is missing or outdated due to communication interruptions or delays, formation planning methods based on accurate status will produce unreliable decision results.

[0007] In heterogeneous clusters, different types of intelligent agents exhibit significant differences in mobility, payload characteristics, and execution capabilities. Existing solutions typically employ pre-defined role allocation strategies (such as assigning specific UAV models to reconnaissance missions) or deterministic task allocation based on the nominal performance parameters of the nodes when handling heterogeneous nodes. However, in practical use, the actual execution capabilities of nodes may degrade due to mechanical wear, environmental degradation, and other factors. This degradation process is slow and difficult to accurately detect using conventional self-checking procedures. Existing solutions lack effective mechanisms to address this, potentially assigning critical tasks to nodes with insufficient actual execution capabilities, leading to mission failure.

[0008] In summary, existing technologies in the field of multi-agent swarm formation share the following limitations: the generation and adjustment of formation configurations rely on preset rules, fixed templates, or manually defined optimization objectives, and cannot automatically generate matching formation schemes based on the on-site task intent expressed in natural language; they lack the ability to proactively respond to the uncertainty of state information under off-network communication conditions; and they lack perception and adaptive adjustment mechanisms for the dynamic degradation of the execution capabilities of heterogeneous nodes. These problems make it difficult for existing technologies to achieve true tactical-level self-organizing collaboration when facing complex environments that are unknown, dynamic, and have limited communication. Summary of the Invention

[0009] To address the aforementioned technical challenges, this invention provides a method for dynamic formation and swarm intelligence reconstruction of off-grid heterogeneous intelligent agents. The aim is to enable heterogeneous intelligent agent clusters to autonomously complete the entire process from semantic understanding to formation generation using only a single natural language command in off-grid environments with limited public network coverage and communication constraints. During this process, the method proactively addresses the uncertainty of cluster state information, the dynamic degradation of node execution capabilities, and cognitive discrepancies among cooperating nodes. It dynamically balances the dual needs of information acquisition and task execution under task time constraints, and maintains semantically coherent formation reconstruction even when local nodes fail. Ultimately, this achieves tactical-level self-organizing and collaborative capabilities in unknown and complex environments.

[0010] To achieve the above objectives, this application proposes a method for dynamic formation and swarm intelligence reconfiguration of off-grid heterogeneous intelligent agents, including: Receive natural language instructions, parse the natural language instructions through a visual-language model deployed at the edge, and generate a task constraint graph containing probabilistic spatiotemporal constraints, time window constraints, and cooperative action relationships; Each agent maintains a distributed probabilistic state estimate locally regarding the cluster's global state and capability factors. The global state includes at least the poses of all agents, and the capability factors characterize the ratio of an agent's execution capability to its nominal value. With the goal of minimizing the joint cost function in the finite time domain, stochastic optimal control is solved in the distributed probabilistic state space to generate control sequences for each agent. The joint cost function includes semantic alignment cost, capability reliability cost, cooperative alignment cost, and information uncertainty cost. The information uncertainty cost is a weighted sum of position probability state entropy, capability probability state entropy, and cooperative probability state entropy. The weights of the weighted sum are dynamically determined by a time-sensitive weighting function based on the remaining time of the task and the current total probability state entropy. The control sequence is sent to each agent for execution to drive the heterogeneous cluster to form a formation that conforms to the task constraint graph.

[0011] As a further solution, the probabilistic spatiotemporal constraints are composed of the expected value of the geometric parameters of the spatial formation, the tolerance variance, and the lower bound of the constraint satisfaction probability extracted by the visual-language model from the natural language instructions. These constraints are used to evaluate the semantic compliance of the formation spatial configuration in probabilistic form in the probabilistic state space.

[0012] As a further solution, the time window constraint includes at least the expected completion time of the task and the upper limit of the allowed timeout probability. The time-sensitive weight function is configured such that as the current time approaches the expected completion time of the task, the weight of the information uncertainty cost monotonically decays, so as to force the optimization focus to shift from reducing uncertainty to quickly achieving the spatiotemporal constraints.

[0013] As a further solution, the capability factor includes at least the maximum speed ratio, sensor detection performance ratio, and communication radius ratio, and the capability reliability cost is constructed in the following manner: Based on the minimum threshold requirement of the role's ability in the task constraint diagram, and combined with the probability state estimation of the agent's ability factor, the reliability of the agent's task execution for that role is calculated. The reliability is compared with the minimum reliability required by the natural language instruction, and the square of any deviation exceeding the allowable deviation is included as a penalty in the semantic alignment cost.

[0014] As a further solution, the collaborative alignment cost is constructed as follows: Extract the key cooperative variables for each pair of cooperative agents in the cooperative action relationship, including relative pose and the current action stage of the other party; Calculate the Bach distance between the probability state estimates of each pair of cooperative agents with respect to the cooperative key variables; The Bach distance is mapped to an exponentially growing cost, which serves as the cost of the collaborative alignment.

[0015] As a further solution, the cooperative probability state entropy is defined as the sum of the differential entropies of the probability state estimates of all cooperative agents with respect to their respective cooperative key variables, and the cooperative probability state entropy is incorporated into the information uncertainty cost.

[0016] As a further solution, the time-sensitive weighting function takes the form of: Λ(t k H total )=λ0·exp(-α·max(0,T end -t k ) / (β+H total )) Where Λ is the time-sensitive weight, t k For the current decision-making moment, T end H is the expected completion time of the task. total λ0 represents the current total probability state entropy, α and β are corresponding positive real numbers, λ0 is the basic exploration coefficient, and is dynamically adjusted by the vision-language model according to the risk-sensitive expressions in the natural language instructions.

[0017] As a further solution, the semantic alignment cost further includes a task progress deviation penalty term, which is calculated based on the deviation between the current agent's actual movement progress and the expected progress based on the linear interpolation constrained by the time window, so that even if the spatial configuration is within the tolerance range, excessively slow movement will incur a cost.

[0018] As a further solution, the method also includes: When any agent fails or becomes abnormal, the probability state estimate of the corresponding agent is updated based on the observations reflecting the failure or abnormality. Based on the updated cluster global probability state, the stochastic optimal control problem is re-solved based on the task constraint graph to generate a reconstructed formation control sequence. During reconstruction, the remaining cooperative pairs with the smallest probability state divergence are selected first to maintain the coherence of cooperative semantics.

[0019] As a further solution, the stochastic optimal control solution adopts the probabilistic state model prediction path integral method. At each control update, multiple control trajectories are sampled based on the current probability state estimate. During the forward propagation of the trajectory, the position probability state and capability probability state are updated synchronously, and the cumulative cost of the trajectory is evaluated using the joint cost function. The control sequence with the lowest cost is then selected for output.

[0020] Compared with related technologies, the off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method provided by this invention has the following advantages: 1. This invention directly parses a single natural language command into a task constraint graph containing probabilistic spatiotemporal constraints, time window constraints, and cooperative action relationships through a visual-language model. This allows the formation configuration to emerge spontaneously from the real-time fusion of the commander's semantic intent, environmental spatial semantics, and the cluster's real-time capabilities, eliminating reliance on preset formation templates and finite state machine rules. When local nodes fail, observational mutation updates based on probabilistic state estimation and optimized re-solution based on the original task constraint graph ensure that the reconstruction scheme remains faithful to the commander's original semantic intent. It utilizes the elastic space provided by tolerance variance to achieve a smooth, semantically closest degradation, rather than mechanically filling in vacant positions, ensuring that the reconstructed formation can still complete its core tasks.

[0021] 2. This invention maintains distributed probabilistic state estimates of the cluster's global state and capability factors locally in each agent. By representing pose and capability information with probability distributions rather than deterministic numerical values, formation planning is directly built upon a state uncertainty model, exhibiting inherent tolerance for high packet loss, long latency, and information loss in ad hoc network environments. Through a time-sensitive weighting function, the weight of information uncertainty cost in the joint cost function is dynamically adjusted based on the remaining task time and the current total probabilistic state entropy. In the early stages of the task, the system drives proactive information acquisition with a higher weight. As the time window narrows, it automatically shifts the optimization focus to quickly achieving spatiotemporal constraints, avoiding timeout failures due to overexploration and rash actions when information is severely lacking. This achieves an adaptive and smooth transition between time urgency and action robustness.

[0022] 3. This invention introduces three capability factors—maximum speed ratio, sensor detection performance ratio, and communication radius ratio—and maintains them in the form of probabilistic state estimation. Combined with capability reliability costs constructed based on task execution reliability, this enables the system to autonomously perceive soft degradation in node performance and automatically avoid risks during task allocation. Simultaneously, by extracting key collaborative variables from each pair of collaborative agents in the collaborative action relationship, quantifying cognitive discrepancies between the parties using Bach distance to construct collaborative alignment costs, and measuring the degree of uncertainty in each party's cognition using collaborative probabilistic state entropy, the health of the collaborative link is comprehensively evaluated from two dimensions: "inconsistency between the parties" and "lack of clarity on each party's part." This drives the system to proactively generate collaborative alignment actions such as mutual observation and mutual positioning, and prioritizes the remaining collaborative pairs with the smallest cognitive discrepancies during reconstruction, significantly improving the success rate of complex tactical coordination tasks.

[0023] 4. All modules of this invention operate offline within a local area network or self-organizing network, without relying on any external base stations, cloud computing power, or fixed command centers. The visual-language model is deployed at the edge after compression and quantization processing, and the parallelization characteristics of the probabilistic state model's path integral solution are adapted to embedded computing resources. By integrating natural language-driven semantic understanding, distributed probabilistic state estimation, time-sensitive multi-objective stochastic optimization, and failure self-healing reconstruction into a complete closed loop, this invention endows heterogeneous clusters with tactical-level self-organizing capabilities in unknown off-network environments with communication denial and infrastructure deficiencies. It has extremely high practical value and patent defense value in high-value-added scenarios such as military reconnaissance, field emergency rescue, and interplanetary mining networks. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram illustrating the steps of a method for dynamic formation and swarm intelligence reconfiguration of off-grid heterogeneous intelligent agents provided by the present invention; Figure 2 This invention provides a schematic diagram of the structure of an off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconfiguration system. The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0028] Example 1 Please see Figure 1 This embodiment provides a method for dynamic formation and swarm intelligence reconstruction of off-grid heterogeneous intelligent agents, including: Receive natural language instructions, parse the natural language instructions through a visual-language model deployed at the edge, and generate a task constraint graph containing probabilistic spatiotemporal constraints, time window constraints, and cooperative action relationships; Each agent maintains a distributed probabilistic state estimate locally regarding the cluster's global state and capability factors. The global state includes at least the poses of all agents, and the capability factors characterize the ratio of an agent's execution capability to its nominal value. With the goal of minimizing the joint cost function in the finite time domain, stochastic optimal control is solved in the distributed probabilistic state space to generate control sequences for each agent. The joint cost function includes semantic alignment cost, capability reliability cost, cooperative alignment cost, and information uncertainty cost. The information uncertainty cost is a weighted sum of position probability state entropy, capability probability state entropy, and cooperative probability state entropy. The weights of the weighted sum are dynamically determined by a time-sensitive weighting function based on the remaining time of the task and the current total probability state entropy. The control sequence is sent to each agent for execution to drive the heterogeneous cluster to form a formation that conforms to the task constraint graph.

[0029] It should be noted that in an off-network environment, the commander inputs a natural language command via voice or text, and the visual-language model deployed at the edge parses the command.

[0030] Unlike traditional methods that simply map instructions to preset formation numbers, this method transforms natural language instructions into a structured task constraint graph, which contains three types of key constraints: First, there are probabilistic spatiotemporal constraints, which describe the spatial configuration requirements of the formation in probabilistic form (such as "forming an equilateral triangle with a side length of about 80 meters", while giving the allowable deviation range and the lower limit of probability). Second, time window constraints describe the time requirements of the task (such as the expected completion time and the allowed probability of timeout). Thirdly, there are cooperative action relationships, which describe the cooperative behaviors that require close cooperation between intelligent agents (such as "A attracts fire, and B takes advantage of the situation to cross over").

[0031] This embodiment uses this probabilistic semantic representation to capture not only the explicit meaning of the instructions in the task constraint graph, but also to preserve the tolerance and flexibility in the command intent, providing a foundation for subsequent optimization under uncertain conditions.

[0032] Furthermore, in an off-grid ad hoc network environment, it is difficult for each agent to obtain accurate global state information of the cluster in real time. To address this, each agent maintains a distributed probabilistic state estimate of the cluster's global state and capability factors locally. The global state includes at least the poses of all agents, but its form is not a single, definite value; rather, it is a probability distribution. Therefore, the system "knows" roughly where a node is located, and can also quantify the degree of uncertainty in this judgment.

[0033] The capability factor further characterizes the ratio of each agent's actual performance level to its nominal value (e.g., how much of its maximum speed remains, how much its sensor detection performance has decreased), also expressed as a probability distribution. Through this probabilistic state estimation, the system possesses an inherent cognitive ability to address issues such as missing, outdated, and capability degradation, rather than blindly assuming that all states are deterministic and accurate.

[0034] Finally, with the goal of minimizing the joint cost function within a finite time domain, the system performs stochastic optimal control solutions in a distributed probabilistic state space to generate control sequences for each agent.

[0035] The joint cost function is one of the core innovations of this method. It consists of four parts: semantic alignment cost evaluates the degree of matching between the current configuration and the task constraint graph; capability reliability cost assesses whether the node assigned the task truly has the required execution capability; cooperative alignment cost measures the degree of consistency in the cooperating pairs' perception of each other's states; and information uncertainty cost integrates the probabilistic state entropy of position, capability, and cooperation to drive the system to actively acquire information when necessary to reduce uncertainty.

[0036] More importantly, the weight of the information uncertainty cost is dynamically adjusted by a time-sensitive weighting function based on the remaining task time and the current total probability state entropy: when time is ample, the system can devote more energy to exploring and eliminating uncertainty, while when time is tight, it can focus on advancing task execution. After the obtained control sequence is issued to each agent for execution, it can drive the heterogeneous cluster to form a formation that is robust in semantic, capability, coordination, and time dimensions.

[0037] Furthermore, the probabilistic spatiotemporal constraints are composed of the expected value of the geometric parameters of the spatial formation, the tolerance variance, and the lower bound of the constraint satisfaction probability extracted by the visual-language model from the natural language instructions. These constraints are used to evaluate the semantic compliance of the formation spatial configuration in probabilistic form in the probabilistic state space.

[0038] Specifically, traditional methods, when dealing with instructions like "form an encirclement on the east side of the building," typically simplify them into a fixed geometric shape (such as a standard annulus centered at a certain coordinate point). This deterministic transformation loses the implicit tolerance and flexibility of the instruction. This method uses a visual-language model to extract three key parameters from natural language instructions to achieve probabilistic transformation: the expected value of the geometric parameters, the tolerance variance, and the lower probability limit for constraint satisfaction. The expected value of the geometric parameters describes the commander's ideal formation (e.g., 80 meters in "an equilateral triangle with sides of 80 meters"), the tolerance variance describes the allowable actual deviation range (e.g., the side length can fluctuate between 70 and 90 meters), and the lower probability limit for constraint satisfaction describes the strictness of the spatial constraint (e.g., "must" corresponds to a high probability lower limit, "as much as possible" corresponds to a low probability lower limit). These three parameters together constitute a probabilistic spatial constraint, ensuring that both the certainty and flexibility of the command intention are fully preserved.

[0039] The three components mentioned above each perform their respective functions within the system, forming a hierarchical constraint system. The expected value of the geometric parameters defines the ideal reference benchmark for the formation spatial configuration, serving as the target center for optimization. This includes ideal values ​​for geometric quantities such as the side lengths, spacing, azimuth angles, and relative distances of the formation. The tolerance variance defines the acceptable deviation range around this ideal benchmark. Its physical meaning is that, in actual execution, limited by the agent's motion accuracy and environmental interference, the formation configuration cannot and does not need to precisely reach the expected value; as long as the deviation is within the tolerance range, it is considered to meet the requirements.

[0040] The magnitude of the tolerance variance reflects the flexibility of the spatial constraint: a larger variance means a wider range of allowable deformations in the configuration and a higher degree of freedom in system optimization. The lower bound of the constraint satisfaction probability is a key threshold for evaluation in the probabilistic state space: since the system's state estimation itself is in the form of a probability distribution, the compliance of the spatial configuration is no longer a binary judgment, but rather a probabilistic expression of "how likely the configuration is to satisfy the constraint." The lower bound of the constraint satisfaction probability sets the minimum acceptable value for this probability; during optimization, the system must ensure that the probability of the formation configuration satisfying the spatial constraint is not lower than this lower bound. Through the coordinated operation of these three elements, a simple command from the commander to "form an encirclement" is precisely quantified into a calculable and optimizable probabilistic constraint.

[0041] Furthermore, the probabilistic evaluation mechanism employed in this embodiment enables this assessment to be performed probabilistically in the probabilistic state space: the system does not require the formation configuration to be exactly equal to the expected value of the geometric parameters. Instead, based on the current probabilistic state estimate, it calculates the probability that the actual configuration falls within the acceptable region defined by the expected value and the tolerance variance. When this probability is higher than the lower probability limit of the constraint, it is considered semantically compliant and incurs no additional cost; only when the probability is lower than the lower limit is a penalty cost calculated based on the extent of the excess. The beneficial effects of this mechanism are: when the system has high certainty about its own state, it can precisely control the formation configuration to pursue higher task execution quality; when the state uncertainty is high, the requirements for configuration accuracy can be appropriately relaxed within the allowable range of the tolerance variance, allowing more resources to be invested in reducing uncertainty or advancing task progress. This probabilistic compliance evaluation method is the key support for the robust formation generation under uncertain conditions in this invention.

[0042] Furthermore, the time window constraint includes at least the expected completion time of the task and the upper limit of the allowed timeout probability. The time-sensitive weight function is configured such that as the current time approaches the expected completion time of the task, the weight of the information uncertainty cost monotonically decays, so as to force the optimization focus to shift from reducing uncertainty to quickly achieving the spatiotemporal constraints.

[0043] Specifically, in actual missions, commanders' instructions often imply time urgency, such as "complete the encirclement within three minutes" or "establish a blockade line as soon as possible." This method uses a visual-language model to extract two key parameters from natural language instructions to construct time window constraints: the expected completion time of the task and the upper limit of the allowed timeout probability.

[0044] The expected completion time of a mission represents the commander's ideal timeframe for the formation to be in position, defining the timeliness benchmark for mission execution. The upper limit of the allowable timeout probability characterizes the stringency of this timeliness requirement: due to uncertainties such as communication delays and environmental interference in actual execution, precise on-time completion is not always feasible. The upper limit of the allowable timeout probability provides a statistical boundary for the latest acceptable completion time. For example, if the expected completion time of a mission is 180 seconds and the upper limit of the allowable timeout probability is 0.1, it means that the system should complete the formation deployment within 180 seconds with a probability of at least 90%. This probabilistic expression makes the time constraint no longer a rigid deadline, but a soft constraint with statistical tolerance, matching the inherent uncertainty of an off-grid environment.

[0045] Furthermore, in the early stages of task execution, with the deadline still far off, time sensitivity weights are high, and information uncertainty costs account for a significant proportion of the joint cost function. The system tends to adopt proactive probing and information gathering behaviors to reduce uncertainty about the environment and its own state, laying the foundation for subsequent precise execution. As time progresses and the expected task completion time approaches, the time sensitivity weights decrease monotonically, the influence of information uncertainty costs gradually weakens, and cost items directly related to task completion, such as semantic alignment costs and capability reliability costs, gradually dominate the optimization direction. The system shifts its focus from "understanding the situation" to "completing the task." This dynamic adjustment mechanism allows the system to adaptively and smoothly transition from "exploration-oriented" to "execution-oriented" without manual intervention.

[0046] More importantly, under conditions of limited communication and incomplete status information, if the system insists on completely eliminating information uncertainty before taking action, it may lead to indefinite waiting and mission timeout failure. Conversely, if it takes rash action when information is severely lacking, it may cause the formation configuration to deviate significantly from the commander's intentions.

[0047] This embodiment establishes an adaptive balance between the two through a time-sensitive weighting function: in the early stages, the system has sufficient time to perform necessary situational awareness and information acquisition, ensuring that actions are based on a relatively reliable foundation; as the time window narrows, the system automatically reduces the requirement for determinism, prioritizing the use of the limited remaining time to meet spatiotemporal constraints, ensuring that the formation mission is completed with an acceptable probability within the time window.

[0048] The mechanism of this embodiment works in conjunction with the tolerance variance design of probabilistic spatiotemporal constraints: when time is ample, the system can use the flexibility space provided by the tolerance variance to explore information; when time is tight, the system can accept a more relaxed configuration accuracy within the tolerance variance range in exchange for a faster deployment speed, thereby jointly ensuring the robust completion of the formation mission in both time and space dimensions.

[0049] Furthermore, the capability factor includes at least the maximum speed ratio, sensor detection performance ratio, and communication radius ratio, and the capability reliability cost is constructed in the following manner: Based on the minimum threshold requirement of the role's ability in the task constraint diagram, and combined with the probability state estimation of the agent's ability factor, the reliability of the agent's task execution for that role is calculated. The reliability is compared with the minimum reliability required by the natural language instruction, and the square of any deviation exceeding the allowable deviation is included as a penalty in the semantic alignment cost.

[0050] Specifically, during long-term offline operation, the actual execution capability of an intelligent agent may slowly degrade due to mechanical wear, sensor contamination, aging of electronic components, etc. Traditional methods treat the capability parameters of nodes as fixed nominal values, resulting in key roles being assigned to nodes whose actual capabilities are insufficient during task allocation.

[0051] This method introduces three specific capability factors to quantify this degradation: the maximum speed ratio, which is the ratio of the node's current actual achievable maximum speed to its factory-specified maximum speed, reflects the health of the power system; the sensor detection performance ratio, which is the ratio of the node's current sensor's actual effective detection distance or sensitivity to its nominal value, reflects the degree of degradation of the sensing system; and the communication radius ratio, which is the ratio of the node's current actual effective communication distance to its nominal value, reflects the performance degradation of the communication system. All three ratios range from zero to one; the closer the value is to one, the closer the capability is to a healthy state, and the closer it is to zero, the more severe the degradation. Because the capability degradation process is slow and difficult to accurately obtain through routine self-testing, these capability factors are not deterministic values ​​but are maintained in the agent's local probabilistic state as probabilistic state estimates. The system can simultaneously know the most likely values ​​of the capability factors and the degree of uncertainty of these estimates.

[0052] Step 1: Based on the minimum threshold requirements for capabilities of each role in the task constraint graph, and combined with the agent's capability factor probabilistic state estimation, calculate the reliability of the agent's task execution for that role. Specifically, each role in the task constraint graph (such as "forward reconnaissance" or "flank cover") is assigned a set of capability requirement thresholds. For example, the reconnaissance role requires a maximum speed ratio of no less than 0.7 and a sensor detection performance ratio of no less than 0.8. Based on the joint probability distribution of these three capability factors, the system calculates the probability that the agent simultaneously meets all capability thresholds; this probability value is the task execution reliability. High reliability means that the agent probabilistically possesses the capability to complete the role's task, while low reliability indicates a significant risk of capability deficiency.

[0053] The second step involves comparing this reliability with the minimum reliability required by the natural language instruction. Any deviation exceeding the square of the allowable deviation is included as a penalty in the semantic alignment cost. The minimum reliability required by the natural language instruction is the level of stringency parsed from the commander's wording (e.g., "ensure completion" corresponds to a higher threshold of 0.9, "do your best" corresponds to a lower threshold of 0.7). If the calculated reliability is higher than this requirement, no penalty is incurred; if it is lower, a penalty is calculated based on the square of the deviation. Using a square function means that minor deviations incur small costs, while severe deviations result in dramatically increased costs, prompting the optimizer to prioritize avoiding high-risk allocation schemes.

[0054] Furthermore, when the probability state entropy of a certain agent's ability is high, it means that the system knows very little about the true ability level of that node. If it is assigned to a role with strict ability requirements, the calculated task execution reliability may be low due to the wide probability distribution, resulting in a large cost to capability reliability.

[0055] The information uncertainty cost in the joint cost function also drives the system to take proactive diagnostic actions when time permits (such as instructing the node to perform a short-term acceleration test to obtain a more accurate velocity factor observation), thereby reducing the capability probability state entropy and improving the calculated reliability of subsequent task execution. As the time window narrows and the time-sensitive weight function decays, if the capability probability state entropy is still high, the system will automatically remove the node from the critical role and select a node with a more certain capability state to take on the heavy responsibility. This mechanism of explicitly incorporating capability uncertainty into the task allocation cost enables heterogeneous clusters to perceive the degree of their own capability decline in off-network degradation environments and make reasonable task allocation decisions accordingly, avoiding the fatal mistake of having a "wounded scout" perform a "full-speed breakthrough".

[0056] Furthermore, the collaborative alignment cost is constructed as follows: Extract the key cooperative variables for each pair of cooperative agents in the cooperative action relationship, including relative pose and the current action stage of the other party; Calculate the Bach distance between the probability state estimates of each pair of cooperative agents with respect to the cooperative key variables; The Bach distance is mapped to an exponentially growing cost, which serves as the cost of the collaborative alignment.

[0057] Specifically, when a commander instructs two or more intelligent agents to perform close spatiotemporal coordination, how can we quantify the degree of consistency in their perception of each other's states? In actual tactical coordination, the success or failure of the coordination often depends on whether both parties have a consistent understanding of the same collaborative scenario. For example, in the instruction "UAV A penetrates from the left to draw fire, while UAV B takes advantage of the situation to cross the gap," the timing and path of B crossing the gap depend entirely on A's current position and the progress of its actions. If B's ​​judgment of A's position is flawed, the crossing action may fail.

[0058] To this end, this embodiment first extracts key cooperative variables for each pair of cooperative agents from the cooperative action relationships in the task constraint graph. These key variables consist of two components: first, relative pose, which describes the relative position and posture of the two agents in space, forming the spatial basis of cooperative actions; and second, the current action stage of the other agent, which describes the stage the other agent is in within the cooperative time sequence (e.g., "currently maneuvering," "has reached the designated position," "is performing a cover action"), forming the temporal basis of cooperative actions. By defining these two elements as key cooperative variables, the abstract problem of "cooperation synergy" is transformed into a concrete, quantifiable problem of variable consistency.

[0059] After extracting the key collaborative variables, this embodiment introduces Bach distance as a mathematical tool to measure the cognitive discrepancies between the collaborating parties. Specifically, the calculation method is as follows: in the local probability state estimates maintained by each pair of collaborating agents, probability distributions for the key collaborative variables are extracted, and the Bach distance between these two probability distributions is calculated. Bach distance is a classic measure of the similarity between two probability distributions, and its value varies between zero and infinity: when both parties have completely identical probability state estimates for the key collaborative variables, the Bach distance is zero, meaning that both parties' cognition of the collaborative scenario is completely synchronized, and their cooperative actions are based on aligned cognition; when the deviation in the probability state estimates of both parties increases and the overlapping area decreases, the Bach distance monotonically increases, meaning that the two parties have discrepancies in their cognition of the same collaborative scenario.

[0060] For example, if node A believes that B is 10 meters due east of it (the probability distribution is concentrated in the vicinity of this location), while B believes that it is 15 meters due east of A due to positioning drift (the probability distribution is concentrated in another location), the overlap area of ​​the two probability distributions is small and the Bach distance is large, indicating that there is a significant difference in the perception of the relative positions between the two parties. At this time, the risk of performing cooperative actions is high.

[0061] After calculating the Bach distance, this embodiment maps it to an exponentially increasing cost as the collaborative alignment cost. When the Bach distance is small, i.e., the cognitive differences between the collaborating parties are within an acceptable range, the exponential mapping produces a small cost, and the collaborative alignment cost accounts for a small proportion in the joint cost function, having no significant impact on the optimization direction. When the Bach distance exceeds a certain threshold, the steepness of the exponential mapping causes the cost to rise sharply, and the collaborative alignment cost becomes the dominant term in the joint cost function, forcing the optimizer to prioritize solving the cognitive difference problem.

[0062] This nonlinear mapping design stems from practical experience in tactical coordination: slight cognitive biases can be tolerated by the robustness of the coordinated action itself (e.g., B can fine-tune the path based on actual observations during the crossing), but severe cognitive discrepancies will lead to the complete failure of the coordinated action (e.g., B completely misunderstands A's position, resulting in a fundamental error in the crossing direction).

[0063] After the collaborative alignment cost is incorporated into the joint cost function, it forms a synergistic effect with the collaborative probability state entropy: when time is sufficient, the system can reduce the Bach distance by actively generating collaborative alignment actions such as mutual observation and mutual positioning, thereby reducing the collaborative alignment cost; when time is tight, the system will prioritize collaborative pairs with smaller cognitive differences to perform key coordination tasks, avoiding risks on combinations with higher collaborative risks.

[0064] Furthermore, the cooperative probability state entropy is defined as the sum of the differential entropies of the probability state estimates of all cooperative agents with respect to their respective cooperative key variables, and the cooperative probability state entropy is incorporated into the information uncertainty cost.

[0065] Specifically, differential entropy is a standard measure of the uncertainty of a continuous probability distribution in information theory. A higher entropy value indicates a more ambiguous understanding of the variable. For example, in a cooperative scenario of "A providing cover, B traversing," even if A and B have completely identical perceptions of each other's relative positions (small Bach distance, small discrepancy), if both parties' judgments are "B is roughly somewhere between 10 and 50 meters east of A" (wide probability distribution, high entropy), then the cooperative action still faces a high risk of failure because the actual relative position could fall anywhere within that broad range. This embodiment incorporates this state of "neither party being certain" into the optimization considerations, compensating for the deficiency of only focusing on "inconsistency between the two parties" while failing to capture the hidden danger of "neither party being clear."

[0066] This embodiment incorporates the limited cooperative probability state entropy into the information uncertainty cost, thus placing it alongside the location probability state entropy and the capability probability state entropy to form the three components of the information uncertainty cost. This incorporation brings two levels of technical benefits.

[0067] First, in terms of driving information acquisition behavior, when the cooperative probability state entropy is high, the cost of information uncertainty increases. In the early stages of a task with high time sensitivity weight, the joint cost function will guide the optimizer to actively generate actions that reduce cooperative uncertainty: for example, assigning one party in the cooperative pair to actively approach the other party to obtain more accurate relative pose observations through sensors, or exchanging confirmation information of their current action stages through communication, thereby narrowing the width of the probability distribution and reducing the cooperative probability state entropy.

[0068] Second, regarding the coordination and alignment costs, the collaborative probability state entropy and Bach distance jointly evaluate the health of the collaborative link from two dimensions: "the degree of ambiguity in each party's cognition" and "the degree of difference in the parties' cognition." Low entropy ensures that all parties have a relatively clear judgment on the collaborative variables, while low Bach distance ensures that the judgments of all parties are consistent. Only when both dimensions are at a healthy level can the collaborative action have a high probability of success. This dual-dimensional evaluation mechanism makes the system's perception of collaborative risks more comprehensive and refined, avoiding the risk omissions that may occur when relying on a single indicator.

[0069] Furthermore, the time-sensitive weighting function is in the following form: Λ(t k H total )=λ0·exp(-α·max(0,T end -t k ) / (β+H total )) Where Λ is the time-sensitive weight, t k For the current decision-making moment, T end H is the expected completion time of the task. total λ0 represents the current total probability state entropy, α and β are corresponding positive real numbers, λ0 is the basic exploration coefficient, and is dynamically adjusted by the vision-language model according to the risk-sensitive expressions in the natural language instructions.

[0070] Specifically, this embodiment implements a dual adjustment mechanism based on the formula structure, enabling the time-sensitive weight to simultaneously respond to changes in both the urgency of time and the degree of information uncertainty. The first adjustment is achieved by max(0,T) in the molecule. end -t k The first term represents the remaining time until the task deadline. In the early stages of the task when there is ample remaining time, this term is relatively large, resulting in a larger independent variable in the exponential function, making Λ approach λ0, and the system drives information acquisition behavior with a higher weight. As the task progresses, the current time gradually approaches T. end As the remaining time approaches zero, the value of the exponential function decays rapidly, Λ approaches zero, and the cost of information uncertainty has almost no longer had an impact on the joint cost function. The optimization focus shifts entirely to cost terms directly related to task completion, such as semantic alignment.

[0071] The second adjustment is achieved by H in the denominator. total Implementation: When the system's understanding of its global state, capabilities, and coordination is highly uncertain, H total When H is relatively large, the increased denominator slows down the rate of exponential decay. Even when little time remains, the system retains a certain degree of exploratory drive to eliminate severe information blind spots as quickly as possible; conversely, when H is relatively small... totalWhen the value is already relatively small, it indicates that the system has a clearer understanding of the current situation. A smaller denominator leads to a faster decay rate, allowing the system to transition to the pure execution phase more quickly. This dual adjustment mechanism means that the system's balance point between exploration and execution is not fixed, but dynamically and adaptively adjusted based on the actual information state and remaining time.

[0072] Furthermore, λ0 is dynamically adjusted by the visual-language model based on risk-sensitive expressions in the natural language instructions. Specifically, when parsing natural language instructions, the visual-language model not only extracts the spatial and temporal constraints of the task but also identifies the risk preferences implied in the commander's wording. For example, when the instruction contains expressions emphasizing reliability such as "absolutely ensure" or "flawless," the model assigns a larger value to λ0, causing the system to prioritize information acquisition and uncertainty elimination during task execution, sacrificing some speed to ensure robustness of action. Conversely, when the instruction contains expressions emphasizing speed such as "quickly," "immediately," or "at all costs," the model assigns a smaller value to λ0, causing the system to operate with a lower information uncertainty weight from the outset, prioritizing task completion within the time window.

[0073] Therefore, through this semantic-to-parameter mapping mechanism, the same time-sensitive weight function can exhibit drastically different exploration-execution balance characteristics under different command styles and task requirements, making the emergence of collective intelligence not only an adaptation to the environment and state, but also a deep understanding and faithful execution of the commander's tactical intentions.

[0074] Furthermore, the semantic alignment cost further includes a task progress deviation penalty term, which is calculated based on the deviation between the current agent's actual movement progress and the expected progress based on the linear interpolation constrained by the time window, so that even if the spatial configuration is within the tolerance range, excessively slow movement will incur a cost.

[0075] Specifically, in the above embodiments, driven by a time-sensitive weighting function, the system can adjust the balance between exploration and execution in the time dimension. However, the semantic alignment cost itself mainly evaluates the compliance of the formation's spatial configuration and does not directly focus on the movement speed of the formation deployment. This may lead to a hidden danger: when information uncertainty is high or environmental constraints are complex, the optimizer may choose to have the agent move towards the target position at an extremely slow speed. This is because slow movement does not trigger spatial configuration penalties (it will still eventually reach the target position) but can reduce the information uncertainty cost (slow movement provides more observation and communication opportunities). However, this overly conservative behavior may result in the formation being spatially compliant but severely lagging in time, ultimately violating the commander's timeliness intentions.

[0076] This embodiment eliminates this potential problem by adding a task progress deviation penalty term to the semantic alignment cost. This penalty term is calculated based on the deviation between the agent's actual movement progress and the expected progress based on linear interpolation using time window constraints. Specifically, the system determines the percentage of task progress that should be completed at the current moment using linear interpolation, based on the expected completion time of the task within the time window constraint. This percentage is then compared with the percentage of progress actually completed by the agent, and the square of the difference is used as the penalty value. This design means that even if the spatial configuration of the formation is completely within the tolerance range, if the agent's actual movement speed is significantly slower than the average speed required to complete the task on time, additional costs will be incurred due to progress lag.

[0077] After the task progress deviation penalty is incorporated into the semantic alignment cost, it forms a three-dimensional synergy with the time-sensitive weight function and the probabilistic spatiotemporal constraint tolerance variance. On the task execution timeline, these three mechanisms regulate formation generation behavior from different perspectives: the tolerance variance of the probabilistic spatiotemporal constraint provides spatial dimensional flexibility for formation configuration, allowing for certain configuration deviations under uncertain conditions; the time-sensitive weight function regulates the overall priority of information acquisition and task execution, prioritizing exploration when time is ample and execution when time is tight; the task progress deviation penalty further refines the regulation within the task execution phase, ensuring that once the system enters the "execution-driven" phase, its movement speed matches the time window requirements, rather than slowly approaching the target at the lowest possible speed. The synergistic logic of the three can be summarized as: the tolerance variance controls "how much shape can differ," the time-sensitive weight controls "when to move," and the progress penalty controls "how fast to move." This multi-layered regulation mechanism ensures that the formation generation process neither misses opportunities due to the pursuit of perfect configuration nor delays the task due to excessive conservatism, achieving a fine balance between spatial flexibility and temporal rigidity.

[0078] Furthermore, the method also includes: When any agent fails or becomes abnormal, the probability state estimate of the corresponding agent is updated based on the observations reflecting the failure or abnormality. Based on the updated cluster global probability state, the stochastic optimal control problem is re-solved based on the task constraint graph to generate a reconstructed formation control sequence. During reconstruction, the remaining cooperative pairs with the smallest probability state divergence are selected first to maintain the coherence of cooperative semantics.

[0079] Specifically, in actual off-network tasks, node failures take various forms: they may be completely disconnected due to physical damage, actively offline due to energy depletion, or in a semi-disconnected state due to communication module failure (other nodes cannot obtain its state updates, but it is still running). In this embodiment, when any agent failure or state anomaly is detected, the probabilistic state estimate of the corresponding agent is updated based on the observation results reflecting the failure or state anomaly. This update is not simply deleting the node from the state list, but performing a Bayesian update in the probabilistic state space: if the node's failure state is directly observed (e.g., confirmed by a neighboring node's sensor), its pose probability distribution is updated to a high-confidence distribution centered on the damage location, and the capability factor is updated to zero or close to zero; if the node is only experiencing a communication interruption and its actual state is unknown, the uncertainty of its pose probability distribution will rapidly increase over time, and the probability distribution of the capability factor will remain in the state before the interruption, but the uncertainty will continue to grow. This observation-based abrupt update mechanism allows the cluster's understanding of failure events to be quickly reflected in the probabilistic state estimate, providing an accurate situational basis for subsequent reconstruction decisions.

[0080] After completing the abrupt update of the probabilistic state, the system in this embodiment re-solves the stochastic optimal control problem based on the updated cluster global probabilistic state and the task constraint graph, generating a reconstructed formation control sequence. That is, the reconstruction process does not abandon the original task instructions and proceeds freely, but rather while maintaining the semantic intent carried by the original task constraint graph. The probabilistic spatiotemporal constraints, time window constraints, and cooperative action relationships contained in the original task constraint graph constitute the semantic boundary of the reconstruction: the system must find a new formation scheme while still satisfying these constraints.

[0081] If some constraints in the original task constraint graph become unsatisfactory due to node failure—for example, if the original instruction required "three drones to form an equilateral triangle," but one drone has lost contact and the remaining two cannot form a triangle—the system does not mechanically adhere to the original constraints leading to no solution. Instead, it utilizes the tolerance variance in probabilistic spatiotemporal constraints and the elastic space provided by the lower bound of constraint satisfaction probability to automatically search for feasible solutions in the belief space that are closest to the original semantics (such as two drones forming a V-formation, which is approximately equivalent to the original triangle in terms of coverage). This semantically preserved constraint processing ensures that the reconstruction result remains faithful to the commander's original intention, rather than simply filling in empty positions.

[0082] Furthermore, this embodiment prioritizes the remaining cooperative pairs with the smallest probability state divergence during reconstruction to maintain the coherence of cooperative semantics. In the original formation scheme, there may be multiple cooperative pairs (e.g., "A covers B's crossing, C is responsible for suppressing flanking fire"). After a node fails, some cooperative pairs disintegrate due to missing members, and the remaining nodes need to re-establish cooperative relationships to continue the mission. Among the possible remaining node pairing schemes, the system prioritizes the combinations with the smallest probability state divergence of the key cooperative variables: that is, the combinations where both sides have the most consistent understanding of the relative pose and the other's action phase, and the smallest Bach distance. The technical rationale for this strategy is that in the emergency reconstruction phase after a node failure, the remaining time window is usually more limited, and the system does not have enough time to reduce cooperative uncertainty from scratch; selecting node pairs with smaller cognitive divergence to perform cooperative tasks can achieve the highest success rate of cooperative actions with the lowest time cost. This strategy forms a complete closed loop with the collaborative alignment cost and collaborative probability state entropy: In daily execution, the system continuously optimizes the consistency of collaborative cognition through the collaborative alignment cost and collaborative probability state entropy; during failure reconstruction, the system utilizes the cognitive results accumulated through this continuous optimization to quickly select the node combination with the best collaborative foundation in the current state, thereby achieving semantically coherent and collaboratively reliable self-healing reconstruction.

[0083] Furthermore, the stochastic optimal control solution employs the probabilistic state model prediction path integral method. During each control update, multiple control trajectories are sampled based on the current probabilistic state estimate. The position probability state and capability probability state are updated synchronously during the trajectory forward propagation process. The cumulative cost of the trajectory is evaluated using the joint cost function, and the control sequence with the lowest cost is selected for output.

[0084] Specifically, when performing optimization in a distributed probabilistic state space, traditional deterministic optimal control methods (such as linear quadratic regulators or gradient-based model predictive control) are difficult to apply directly because the state variables are probability distributions rather than deterministic values, and the joint cost function contains multiple nonlinear cost terms (such as exponentially growing co-alignment costs and information uncertainty costs of time-sensitive weight dynamic adjustment).

[0085] This embodiment uses the probabilistic state model prediction path integral method as the solution tool. This method belongs to the category of sampling-based stochastic optimization algorithms. Its basic workflow is as follows: In each control update cycle, the algorithm generates multiple candidate control trajectories in the control space based on the current probabilistic state estimate. Each trajectory represents a possible future action sequence. Then, these control trajectories are substituted into the system model for forward propagation to deduce the future state evolution process corresponding to each trajectory. Finally, the cumulative cost of each trajectory is evaluated using the joint cost function, and the control sequence with the lowest cost is selected as the output. This sampling-based method does not require gradient calculation of the cost function, making it naturally suitable for handling complex cost structures that are nonlinear, nonconvex, and multi-objective. Furthermore, the stochasticity of the sampling process can effectively avoid getting trapped in local optima.

[0086] The position probability state and capability probability state are updated synchronously during the trajectory forward propagation process, so that the trajectory evaluation of the algorithm is no longer based on fixed state assumptions, but dynamically evolves the system's perception of its own state as different actions are performed.

[0087] Specifically, for each sampled control trajectory, as the algorithm extrapolates along the time axis, it not only updates the agent's expected position based on the kinematic model, but also synchronously updates the distribution parameters of the probability state estimate using Bayesian filtering based on the observation results that may be generated by the specific actions performed in the trajectory.

[0088] For example, if a sampled trajectory includes a short, rapid acceleration, this action will generate effective observations about the agent's maximum velocity factor during execution, thus narrowing the probability distribution of the velocity factor in the capability probability state and reducing uncertainty. Conversely, if a trajectory moves at a constant, slow speed throughout, the probability distribution of the velocity factor will remain unchanged, and uncertainty cannot be eliminated. Through this synchronous update mechanism, each sampled trajectory not only generates different physical motion paths but also different information acquisition effects: trajectories that actively execute diagnostic maneuvers may consume more energy in the early stages but can significantly reduce the uncertainty of subsequent decisions; trajectories with conservative movements have lower costs in the early stages but may face higher risk costs in the later stages due to insufficient information. By accumulating these costs over the entire time domain, the algorithm automatically selects the control sequence that achieves the optimal balance between exploration and execution.

[0089] Furthermore, the solution method in this embodiment is computationally efficient and matches the deployment conditions of the off-grid edge. The core computational load of the probabilistic state model prediction path integral method is concentrated on the forward propagation and cost evaluation of the sampled trajectories. Both of these operations can be highly parallelized: the derivation of multiple sampled trajectories is independent of each other and can be executed simultaneously on multiple processing cores of the edge computing device; the derivation of each time step within each trajectory can also be partially pipelined.

[0090] Furthermore, due to the low dimensionality of latent states such as capability factors (three ratio scalars), even if Bayesian updates of capability probability states are added to the forward propagation of each trajectory, the increased computational cost is almost negligible compared to the derivation of position states. This allows the entire solution process to be optimized within a finite time-domain prediction window with a sufficiently high number of sampled trajectories, meeting the periodic requirements of real-time formation control.

[0091] Example 2 Please see Figure 2 This embodiment provides a complete implementation process for a semantic-probabilistic constraint-driven cooperative formation method for heterogeneous intelligent agent clusters in an off-grid environment.

[0092] The system consists of four parts: an edge vision-language model inference engine deployed on the mission navigation node, a distributed probability state estimation and maintenance module, a time-sensitive multi-objective stochastic optimal control solution module, and a formation execution and monitoring module. All modules run offline within a local area network or self-organizing network and do not rely on any external base station or cloud service.

[0093] Step 1: Receive natural language instructions and parse them to generate a task constraint graph. Commanders input natural language instructions via voice or text, such as: "Three infrared drones advance 300 meters within three minutes to form an equilateral triangle blockade area, two transport vehicles maintain a diamond-shaped protection 200 meters behind, drone A breaks through from the left to attract fire, and drone B takes advantage of the opportunity to cross the gap. Make sure the encirclement is completed." The visual-language model deployed at the edge receives and parses the natural language instruction, generating a structured task constraint graph containing three types of constraints: (1) Probabilistic spatiotemporal constraints: including the expected value of the geometric parameters of the spatial array, the tolerance variance, and the lower limit of the constraint satisfaction probability. For example, for the "equilateral triangle", the expected value of the side length is 80 meters, the tolerance variance is 100 square meters (that is, the side length can fluctuate between about 70 and 90 meters), and the lower limit of the constraint satisfaction probability is 0.9, that is, the system should maintain the side length of the triangle configuration within the tolerance range with a probability of not less than 90%; for "300 meters forward", the expected value of the forward distance is 300 meters, the tolerance range is 270 to 330 meters, and the lower limit of the constraint satisfaction probability is 0.85; for "200 meters backward", the expected value of the backward offset between the centroid of the triangle and the centroid of the rhombus is 200 meters, and the Rayleigh distribution scale parameter of the radial error is allowed to be 25 meters.

[0094] (2) Time window constraint: including the expected completion time T of the task end And the upper limit of the allowed timeout probability. The expected completion time of the task is extracted from the instruction "complete within three minutes" as 180 seconds, and the upper limit of the allowed timeout probability is 0.1, that is, the system should complete the formation deployment within 180 seconds with a probability of not less than 90%.

[0095] (3) Cooperative action relationship: Extract the cooperative pair A and B from "UAV A breaks through from the left to attract fire and UAV B takes advantage of the gap to cross the gap". The cooperative action relationship requires A to perform the breakthrough action before B, with a time difference between the two between 2 and 5 seconds (probability not less than 0.85). B's crossing path must pass through the gap area formed after A attracts fire. A must be within B's line of sight and the distance must not exceed 50 meters when B enters the gap.

[0096] Meanwhile, the visual-language model extracts a basic exploration coefficient λ0 of 0.8 from the risk-sensitive expression "make sure the encirclement is completed" in the instruction, making the system pay more attention to information acquisition and uncertainty elimination during task execution.

[0097] Step 2: Each agent maintains a distributed probabilistic state estimate. Each agent maintains a distributed probabilistic state estimate locally regarding the cluster's global state and capability factors. The cluster's global state includes at least the poses of all agents, represented as a probability distribution. For example, the position of a node is described as a two-dimensional Gaussian distribution centered at a certain coordinate, and the covariance matrix reflects the uncertainty of that position estimate.

[0098] The capability factor characterizes the ratio of each agent's execution capability to its nominal value, including the maximum speed ratio (the ratio of the current actual maximum speed to the factory nominal maximum speed), the sensor detection performance ratio (the ratio of the current actual effective detection distance of the sensor to the nominal value), and the communication radius ratio (the ratio of the current actual communication distance to the nominal value). All three ratios take values ​​between 0 and 1 and are maintained in the form of probability distribution.

[0099] Distributed probabilistic state estimation is updated as follows: When an agent obtains local observation data such as its own odometry or relative measurements to neighboring nodes, and receives compressed probabilistic state messages from neighboring nodes through an ad hoc network, it runs a Bayesian filtering algorithm locally to fuse the observation likelihood with neighboring node messages, updating the probability distribution of the global state and capability factors. When an agent performs a specific maneuver (such as rapid acceleration or sensor self-check) and observes the corresponding result, the corresponding capability factors are updated using a Bayesian approach based on the observation model. For example, when an agent executes a command with a speed of v... cmd The maneuver was observed and the actual speed measurement value v was recorded. hat At that time, using the observation model p(v) hat |θ v ,v cmd ) for the maximum speed ratio θ v The observation model is updated to assume that the measurements follow a pattern with respect to θ. v ·v cmd It is a Gaussian distribution with mean .

[0100] Step 3: Construct the joint cost function and perform stochastic optimal control to solve. The system aims to minimize the joint cost function within a finite time domain, and solves for stochastic optimal control in a distributed probabilistic state space. The solution employs the probabilistic state model prediction path integral method.

[0101] The construction of the joint cost function includes the following components: (1) Semantic alignment cost: Based on the probabilistic spatiotemporal constraints in the task constraint graph, the degree of matching between the current formation spatial configuration and the commander's intention is evaluated. Taking the equilateral triangle constraint as an example, the Mahalanobis distance between the current configuration and the desired configuration is calculated. A cost is only incurred when this distance exceeds the chi-square quantile corresponding to the lower limit of the constraint satisfaction probability of 0.9. The cost value is the square of the excess. The semantic alignment cost further includes a task progress deviation penalty term, which is calculated based on the deviation between the actual forward distance of each agent and the expected forward distance based on the linear interpolation of the time window constraint. This ensures that even if the spatial configuration is within the tolerance range, excessively slow movement will incur a cost.

[0102] (2) Capability Reliability Cost: Based on the minimum threshold requirements for capabilities of each role in the task constraint diagram, the task execution reliability is calculated by combining the capability factor probability state estimation of the agent. The squared deviation of the reliability being lower than the minimum reliability required by the natural language instruction is used as a penalty term. For example, the forward reconnaissance role requires a maximum speed ratio of not less than 0.7 and a sensor detection performance ratio of not less than 0.8. The joint probability that the three capability factors of the agent simultaneously meet each threshold is calculated as the reliability and compared with the minimum reliability of 0.9 required by the instruction.

[0103] (3) Cooperative Alignment Cost: Extract the key cooperative variables (including relative pose and the other party's current action stage, which is divided into standby, maneuvering, in position, and task execution) for each pair of cooperative agents in the cooperative action relationship. Calculate the Bach distance between the probability state estimates of each pair of cooperative agents regarding the key cooperative variables, and map the Bach distance to an exponentially increasing cost. When the Bach distance is small, the cost is almost negligible; after exceeding the threshold, the cost increases sharply.

[0104] (4) Cost of information uncertainty: This is the weighted sum of position probability state entropy, capability probability state entropy, and cooperation probability state entropy. Position probability state entropy measures the overall uncertainty of the cluster's global pose estimation; capability probability state entropy measures the uncertainty of each agent's perception of its own execution capabilities; cooperation probability state entropy is defined as the sum of the differential entropies of the probability state estimates of all cooperative agents regarding their respective cooperative key variables, measuring the degree of uncertainty of each cooperative party's perception of the cooperative key variables.

[0105] The weights of the weighted sum of information uncertainty costs are determined by the time-sensitive weighting function Λ based on the remaining task time and the current total probability state entropy H. total The time-sensitive weighting function is dynamically determined and takes the following form: Λ(t k H total )=λ0·exp(-α·max(0,T end -t k ) / (β+H total )) Among them, t k For the current decision-making moment, T end Given a target completion time of 180 seconds, H total Let λ be the sum of the position probability state entropy, capability probability state entropy, and cooperation probability state entropy; λ0 be the basic exploration coefficient extracted from the instruction by the visual-language model (0.8); α be 0.05; and β be 0.1. As time t increases... k Gradually approaching T end As Λ monotonically decays, the weight of information uncertainty cost in the joint cost function gradually decreases, and the optimization focus shifts from reducing uncertainty to quickly achieving spatiotemporal constraints.

[0106] During the solution process, in each control update cycle, the algorithm samples and generates multiple candidate control trajectories in the control space based on the current probability state estimate. During the forward propagation of each trajectory, the position probability state and capability probability state are updated synchronously: based on the type of maneuver performed in the trajectory, the probability distribution of the capability factor is updated using the corresponding observation model via Bayesian algorithm. For example, a trajectory containing a rapid acceleration maneuver will generate effective observations about the maximum velocity factor during execution, narrowing the probability distribution of this factor; while a trajectory with uniform motion cannot effectively reduce the uncertainty of the velocity factor. The cumulative cost of each sampled trajectory is evaluated cumulatively in the time domain through a joint cost function, and the control sequence with the lowest cost is selected as the output.

[0107] Step 4: Control sequence distribution and queuing execution The solved control sequence is then distributed to each agent for execution. The control sequence contains a series of timestamped poses and control commands for each agent, and can be directly injected into the autopilot or motion controller. Each agent moves according to the control sequence, forming a formation that conforms to the task constraint graph.

[0108] Step 5: Failure Detection and Self-Healing Reconstruction During formation execution, the system continuously monitors the status of each agent. When any agent fails or becomes abnormal, such as a drone going offline for some reason, the following reconfiguration process is initiated: First, the probabilistic state estimation of the corresponding agent is updated based on observations reflecting failure or state anomalies. If the failed node is confirmed as damaged by the sensors of neighboring nodes, its pose probability distribution is updated to a high-confidence distribution centered on the damaged location, and the capability factor is updated to zero; if it is only a communication interruption and the actual state is unknown, the uncertainty of its pose probability distribution increases rapidly over time.

[0109] Secondly, based on the updated global probability state of the cluster, the stochastic optimal control problem is resolved based on the task constraint graph to generate a reconstructed formation control sequence. During the reconstruction process, the spatial and temporal constraints of the original task constraint graph are maintained. If the original constraints cannot be satisfied due to node failure (e.g., one drone in the original triangular formation loses contact, and the remaining two cannot form a triangle), the system utilizes the elastic space provided by the tolerance variance in the probabilistic spatiotemporal constraints to automatically generate a feasible solution that is closest to the original semantics (e.g., two drones forming a V-shaped formation, which is approximately equivalent to the original triangle in terms of coverage).

[0110] Finally, during reconstruction, the remaining cooperative pairs with the smallest probability state divergence are prioritized. For example, when the original cooperative pair AB collapses due to the failure of A, and a replacement needs to be selected from the remaining nodes, the system evaluates the Baccarat distance between each candidate node and B, and selects the combination with the smallest Baccarat distance to achieve the highest success rate of cooperative actions with the lowest time cost, thus maintaining the coherence of cooperative semantics.

[0111] Example 3 This embodiment further illustrates the key calculation steps in the above process using a numerical approach.

[0112] Assume the current cluster consists of three infrared UAVs (UAV-1, UAV-2, and UAV-3) and two transport vehicles (UGV-1 and UGV-2). The expected completion time T of the task is... end The duration is 180 seconds, with the initial time being t=0.

[0113] At t=30 seconds, the maximum velocity ratio of UAV-1 to itself is θ. v One valid observation was conducted: the commanded speed was 10 m / s, and the measured speed was 8.5 m / s. Based on the observation model, UAV-1 updated the probability distribution of its maximum speed ratio, with the mean changing from the prior 0.95 to 0.88, the variance decreasing from 0.04 to 0.01, and the capability probability state entropy significantly decreasing.

[0114] At time t=60 seconds, the time-sensitive weighting function Λ is calculated as follows: the remaining time is T. end -t k=120 seconds, assuming the current total probability state entropy H_total is 3.2 Knights, λ0=0.8, α=0.05, β=0.1. Substituting into the formula, we get Λ=0.8·exp(-0.05·120 / (0.1+3.2))=0.8·exp(-1.82)≈0.13. The weight of the information uncertainty cost has decreased from the initial approximately 0.8 to 0.13, and the system begins to shift its optimization focus to task execution.

[0115] At t=120 seconds, UAV-2 lost contact due to an unforeseen event. Upon detecting the failure, the system immediately updated the probability state estimate of UAV-2. The remaining infrared UAVs, UAV-1 and UAV-3, could not maintain the original equilateral triangular constraints. The system re-solved the problem based on the current mission constraint graph, utilizing the elastic space provided by the tolerance variance to generate a V-shaped formation alternative. With only 60 seconds remaining, the time-sensitive weight Λ further decayed to near zero. The reconstruction solution was almost entirely focused on quickly achieving the spatial constraints, ensuring formation deployment was completed within the time window.

[0116] During the collaborative reconfiguration, the original collaborative relationship between UAV-2 and UAV-3 was disrupted, while the diamond-shaped protection collaborative relationship between UGV-1 and UGV-2 remained unaffected. The system evaluated the Bartholomew's distance, a key collaborative variable between UAV-1 and UAV-3. Because the two aircraft had established a relatively consistent relative pose recognition through mutual observation in previous executions, and their Bartholomew's distance was small, they were preferentially selected as the new collaborative pair to continue executing the collaborative task.

[0117] Example 4 This embodiment illustrates how the system is deployed at the edge.

[0118] The visual-language model employs a lightweight, compressed and quantized multimodal model, deployed on the embedded GPU or NPU of the lead node, with inference latency controlled within 100 milliseconds. The distributed probabilistic state estimation and maintenance module runs on the flight control computer or associated coprocessor of each agent. Each agent exchanges compressed probabilistic state summary information via periodic broadcasts through a self-organizing network, with communication overhead controlled within 10% of available bandwidth. The probabilistic state model prediction path integral solving module is deployed on the lead node, with the number of sampled trajectories set to 200 to 500 per control cycle. The prediction time domain length is dynamically adjusted between 10 and 30 seconds according to mission requirements, and the control cycle is set to 0.5 seconds to meet real-time formation control requirements.

[0119] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for dynamic formation and swarm intelligence reconfiguration of off-grid heterogeneous intelligent agents, characterized in that, include: Receive natural language instructions, parse the natural language instructions through a visual-language model deployed at the edge, and generate a task constraint graph containing probabilistic spatiotemporal constraints, time window constraints, and cooperative action relationships; Each agent maintains a distributed probabilistic state estimate locally regarding the cluster's global state and capability factors. The global state includes at least the poses of all agents, and the capability factors characterize the ratio of an agent's execution capability to its nominal value. With the goal of minimizing the joint cost function in the finite time domain, stochastic optimal control is solved in the distributed probabilistic state space to generate control sequences for each agent. The joint cost function includes semantic alignment cost, capability reliability cost, cooperative alignment cost, and information uncertainty cost. The information uncertainty cost is a weighted sum of position probability state entropy, capability probability state entropy, and cooperative probability state entropy. The weights of the weighted sum are dynamically determined by a time-sensitive weighting function based on the remaining time of the task and the current total probability state entropy. The control sequence is sent to each agent for execution to drive the heterogeneous cluster to form a formation that conforms to the task constraint graph.

2. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The probabilistic spatiotemporal constraints are composed of the expected value of the geometric parameters of the spatial formation, the tolerance variance, and the lower bound of the constraint satisfaction extracted by the visual-language model from the natural language instructions. They are used to evaluate the semantic compliance of the formation spatial configuration in probabilistic form in the probabilistic state space.

3. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The time window constraint includes at least the expected completion time of the task and the upper limit of the allowed timeout probability. The time-sensitive weight function is configured such that as the current time approaches the expected completion time of the task, the weight of the information uncertainty cost monotonically decays, so as to force the optimization focus to shift from reducing uncertainty to quickly achieving the spatiotemporal constraints.

4. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The capability factor includes at least the maximum speed ratio, sensor detection performance ratio, and communication radius ratio, and the capability reliability cost is constructed in the following manner: Based on the minimum threshold requirement of the role's ability in the task constraint diagram, and combined with the probability state estimation of the agent's ability factor, the reliability of the agent's task execution for that role is calculated. The reliability is compared with the minimum reliability required by the natural language instruction, and the square of any deviation exceeding the allowable deviation is included as a penalty in the semantic alignment cost.

5. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The collaborative alignment cost is constructed as follows: Extract the key cooperative variables for each pair of cooperative agents in the cooperative action relationship, including relative pose and the current action stage of the other party; Calculate the Bach distance between the probability state estimates of each pair of cooperative agents with respect to the cooperative key variables; The Bach distance is mapped to an exponentially growing cost, which serves as the cost of the collaborative alignment.

6. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 5, characterized in that, The cooperative probability state entropy is defined as the sum of the differential entropies of the probability state estimates of all cooperative agents with respect to their respective cooperative key variables, and the cooperative probability state entropy is incorporated into the information uncertainty cost.

7. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The time-sensitive weighting function is in the form of: L(t k ,H total )=λ0·exp(-α·max(0,T end -t k ) / (β+H total )) Where Λ is the time-sensitive weight, t k For the current decision-making moment, T end H is the expected completion time of the task. total λ0 represents the current total probability state entropy, α and β are corresponding positive real numbers, λ0 is the basic exploration coefficient, and is dynamically adjusted by the vision-language model according to the risk-sensitive expressions in the natural language instructions.

8. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The semantic alignment cost further includes a task progress deviation penalty term, which is calculated based on the deviation between the current agent's actual movement progress and the expected progress based on the linear interpolation constrained by the time window, so that even if the spatial configuration is within the tolerance range, excessively slow movement will incur a cost.

9. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The method further includes: When any agent fails or becomes abnormal, the probability state estimate of the corresponding agent is updated based on the observations reflecting the failure or abnormality. Based on the updated cluster global probability state, the stochastic optimal control problem is re-solved based on the task constraint graph to generate a reconstructed formation control sequence. During reconstruction, the remaining cooperative pairs with the smallest probability state divergence are selected first to maintain the coherence of cooperative semantics.

10. The off-grid heterogeneous intelligent agent dynamic formation and swarm intelligence reconstruction method according to claim 1, characterized in that, The stochastic optimal control solution employs the probabilistic state model prediction path integral method. During each control update, multiple control trajectories are sampled based on the current probabilistic state estimate. The position probability state and capability probability state are updated synchronously during the trajectory forward propagation process. The cumulative cost of the trajectory is evaluated using the joint cost function, and the control sequence with the lowest cost is selected for output.

Images