Ship multi-agent cooperative control large model system

Abstract

The application discloses a ship multi-agent cooperative control large model system, and belongs to the technical field of cross of artificial intelligence and ocean engineering. The system comprises a multi-source marine situation awareness module, a semantic cognitive decision-making large model module, a distributed cooperative control module and a dynamic feedback optimization module. The multi-source marine situation awareness module collects multi-source heterogeneous data and performs space-time feature extraction and semantic coding. The semantic cognitive decision-making large model module performs scene semantic analysis and intention inference based on a pre-trained large language model, and generates cooperative decision-making instructions. The distributed cooperative control module decomposes the decision-making instructions into local control strategies and realizes consistent execution. The dynamic feedback optimization module evaluates the cooperative effect and dynamically adjusts the system parameters, forming a closed-loop feedback optimization mechanism. The application realizes intelligent cooperative control of a ship formation by constructing a deep coupling closed-loop cooperative architecture, and has strong scene generalization ability and self-adaptive optimization ability.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and marine engineering, and in particular to a large-scale model system for collaborative control of multiple intelligent agents on ships. Background Technology

[0002] With the rapid development of intelligent shipping technology, autonomous ship navigation and collaborative operations of unmanned surface vessels have become important research directions in the field of marine engineering. In complex mission scenarios such as port inspection, marine resource exploration, and maritime emergency rescue, traditional single-ship control modes are insufficient to meet the requirements of efficiency, flexibility, and reliability in mission execution. Multi-ship collaborative control technology, by enabling information sharing, task coordination, and behavior synchronization among multiple intelligent ship agents, can significantly improve operational capabilities in complex marine environments.

[0003] In the prior art, Chinese invention patent CN108229685A discloses an air-ground integrated unmanned intelligent decision-making method. This method achieves intelligent decision support for unmanned systems by combining an air-ground collaborative working mode with knowledge construction, evolution and use based on a knowledge center, as well as a collaborative reasoning engine of rule reasoning and model reasoning. However, this technical solution has the following shortcomings: First, the solution is mainly designed for air-ground collaborative scenarios, and its knowledge representation and reasoning mechanisms are difficult to adapt to the complexity and uncertainty of the marine environment, especially lacking targeted design in areas such as changing sea conditions, dynamic ship interactions, and obstacle avoidance on the water surface. Second, the solution adopts a traditional rule-based reasoning and model-based reasoning collaborative mechanism, and knowledge acquisition relies on predefined rule and model libraries, which has limited generalization ability for unknown scenarios and makes it difficult to achieve deep understanding and intent inference of complex marine semantic information. Third, the multi-agent collaboration of this solution mainly relies on static task planning and resource allocation, lacking a dynamic and adaptive collaborative decision-making mechanism. When faced with sudden changes in sea conditions or adjustments in task requirements, the system's response efficiency and collaborative accuracy are severely constrained. Finally, the solution lacks effective utilization of the cognitive capabilities of large language models, and cannot achieve scene understanding and decision generation based on natural language semantics, thus limiting the system's intelligence level in complex multimodal marine information processing.

[0004] In recent years, large language model technology has demonstrated outstanding performance in natural language processing, knowledge reasoning, and decision support. Research shows that large language models based on the Transformer architecture can effectively capture long-range dependencies in text, achieving semantic-level information understanding and reasoning. In cutting-edge international research, a ship trajectory prediction framework based on semantic cognition and intent context awareness significantly improves the accuracy of long-term trajectory prediction by combining the semantic cognitive capabilities of large language models with the temporal modeling capabilities of deep learning models. Recent advances in multi-agent collaboration demonstrate that multi-agent systems based on large language models can achieve flexible coordinated behavior and emergent swarm intelligence through natural language as a communication medium. Research on the application of deep reinforcement learning technology in collaborative path planning for unmanned surface vessel swarms proves that distributed decision-making frameworks combined with adaptive learning mechanisms can effectively cope with high-dimensional, dynamic, and multi-constrained marine mission environments. However, how to deeply integrate the powerful cognitive capabilities of large language models with multi-agent collaborative control technology to construct an intelligent collaborative control system for ship formations remains a pressing technical challenge. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a large-scale model system for collaborative control of ships by multiple intelligent agents. By constructing a deep-coupled closed-loop architecture of a semantic cognitive decision-making model and distributed collaborative control, intelligent collaborative control of ship formations can be achieved.

[0006] This invention provides a large-scale model system for multi-agent cooperative control of ships, comprising:

[0007] The multi-source marine situation awareness module is used to collect the self-state data, surrounding environment data and communication interaction data of each intelligent agent in the ship formation, and to extract spatiotemporal features and semantically encode the collected multi-source heterogeneous data to generate a structured marine situation representation vector.

[0008] The semantic cognitive decision-making big model module is connected to the multi-source ocean situation awareness module. It receives ocean situation representation vectors, and based on the semantic understanding ability of the pre-trained big language model, it performs scene semantic analysis and intent inference on the current ocean situation. Combined with task objective constraints, it generates multi-agent collaborative decision-making instructions.

[0009] The distributed collaborative control module is connected to the semantic cognitive decision-making big model module. It receives multi-agent collaborative decision-making instructions, decomposes the decision instructions into local control strategies of each agent based on the distributed control architecture, achieves consistent execution of collaborative behavior through information interaction between agents, and outputs motion control instructions and collaborative state information of each agent.

[0010] The dynamic feedback optimization module is connected to the distributed collaborative control module, the multi-source marine situational awareness module, and the semantic cognitive decision-making big model module, respectively. It receives collaborative status information, evaluates the deviation between the collaborative execution effect and the expected goal, and dynamically adjusts the feature extraction parameters of the multi-source marine situational awareness module and the inference strategy parameters of the semantic cognitive decision-making big model module based on the deviation information, forming a closed-loop feedback optimization mechanism.

[0011] The beneficial effects of this invention are as follows: First, by introducing a large language model as the core decision engine, the system can achieve a deep understanding of complex marine semantic information, exhibiting stronger scene generalization ability and intent inference accuracy compared to traditional rule-based reasoning methods. Second, by constructing a deeply coupled closed-loop architecture among the multi-source marine situational awareness module, the semantic cognitive decision-making large model module, the distributed collaborative control module, and the dynamic feedback optimization module, close correlations are formed between the modules at the parameter and state levels, achieving synergistic effects of perception, decision-making, execution, and optimization, with the overall system performance exhibiting a non-linear improvement characteristic of 1+1>2. Third, the dynamic feedback optimization module can adaptively adjust system parameters based on real-time collaborative effects, enabling the system to quickly adapt to changing marine environments and mission requirements, significantly improving the accuracy and efficiency of collaborative control. Fourth, the system adopts a distributed collaborative control architecture, where each agent can independently execute local control strategies and achieve global collaborative optimization through information interaction, exhibiting good scalability and robustness. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the overall structure of the large-scale ship multi-agent cooperative control system of the present invention;

[0013] Figure 2 This is a detailed structural diagram of the multi-source marine situational awareness module of the present invention;

[0014] Figure 3 This is a detailed structural diagram of the semantic cognitive decision-making large model module of the present invention;

[0015] Figure 4 This is a detailed structural diagram of the distributed collaborative control module of the present invention;

[0016] Figure 5 This is a detailed structural diagram of the dynamic feedback optimization module of the present invention. Detailed Implementation

[0017] Please refer to the attached document. Figures 1-5 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0018] Reference Figure 1The ship multi-agent cooperative control large-scale model system of the present invention includes a multi-source marine situational awareness module 1, a semantic cognitive decision-making large-scale model module 2, a distributed cooperative control module 3, and a dynamic feedback optimization module 4. These four modules constitute a deeply coupled closed-loop cooperative system. The multi-source marine situational awareness module 1 is responsible for data acquisition and feature extraction, and its output serves as the input to the semantic cognitive decision-making large-scale model module 2. The semantic cognitive decision-making large-scale model module 2 performs semantic understanding and decision generation based on a large language model, and its output drives the distributed cooperative control module 3 to execute cooperative control. The execution status of the distributed cooperative control module 3 is fed back to the dynamic feedback optimization module 4. The dynamic feedback optimization module 4 adjusts the parameters of the multi-source marine situational awareness module 1 and the semantic cognitive decision-making large-scale model module 2 in reverse according to the evaluation results, forming a complete closed-loop optimization mechanism.

[0019] Reference Figure 2 The multi-source marine situational awareness module 1 includes a multi-sensor data acquisition unit, a spatiotemporal feature extraction unit, and a semantic coding unit.

[0020] The multi-sensor data acquisition unit is used to acquire multi-source heterogeneous data from each agent in the ship formation. Specifically, the ship's own state data includes motion parameters such as position coordinates, heading angle, speed, acceleration, and rudder angle, as well as operational parameters such as fuel consumption, load, and equipment status. Surrounding environment data includes environmental features such as the distribution of sea surface obstacles, the positions and trajectories of other ships, sea state information (such as wind speed, wave height, current direction and speed), and visibility, acquired through sensors such as radar, sonar, and cameras. Communication interaction data includes interactive content such as intent information, cooperation requests, and status reports exchanged between agents through the shipborne automatic identification system and dedicated communication links. In one embodiment of the invention, the data acquisition frequency is set to 10Hz, which can meet the timeliness requirements of real-time collaborative control.

[0021] The spatiotemporal feature extraction unit performs temporal modeling and spatial relationship encoding on the collected multi-source heterogeneous data. Temporal modeling employs a sliding time window method, organizing the state data from consecutive moments into a time series format. Preferably, the time window length is set to 30 sampling periods, i.e., 3 seconds of historical data, which captures sufficient motion trend information while avoiding the computational burden of excessively long historical data. Spatial relationship encoding converts the absolute positions of each agent into relative positional relationships, constructing a spatial topology between agents. In a preferred embodiment of the invention, a graph structure is used to represent the spatial relationships between agents, where nodes represent agents and edges represent communication connections and relative distances between agents.

[0022] The semantic encoding unit transforms the extracted spatiotemporal features into semantic representations suitable for processing by large language models. Specifically, numerical spatiotemporal features are converted into structured natural language descriptions using cueing engineering techniques. For example, the state vector of an agent is encoded as the semantic statement: "Agent A's current position is 30.5 degrees North latitude, 122.3 degrees East longitude, heading 225 degrees, speed 12 knots. There are two ships within a 3-nautical-mile radius, located 1.2 nautical miles to its port side at a 45-degree angle and 2.1 nautical miles to its starboard side at a 30-degree angle." Simultaneously, the semantic encoding unit generates a structured ocean situational awareness vector, which comprises both spatiotemporal numerical features and semantic textual descriptions, providing multimodal input for subsequent semantic cognitive decision-making.

[0023] In one embodiment of the present invention, the multi-source marine situational awareness module 1 also has an adaptive sampling function. When the dynamic feedback optimization module 4 reports insufficient sensing accuracy, this module can automatically increase the sampling frequency of a specific area or a specific intelligent agent, thereby achieving dynamic optimization of sensing resources.

[0024] Reference Figure 3 The semantic cognitive decision-making big model module 2 includes a scene semantic parsing unit, an intent inference unit, and a decision generation unit.

[0025] The scene semantic parsing unit receives ocean situation representation vectors from the multi-source ocean situation awareness module 1 and uses the semantic understanding capabilities of a pre-trained large language model to perform deep analysis of the current ocean scene. In a preferred embodiment of the invention, a large language model based on the Transformer architecture is used as the core inference engine. The scene semantic parsing unit first fuses the input multimodal features, aligning and integrating numerical spatiotemporal features with semantic text descriptions. Then, through the contextual understanding capabilities of the large language model, it identifies key elements of the current scene, including the overall situation of the ship formation, the relative positional relationships of each agent, potential collision risk areas, and favorable navigation channels. The results of the scene semantic parsing are output in the form of a structured scene description, including scene type labels, a list of key elements, and a situation assessment score.

[0026] The intent inference unit infers the future behavioral intentions of each agent based on scene semantic parsing results and historical behavior data. Intent inference is a crucial step in achieving multi-agent collaboration; only by accurately understanding the potential intentions of each agent can effective collaborative strategies be formulated. In this invention, the intent inference unit employs an innovative semantically enhanced spatiotemporal intent reasoning algorithm. This algorithm first constructs an intent semantic space, mapping the ship's possible behavioral intentions into semantic vectors, including typical intent types such as maintaining straight course, turning left to avoid a collision, turning right to avoid a collision, accelerating to overtake, decelerating to give way, and emergency stopping. Then, based on the reasoning capabilities of a large language model, combined with the current scene semantics and historical behavior trajectories, it calculates the probability distribution of each agent in the intent semantic space.

[0027] The core innovation of semantically enhanced spatiotemporal intent reasoning lies in the introduction of semantic prior knowledge and spatiotemporal association constraints. Specifically, the computational process of intent inference is as follows:

[0028] ,

[0029] in, In the current scenario and historical trajectory Under the conditions The posterior probability of such intention, For the purpose semantic embedding vector, This is the semantic embedding vector for the current scene. This is a function for calculating semantic similarity. Scoring is given based on the consistency between historical trajectory and intention. For semantic weight coefficients, For trajectory weighting coefficients, This represents the total number of candidate intent types. In a preferred embodiment, The value is 0.6. A value of 0.4 allows for a good balance between semantic understanding and trajectory analysis.

[0030] The decision generation unit generates multi-agent collaborative decision-making instructions based on the scene semantic parsing results and intent inference results, combined with the current task objective constraints. Task objective constraints include target position, arrival time requirements, formation requirements, safe distance constraints, and energy consumption limits. The decision generation unit adopts a hierarchical decision architecture. First, at the global level, it determines the overall navigation strategy of the formation, including target heading, formation speed, and formation adjustment direction. Then, at the local level, it assigns specific behavioral instructions to each agent, including heading adjustment amounts, speed adjustment amounts, and collaborative actions with neighboring agents.

[0031] In one embodiment of the present invention, the decision generation unit utilizes the reasoning capabilities of a large language model to perform multi-step decision planning. Specifically, by constructing a prompt template, the current scene, intent inference results, and task constraints are organized into structured input text, and then the large language model is requested to generate a decision scheme. Based on its rich knowledge and reasoning capabilities acquired through pre-training, the large language model can comprehensively consider multiple factors and generate reasonable and executable collaborative decision instructions. Preferably, the decision generation process employs a bundle search strategy, simultaneously generating multiple candidate decision schemes, and then selecting the optimal scheme based on multi-dimensional evaluation indicators such as safety, efficiency, and feasibility.

[0032] Reference Figure 4 The distributed collaborative control module 3 includes a decision decomposition unit, an information interaction unit, and a consistency execution unit.

[0033] The decision decomposition unit receives multi-agent collaborative decision-making instructions from the semantic cognitive decision-making big model module 2, and decomposes the global decision into local control strategies for each agent. The decomposition process needs to consider the heterogeneity of each agent, including factors such as ship type, performance parameters, and current state. In a preferred embodiment of the invention, the decision decomposition adopts a role-based task allocation mechanism. First, role labels are assigned to agents based on their capability characteristics, such as lead ship, follower ship, and reconnaissance ship; then, the global decision is mapped to the corresponding local control strategy according to the role definition. For example, when the global decision is "the formation turns 30 degrees to the left," the lead ship needs to first execute the turning action and broadcast the turning information, and the follower ship needs to calculate its own turning parameters based on its relative position to the lead ship and the preset formation.

[0034] The information interaction unit is responsible for managing communication and data exchange between the agents. In a distributed cooperative control architecture, information interaction is the foundation for achieving cooperative consistency. This invention adopts a hybrid communication topology, combining the advantages of centralized and distributed communication. Specifically, the system sets up a virtual coordinator responsible for maintaining global state information and coordinating major decisions, while agents can communicate directly point-to-point to exchange local information. The information interaction unit defines standardized communication protocols, including state synchronization messages, cooperation request messages, and decision confirmation messages. The communication frequency is dynamically adjusted according to the urgency of the task, normally 2Hz, and can be increased to 10Hz in emergency situations.

[0035] In one embodiment of the present invention, the information interaction unit also implements a communication fault tolerance mechanism. When a communication link interruption of a certain agent is detected, the system automatically activates a backup communication scheme, such as information relay through neighboring agents, to ensure the continuity of collaborative control. Furthermore, the information interaction unit employs timestamp and sequence number mechanisms to handle communication delays and message out-of-order issues, ensuring information consistency and real-time performance.

[0036] The consensus execution unit is responsible for coordinating the behavior of each agent, ensuring that the implementation of local control strategies achieves the global collaborative goal. The core challenge of consensus execution lies in achieving global optimization in a distributed environment. This invention employs an innovative multi-agent collaborative coupling degree calculation method based on an attention mechanism. This method first calculates the collaborative coupling degree between agents, reflecting the degree of collaboration and mutual influence among them; then, it dynamically adjusts the execution priority and coordination weight of the control strategy based on the coupling degree.

[0037] The calculation process for the degree of cooperative coupling is as follows:

[0038] ,

[0039] in, For intelligent agents For intelligent agents The degree of cooperative coupling, For intelligent agents The query vector, For intelligent agents The key vector, For intelligent agents The value vector, The dimension of the key vector, superscript This represents the transpose of a vector. For intelligent agents and The Euclidean distance between them This is the distance attenuation coefficient. In a preferred embodiment, The value is 64. The value is set to 500m. This configuration effectively captures the cooperative relationships between agents while avoiding excessive influence from distant agents. The formula captures semantic associations between agents through an attention mechanism, and introduces spatial constraints through a Gaussian distance decay function, thus achieving an organic unity of semantic and spatial cooperation.

[0040] The consensus execution unit adjusts the control command execution parameters of each agent based on the calculated cooperative coupling degree. Specifically, when an agent's cooperative coupling degree is high, its control strategy execution needs to consider the state and intentions of neighboring agents more to ensure the consistency of cooperative actions; when the cooperative coupling degree is low, the agent can execute its local control strategy more independently. Furthermore, the consensus execution unit implements a cooperative lockstep mechanism. For cooperative actions requiring synchronous execution, such as formation changes and cooperative obstacle avoidance, a distributed consensus algorithm ensures the temporal coordination and synchronization of each agent.

[0041] Reference Figure 5The dynamic feedback optimization module 4 includes a collaborative effect evaluation unit, a deviation analysis unit, and a parameter adjustment unit.

[0042] The collaborative performance evaluation unit receives collaborative status information from the distributed collaborative control module 3 and performs a multi-dimensional evaluation of the current collaborative execution effect. Evaluation indicators include formation maintenance accuracy, task completion progress, energy efficiency, and safety margin. In a preferred embodiment of the invention, the collaborative performance evaluation employs a composite evaluation function, weighting and combining multiple individual indicators into a total score. The weight of each indicator is dynamically determined according to the task type; for example, in emergency rescue tasks, the weight of task completion progress is set higher; in routine inspection tasks, the weight of energy efficiency is set higher.

[0043] The formation maintenance accuracy is assessed based on the deviation between the actual formation position and the target formation position. Specifically, the Euclidean distance between the actual and target positions of each agent is calculated, and the root mean square of the deviations of all agents is taken as the formation maintenance accuracy index. Task completion progress is determined by calculating the ratio of the number of completed sub-tasks to the total number of sub-tasks. Energy efficiency is calculated based on fuel or electricity consumption per unit distance. Safety margin is assessed by statistically analyzing the minimum safe distances between each agent and obstacles or other vessels during the collaborative process.

[0044] The deviation analysis unit performs in-depth analysis of the collaborative effect evaluation results to identify the main factors causing the deviations. The deviation analysis employs attribution analysis, decomposing the collaborative effect deviations into three components: perception deviation, decision-making deviation, and execution deviation. Perception deviation reflects the accuracy and completeness of feature extraction from the multi-source marine situational awareness module 1; decision-making deviation reflects the rationality and effectiveness of decisions made by the semantic cognitive decision-making big model module 2; and execution deviation reflects the precision and reliability of control execution by the distributed collaborative control module 3. Through deviation attribution analysis, the system can accurately pinpoint performance bottlenecks, providing a clear direction for subsequent parameter adjustments.

[0045] In one embodiment of the present invention, the deviation analysis unit also implements a trend prediction function. By analyzing deviation data over multiple consecutive time periods, the system predicts the development trend of the deviation and achieves feedforward optimization. For example, when a continuous downward trend in formation maintenance accuracy is detected, the system proactively adjusts relevant parameters before the deviation exceeds the allowable range, thus avoiding significant fluctuations in coordination performance.

[0046] Based on the deviation analysis results, the parameter adjustment unit dynamically adjusts the feature extraction parameters of the multi-source marine situational awareness module 1 and the inference strategy parameters of the semantic cognitive decision-making big model module 2. The parameter adjustment employs an innovative adaptive gradient optimization method. The core idea of ​​this method is to adaptively calculate the parameter adjustment amount according to the deviation type and magnitude, achieving rapid convergence while avoiding parameter oscillations.

[0047] The calculation process for adaptive weight optimization is as follows:

[0048] ,

[0049] in, For parameter adjustment amount, Based on the learning rate, loss function For parameters gradient, For the target loss value, For adaptive coefficients, It is the hyperbolic tangent function. This refers to the temperature parameter. In a preferred embodiment, The value is 0.001. The value is 0.5. The value is set to 0.1. This formula achieves adaptive adjustment of the deviation through the hyperbolic tangent function. When the deviation between the actual loss and the target loss is large, the parameter adjustment amount increases accordingly, accelerating the convergence process; when the deviation is small, the parameter adjustment amount tends to be stable, avoiding oscillations caused by over-adjustment.

[0050] For the multi-source marine situational awareness module 1, the parameter adjustment unit mainly adjusts the weight parameters of feature extraction, including the fusion weights of different sensor data, the length of the time window, and the resolution of spatial relationship encoding. For the semantic cognitive decision-making large model module 2, the parameter adjustment unit mainly adjusts the inference strategy parameters, including the semantic weight coefficients and trajectory weight coefficients for intent inference, the beam search width for decision generation, and the risk threshold setting. Through continuous parameter optimization, the system can adaptively adjust its behavior to match the current marine environment and mission requirements.

[0051] The data flow and closed-loop feedback mechanism of this invention system is as follows: First, the multi-source marine situational awareness module 1 collects and processes multi-source heterogeneous data to generate a marine situational representation vector; then, this representation vector is input to the semantic cognitive decision-making big model module 2 for semantic parsing and intent inference, generating multi-agent collaborative decision-making instructions; next, the distributed collaborative control module 3 decomposes and executes the decision instructions, outputting motion control instructions and collaborative status information; finally, the dynamic feedback optimization module 4 evaluates the collaborative effect and adjusts the parameters of the preceding modules. The entire process forms a complete closed loop of "perception → decision → execution → evaluation → optimization," and the deep coupling between the modules ensures the synergistic efficiency of the system.

[0052] In specific embodiments of the present invention, the deep coupling between the modules is manifested at multiple levels. First, there is coupling at the data level: the ocean situation representation vector output by the multi-source ocean situation awareness module 1 comprises two components—spatiotemporal numerical features and semantic text descriptions. These two components correspond to the input requirements of the scene semantic parsing unit and the intent inference unit in the semantic cognitive decision-making large model module 2, respectively, forming precise data integration. Second, there is coupling at the parameter level: the parameter adjustment amount output by the dynamic feedback optimization module 4 directly affects the feature extraction weights of the multi-source ocean situation awareness module 1 and the inference strategy coefficients of the semantic cognitive decision-making large model module 2, achieving real-time parameter-level optimization. Finally, there is coupling at the state level: the execution state information of the distributed cooperative control module 3 not only includes the motion control results of each agent but also cooperative features such as cooperative coupling degree and formation maintenance accuracy. This information constitutes the core basis for the dynamic feedback optimization module 4 to evaluate the cooperative effect.

[0053] In one embodiment of the present invention, the system employs a hierarchical information transmission mechanism to ensure efficient collaboration between modules. Specifically, the multi-source ocean situational awareness module 1 transmits ocean situational representation vectors to the semantic cognitive decision-making large model module 2 at a frequency of 10Hz; the semantic cognitive decision-making large model module 2 transmits collaborative decision-making commands to the distributed collaborative control module 3 at a frequency of 2Hz; the distributed collaborative control module 3 transmits motion control commands to each agent at a frequency of 20Hz, and simultaneously transmits collaborative state information to the dynamic feedback optimization module 4 at a frequency of 1Hz; the dynamic feedback optimization module 4 feeds back parameter adjustment information to the front-end module at a frequency of 0.5Hz. This differentiated information transmission frequency design ensures the response speed of control links with high real-time requirements while avoiding excessive consumption of system resources.

[0054] In another embodiment of the present invention, the multi-sensor data acquisition unit of the multi-source marine situational awareness module 1 sets differentiated data processing strategies for different types of sensors. For shipborne radar data, a constant false alarm rate (CFAR) detection algorithm is used for target extraction. The detection threshold is dynamically adjusted according to the sea state level. When the sea state level is 1-3, the threshold is set to a lower value to improve detection sensitivity, and when the sea state level is 4-6, the threshold is set to a higher value to reduce the false alarm rate. For shipborne AIS data, a data integrity verification and outlier filtering strategy is adopted. When there is a large deviation between the ship position reported by AIS and the radar detection result, the system automatically marks the target as a suspected data anomaly target and increases the monitoring frequency of the target. For camera visual data, a target detection algorithm based on convolutional neural networks is used, which can identify various target types such as sea surface buoys, ships, and people in the water. The detection confidence threshold is set to 0.7. Detection results below this threshold will be filtered out to reduce false alarms.

[0055] The spatiotemporal feature extraction unit employs a dynamic graph update mechanism when constructing the spatial topology. Node weights in the graph structure are dynamically adjusted based on the importance of the agents. The weight of the mother ship node is set to 1.0, while the weight of the unmanned surface vessel node is dynamically calculated based on its current task load and remaining energy, ranging from 0.5 to 0.9. Edge weights in the graph structure are determined based on the communication quality and coordination tightness between agents. Communication quality is comprehensively evaluated using signal strength and packet error rate, while coordination tightness is calculated using the success rate of historical collaborative tasks. When the communication link quality of an agent drops below a preset threshold, the system automatically adjusts the graph structure, maintaining overall network connectivity by adding relay nodes or adjusting communication paths.

[0056] The semantic encoding unit employs a template-based prompting engineering strategy when converting numerical features into natural language descriptions. The system predefines various scenario templates, including normal navigation scenario templates, collision avoidance scenario templates, formation adjustment scenario templates, and emergency situation scenario templates. The semantic encoding unit automatically selects the appropriate template based on the currently detected scenario features and fills the corresponding positions in the template with specific numerical parameters. For example, the collision avoidance scenario template structure is: "Agent [ID] detects [number] potential collision targets within [distance] range. The nearest target is located at [distance] in the [direction] direction, with a relative speed of [speed], and is expected to reach the nearest encounter point in [time]." This template-based semantic encoding strategy ensures both the standardization and consistency of semantic descriptions and facilitates subsequent semantic understanding and reasoning by large language models.

[0057] In a specific embodiment of the present invention, the scene semantic parsing unit of the semantic cognitive decision-making big model module 2 adopts a multi-turn dialogue reasoning mode. The first round of dialogue is used to obtain an overall overview of the scene. The system inputs the semantic description part of the ocean situation representation vector into the big language model and requests the model to identify the type and main features of the current scene. The second round of dialogue is used to analyze key elements in depth. Based on the scene type identification results of the first round, the system constructs targeted prompt questions to guide the model to analyze the key risk points and opportunity elements under this type of scene. The third round of dialogue is used to comprehensively assess the situation. The system requests the model to integrate the analysis results of the first two rounds and output a structured scene description. This multi-turn dialogue reasoning mode can give full play to the deep reasoning ability of the big language model and avoid the problems of incomplete understanding or insufficient analysis that may exist in single-turn dialogue.

[0058] The intent inference unit employs a hierarchical intent classification system when constructing the intent semantic space. Top-level intent types include four main categories: forward movement, turning, speed adjustment, and emergency response. Mid-level intent types further subdivide the top-level categories; for example, turning is further divided into left turn and right turn. Bottom-level intent types quantify and refine the mid-level categories; for example, left turn is further subdivided into small-amplitude left turn (turning angle less than 30 degrees), medium-amplitude left turn (turning angle between 30 and 60 degrees), and large-amplitude left turn (turning angle greater than 60 degrees). This hierarchical intent classification system ensures fine-grained intent inference while allowing for flexible adjustment of the inference granularity in scenarios with varying precision requirements. In emergency scenarios with high real-time requirements, the system can infer only the top-level intent to accelerate decision-making; in fine-grained control scenarios with high precision requirements, the system can infer the bottom-level intent to provide more accurate collaborative strategies.

[0059] The decision generation unit employs a constraint-satisfaction optimization framework when generating multi-agent collaborative decision-making instructions. The system first transforms the task objective constraints into an objective function and constraints for a mathematical optimization problem. Then, it uses the reasoning capabilities of a large language model to find feasible solutions that satisfy all constraints. The objective function typically includes multiple optimization objectives such as minimizing task completion time, minimizing energy consumption, and maximizing safety margin. Constraints include minimum safe distance constraints between agents, constraints on the movement capabilities of each agent, and constraints on communication range. The large language model generates collaborative decision-making schemes that satisfy the constraints by analyzing the semantic relationships between the objective function and the constraints. In a preferred embodiment, the decision generation unit simultaneously generates 3 to 5 candidate schemes, each with a different optimization focus, such as scheme A focusing on time efficiency, scheme B focusing on energy saving, and scheme C focusing on safety. Then, the most suitable scheme is selected based on the priority of the current task.

[0060] The decision decomposition unit of the distributed cooperative control module 3 considers the heterogeneity of agents when mapping global decisions to local control strategies. Different types of agents have different motion performance parameters, such as maximum speed, turning radius, and acceleration limits. The decision decomposition unit maintains an agent capability database, storing the performance parameters and current state information of each agent. Upon receiving a global decision command, the decision decomposition unit first queries the capability parameters of each agent, and then adaptively decomposes the global decision based on the parameter differences. For example, when the global decision requires the formation to advance towards a target at a speed of 15 knots, for an agent with a maximum speed of 20 knots, its local control strategy is directly set to 15 knots; for an agent with a maximum speed of only 12 knots, its local control strategy is set to 12 knots, and the system adjusts the formation to adapt to the speed limit of that agent.

[0061] The hybrid communication topology of the information interaction unit follows the principle of balancing reliability and efficiency. A virtual coordinator, deployed on the mothership, is responsible for maintaining global formation status information, coordinating major decision-making conflicts, and distributing global mission instructions. Each unmanned surface vessel (USV) exchanges local information via point-to-point communication, including its position and motion status, local environmental perception results, and collaborative execution progress. The information interaction unit implements a priority queue mechanism, categorizing communication messages into three levels based on urgency: high priority (e.g., collision warnings, emergency avoidance instructions), medium priority (e.g., routine status synchronization, mission progress reports), and low priority (e.g., performance optimization suggestions, non-urgent maintenance information). High-priority messages can preempt communication resources for priority transmission, ensuring the real-time nature of urgent information. Furthermore, the information interaction unit employs message compression and incremental update strategies to reduce communication bandwidth consumption, transmitting only information that has changed since the previous moment, achieving a compression ratio of over 60%.

[0062] The cooperative lockstep mechanism of the consistency execution unit ensures precise temporal coordination of cooperative actions that require synchronous execution. This mechanism employs a distributed consensus algorithm, where agents exchange timestamps and synchronization signals to achieve consistency in execution timing. Specifically, when the system initiates a cooperative action requiring synchronous execution, the virtual coordinator first broadcasts a synchronization initiation signal. Upon receiving the signal, each agent reports its readiness status to the coordinator. Once all agents report readiness, the coordinator broadcasts a unified execution time, and each agent simultaneously executes the cooperative action at the specified time. The cooperative lockstep mechanism also implements fault tolerance. If an agent fails to report readiness within a specified time, the system automatically determines that the agent is abnormal, removes it from the current synchronous action, and initiates an anomaly diagnosis process. In a preferred embodiment, the timing accuracy of synchronous execution can reach within 50ms, meeting the requirements of high-precision control tasks such as precise formation changes and cooperative obstacle avoidance.

[0063] The collaborative effect evaluation unit of the dynamic feedback optimization module 4 adopts a sliding window evaluation strategy. The evaluation window length is set to 60 seconds, and the system calculates the average value of each performance indicator within the window in each evaluation cycle. This sliding window evaluation strategy can reflect the overall trend of collaborative effect while avoiding the impact of random fluctuations in single sampling data on the evaluation results. The weights of the evaluation indicators are automatically configured according to the task type. The weights for port inspection tasks are configured as follows: coverage efficiency 0.4, energy efficiency 0.3, safety margin 0.2, and formation maintenance 0.1; the weights for emergency rescue tasks are configured as follows: task completion progress 0.5, safety margin 0.3, response speed 0.15, and resource utilization 0.05. The weight configurations are stored in the system's task configuration database, and the corresponding weight configurations are automatically loaded when the task type changes.

[0064] The attribution analysis method of the deviation analysis unit adopts a gradient backtracking strategy. Specifically, the system traces the deviation of the collaborative effect backward along the data flow path to analyze the propagation and amplification of the deviation at each stage. Perception deviation is identified by comparing the consistency between the situational representation vector output by the perception module and the actual environmental state; a consistency score below 0.85 indicates the existence of a perception deviation. Decision deviation is identified by analyzing the matching degree between the execution effect of the decision command and the expected effect; a matching degree below 0.8 indicates the existence of a decision deviation. Execution deviation is identified by comparing the deviation between the actual execution trajectory of the agent and the target execution trajectory; a trajectory deviation exceeding a preset threshold indicates the existence of an execution deviation. The results of the attribution analysis are output as percentages, representing the contribution of each type of deviation to the overall deviation, providing precise guidance for subsequent parameter adjustments.

[0065] The parameter adjustment unit employs a layered adjustment strategy when adjusting the parameters of the multi-source marine situational awareness module 1. To address insufficient perception accuracy, the data acquisition frequency parameter is first adjusted, increasing the sampling frequency of the target area by 20% to 50%. If the effect is still unsatisfactory, the feature extraction parameters are further adjusted, including the time window length and spatial resolution. Finally, the semantic encoding parameters are adjusted to optimize the level of detail in the prompt template. For the parameter adjustment of the semantic cognitive decision-making large model module 2, this mainly includes adjusting the weight coefficients for intent inference and the search parameters for decision generation. When the accuracy of intent inference decreases, the system analyzes the type of deviation between historical inference results and actual intent. If the deviation mainly comes from semantic understanding, the semantic weight coefficient is increased; if the deviation mainly comes from trajectory prediction, the trajectory weight coefficient is increased. The search parameters for decision generation include the beam search width and temperature parameter. When decision quality decreases, the system increases the beam search width to generate more candidate solutions; when decision diversity is insufficient, the system increases the temperature parameter to increase the diversity of solutions.

[0066] In one specific application embodiment of the present invention, a large-scale model system for multi-agent collaborative control of ships is applied to port inspection tasks. The task scenario is the inspection of the waters of a large port, with the formation consisting of one mother ship and four unmanned surface vessels (USVs). The mother ship acts as the central coordinator, responsible for overall task planning and system monitoring, while the USVs act as the executing agents, responsible for specific inspection operations. The task objective is to complete a full-coverage inspection of the port waters within 4 hours, while avoiding busy waterways and anchored vessels.

[0067] In this application scenario, the multi-source marine situational awareness module 1 collects environmental data through radar, cameras, and AIS receivers mounted on each intelligent agent. Each unmanned surface vessel collects its own position and motion status, the distribution of obstacles within a 100m radius, and information on other vessels obtained via AIS. The data acquisition frequency is 10Hz, and the time window is set to 3s. The spatiotemporal feature extraction unit organizes the data from 30 consecutive time points into temporal features and simultaneously constructs a spatial relationship graph among the five intelligent agents. The semantic encoding unit converts the numerical features into natural language descriptions, generating a structured marine situational awareness vector.

[0068] After receiving the situational representation vector, the semantic cognitive decision-making module 2 first identifies the current scene type as "port inspection - complex obstacle environment" and extracts key elements including "3 anchored vessels", "1 busy waterway", and "multiple buoys". The intent inference unit, based on a semantically enhanced spatiotemporal intent reasoning algorithm, infers that the intent of each unmanned surface vessel is "maximize inspection coverage", while also predicting possible vessel traffic flows. The decision generation unit, combined with task constraints, generates collaborative decision instructions, including specific allocation schemes such as "the formation adopts a dispersed search formation", "Agent 1 is responsible for the northern region", and "Agent 2 is responsible for the eastern region".

[0069] The decision decomposition unit of the distributed cooperative control module 3 decomposes the global partitioning task into inspection path planning tasks for each agent. The information interaction unit coordinates boundary coordination communication between agents to avoid overlap and omission of inspection areas. The consistency execution unit ensures the coordinated cooperation of adjacent agents in the boundary area through cooperative coupling degree calculation. For example, when agent 1 and agent 2 meet in the boundary area, the system automatically adjusts their navigation paths according to the cooperative coupling degree to avoid collisions while maximizing coverage efficiency.

[0070] The dynamic feedback optimization module 4 continuously monitors the collaborative effect. When a decrease in coverage efficiency is detected in a certain area, the deviation analysis unit identifies the cause as the presence of unexpected dynamic obstacles in that area. The parameter adjustment unit then adjusts the perception parameters of the multi-source marine situational awareness module 1 to increase the sampling frequency in that area; simultaneously, it adjusts the risk threshold parameters of the semantic cognitive decision-making big model module 2 to generate a more conservative obstacle avoidance strategy. Through closed-loop feedback optimization, the system achieved an inspection coverage rate of over 95%, an average obstacle avoidance success rate of 98.5%, and a formation maintenance accuracy error of less than 5m in experimental verification, with collaborative operation efficiency improved by more than 200% compared to traditional methods.

[0071] In another application embodiment of the present invention, the system is applied to maritime emergency rescue missions. The mission scenario involves a ship in distress in a certain sea area, requiring a mother ship and multiple unmanned surface vessels to coordinate search and rescue operations. In this scenario, the scene semantic parsing unit of the semantic cognitive decision-making big model module 2 quickly identifies the scenario type as "emergency rescue - time-sensitive mission," the intent inference unit predicts the possible drift direction of the distressed ship and the distribution area of ​​survivors, and the decision generation unit generates decision instructions for zonal search and coordinated containment. The dynamic feedback optimization module 4 sets the weight of mission completion progress as the highest priority and continuously optimizes the search strategy to locate the distressed target as quickly as possible. Experimental verification shows that the system reduces the search and rescue response time by 60% and improves the search coverage efficiency by 180% in complex sea conditions compared to traditional methods.

[0072] The technical solution of this invention achieves an intelligent upgrade of ship multi-agent collaborative control by constructing a deeply coupled closed-loop architecture among a multi-source marine situational awareness module, a semantic cognitive decision-making large-scale model module, a distributed collaborative control module, and a dynamic feedback optimization module. A tight parameter-level coupling relationship is formed among the four core modules: the output of the multi-source marine situational awareness module directly drives the input of the semantic cognitive decision-making large-scale model module; the decision commands of the semantic cognitive decision-making large-scale model module directly control the execution strategy of the distributed collaborative control module; the execution status feedback of the distributed collaborative control module affects the evaluation results of the dynamic feedback optimization module; and the parameter adjustment of the dynamic feedback optimization module inversely optimizes the feature extraction and decision generation process of the preceding modules. This deep coupling relationship means that the modules are no longer independent functional units, but form an organically unified collaborative whole, realizing a virtuous cycle where improved perception accuracy promotes improved decision quality, improved decision quality drives optimized execution effect, and optimized execution effect feedback parameters are adaptively adjusted. The overall system performance exhibits a significant 1+1>2 nonlinear growth characteristic.

[0073] In the technical solution of this invention, the system's scalability is reflected in multiple dimensions. First, there is the scalability in the number of agents. The system architecture adopts a distributed design, with each agent's control strategy relatively independent. When a new agent needs to be added, it only needs to be registered in the system and its capability parameters configured, and the system can automatically incorporate it into the collaborative control framework. Second, there is the scalability in task types. By adding new task templates and decision-making strategies to the semantic cognitive decision-making big model module, the system can support more types of marine operational tasks. Finally, there is the scalability of functional modules. The system adopts a modular architecture design, with modules interacting through standardized interfaces. When it is necessary to upgrade the function of a module or replace it with a more advanced algorithm, a smooth upgrade can be achieved simply by maintaining interface compatibility.

[0074] The system's robust design is reflected in its fault-tolerant handling capabilities for various anomalies. At the communication level, the information interaction unit implements a multi-path redundancy communication mechanism, automatically switching to a backup link when the main communication link fails, ensuring the continuity of collaborative control. At the perception level, the multi-sensor data acquisition unit employs a multi-source data fusion strategy; when a sensor fails, the system relies on data from other sensors to maintain situational awareness. At the decision-making level, the semantic cognitive decision-making big model module has a decision fallback mechanism; when the big language model inference fails or times out, the system automatically adopts a preset emergency decision-making strategy. At the execution level, the distributed collaborative control module implements an agent fault isolation mechanism; when an agent fails, the system automatically removes it from the collaborative formation and reassigns tasks to the remaining agents, ensuring the completion of the overall task.

[0075] The system's real-time performance is ensured through a multi-level parallel processing architecture. The multi-source marine situational awareness module employs a pipelined processing model, with data acquisition, feature extraction, and semantic encoding executed in parallel. Semantic encoding of the previous batch of data is performed concurrently with feature extraction of the next batch, effectively improving processing throughput. The semantic cognitive decision-making large-scale model module uses an asynchronous inference model, with decision generation occurring asynchronously in the background without blocking the main control loop. The distributed collaborative control module adopts an event-driven architecture, triggering control strategy updates only when a state change is detected or a new decision command is received, avoiding unnecessary computational overhead. Through these real-time optimization strategies, the system's end-to-end latency is controlled within 200ms, meeting the timeliness requirements of real-time collaborative control.

Claims

1. A large-scale model system for multi-agent collaborative control of ships, characterized in that, include: The multi-source marine situation awareness module is used to collect the self-state data, surrounding environment data and communication interaction data of each intelligent agent in the ship formation, and to extract spatiotemporal features and semantically encode the collected multi-source heterogeneous data to generate a structured marine situation representation vector. The semantic cognitive decision-making big model module is connected to the multi-source ocean situation awareness module, receives the ocean situation representation vector, performs scene semantic analysis and intent inference on the current ocean situation based on the semantic understanding capability of the pre-trained big language model, and generates multi-agent collaborative decision-making instructions in combination with task objective constraints. The distributed collaborative control module is connected to the semantic cognitive decision-making big model module, receives the multi-agent collaborative decision-making instructions, decomposes the collaborative decision-making instructions into local control strategies of each agent based on the distributed control architecture, realizes the consistent execution of collaborative behavior through information interaction between agents, and outputs the motion control instructions and collaborative status information of each agent. The dynamic feedback optimization module is connected to the distributed collaborative control module, the multi-source marine situational awareness module, and the semantic cognitive decision-making big model module, respectively. It receives the collaborative status information, evaluates the deviation between the collaborative execution effect and the expected goal, and dynamically adjusts the feature extraction parameters of the multi-source marine situational awareness module and the inference strategy parameters of the semantic cognitive decision-making big model module based on the deviation information, thus forming a closed-loop feedback optimization mechanism.

2. The large-scale ship multi-agent cooperative control model system according to claim 1, characterized in that, The multi-source ocean situational awareness module includes: A multi-sensor data acquisition unit is used to acquire self-state data, surrounding environment data and communication interaction data from each intelligent agent in the ship formation. The self-state data includes position coordinates, heading angle, speed and rudder angle. The surrounding environment data includes the distribution of obstacles on the sea surface, the position and trajectory of other ships and sea state information. The communication interaction data includes intention information and cooperation requests exchanged between intelligent agents. The spatiotemporal feature extraction unit is connected to the multi-sensor data acquisition unit. It uses a sliding time window method to perform temporal modeling on the state data at continuous moments and converts the absolute positions of each agent into relative positional relationships to construct a spatial topology. The semantic encoding unit, connected to the spatiotemporal feature extraction unit, converts numerical spatiotemporal features into structured natural language descriptions through prompting engineering techniques, generating an ocean situational representation vector containing spatiotemporal numerical features and semantic text descriptions.

3. The large-scale ship multi-agent cooperative control model system according to claim 2, characterized in that, The length of the sliding time window is set to 30 sampling periods, and the spatial topology is represented by a graph structure, where nodes represent agents and edges represent communication connections and relative distances between agents.

4. The large-scale ship multi-agent cooperative control model system according to claim 1, characterized in that, The semantic cognitive decision-making large model module includes: The scene semantic parsing unit receives the ocean situation representation vector, uses a pre-trained large language model based on the Transformer architecture to perform deep parsing of the current ocean scene, identifies key elements of the scene, and outputs a structured scene description containing scene type labels, a list of key elements, and a situation assessment score. The intent inference unit is connected to the scene semantic parsing unit. Based on the scene semantic parsing results and historical behavior data, it uses a semantically enhanced spatiotemporal intent reasoning algorithm to infer the future behavioral intent of each agent and generate the probability distribution of each agent in the intent semantic space. The decision generation unit, connected to the intent inference unit, generates hierarchical multi-agent collaborative decision instructions based on the scene semantic parsing results and intent inference results, combined with task objective constraints. These instructions include global-level formation navigation strategies and local-level individual agent behavior instructions.

5. The large-scale ship multi-agent cooperative control model system according to claim 4, characterized in that, The semantically enhanced spatiotemporal intent reasoning algorithm maps the ship's possible behavioral intents into semantic vectors by constructing an intent semantic space. Based on semantic similarity calculation and consistency score between historical trajectory and intent, it determines the posterior probability of each agent under different intent types. The semantic weight coefficient and trajectory weight coefficient are used to balance the contributions of semantic understanding and trajectory analysis, respectively.

6. The large-scale ship multi-agent cooperative control model system according to claim 1, characterized in that, The distributed collaborative control module includes: The decision decomposition unit receives the multi-agent collaborative decision-making instruction, adopts a role-based task allocation mechanism, assigns role labels to each agent according to their ability characteristics, and maps the global decision to the corresponding local control strategy. The information interaction unit, connected to the decision decomposition unit, adopts a hybrid communication topology to manage communication and data exchange between agents, including maintaining global state information through a virtual coordinator and exchanging local information through point-to-point communication. The consistency execution unit, connected to the information interaction unit, calculates the degree of cooperative coupling between agents based on the attention mechanism, and dynamically adjusts the execution priority and coordination weight of the control strategy according to the degree of cooperative coupling to ensure that the implementation of the local control strategy achieves the global cooperative goal.

7. The large-scale ship multi-agent cooperative control model system according to claim 6, characterized in that, The calculation of the cooperative coupling degree is based on the attention calculation of the query vector, key vector and value vector of the agent, combined with the Gaussian decay function of the Euclidean distance between agents, to achieve the unity of semantic cooperation and spatial cooperation.

8. The large-scale ship multi-agent cooperative control model system according to claim 1, characterized in that, The dynamic feedback optimization module includes: The collaborative performance evaluation unit receives the collaborative status information and uses a composite evaluation function to evaluate the collaborative execution performance in multiple dimensions. The evaluation indicators include formation maintenance accuracy, task completion progress, energy efficiency, and safety margin. The deviation analysis unit, connected to the synergy effect evaluation unit, uses attribution analysis to decompose the synergy effect deviation into perception deviation, decision-making deviation, and execution deviation, and identifies the main factors causing the deviation. The parameter adjustment unit is connected to the deviation analysis unit and dynamically adjusts the feature extraction parameters of the multi-source marine situational awareness module and the inference strategy parameters of the semantic cognitive decision-making big model module based on the deviation analysis results using an adaptive gradient optimization method.

9. The large-scale ship multi-agent cooperative control model system according to claim 8, characterized in that, The adaptive gradient optimization method calculates the parameter adjustment amount by using the gradient of the parameters with the loss function, and introduces an adaptive adjustment factor based on the deviation between the actual loss and the target loss. When the deviation is large, the parameter adjustment amount is increased to accelerate convergence, and when the deviation is small, the parameter adjustment amount is decreased to avoid oscillation.

10. The large-scale ship multi-agent cooperative control model system according to claim 8, characterized in that, The deviation analysis unit also includes a trend prediction function, which predicts the development trend of deviation by analyzing deviation data over multiple consecutive time periods, and actively adjusts relevant parameters to achieve feedforward optimization before the deviation exceeds the allowable range.

Images