Autonomous Control Method and System for Unmanned Vessels Based on Embodied Nominal World Model

Abstract

This invention discloses an autonomous control method and system for unmanned surface vessels (USVs) based on an embodied normative world model, relating to the fields of intelligent ships and autonomous control technology. The method includes: receiving and parsing natural language task instructions; driving a normative automaton to run based on task objectives, constraints, and maritime rules, generating normative embedding vectors; constructing an embodied normative world model, combining the physical latent states encoded by multi-source observations with the normative embedding vectors to form a unified embodied normative latent state; using this latent state as input, employing a safety reinforcement learning algorithm to solve for the control strategy. This algorithm transforms maritime rules into independent constraint cost functions and performs Lagrangian optimization, maximizing long-term task rewards while strictly satisfying safety thresholds; finally, executing control instructions and updating the model and automaton in a closed loop. This invention deeply embeds normative knowledge into the world model and decision-making process, achieving safe, efficient, and interpretable autonomous control of USVs in complex maritime environments.

Description

Technical Field

[0001] This invention relates to the field of intelligent ships and autonomous control systems, and more specifically, to an autonomous control method and system for unmanned ships based on an embodied normative world model. Background Technology

[0002] With the rapid development of smart ports and intelligent shipping, unmanned surface vessels (USVs) are increasingly being used in port inspections, waterway patrols, and offshore facility maintenance. However, the current level of intelligence in USV systems still faces significant challenges, mainly in the following three aspects: First, mission configuration is complex. Most existing systems rely on operators manually setting waypoints, speeds, and other parameters on electronic nautical charts, a cumbersome and error-prone process that cannot respond to rapidly changing operational demands. Furthermore, operators require highly specialized skills, limiting the widespread application of unmanned surface vessels (USVs).

[0003] Secondly, there is a disconnect between environmental understanding and rule adherence. Traditional path planning algorithms (such as A) Dijkstra's algorithm is typically based on simplified static cost maps, making it difficult to systematically incorporate complex constraints such as the International Maritime Collision Prevention Regulations (COLREGs), port management regulations, and water depth safety margins. These rules are often used as retrospective correction logic or penalty terms in reinforcement learning reward functions, making it difficult to provide verifiable safety guarantees during the planning phase.

[0004] Finally, the system architecture is rigid. Existing systems mostly adopt a sequential pipeline of "perception-planning-control," with each module relatively independent. When the environment changes dynamically, there is a lack of a unified environmental cognition model to support continuous prediction and adaptive decision-making, resulting in insufficient system robustness and heavy reliance on human intervention.

[0005] In the relevant prior art, reference document 1 (CN116954232A) discloses a multi-vehicle collision avoidance decision-making method for unmanned surface vessels based on reinforcement learning. It employs the VO / RVO algorithm for collision risk assessment and utilizes a BiGRU network for decision-making. However, this method still relies on preset target points, does not involve natural language task input, and its rules are indirectly reflected through a reward function, lacking endogenous modeling of the rule states. Reference document 2 (CN120875431A) discloses a multi-machine cooperative scheduling system that uses a large language model to parse natural language task instructions, achieving task semantic understanding. However, this system is applied to ground / air robot cooperative scenarios, and its environmental model (random forest classification and surface fitting) and decision-making algorithm (improved A) are problematic. Neither game theory nor other methods are suitable for maritime environments with complex maritime rules. Reference document 3 (CN119088044B) discloses an autonomous navigation control method for unmanned vessels, which integrates multi-source sensor fusion, reinforcement learning strategy optimization, and a hierarchical control architecture. However, this method does not propose a unified world model concept, lacks deep integration between its environmental perception, global planning, and local control modules, and does not utilize natural language as a task input interface.

[0006] In summary, there is an urgent need for an unmanned surface vessel (USV) mission planning and safety control method and system that can integrate natural language mission instructions, complex and ever-changing aquatic environments, and written maritime rules into a unified framework for modeling and decision-making, so as to achieve an intelligent leap in the efficiency and safety of USV mission planning. Summary of the Invention

[0007] This invention provides an autonomous control method and system for unmanned vessels based on an embodied normative world model, aiming to solve the technical problem that existing autonomous control systems for unmanned vessels are unable to simultaneously achieve accurate understanding of high-level mission intentions, strict yet flexible compliance with multiple maritime rules, and synergistic optimization of safety and efficiency in dynamic and complex real maritime environments.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an autonomous control method for unmanned vessels based on an embodied canonical world model, comprising the following steps: S1. Receive natural language task instructions and parse them to generate the corresponding task objectives and constraints; S2. Based on the task objective, constraints, and predefined maritime rules, drive the normative automaton to generate normative embedding vectors representing the current task stage and rule state. The state transitions of the canonical automaton are driven by the current physical state's satisfaction of the task and rule predicates. S3. Construct and maintain an embodied canonical world model, which obtains physical hidden states by processing multi-source observation data through an encoder network. and the physical hidden state With the canonical embedding vector By combining them, a embodied normative latent state is formed. ; S4, using the aforementioned embodied normative hidden state As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold, and the constraint cost function at time... The value of is denoted as := ; S5. Execute the control strategy described above. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

[0009] Furthermore, in step S2, the operation of the canonical automaton satisfies the state transition equation: ; in, and For adjacent time step indices, For the canonical automaton at time... discrete states, Let be the discrete state of the canonical automaton at time t+1. This is the state transition function. For a moment The physical state, To be based on the physical state The labeling results obtained after judging the truth value of the task predicate and the rule predicate.

[0010] Furthermore, in step S3, the physical hidden state is obtained by encoding the historical multi-source observation data sequence and the historical control command sequence through an encoder network, and its functional form is: ; in, Index for the current time step, Let be the physical hidden state at time t. For encoder networks, The network weights of the encoder network are... This represents a multi-source observation data sequence from initial time 1 to current time t. For a moment Multi-source observation data, This represents the time from the initial time 1 to the previous time. The control command sequence, the training objective of the model includes minimizing observation reconstruction error and state prediction error.

[0011] Furthermore, in step S4, independent constraint optimization is performed by introducing Lagrange multipliers. The constrained optimization problem is transformed into an unconstrained optimization problem, and its objective function takes the form:

[0012] in, To optimize the objective function using the Lagrange multiplier, For immediate task rewards and For the first Class-specific instant constraint cost, where t is the time step index. For the Lagrange multipliers corresponding to the constraints, For the corresponding constraint threshold, As a discount factor, expected Based on strategy The state transition probabilities of the embodied canonical world model calculate.

[0013] Furthermore, the Actor-Critic framework is used to solve the optimization objective function, which includes a policy network. Task value network and at least one security value network Update gradient of policy network and task advantage function and security advantage functions Related, of which:

[0014]

[0015] in, For task advantage function; For the first Class security advantage functions; For a moment Instant task rewards; For the first Class safety constraints at time The immediate cost; Discount factor; For task value networks, used to estimate from Long-term task rewards based on state. Its parameters; For the first A security-like value network is used to estimate the value of security from... The long-term cumulative constraint cost starting from the state, Its parameters; For time step index, For constraint indexes.

[0016] Furthermore, the training data for the safety reinforcement learning algorithm comes from three sources: real navigation data streams, canonical data streams generated by sampling near rule boundaries based on the world model, and counterfactual data streams generated by simulating violations in the world model.

[0017] Furthermore, in step S5, the update includes: based on new observations Update physical stealth state to According to the new physical state Update the state of the standard automaton to and standard embedding vectors to When the predicted security constraint cost exceeds the threshold or environmental changes exceed the tolerance, a new decision-making process based on the updated world model is triggered.

[0018] Secondly, the present invention provides an autonomous control system for unmanned vessels based on an embodied normative world model, comprising: The task parsing module is configured to receive natural language task instructions and parse them to generate corresponding task objectives and constraints. The normative state management module is configured to drive the normative automaton based on task objectives, constraints, and predefined maritime rules to generate normative embedding vectors representing the current task stage and rule state. In this process, the state transitions of the canonical automaton are driven by the satisfaction of the task predicate and rule predicate by the current physical state. The world model module is configured to build and maintain embodied canonical world models, which obtain physical hidden states by processing multi-source observation data through an encoder network. and physical hidden state With canonical embedding vectors By combining them, a embodied normative latent state is formed. ; The security decision module is configured to be in an embodied, specification-based hidden state. As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold, and the constraint cost function at time... The value of is denoted as := ; The execution and update module is configured to execute the control strategy. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

[0019] In another aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described autonomous control methods for unmanned vessels based on an embodied specification world model.

[0020] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements any of the above-mentioned autonomous control methods for unmanned vessels based on an embodied canonical world model.

[0021] The above technical solution has the following technical effects: The technical solution proposed in this invention combines natural language task parsing with normative automata to automatically transform high-level human commands and maritime rules into machine-traceable internal states (normative embedding vectors), thereby significantly improving the intuitiveness of task issuance and the system's real-time compliance with complex norms. By constructing an embodied normative world model that integrates physical perception and normative states, a unified representation of environmental dynamics and rule logic is achieved, providing more complete and interpretable state inputs for subsequent decision-making. Furthermore, by employing a safe reinforcement learning algorithm and performing independent constraint optimization, various maritime rules can be independently and reliably satisfied as explicit constraints while pursuing task efficiency, thus fundamentally strengthening the behavioral safety and compliance of unmanned vessels in complex and uncertain sea areas. Finally, through execution and closed-loop update mechanisms, the world model and normative states can continuously evolve during operation, possessing the ability to adapt online and ensure continuous safety. This invention, by deeply embedding normative knowledge into the world model and decision-making process, enables safe, compliant, efficient, and interpretable autonomous control of unmanned vessels in complex maritime environments. Attached Figure Description

[0022] Figure 1 A flowchart of an autonomous control method for unmanned vessels based on an embodied canonical world model provided for an embodiment of the present invention; Figure 2 A schematic diagram of the structure of an autonomous control system for an unmanned vessel based on an embodied normative world model, provided for another embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present invention. Detailed Implementation

[0023] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0024] With the increasing demands for marine resource development, intelligent shipping, and maritime safety, unmanned surface vessels (USVs), as key intelligent carriers for performing various maritime missions, face significant challenges in the reliability, safety, and intelligence level of their autonomous control technologies, which have become core bottlenecks restricting their large-scale application. USVs need to independently complete complex tasks such as patrolling, inspection, and transportation in dynamic, uncertain, and strictly regulated real-world maritime environments (such as busy ports and narrow waterways). This places extremely high demands on the autonomous system to simultaneously perform efficient environmental perception, compliant behavior decision-making, and safe action execution. Therefore, how to enable USVs to possess human-like "mission understanding" and "rule compliance" capabilities, while achieving intelligent autonomy under the premise of ensuring absolute safety, is a key scientific problem that urgently needs to be solved in this field.

[0025] Existing autonomous control methods for unmanned surface vessels (USVs) mainly follow two technical paths: traditional methods based on rules and optimization models, and data-driven deep learning methods. Rule-based methods typically hard-code maritime regulations such as the International Maritime Collision Avoidance Codes (COLREGs) into state machines or set constraints for cost functions. While this provides interpretability in rule compliance, its behavior is rigid, its environmental adaptability is poor, and it struggles to handle rule conflicts and unforeseen scenarios. Deep learning-based methods (such as end-to-end reinforcement learning) learn strategies from data, offering high flexibility. However, their "black box" decision-making process is difficult to verify, cannot strictly guarantee compliance with key safety rules, and suffers from low learning efficiency and questionable safety. Both approaches suffer from the following fundamental disconnects: First, there is a lack of a logical bridge that can be reasoned and learned between the semantics of high-level task instructions (often in natural language) and the underlying control actions, making it difficult for the system to understand the stages and purposes of complex tasks. Second, discrete and symbolic maritime rules are difficult to deeply integrate with continuous and high-dimensional environmental perception and action space, resulting in either sacrificing flexibility or safety when making decisions. Third, there is a lack of a predictive model that can endogenously simulate the interaction between physical dynamics and normative logic, making it impossible for the system to foresee potential risks before taking action, especially the long-term consequences caused by rule violations.

[0026] From a technical implementation perspective, existing solutions suffer from three core flaws. First, environmental representation and rule representation are separated. The perception module typically outputs physical features (such as target position and velocity), while the rule compliance module relies on logical states (such as "having avoidance responsibility"). The lack of a unified, learnable joint representation makes it difficult for decision-making algorithms to balance physical optimality and rule compliance within the same semantic space. Second, the handling of safety constraints is passive and crude. Existing methods often transform safety rules into sparse, post-hoc penalty signals or set them as insurmountable hard boundaries. The former fails to provide gradient guidance throughout the decision-making process, resulting in low learning efficiency; the latter leads to unstable policy behavior at boundaries and difficulty in handling multiple potentially conflicting constraints. Third, there is a lack of targeted learning mechanisms for safety and rules. Relying solely on real navigation data or random simulation exploration is insufficient to efficiently cover dangerous but rare "rule-critical" scenarios, resulting in poor robustness to edge cases and weak generalization ability in the trained policies.

[0027] In summary, there is an urgent need to propose a new autonomous control method for unmanned vessels, constructing an internal world model that can uniformly represent the physical dynamics of the environment and the logic of maritime regulations, providing the decision-making module with state input that integrates real-time perception information and the current regulatory context; based on a safety reinforcement learning-based decision framework, multiple maritime rules are transformed into explicit and independent constraints and optimized, thereby ensuring that the control strategy can meet various safety and compliance requirements at the algorithm level; and a training mechanism is established that integrates real navigation data, regulatory data actively sampled near the rule boundaries, and counterfactual data simulating violations, to systematically improve the robustness and safety of the control strategy in complex and high-risk maritime scenarios.

[0028] The following is combined Figure 1 The flowchart below provides a detailed explanation of the method of the present invention.

[0029] A first aspect of the present invention provides an autonomous control method for unmanned vessels based on an embodied canonical world model, comprising the following steps: S1. Receive natural language task instructions and parse them to generate the corresponding task objectives and constraints; S2. Based on the task objectives, constraints, and predefined maritime rules, drive the operation of the normative automaton to generate normative embedding vectors representing the current task stage and rule state. In this context, the state transitions of a canonical automaton are driven by the satisfaction of the current physical state with respect to the task and rule predicates. S3. Construct and maintain the embodied canonical world model. The model obtains the physical hidden states by processing multi-source observation data through an encoder network. and physical hidden state With canonical embedding vectors By combining them, a embodied normative latent state is formed. ; S4, In the state of embodied norms concealment As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold. To maintain sign consistency, the constraint cost function is adjusted at time t. The value of is denoted as := ; S5. Execute the control strategy described above. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

[0030] Based on the above embodiments, exemplarily, in practical applications, an unmanned surface vessel (USV) performs the task of "navigating from anchorage A to berth B while adhering to collision avoidance rules and speed limits throughout the journey" in a busy waterway. Specifically, after receiving the aforementioned natural language instructions, the core mission objective (reaching the coordinates of berth B) and constraints (adhering to COLREGs collision avoidance rules and channel speed limits) are parsed. Subsequently, a normative automaton combining mission phases (e.g., "departure," "channel navigation," "berthing") and collision avoidance rule states (e.g., "no risk," "crossing encounter on starboard," "overtaking") is driven to run, outputting an embedding vector representing the current composite normative state. Simultaneously, real-time data from shipborne radar, AIS, and visual sensors are encoded into a physical stealth state. The two, when combined, form the embodied normative implicit state. The data is input into a safety reinforcement learning policy network. This network outputs rudder angle and throttle commands to maximize navigation efficiency while ensuring that the values ​​of the two independent constraint cost functions derived from collision avoidance and overspeed rules remain below a safety threshold. After the control commands are executed, the physical latent state of the world model and the state of the normative automaton are updated synchronously based on the new sensor data and ship position, forming a closed loop of perception-decision-execution. The "embodied normative world model" described in this application is sometimes simply referred to as the "world model" for ease of description.

[0031] The technical essence of the above embodiments lies in the systematic integration of traditionally separate task logic, security rules, and environmental awareness through two core components: a "canonical automaton" and a "world model." The canonical automaton transforms discrete, symbolic task stages and rule clauses into continuous canonical embeddings, providing high-level semantics and logical context; the world model compresses high-dimensional, heterogeneous real-time perception data into physical dynamic latent variables. The combined state of the two... This constitutes a unified decision representation that possesses both physical and normative significance. Based on this, safety reinforcement learning algorithms can process, in parallel, two traditionally potentially conflicting objectives: pursuing task goals (such as arriving in the shortest time) and adhering to multiple hard safety constraints (such as never colliding and never exceeding speed limits) within a shared state space. This architecture transforms the decision-making process of unmanned vessels from a simple mapping of perception into action into a long-term planning process with clearly defined safety boundaries within a "normative world" model that embeds task and rule understanding.

[0032] In the above embodiments, "multi-source observation data" specifically includes, but is not limited to: point cloud data of surrounding targets generated by millimeter-wave radar or lidar (providing distance, bearing, and contour information), dynamic information of other vessels received by the Automatic Identification System (AIS) (such as identity, position, heading, and speed), video streams acquired by visible light and infrared cameras (used for target identification and visual positioning), and its own pose and motion state data from the shipborne inertial measurement unit (IMU) and Global Navigation Satellite System (GNSS). The "maritime rules" include not only international Collision Avoidance Regulations at Sea (COLREGs), such as specific clauses regarding cross encounters, head-on encounters, and overtaking; but also local waterway management rules (such as speed limits and no-passage zones in specific areas), and mission-related safety constraints (such as maintaining a safe distance from specific targets). The construction of the "normative automaton" is based on the following: its discrete state set is designed as the Cartesian product of the task phase (such as "departure", "transit", "operation", "return") and the relevant maritime rule activation state, thus forming a composite state such as "transit_comply with COLREGs starboard crossing rule"; its state transition rule is driven by a labeling function, which continuously evaluates whether the current physical state (integrating multi-source observations) satisfies the logical predicates derived from the task objective and maritime rules (such as "there is a starboard approaching vessel and the distance is less than the safety threshold", "entering the speed limit zone", "reaching within 100 meters of the target point"). Once the truth value of the predicate changes, the automaton is triggered to transition to the corresponding subsequent state.

[0033] In some embodiments, in step S2, the operation of the canonical automaton satisfies the state transition equation: ; in, and For adjacent time step indices, For the canonical automaton at time... discrete states, Let be the discrete state of the canonical automaton at time t+1. This is the state transition function. For a moment The physical state, To be based on the physical state The labeling results obtained after judging the truth value of the task predicate and the rule predicate.

[0034] This embodiment details the operating mechanism of a canonical automaton. In the "intersection encounter" scenario, the state of the canonical automaton... Possibly "Freedom of navigation (Free)". (Tag function) Continuous assessment of physical condition (Including the position, speed, and heading of the ship and other ships), when the predicate "There is a ship from the starboard side at a distance less than the safe distance" is true, the automaton state is determined according to the transfer function. Move to "Cross-Starboard". Status Then, through a trainable embedding function Mapped to a low-dimensional, dense vector During training, this function learns to arrange semantically similar states (such as "overtaking" and "head-on") closer together in the vector space.

[0035] The essence of the above embodiments is to combine the precise logical controllability of discrete state machines with the flexibility and differentiability of continuous vector representations. While a simple finite state machine (FSM) can rigorously encode rules, its discrete states are difficult to directly use as input to deep neural networks for gradient optimization. This is addressed by introducing a trainable embedding function. Discrete state Transformed into a canonical embedding vector This allows downstream neural networks (such as policy networks) to "understand" and smoothly respond to changes in the canonical state. More importantly, this embedding can be learned through data, enabling the system to automatically discover potential correlations between different rule states (e.g., both "crossing encounter" and "meeting" require significant avoidance actions), thus generating more coherent and generalized behavior in decision-making. This addresses the shortcomings of traditional rule-based systems, such as rigid state representations and the inability to co-train with data-driven modules.

[0036] In some embodiments, in step S3, the physical latent state is obtained by encoding the historical multi-source observation data sequence and the historical control command sequence through an encoder network, and its functional form is:

[0037] in, Index for the current time step, Let be the physical hidden state at time t. For encoder networks, The network weights of the encoder network are... This represents a multi-source observation data sequence from initial time 1 to current time t. For a moment Multi-source observation data, This represents the time from the initial time 1 to the previous time. The control command sequence, the training objective of the model includes minimizing observation reconstruction error and state prediction error.

[0038] This embodiment specifies the method for obtaining the physical hidden state. Encoder network A recurrent neural network (RNN) with an attention mechanism is used, and its input is the historical observation sequence up to time t. (Including radar point cloud features, AIS vectors, forward image features) and historical action sequences Network outputs physical hidden state The hidden state is required to be able to reconstruct the current observation through the decoder. It can accurately predict the next hidden state through the feedforward prediction network. (Given action) The model is trained end-to-end by jointly minimizing observation reconstruction error and hidden state prediction error. Regarding the unified modeling of environment and rules, traditional world models often only model the physical environment; this application introduces an embodied normative world model as its core. The unmanned vessel's observations are processed through a representation-prediction network. and actions Mapping to hidden state And model the environmental evolution process in the latent space:

[0039]

[0040]

[0041] In the above formula, The data are from multiple sources at time t. for Time control commands, This is a physically hidden state. For encoding networks, This is a hidden state transition and observation generation model. These are the observation predictions obtained from the reconstruction of the hidden states. The above three formulas are used to describe the basic working mechanism of the world model "from observation-action sequence to hidden state, and then from hidden state to predict observation", and its structure is based on the general form of existing world models (such as variational state space models).

[0042] During the training phase, the world model is optimized by minimizing the reconstruction error and the state prediction error. A typical loss function can be written as follows:

[0043] In the formula, The total loss function of the world model. To reconstruct the probability term from observation, and Let be the posterior and prior distributions of the hidden states, respectively. For Kullback–Leibler divergence, To balance the reconstruction error with the coefficients of the regularization term, and with the physical hidden state. In combination, this application will standardize the embedding vector Included in the state, the embodied normative implicit state is defined as:

[0044] This formula employs a structure of "reconstruction error + KL regularization," a common training method in the field of variational world models. It achieves this by transferring physics... Transfer with standard automata Working together to construct a joint state transition kernel In this model, each hidden state Simultaneously carrying the dynamics information of the unmanned vessel, the semantic information of the local environment, and the satisfaction of tasks and rules, it serves as the embodied state space for the unified operation of subsequent planning and decision-making algorithms.

[0045] The essence of the above embodiments is to construct a generative world model, the core objective of which is to learn the compressed representation and predictive ability of environmental dynamics. Physical hidden states. It's not a simple compression of the original observations, but rather a trained set of latent variables that contain key environmental dynamics factors (such as the intentions of other vessels and the influence of water currents). Minimizing the reconstruction error ensures... No key observation information was lost; minimizing prediction error is forced It must contain dynamic information useful for predicting future states. Therefore, It is an abstract representation with strong physical meaning and predictive power. Embedding it with specifications... After combination, the state on which the decision system is based It not only knows "what the current physical environment is like," but also predicts "how the environment will change," and understands "what state it should be in under the current task and rules." This combination provides a foundation of information with temporal coherence and task context that goes far beyond the current instantaneous observations for subsequent reinforcement learning planning.

[0046] In some embodiments, in step S4, independent constraint optimization is performed by introducing Lagrange multipliers. The constrained optimization problem is transformed into an unconstrained optimization problem, and its objective function takes the form:

[0047] in, To optimize the objective function using the Lagrange multiplier, For immediate task rewards and For the first Class-specific instant constraint cost, where t is the time step index. For the Lagrange multipliers corresponding to the constraints, For the corresponding constraint threshold, As a discount factor, For immediate task rewards abbreviation, Cost of instantaneous constraints of type k abbreviation, expected Based on strategy State transition probabilities with embodied canonical world model calculate.

[0048] This embodiment performs independent constraint optimization for two types of rules: "collision avoidance" and "channel boundary". The algorithm defines a constraint cost function for each type of rule k. ,For example, Related to the reciprocal of the distance to the nearest ship, Related to the distance from the centerline of the shipping channel. Safety threshold. The upper limit of the allowed long-term discount cost is set. In the optimization objective function, the Lagrange multipliers... This serves as an adaptive penalty coefficient. During training, if the expected actual cost of a certain type of constraint exceeds... Corresponding It will automatically increase, thereby strengthening the penalty for this type of violation in policy updates; conversely, if the security margin is sufficient, Reducing the size of the task allows the strategy to focus more on task rewards.

[0049] In terms of path planning and control decision-making, this invention formalizes the local task execution problem of unmanned vessels into a constrained Markov decision process, and performs hierarchical safety reinforcement learning on the embodied normative hidden state space. In the decision-making state, control commands are used. As an action, this invention sets the instant task reward as... It is used to characterize comprehensive indicators such as the reduction of distance to sub-targets, the improvement of mission coverage, energy consumption, and time; it sets an immediate constraint cost. , used to indicate the first The degree of violation of rules or safety constraints, such as collision risk, extent of entry into restricted areas, severity of insufficient water depth safety margin, and degree of violation of traffic separation regulations. Discount factor is taken as... ,Strategy By parameters The policy network is given. Under this setting, the objective of this invention is to maximize long-term task rewards while ensuring that the long-term expectations of various security constraints do not exceed a preset threshold, which can be expressed as:

[0050]

[0051] In the above formula, For embodied normative state The control strategy For long-term mission reward expectations, For immediate task rewards, The immediate cost of the k-th type of security constraint, This is the upper limit allowed by the constraint. As a discount factor, The state transition probabilities given for the embodied canonical world model.

[0052] This set of formulas follows the common notation of constrained MDPs in secure reinforcement learning, with constraint costs in the formulas. It is specifically designed for COLREGs, port management regulations and task specifications, and is solved uniformly in the embodied specification state space.

[0053] To solve this multi-constraint optimization problem, this invention introduces Lagrange multipliers. Construct the Lagrange objective function:

[0054] In the formula, To optimize the objective function using the Lagrange multiplier, For immediate task rewards, For the first Class-specific instant constraint cost, where t is the time step index. For the Lagrange multipliers corresponding to the constraints, the meanings of the other symbols are the same as before.

[0055] This formula belongs to the standard Lagrangian form in constrained optimization theory. This invention combines it with the embodied normative world model and multidimensional safety constraints for unmanned vessel mission planning, thereby expanding its engineering applications based on existing theoretical foundations.

[0056] The above embodiments essentially employ the Lagrange duality method to handle multi-constraint secure reinforcement learning problems. This differs from traditional methods that directly incorporate penalty terms into the reward (…). -β Unlike other methods, the Lagrange method establishes an independent "budget" for each constraint. and adaptive multipliers The key differences in this approach are: 1) Explicitness and independence of constraints: Each security rule has its own independent and inviolable cost budget, avoiding interference or masking between different rules in the mixed penalty terms due to different cost scales. 2) Guaranteed convergence: Theoretically, it is expected to converge to the optimal policy point that satisfies all constraints, while the fixed penalty coefficient method may require tedious parameter tuning and cannot guarantee strict constraint satisfaction. 3) Adaptive balancing: The dynamic adjustment enables an automatic and precise trade-off between mission rewards and multiple safety constraints. This allows the system to respond with varying degrees of stringency when faced with "ships that must be avoided" (where violating collision constraints is extremely costly) and "ships that are advised to be avoided" (where violating certain navigation advice is less costly).

[0057] In some embodiments, an Actor-Critic framework is used to solve the optimization objective function, and the framework includes a policy network. Task value network and at least one security value network Update gradient of policy network and task advantage function and security advantage functions Related, of which:

[0058] .

[0059] This embodiment employs an Actor network (strategy) A task Critic network ( ) and two secure Critic networks ( For collisions, An architecture for out-of-bounds scenarios. The gradient update of the policy network depends not only on the task advantage function. (Measuring the improvement of the action's value to the task) also relies on two safety advantage functions. (Measures the impact of actions on safety costs). The updated formula integrates guidance signals from the task criterion and two safety criters, and uses Lagrange multipliers. Weight the safety gradient.

[0060] Building upon specific reinforcement learning structures, this invention employs an Actor-Critic framework based on an embodied canonical world model. (Policy Network) Responsible for generating the action distribution in the current state; Task value network Used to estimate from Used to estimate from Departure at The long-term cumulative cost on the constraints. To reduce variance and accelerate convergence, this invention introduces the form of an advantage function, defining the task advantage as follows:

[0061] The security advantages are as follows:

[0062] The update direction of the strategy parameters can be written as:

[0063] in, For task advantage function; For the first Class security advantage functions; For a moment Instant task rewards; For the first Class safety constraints at time The immediate cost; Discount factor; For task value networks, used to estimate from Long-term task rewards based on state. Its parameters; For the first A security-like value network is used to estimate the value of security from... The long-term cumulative constraint cost starting from the state, Its parameters; For time step index, For constrained indexes, This represents the logarithmic gradient of the policy; the other symbols have the same meaning as before.

[0064] The basic form of the advantage function and policy gradient comes from the general methods of Actor-Critic and safety reinforcement learning. In this invention, a structure of "one task value channel + multiple safety value channels" is adopted in the embodied normative state space, so that different maritime rules are mapped to different safety advantage items to participate in policy updates.

[0065] Task value networks utilize time difference error:

[0066] To update, the security value network utilizes the following formula:

[0067] The update will be performed in the following form:

[0068] Lagrange multipliers are adjusted according to the duality ascent principle:

[0069] In the formula, , For time difference error, , , To update the step size accordingly, Estimated from imagined trajectories within the world model. This indicates a truncation operation on zero. The meanings of the other symbols are the same as before.

[0070] The above formulas employ temporal difference updates and dual gradient updates, common in reinforcement learning. All updates in this invention are performed on the latent space trajectory provided by the embodied canonical world model, and the safety value function and Lagrange multipliers are updated separately for multiple safety constraints, used for unmanned surface vessel natural language task planning and safety control. Through these joint updates, this invention achieves multi-objective, multi-constraint policy optimization within the latent space of the embodied canonical world model, enabling the reinforcement learning algorithm to gradually tend towards a policy that is both efficient and rule-compliant during the training phase.

[0071] The essence of the above embodiments is to construct a multi-objective policy evaluation and improvement framework. Traditional Actor-Critic frameworks typically use only one value function to evaluate the long-term expected return of a state / action. This embodiment introduces a dedicated security value network. Its function is to accurately assess the potential future accumulation of a specific type (k-th type) of security constraint costs under the current state. Security Advantage Function This further quantifies whether a specific action, compared to the average action, increases or decreases the long-term security risk. The policy network simultaneously receives gradient signals from both task advantages and security advantages, meaning that policy improvement proceeds along the vector resultant force of "enhancing task performance" and "reducing various security risks." This design allows the policy to be explicitly and structurally guided towards a safe behavioral space during the learning process, rather than relying solely on sparse, post-incident penalties. It achieves deep integration and synergistic optimization of security and task objectives at the policy gradient level.

[0072] In some embodiments, the training data for the safety reinforcement learning algorithm comes from three sources: real navigation data streams, canonical data streams generated by sampling near rule boundaries based on a world model, and counterfactual data streams generated by simulating violations in a world model.

[0073] In this embodiment, the training data for the safety reinforcement learning strategy comes from three parallel data streams. The term "path" can be understood as a data source or data stream. Path 1: Real navigation log data stream, recording sensor observations, actual actions, and results. Path 2: Normative data stream, utilizing a trained world model to perform extensive simulation sampling under rule boundary conditions (such as the distance between two ships being exactly at the safety distance threshold), generating a large amount of "critical compliance" scenario data. Path 3: Counterfactual data stream, also within the world model, actively executing known violations (such as not turning or accelerating through), generating "violation-consequence" sequence data.

[0074] The essence of the above embodiments is to overcome the data sparsity problem in security-critical scenarios through structured data augmentation. While real-world data streams are reliable, extremely dangerous or rule-boundary situations are rare. Simply generating data through random exploration within the model is inefficient and fails to cover critical areas. The unique feature of this embodiment is the targeted generation of two types of critical data within the world model: 1) Boundary data: Proactively sampling near rule-permitted boundaries forces the policy to learn how to maintain balance on a "tightrope," greatly enhancing its decision robustness and accuracy in critical states. 2) Counterfactual data: By simulating violations and their consequences, the security value network... This allows for more accurate learning of the long-term, high-risk consequences of violations, even if such behaviors are rarely or never seen in real-world data. This proactive, targeted data generation mechanism provides strategy and security criters with a wealth of "high-risk case studies," deepening their understanding of rules from "memorizing clauses" to "understanding the specific consequences of violating clauses," thereby significantly improving the proactive risk avoidance capabilities and generalization performance of the implementation methods.

[0075] In some embodiments, step S5, the update includes: based on new observations Update physical stealth state to According to the new physical state Update the state of the standard automaton to and standard embedding vectors to When the predicted security constraint cost exceeds the threshold or environmental changes exceed the tolerance, a new decision-making process based on the updated world model is triggered.

[0076] This embodiment describes the online runtime update and monitoring mechanism. Each control command... After execution, the system obtains new observations. The world model encoder updates the hidden physical state accordingly. Simultaneously, based on the new physical state obtained from the sensors and the state estimator... (e.g., latest relative distance), standardize the automaton status update to And generate new specification embeddings. Furthermore, the system uses a post-hoc state. Starting with a world model, a short-term (e.g., 5 steps ahead) "forward search" is performed. If the predicted average safety constraint cost over the next few steps (e.g., collision risk) exceeds a preset alarm threshold, or if the predicted environmental changes (e.g., the speed of another vessel suddenly appearing) exceed the empirical tolerance during the training of the current strategy, the continuous execution of the current strategy is immediately interrupted, and the latest strategy is used. To revert to the initial state, the security decision module is invoked again for a new round of planning.

[0077] The essence of the above embodiment is to realize a dynamic and forward-looking closed loop of safety monitoring and replanning. It goes beyond simply passively executing strategies and updating states; more importantly, it introduces forward simulation and risk prediction based on a world model. This gives the system two key capabilities: 1) Predictive risk avoidance: It can predict risks seconds before physical collisions or rule violations actually occur and proactively trigger replanning, transforming passive reaction into proactive prevention. 2) Adaptation to non-stationary environments: When the environment undergoes a sudden change, exceeding the strategy's previous experience ("out of tolerance"), the system can identify this out-of-distribution (OOD) situation, avoiding blindly trusting learned strategies and instead making new decisions based on the latest and most accurate world model state. This solves the problems of "reaction lag" and "poor robustness to model errors and environmental changes" that may exist in traditional end-to-end strategies or fixed-period replanning methods, enabling the system to maintain a safety baseline in dynamic uncertainty.

[0078] refer to Figure 2 To facilitate understanding of the second aspect of the invention, an autonomous control system for unmanned vessels based on an embodied normative world model is provided, comprising: The task parsing module is configured to receive natural language task instructions and parse them to generate corresponding task objectives and constraints. The normative state management module is configured to drive the normative automaton based on task objectives, constraints, and predefined maritime rules to generate normative embedding vectors representing the current task stage and rule state. In this process, the state transitions of the canonical automaton are driven by the satisfaction of the task predicate and rule predicate by the current physical state. The world model module is configured to build and maintain embodied canonical world models, which obtain physical hidden states by processing multi-source observation data through an encoder network. and physical hidden state With canonical embedding vectors By combining them, a embodied normative latent state is formed. ; The security decision module is configured to be in an embodied, specification-based hidden state. As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold. To maintain sign consistency, the constraint cost function is adjusted at time t. The value of is denoted as := ; The execution and update module is configured to execute the control strategy. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

[0079] This embodiment provides a hardware / software integrated system. Deployed in the computing unit of an unmanned surface vessel, the system comprises five core modules: a task parsing module (receiving and parsing natural language commands); a canonical state management module (embedding a canonical automaton library and embedding functions to output canonical embeddings); a world model module (running the encoder network, maintaining and updating physical latent states, and completing state fusion); a safety decision-making module (deploying a policy network, value network, and optimizer to perform safety reinforcement learning inference); and an execution and update module (sending control commands to the servo and throttle, and coordinating the online update process of each module). These modules communicate through predefined APIs and data interfaces, working collaboratively in a pipelined manner.

[0080] The technical essence of the system described in the above embodiments lies in mapping the functional logic of the autonomous control method for unmanned vessels based on the embodied canonical world model to a loosely coupled modular architecture. Each module encapsulates a set of closely related functions and internal states (e.g., the world model module encapsulates encoder network parameters and the current hidden state), and the modules are connected through clearly defined interfaces (e.g., canonical embedding vectors). Embodied norms hidden state This design allows for interaction with the system. Its unique feature lies in ensuring both the continuity and efficiency of the entire autonomous control process, while also facilitating system development, debugging, and upgrades. For example, the encoder architecture of the world model can be improved independently without affecting the logic of the safety decision-making module; the rule base in the canonical automaton can be updated without retraining the entire deep network. The modular design enables technologies of different natures—natural language understanding, rule logic, environment modeling, and safety decision-making—to develop and integrate relatively independently, collectively constructing a complex and reliable autonomous system, rather than a difficult-to-maintain and evolve "black box" whole.

[0081] As another possible implementation, based on the Transformer architecture and offline security policy learning, the world model is constructed using the Transformer architecture instead of a recurrent neural network (RNN), and an offline security policy learning mechanism is introduced. Specifically, the encoder network... A hybrid architecture combining Vision Transformer and Temporal Transformer is employed to process multi-source observation sequences. The spatiotemporal feature blocks, assembled from these elements, directly output the physical hidden states. Embedding functions for standard automata Similarly, using a Transformer encoder, discrete state sequences and their contextual relationships are mapped to canonical embeddings. In the decision-making section, this embodiment employs a conservative Q-learning (CQL) framework for offline security policy learning. Specifically, based on a large-scale historical navigation dataset (containing compliant and a small number of non-compliant scenarios), a conservative action value function is pre-trained. When running online, the security decision module operates in an embodied, specification-based implicit state. As input, a weighted fusion online security reinforcement learning strategy is used. Output and offline strategies The output is used to generate the final control command, and the weights are dynamically determined by the similarity between the current state and the distribution of the offline dataset.

[0082] The essence of the above embodiments lies in enhancing the robustness and safety ceiling of the system through more powerful sequence modeling capabilities and the injection of prior safety knowledge. Due to its self-attention mechanism, the Transformer architecture can more effectively model complex relationships between long-distance dependencies and multi-source heterogeneous data, thereby improving the prediction accuracy of the world model for dynamic scenarios (such as long-term interactions between multiple objectives). The introduction of offline safety policy learning essentially injects the expert (or safety) driving priors implicit in large-scale historical data into the online decision-making process in the form of a quantifiable conservative value function. This design provides a safety fallback strategy based on empirical knowledge when facing rare scenarios with high uncertainty or insufficient training data coverage, avoiding the high-risk trial and error that may result from purely online exploration. Unlike online safety reinforcement learning, which focuses on constraint optimization, offline learning focuses more on extracting safety patterns from given data; the fusion of the two achieves a complementarity between online adaptability and offline robustness.

[0083] In other embodiments, the canonical automaton can be designed as a hierarchical structure and a distributed constraint optimization method can be used for decision-making. Specifically, the canonical state is divided into two layers: the top layer represents the macroscopic task stage (e.g., "crossing," "operation"), and the bottom layer represents the specific rule activation states (e.g., "intersection," "speed limit zone"). Top-level state transitions are driven by the task objective predicate, while bottom-level state transitions are driven by the satisfaction of the specific rule predicates by the physical state. The set of bottom-level states is constrained by the top-level states. The resulting hierarchical canonical embedding... It is composed of top-level embeddings and bottom-level embeddings. In the safety decision-making module, this embodiment adopts a distributed constraint optimization method to decompose the problem into multiple parallel sub-decision-makers. For example, independent "collision avoidance sub-decision-makers," "routes sub-decision-makers," and "task efficiency sub-decision-makers" are set up, each of which is based on a complete embodied specification implicit state. However, it focuses on optimizing its corresponding local objectives (or constraints). A meta-coordinator is responsible for fusing and arbitrating the action suggestions output by each sub-decision-maker. The arbitration rules are dynamically adjusted based on the current hierarchical specification state (for example, when the "cross-encounter" bottom-level state is activated, the meta-coordinator significantly increases the weight of the "collision avoidance sub-decision-maker" suggestion).

[0084] The technical essence of the above embodiments is to address the decision-making complexity brought about by the intertwining of complex tasks and rules through explicit hierarchical decomposition of states and decisions. The hierarchical canonical automaton structurally separates the "task context" from the "immediate rules," enabling the system to more clearly distinguish when to follow general task flows and when to prioritize emergency rules, thereby reducing semantic confusion at the state representation level. Distributed constraint optimization decomposes the global, multi-objective decision problem into a series of parallel-computable, relatively single-objective local problems. This decomposition not only improves the computational efficiency and scalability of the decision-making module, but more importantly, it allows different types of constraints (such as hard safety constraints and soft efficiency constraints) to be optimized more professionally and thoroughly in different subspaces. The meta-coordinator, based on a dynamic arbitration mechanism of canonical states, essentially implements a context-aware resource (here, control authority) allocation strategy, ensuring that safety-related sub-decision-makers gain dominance at critical moments. This architecture is particularly suitable for complex maritime scenarios with clear task phases, diverse rule types, and potential conflicts.

[0085] Based on the foregoing embodiments, there are also some embodiments with essentially the same or similar technical concepts, as shown in the following examples: The implementation scheme based on model predictive control maintains the same overall structure as the aforementioned implementation scheme, with the difference being that the decision algorithm in the "route planning and action generation module" is replaced from "hierarchical safety reinforcement learning based on a world model" to "model predictive control (MPC) based on a world model." Specifically, the embodied specification world model also provides the joint state transition probability. The differentiable approximation. In each control cycle, the decision module uses the current embodied canonical hidden state. As initial conditions, in the model prediction time domain Within this framework, an online solution is provided for a finite-time optimization problem. The objective function of this problem is to minimize the task cost (such as distance to sub-objectives and energy consumption) along the predicted trajectory. The constraints are multinomial safety constraints directly calculated from the world model. These functions are composed of factors such as collision risk, no-navigation zone incursion, water depth safety margin, and encounter rules. The optimal action sequence is obtained through numerical optimization. Only the first item is executed. The process is then repeated in the next cycle. To enhance theoretical security, this scheme can combine a world model to pre-calculate the safety control invariant set or inversely reachable set associated with various rules offline, and use the boundary of this set as the terminal constraint or hard constraint of the MPC optimization problem. The scheme replaces the "offline learning and online generalization" mode of the policy with an "online rolling optimization" mode. Utilizing the differentiable state prediction provided by the embodied canonical world model, maritime rules are directly encoded as constraints of the optimization problem, and a locally optimal and explicitly constraint-satisfying action sequence is obtained at each time step through online solving. This method does not rely on a pre-trained policy network, but rather on the real-time computing power of the online solver. It provides higher transparency and interpretability of the decision-making process because each action is a direct result of explicit optimization of the finite future at the current time step, making it particularly suitable for application scenarios with strict requirements for the verifiability of the control law. This proves that the core framework of this invention (natural language parsing → canonical world model construction) is compatible with different types of decision-makers and is not limited to specific learning or optimization algorithms.

[0086] Another embodiment is an extended example for multi-unmanned vessel collaboration, expanding the application scope of the invention from a single vessel to a multi-unmanned vessel collaborative system. This represents a higher-level, scenario-based extension of the inventive concept. The system architecture retains the core modules while adding a "cooperative scheduling and information sharing module," and extends the "embodied canonical world model" to a "multi-agent joint world model." The task parsing module not only parses the overall task but also automatically generates multi-vessel task decomposition and role allocation schemes (such as region division and master-slave relationships). The world model's state representation... The model is extended to include the joint hidden state of all friendly vessels (position, speed, mission phase) and their interactions. A graph neural network is typically used to model the topology of the multi-agent system. The decision-making part employs a cooperative safety reinforcement learning framework of "centralized training and distributed execution." During training, the joint trajectory of multiple vessels is simulated within the world model, optimizing the joint value function, which includes overall mission efficiency, formation maintenance, and cooperative coverage, while maintaining an independent safety constraint channel for each vessel. During execution, each vessel generates control commands through its own deployed policy network based on its local observations and localized sub-states derived from the joint world model, achieving autonomous distributed decision-making. This embodiment extends the concept of "embodied norms" for single agents to the level of "cooperative norms" for multiple agents. Through this extension, the world model not only internalizes the physical dynamics of the environment and maritime rules but also internalizes the cooperative constraints among multiple agents (such as communication protocols, formation requirements, and collision avoidance responsibility allocation) and their mutual influence models. By summarizing the relationships between joint states using a graph structure, the model can understand how "the avoidance maneuver of vessel A affects the option space of vessel B." This enables cooperative strategies trained based on this world model to exhibit complex coordinated behaviors, such as forming ordered passage sequences during multi-vessel encounters or automatically achieving non-overlapping coverage of regions in cooperative search. This embodiment fully demonstrates the good scalability of the basic framework of this invention (a specification-driven world model), which can naturally extend from individual intelligence to swarm intelligence, providing a unified technical path for solving the planning and safety control problems of cooperative operations of multiple unmanned systems at sea.

[0087] In some embodiments, such as Figure 3 This application also provides an electronic device, including a processor 301, a memory 302 and a bus 303, and a computer program stored in the memory. When the processor executes the program, it implements any of the above-described autonomous control methods for unmanned vessels based on an embodied specification world model.

[0088] Furthermore, as an executable solution, the autonomous control system of an unmanned vessel based on an embodied normative world model can be a computer unit, which can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit; it may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., which are not limited in this embodiment of the invention.

[0089] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.

[0090] In some embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the autonomous control method for unmanned vessels based on an embodied canonical world model described above in the embodiments of the present invention.

[0091] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0092] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. An autonomous control method for unmanned surface vessels based on an embodied normative world model, characterized in that, Includes the following steps: S1. Receive natural language task instructions and parse them to generate the corresponding task objectives and constraints; S2. Based on the task objective, constraints, and predefined maritime rules, drive the normative automaton to generate normative embedding vectors representing the current task stage and rule state. The state transitions of the canonical automaton are driven by the current physical state's satisfaction of the task and rule predicates. S3. Construct and maintain an embodied canonical world model, which obtains physical hidden states by processing multi-source observation data through an encoder network. and the physical hidden state With the canonical embedding vector By combining them, a embodied normative latent state is formed. ; S4, using the aforementioned embodied normative hidden state As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold, and the constraint cost function at time... The value of is denoted as := The independent constraint optimization is achieved by introducing Lagrange multipliers. The constrained optimization problem is transformed into an unconstrained optimization problem, and its objective function takes the form: in, To optimize the objective function using the Lagrange multiplier, For immediate task rewards and For the first Class-specific instant constraint cost, where t is the time step index. For the Lagrange multipliers corresponding to the constraints, For the corresponding constraint threshold, As a discount factor, expected Based on strategy The state transition probabilities of the embodied canonical world model calculate; The training data for the security reinforcement learning algorithm comes from three sources: real navigation data streams, canonical data streams generated by sampling near rule boundaries based on the world model, and counterfactual data streams generated by simulating violations in the world model. S5. Execute the control strategy described above. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

2. The method according to claim 1, characterized in that, In step S2, the operation of the canonical automaton satisfies ,in For the discrete states of the automaton, Let be the discrete state of the canonical automaton at time t+1. This is the state transition function. To be based on physical state A tagging function to determine the truth value of task and rule predicates.

3. The method according to claim 1, characterized in that, In step S3, the physical hidden state is achieved through... Received, among which For encoder networks, The data represents the multi-source observation data at time t; the training objectives of the model include minimizing the observation reconstruction error and the state prediction error.

4. The method according to claim 1, characterized in that, The optimization objective function is solved using an Actor-Critic framework, which includes a policy network. Task value network and at least one security value network The update gradient of the policy network and the task advantage function and security advantage functions Related, of which: in, For task advantage function; For the first Class security advantage functions; For a moment Instant task rewards; For the first Class safety constraints at time The immediate cost; Discount factor; For task value networks, used to estimate from Long-term task rewards based on state. Its parameters; For the first A security-like value network is used to estimate the value of security from... The long-term cumulative constraint cost starting from the state, Its parameters; For time step index, For constraint indexes.

5. The method according to claim 1, characterized in that, In step S5, the update includes: based on new observations Update physical stealth state to According to the new physical state Update the state of the standard automaton to and standard embedding vectors to When the predicted security constraint cost exceeds the threshold or environmental changes exceed the tolerance, a new decision-making process based on the updated world model is triggered.

6. An autonomous control system for unmanned vessels based on an embodied canonical world model, characterized in that, include: The task parsing module is configured to receive natural language task instructions and parse them to generate corresponding task objectives and constraints. The normative state management module is configured to drive the normative automaton to run based on the task objectives, constraints, and predefined maritime rules, in order to generate normative embedding vectors representing the current task stage and rule state. The state transitions of the canonical automaton are driven by the satisfaction of the task predicate and rule predicate by the current physical state. The world model module is configured to build and maintain an embodied canonical world model, which obtains physical hidden states by processing multi-source observation data through an encoder network. and the physical hidden state With the canonical embedding vector By combining them, a embodied normative latent state is formed. ; The security decision module is configured to be in an embodied, specification-based hidden state. As input, the control strategy is solved based on a security reinforcement learning algorithm. The security reinforcement learning algorithm optimizes long-term task rewards while transforming at least one class of maritime rules into explicit constraint cost functions. Independent constraint optimization is performed to meet a preset safety threshold, and the constraint cost function at time... The value of is denoted as := The independent constraint optimization is achieved by introducing Lagrange multipliers. The constrained optimization problem is transformed into an unconstrained optimization problem, and its objective function takes the form: in, To optimize the objective function using the Lagrange multiplier, For immediate task rewards and For the first Class-specific instant constraint cost, where t is the time step index. For the Lagrange multipliers corresponding to the constraints, For the corresponding constraint threshold, As a discount factor, expected Based on strategy The state transition probabilities of the embodied canonical world model calculate; The training data for the security reinforcement learning algorithm comes from three sources: real navigation data streams, canonical data streams generated by sampling near rule boundaries based on the world model, and counterfactual data streams generated by simulating violations in the world model. The execution and update module is configured to execute the control strategy. The embodied normative hidden state Down-output control commands The embodied normative world model and the normative automaton are updated based on the new observation data after execution to achieve closed-loop control.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5.

Images