Intelligent ship control system

Abstract

The invention discloses an intelligent ship control system which comprises a ship interaction data acquisition module, a ship state extraction module, a navigation experience screening module, a multi-ship cooperative control agent training module, a ship parameter adaptive adjustment module and a ship control module. The invention belongs to the field of intelligent control, and particularly relates to an intelligent ship control system. According to the scheme, core features are extracted through double-branch extraction, and a dynamic transition probability similarity coefficient is introduced, so that similarity judgment adapts to interaction strength; for positive sample pairs, ship low-dimensional features of similar behaviors are forced to approach, and for negative sample pairs, the feature distance is increased, so that a collision avoidance strategy can migrate across ship types; navigation experience screening is carried out through safe sampling guided by a reference path, and dangerous data interference is avoided; extreme risk punishment is added to strengthen fitting of the close-range collision sample; and an environment perception dynamic period is introduced, an adjusted safety reward is set, and environment change is dynamically adapted, so that the ship control effect is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the field of intelligent control, and particularly discloses an intelligent ship control system. BACKGROUND

[0002] Generally, a ship control system is a system that realizes accurate regulation and control of ship movement by collecting ship state and marine environment information through sensors, outputting instructions to rudders, throttles and other actuators after control algorithm operation. However, the general ship control system has the problems of ignoring the dynamics of environmental disturbance, static similarity measurement of multi-ship behavior, weak migration ability and insufficient anti-interference ability, which further aggravates energy consumption and unstable heading, resulting in low ship control reliability. The general ship control system also has the problems of blind experience screening, low utilization rate of high-value data, single multi-ship cooperative target, imbalance between individual and group needs, rigid parameter adjustment and difficulty in adapting to dynamic environment, which further leads to poor ship control effect. SUMMARY

[0003] In view of the above problems, the application provides an intelligent ship control system to overcome the defects of the prior art. The application solves the problems of ignoring the dynamics of environmental disturbance, static similarity measurement of multi-ship behavior, weak migration ability and insufficient anti-interference ability of the general ship control system, which further aggravates energy consumption and unstable heading, resulting in low ship control reliability. The application introduces a dynamic transition probability similarity coefficient in the dynamic behavior similarity measurement by double-branch extraction of core features, so that the similarity judgment adapts to the interaction intensity and scene type. The application sets a loss function for positive and negative sample pairs of ships. For positive sample pairs, the low-dimensional features of similar behaviors of ships are forced to be close to each other. For negative sample pairs, the feature distance is enlarged, so that the collision avoidance strategy can be transferred across ship types. The application realizes physical modeling-dynamic measurement-full data three-layer design, so that the ship control strategy has physical feasibility, scene adaptability and cross-ship type migration in complex sea conditions and multi-ship interaction scenes, thereby improving the ship control reliability. The application solves the problems of blind experience screening, low utilization rate of high-value data, single multi-ship cooperative target, imbalance between individual and group needs, rigid parameter adjustment and difficulty in adapting to dynamic environment of the general ship control system, which further leads to poor ship control effect. The application performs navigation experience screening by reference path guided safe sampling to avoid dangerous data interference. The application also constrains the path tracking accuracy, energy consumption and collision avoidance safety to guide the multi-objective optimization of the main network. The application avoids value aliasing by Q value separation and risk punishment of the learning sub-network. The application strengthens the fitting of close-range collision samples by adding extreme risk punishment. The application introduces a dynamic period of environmental perception by hierarchical parameter adjustment, sets a safe reward after adjustment, dynamically adapts to environmental changes and solves the problems of strategy rigidity and safety risk. The application further improves the ship control effect.

[0004] The technical solutions adopted by the application are as follows: The application provides an intelligent ship control system, which comprises a ship interactive data acquisition module, a ship state refining module, a navigation experience screening module, a multi-ship cooperative control intelligent agent training module, a ship parameter self-adaptive adjustment module and a ship control module.

[0005] The ship interactive data acquisition module acquires ship state data and surrounding ship observation data, and constructs a ship navigation experience storage library.

[0006] The ship state refining module optimizes the characteristics of the samples in the ship navigation experience storage library through double-branch training.

[0007] The navigation experience screening module defines a value refining vector of a ship path segment, and screens the samples in the ship navigation experience storage library.

[0008] The multi-ship cooperative control intelligent agent training module constructs a ship guiding main network and a control learning sub-network based on the screened and optimized ship navigation experience storage library.

[0009] The ship parameter self-adaptive adjustment module divides the control network parameters into a core safety layer and a non-core optimization layer, and calculates a dynamic adjustment period according to the environmental disturbance intensity.

[0010] The ship control module acquires real-time ship operation data, inputs the optimized ship state refining module, and then inputs the multi-ship cooperative control intelligent agent to realize real-time ship control.

[0011] Further, the ship interactive data acquisition module acquires interactive data of the ship in a digital twin environment, each ship acquires the state of the ship itself and the observation state of the surrounding ships through sensors in real time; records the next state and the reward after executing a control action; and stores the multi-ship joint data into the ship navigation experience storage library.

[0012] Further, the ship state refining module specifically comprises:

[0013] Branch one, state refining training based on ship dynamics: based on a ship MMG model, the next state and environmental disturbance force are predicted through the current state and action, and the initial refining of the ship is obtained; the loss function is as follows: ; wherein, is the predicted next state of the ship, the superscript te represents a true value, the superscript pd represents a predicted value, t represents a time, and i represents a ship; is the predicted water flow force and wind force, c represents the water flow force, and d represents the wind force; is the weight of the balance state prediction and the disturbance force prediction;

[0014] Branch two, state abstraction training based on ship mutual simulation: measure the behavior similarity of two ships, and introduce a dynamic transition probability similarity coefficient; denoted as: ; ; wherein, is the behavior similarity measure of ship i and ship j, the smaller the value, the more similar the behavior; a is the control action, A is the control action set; and are the rewards obtained by ships i and j performing action a; is the probability of ship i transitioning to state under action a, water flow , wind speed ; is the probability of ship j transitioning to state ; is the transition probability similarity coefficient; is the reference coefficient; is the reference distance; is the distance between the centers of mass of the two ships; is the JS divergence; is the scene correction coefficient; learn a behavior feature encoder to map the initial abstraction of the ship to a low-dimensional behavior feature, so that ships with similar behaviors have similar low-dimensional behavior features, and divide the ship data into positive and negative sample pairs through a behavior threshold; define the core loss for the positive sample pair; for the negative sample pair, define the contrast loss; sample equal amounts of positive and negative sample pairs each iteration, update the behavior feature encoder parameters with the Adam optimizer, and iterate until the loss converges.

[0015] Further, the navigation experience screening module specifically includes:

[0016] Ship path segment abstraction definition, abstracted as the difference between the initial state abstraction and the end state abstraction;

[0017] Similarity sampling guided by reference path segments: randomly select one path segment from the experience storage pool; calculate the similarity of all candidate path segments with the selected path segment, and screen the similarity ∈ [0.3, 0.7] path segments; if the number of screened path segments is greater than the batch size, keep the top K path segments in descending order of reward value, and divide each path segment into single-step training samples.

[0018] Further, the multi-ship cooperative control intelligent agent training module specifically includes:

[0019] Ship guidance main network construction: taking path tracking under multi-ship collision avoidance constraints as the goal, construct an optimization objective function;

[0020] The ship control learning subnetwork is constructed, and a Q value function of the ship control learning subnetwork is divided into a state value and an action advantage.

[0021] The ship control learning subnetwork loss adds an extreme risk penalty term.

[0022] Further, the ship parameter adaptive adjustment module specifically comprises:

[0023] Partial parameter adjustment, the ship control network parameters are divided into a core safety layer and a non-core optimization layer, and only the non-core layer is adjusted;

[0024] A dynamic adjustment cycle mechanism of environment perception is introduced; and a safety reward is obtained after adjustment.

[0025] Further, the ship control module acquires real-time ship operation data, inputs the real-time ship operation data into the ship state refining module for optimization, and inputs the real-time ship operation data into the multi-ship cooperative control intelligent agent module, so as to realize real-time ship control based on the module output.

[0026] The above scheme has the following beneficial effects:

[0027] (1) In view of the problems that a general ship control system ignores the dynamics of environmental disturbance, the static similarity of multi-ship behaviors is measured, the migration ability is weak, and the anti-interference ability is insufficient, thereby aggravating energy consumption and unstable heading, and leading to low ship control reliability, the scheme extracts core features through double-branch refining, introduces a dynamic transition probability similarity coefficient in dynamic behavior similarity measurement, so that the similarity judgment adapts to the interaction intensity and the scene type; by dividing the positive and negative sample pairs of the ship, the loss function is set respectively, for the positive sample pair, the low-dimensional features of the similar behaviors of the ship are forced to be close, and for the negative sample pair, the feature distance is enlarged, and then the collision avoidance strategy can be transferred across ship types; through the three-layer design of physical modeling-dynamic measurement-full-quantity data, the ship control strategy has physical feasibility, scene adaptability and cross-ship type migration in complex sea conditions and multi-ship interaction scenes, thereby improving the ship control reliability.

[0028] (2) In view of the problems that the experience screening of the general ship control system is blind, the utilization rate of high-value data is low, the multi-ship cooperation target is single, the individual and group demand is unbalanced, the parameter adjustment is rigid, and it is difficult to adapt to the dynamic environment, thereby leading to poor ship control effect, the scheme screens the sailing experience through the safe sampling guided by the reference path, avoids the interference of dangerous data, and simultaneously constrains the path tracking accuracy, energy consumption and collision avoidance safety, guides the multi-objective optimization of the main network, separates the Q value of the learning sub-network and avoids the value aliasing through the risk punishment, strengthens the fitting of the near-distance collision sample by adding the extreme risk punishment, dynamically adapts the environmental changes through the hierarchical parameter adjustment, the introduction of the dynamic period of environmental perception and the setting of the safety reward after adjustment, solves the problems of strategy rigidity and safety risk, and further improves the ship control effect. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A flowchart of an intelligent ship control system provided by the present application is shown.

[0030] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, and are used to explain the present application together with embodiments of the present application, and do not constitute a limitation on the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0032] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0033] Embodiment one, refer to Figure 1 The intelligent ship control system provided by the present application comprises a ship interactive data acquisition module, a ship state extraction module, a sailing experience screening module, a multi-ship cooperation control intelligent agent training module, a ship parameter self-adaptive adjustment module and a ship control module.

[0034] The ship interactive data acquisition module acquires ship state data and surrounding ship observation data, and constructs a ship sailing experience storage library; and sends the data to the ship state extraction module.

[0035] The ship state extraction module optimizes the features of the samples in the ship navigation experience repository through double-branch training, and sends data to the navigation experience screening module;

[0036] The navigation experience screening module defines a value extraction vector of a ship path segment, screens the samples in the ship navigation experience repository, and sends data to the multi-ship cooperative control intelligent agent training module;

[0037] The multi-ship cooperative control intelligent agent training module constructs a ship guiding main network and a control learning sub-network based on the screened and optimized ship navigation experience repository, and sends data to the ship parameter self-adaptive adjustment module;

[0038] The ship parameter self-adaptive adjustment module divides the control network parameters into a core safety layer and a non-core optimization layer, calculates a dynamic adjustment period according to the environmental disturbance intensity, and sends data to the ship control module;

[0039] The ship control module obtains real-time ship operation data, inputs the optimized ship state extraction module, and then inputs the multi-ship cooperative control intelligent agent to realize real-time ship control.

[0040] Embodiment two, refer to Figure 1 Based on the above embodiment, the ship interaction data acquisition module acquires interaction data of the ship in the digital twin environment, covering different sea conditions and scenes;

[0041] The specific operation is: each ship collects the ship's own state in real time through a sensor , and the surrounding ship observation vector is ; is the ship position coordinate; is the ship speed; is the ship heading angle; is the draft; is the heading of ship j relative to ship i, is the relative speed; is the distance between the two ship centers of mass; execute the control action After that, record the next state and the reward ; store the multi-ship joint data to the ship navigation experience repository; , and are the rudder angle change, throttle opening and propeller speed change, respectively; , , are the ship joint state, joint action and joint reward at time t, respectively; is the joint state of ships at time t+1; the data needs to be normalized.

[0042] Embodiment three, see Figure 1 , which is based on the above embodiment, the ship state refining module for the existence of ship state including speed change affecting the strong coupling of the stability of the heading and the environmental disturbance including water flow, wind and waves, through the double branch refining to extract the core features; realize the feature reorganization and physical alignment of the multi-ship joint data;

[0043] The specific operation is:

[0044] Branch one, state refinement training based on ship dynamics: based on the ship MMG model, the next state and environmental disturbance force are predicted through the current state and action, the state includes position, heading, speed, the environmental disturbance force includes water flow force and wind force, the initial refinement of the ship is obtained; the loss function is: ; wherein, is the predicted next state of the ship, the superscript te represents the true value, the superscript pd represents the predicted value, t represents the time, and i represents the ship; is the predicted water flow force and wind force, c represents the water flow force, and d represents the wind force; is the weight of the balance state prediction and disturbance force prediction;

[0045] Training data: a small batch of ship data is sampled from the experience storage pool, and the ship space-time feature extraction network is used to train the refinement model, and the CNN layer is used to extract the spatial features such as relative position, and the LSTM layer is used to capture the time sequence dynamics such as speed change; the model structure: the CNN layer includes 2 convolution layers+pooling layers, and outputs the spatial feature vector; the LSTM layer includes 1 bidirectional LSTM, and outputs the time sequence feature vector; the prediction head includes a state prediction head: a fully connected layer outputs the next state prediction , an interference force prediction head: a fully connected layer outputs the environmental disturbance force prediction , forward propagation: the output of the ship space-time feature extraction is calculated ; use the Adam optimizer to perform back propagation until convergence;

[0046] Branch two, state refinement training based on ship mutual simulation: measure the behavior similarity of two ships i and j, which is to measure whether the responses of two ships to the same control action are consistent under similar sea conditions, and introduce a dynamic transition probability similarity coefficient, which is expressed as: ; ; wherein, is the behavior similarity measure of ship i and ship j, the smaller the value, the more similar the behavior; a is the control action, and A is the control action set; and is the reward obtained by ship i and j performing action a; is the probability of ship i transferring to state under action a, water flow , wind speed ; is the probability of ship j transferring to state ; is the transition probability similarity coefficient; is the baseline coefficient; is the reference distance, used to define the threshold between interaction and no interaction, and beyond this distance is considered as no mutual influence between two ships; is the JS divergence; is the scene correction coefficient, when free sailing, the ship behavior is independent, ; when general collision avoidance ; when emergency collision avoidance ; when close distance strong interaction, focus on the JS divergence of state transition probability; when emergency collision avoidance scene, strengthen the weight of physical response to ensure the compliance of collision avoidance action;

[0047] Learn a behavior feature encoder to map the initial extraction of the ship to a low-dimensional behavior feature, so that ships with similar behaviors have similar low-dimensional behavior features. Divide the ship data into positive and negative sample pairs through the behavior threshold; construct a behavior feature mapping network, including a 3-layer fully connected network, which outputs low-dimensional behavior features; predict the state transition probability distribution based on the low-dimensional behavior features, modeled by a Gaussian distribution, outputting the mean and variance;

[0048] For the positive sample pair, define the core loss, denoted as: ; and are the low-dimensional state extraction of the ship; and are the rewards obtained by the ship; and are the conditional state transition probability distributions based on low-dimensional features, actions and water flow, for all possible states;

[0049] For the negative sample pair, define the contrast loss , denoted as: ; m is the marginal parameter;

[0050] Sample equal amounts of positive and negative sample pairs each time, update the behavior feature encoder parameters with the Adam optimizer, and iterate until the loss converges;

[0051] The loss function based on the MMG model avoids black box extraction and disconnection from the real motion of the ship, ensuring that the subsequent control strategy conforms to the physical laws of ship maneuvering. Through mutual simulation extraction, support migration, capture the common behavior of multi-ship cooperation, and can migrate the collision avoidance strategy of cargo ships to passenger ships, reducing repeated training.

[0052] By performing the above operation, the general ship control system ignores the dynamic nature of environmental disturbance, the similarity measure of multi-ship behavior is static, the migration ability is weak, and the anti-interference ability is insufficient, which further aggravates energy consumption and unstable heading, resulting in low reliability of ship control. The scheme extracts core features through double branch extraction, introduces dynamic transition probability similarity coefficient in dynamic behavior similarity measurement, and adapts similarity judgment to interaction intensity and scene type; by dividing the positive and negative sample pairs of the ship, the loss function is set respectively, for the positive sample pair, the low-dimensional features of the similar behavior of the ship are forced to be close, and for the negative sample pair, the feature distance is enlarged, thereby realizing the cross-ship type migration of collision avoidance strategy; through the three-layer design of physical modeling-dynamic measurement-full-quantity data, the ship control strategy has physical feasibility, scene adaptability and cross-ship type migration in complex sea conditions and multi-ship interaction scenes, thereby improving the reliability of ship control.

[0053] Embodiment four, see Figure 1 This embodiment is based on the above embodiment. The sailing experience screening module has a large amount of redundant data in the ship experience storage pool. High-value path segments, i.e., path segments containing collision avoidance decisions, extreme sea condition responses, and path corrections, are preferentially sampled. A sampling method using ship path segment level value extraction + reference path segment guidance is adopted. The multi-ship joint data set is optimized, and the most valuable path segment data for the control strategy is screened;

[0054] The specific operation is: ship path segment extraction definition, ship path segment is a continuous sailing segment, which is extracted as the difference between the initial state extraction and the end state extraction, directly reflecting the control significance of the path segment. The larger the difference, the more critical the decisions contained in the path segment. It is represented as: ; is the ship extraction at the end point of the path segment, which includes the end position, heading, and cumulative reward; is the ship extraction at the initial time of the path segment; is a single continuous sailing path segment of ship i; is the value extraction vector of the path segment;

[0055] Reference path segment guided similarity sampling, randomly select 1 safe high value path segment from the experience storage pool, which satisfies the collision avoidance success rate = 100%, the path deviation is less than 50m and the energy consumption is lower than the average 10%, to ensure the safety of the sampling direction; calculate the similarity of all candidate path segments and , screen the similarity ∈ [0.3, 0.7] path segment, avoid completely redundant straight sailing path segment, also avoid dangerous path segment with too large difference from safe path segment, represented as: If the number of filtered path segments is greater than the batch size, the first K path segments are retained in descending order of reward value, and each path segment is divided into single-step training samples; is the path segment similarity.

[0056] Embodiment five, see Figure 1 This embodiment is based on the above-mentioned embodiments. The multi-ship cooperative control intelligent agent training module needs to consider both the individual accuracy of single-ship heading tracking and the cooperative safety of multi-ship collision avoidance. For the ship navigation experience repository optimized by the ship state extraction module and the navigation experience screening module, a ship guidance main network-ship control learning subnetwork is established.

[0057] The specific operation is:

[0058] The ship guidance main network is constructed to track the path under the constraint of multi-ship collision avoidance, realize group cooperative safety, and construct an optimization objective function, which is represented as: ; is the control action sequence in the future H steps; is the path deviation square of ship i in the t+k step; is the throttle opening square; H is the total number of steps, and k is the step index; 、 and are target weight coefficients; is the collision avoidance safety distance; is the distance between the centers of mass of ship i and ship j in the t+k step;

[0059] The ship control learning subnetwork is constructed. The Q value function of the ship control learning subnetwork is divided into state value and action advantage. The state value corresponds to the overall value of the sea environment, and the action advantage corresponds to the local value of the control action, which strengthens the individual control effect. It is represented as: ; wherein, is the state value, which selects the safety level of the current sea state; is the action advantage, which selects the effect of rudder angle adjustment on heading correction; is the Q value function of ship i, which is the output of the ship control learning subnetwork. The state value and the action advantage are obtained by fitting through a neural network, and are independently trained. The network structure is a fully connected network + batch normalization layer, and are network parameters; is any action in the action space ;

[0060] The loss of the ship control learning subnetwork adds an extreme risk penalty term to strengthen the fitting of close-range collision samples, which is represented as: ; ; wherein is the guiding weight of the ship cooperative control guiding master network; is the Q value evaluation of the ship cooperative control guiding master network to the action of the learning sub-network, which ensures that the learning sub-network strategy does not deviate from the safe direction; M is the number of samples; j is the sample index; is the target Q value of the jth sample.

[0061] Embodiment six, refer to Figure 1 , which is based on the above-mentioned embodiment, the ship parameter adaptive adjustment module is designed to solve the problem that the control strategy is easy to be trapped in strategy rigidification due to environmental dynamic changes in long-term ship navigation. A three-layer mechanism of partial parameter adjustment + periodic opportunity + safety reward is designed. After adjustment, the reward is guided to small action to prevent loss of control;

[0062] The specific operation is: partial parameter adjustment, the ship control network parameters are divided into core safety layer (decide navigation safety, not adjustable) and non-core optimization layer (affect efficiency / energy consumption, adjustable), only the non-core layer is adjusted to ensure that the core functions such as collision avoidance and heading stability are not interrupted, which is represented as: ; is the control network parameter of ship i at time t; is the initial parameter of the non-core layer; is the control network parameter of ship i at time t+1, which is the adjusted control network parameter; the non-core layer includes all weight coefficients; the core layer includes network parameters and collision avoidance safety distance;

[0063] A dynamic adjustment period mechanism of environmental perception is introduced to prevent response lag when the environment changes suddenly and over-adjustment when the environment is stable, which is represented as: ; ; is the time when the parameter adjustment is triggered; is the basic adjustment period; n is the adjustment period number; is the interference coefficient; is the environmental disturbance intensity; , and are the current of the water, the wind speed and the significant wave height at time t, which need to be normalized;

[0064] After adjustment, a safety reward is obtained. After parameter adjustment, the ship control strategy may appear short-term fluctuations, which need to be guided by additional reward to keep the ship safe and avoid entering dangerous encounter situation, which is represented as: ; wherein is the total reward of ship i after parameter adjustment; is the basic safety reward; is the indicator function; is the rudder angle change threshold value; is a rudder angle change amount;

[0065] The task termination condition, the cooperative task, is determined to be terminated only when all three conditions are met simultaneously, and if any one condition is not met, the task does not terminate, and the multi-ship cooperative interaction data collection needs to be supplemented to collect new experience and retrain the multi-ship cooperative control model; the conditions include: the berthing position accuracy of the whole ship meets the standard, the single-ship berthing position deviation is lower than the parking threshold; the total task duration does not exceed the limit; there is no collision risk between the whole ships.

[0066] By performing the above operations, in view of the problems of experience screening blindness, low utilization rate of high-value data, single multi-ship cooperative target, imbalance between individual and group needs, parameter adjustment rigidity, difficulty in adapting to dynamic environment, and poor ship control effect of general ship control system, the scheme performs navigation experience screening through safe sampling guided by the reference path, avoids dangerous data interference; at the same time, the path tracking accuracy, energy consumption and collision avoidance safety are constrained to guide the multi-objective optimization of the main network; the value aliasing is avoided through Q value separation and risk punishment of the learning sub-network; the fitting of near-distance collision samples is strengthened by adding extreme risk punishment; through hierarchical parameter adjustment, the dynamic period of environmental perception is introduced, the safety reward after adjustment is set, and the environmental changes are dynamically adapted to solve the problems of strategy rigidity and safety risk; and the ship control effect is improved.

[0067] Embodiment seven, refer to Figure 1 This embodiment is based on the above-mentioned embodiments. The ship control module acquires real-time ship operation data, inputs it to the ship state refining module for optimization, and inputs it to the multi-ship cooperative control intelligent agent module. Real-time ship control is realized based on the module output; the intelligent agent combines the optimized ship state and multi-ship collision avoidance rules to generate control strategies for rudder angle, throttle, and propeller speed, and then issues the strategy instructions to the hardware of the rudder and actuator, while real-time feedback of the execution effect is used for dynamic fine-tuning, finally realizing real-time and accurate control of the ship, ensuring stable single-ship operation and safe multi-ship cooperation.

[0068] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application.

[0069] The above describes the present application and its embodiments, which are not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto. In summary, if a person skilled in the art is inspired thereby, without departing from the purpose of the present application, without creative design, similar structure and embodiments of the technical solution can be designed, which shall belong to the protection scope of the present application.

Claims

1. An intelligent ship control system, characterized by: The system includes a ship interaction data acquisition module, a ship status extraction module, a navigation experience screening module, a multi-ship collaborative control agent training module, a ship parameter adaptive adjustment module, and a ship control module; The ship interaction data acquisition module acquires the ship's own status data and surrounding ship observation data to build a ship navigation experience repository; The ship state extraction module optimizes the features of samples in the ship navigation experience repository through dual-branch training; The navigation experience screening module defines the value extraction vector of the ship path segment and screens the samples of the ship navigation experience repository; The multi-ship collaborative control agent training module constructs a ship guidance main network and a control learning sub-network based on the filtered and optimized ship navigation experience repository; The ship parameter adaptive adjustment module divides the control network parameters into a core security layer and a non-core optimization layer, and calculates a dynamic adjustment period according to the environmental interference intensity; The ship control module obtains real-time ship operation data, inputs it into the ship status extraction module for optimization, and then inputs it into the multi-ship collaborative control intelligent agent to achieve real-time ship control; The ship status refining module specifically includes: Branch 1, state extraction training based on ship dynamics: Based on the ship MMG model, the next state and environmental interference force are predicted through the current state and action to obtain the initial extraction of the ship; loss function for: ;in, is the predicted next state of the ship, the superscript te represents the true value, the superscript pd represents the predicted value, t represents the time, and i represents the ship; is the predicted water flow force and wind force, c represents the water flow force, Indicates wind force; is the weight of the equilibrium state prediction and the disturbance force prediction; Branch 2, state extraction training based on ship mutual simulation: measures the behavioral similarity of two ships and introduces the dynamic transition probability similarity coefficient; expressed as: ; ;in, is the similarity measure of the behavior of ship i and ship j. The smaller the value, the more similar the behavior. a is the control action, and A is the set of control actions. and is the reward obtained by ships i and j for performing action a; In action a, water flow , wind speed Next, the vessel i is transferred to state probability; Is the ship j transferred to state probability; is the transition probability similarity coefficient; is the benchmark coefficient; is the reference distance; is the distance between the centers of mass of the two ships; is the JS divergence; is the scene correction coefficient; learn a behavior feature encoder to map the initial extraction of ships into low-dimensional behavior features, so that ships with similar behaviors have similar low-dimensional behavior features, and divide the ship data into positive and negative sample pairs through the behavior threshold; define the core loss for the positive sample pairs; define the contrast loss for the negative sample pairs; sample an equal number of positive and negative sample pairs in each iteration, use the Adam optimizer to update the behavior feature encoder parameters, and iterate until the loss converges.

2. The intelligent ship control system according to claim 1, characterized in that: The navigation experience screening module specifically includes: The definition of ship path segment refinement is that the refinement is the difference between the initial state refinement and the terminal state refinement; Refer to the similarity sampling guided by path segments and randomly select a path segment from the experience storage pool; calculate the similarity between all candidate path segments and the selected path segment, and filter path segments with similarity ∈ [0.3, 0.7]; if the number of filtered path segments is greater than the batch size, retain the first K path segments in descending order of reward value, and split each path segment into single-step training samples.

3. The intelligent ship control system according to claim 2, characterized in that: The multi-ship collaborative control agent training module specifically includes: The main ship guidance network is constructed, with the goal of path tracking under multi-ship collision avoidance constraints, and the optimization objective function is constructed; The ship control learning sub-network is constructed. The Q-value function of the ship control learning sub-network is divided into state value and action advantage. The state value corresponds to the overall value of the sea environment, and the action advantage corresponds to the local value of the control action. Both state value and action advantage are obtained through neural network fitting. An extreme risk penalty term is added to the loss of the ship control learning sub-network.

4. An intelligent ship control system according to claim 3, characterized in that: The ship parameter adaptive adjustment module specifically includes: Adjust some parameters, dividing the ship control network parameters into a core security layer and a non-core optimization layer, and only adjusting the non-core layer; Introduce a dynamic adjustment cycle mechanism based on environmental awareness; obtain safety rewards after adjustment.

5. The intelligent ship control system according to claim 4, characterized in that: The ship interaction data acquisition module obtains the interaction data of ships in the digital twin environment. Each ship collects its own status and the observed status of surrounding ships in real time through sensors; after executing the control action, it records the next status and reward; and stores the joint data of multiple ships in the ship navigation experience repository.

6. The intelligent ship control system according to claim 5, characterized in that: The ship control module obtains real-time ship operation data, inputs it into the ship status extraction module for optimization, and inputs it into the multi-ship collaborative control intelligent body module to achieve real-time ship control based on the module output.

Images