Unmanned ship control system and method based on digital twin and deep learning technology

Abstract

The invention discloses an unmanned ship control system based on a digital twin and deep learning technology in the technical field, and the system comprises a perception information fusion module which collects real-time data in real time, and transmits the real-time data to a digital twin module; the digital twin module predicts the change of the marine environment in a period of time in the future through the real-time data collected by the perception information fusion module, and the prediction result provides a basis for subsequent path planning and navigation control; the deep learning module is used for extracting a feature recognition result in the marine environment, and the feature recognition result is used for optimizing path planning and navigation control of the unmanned ship; and the controller execution module is used for receiving a prediction result from the digital twin module and a feature recognition result from the deep learning module, realizing accurate control of a main engine, a steering engine and a load of the unmanned ship, and realizing high-precision and self-adaptive control of the unmanned ship by simulating and predicting a marine environment in real time. And the adaptability and robustness of the unmanned ship in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned ships, and in particular to an unmanned ship control system and method. Background Art

[0002] Traditional unmanned ship control systems usually use fixed PID parameters for control, but this control method often shows certain limitations and shortcomings when facing complex marine environments and operational requirements. In addition, traditional control systems cannot make full use of a large amount of real-time data and historical data, which limits the performance and intelligence level of unmanned ships.

[0003] With the development of science and technology, unmanned ships are increasingly used in the fields of ocean exploration, environmental monitoring, cargo transportation, etc. However, the navigation and control of unmanned ships have always been a technical problem, and traditional navigation systems are difficult to cope with the complex and changeable marine environment. Therefore, a control system that can perceive, learn and make decisions in real time is needed to improve the navigation accuracy and stability of unmanned ships. As an advanced technology, the integration of digital twins and deep learning can realize the interaction between the real world and the virtual world, providing new ideas for the design and optimization of unmanned ship control systems. Summary of the invention

[0004] In response to the shortcomings of the prior art, the present invention provides an unmanned ship control system and method based on digital twin and deep learning technology. By real-time simulation and prediction of the marine environment and combining deep learning algorithms to identify environmental characteristics, high-precision and adaptive control of the unmanned ship can be achieved, thereby improving the adaptability and robustness of the unmanned ship in complex environments.

[0005] The object of the present invention is achieved as follows: an unmanned ship control system based on digital twin and deep learning technology, comprising:

[0006] The perception information fusion module is installed on the unmanned ship to collect real-time data, including the status data of the unmanned ship and the marine environment data, and transmit the real-time data to the digital twin module. The selection of the perception information fusion module depends on the actual application requirements;

[0007] The digital twin module builds and updates the digital twin model through the real-time data collected by the perception information fusion module to simulate the marine environment in the area where the unmanned ship is located. The digital twin module uses historical data, real-time data and ocean dynamics models to predict changes in the marine environment in the future. The prediction results provide a basis for subsequent path planning and navigation control;

[0008] A deep learning module, which establishes a marine environment feature recognition model through training and learning. This module uses marine environment data to train and learn through a deep learning algorithm to extract feature recognition results in the marine environment. The feature recognition results are used to optimize the path planning and navigation control of the unmanned ship;

[0009] The controller execution module receives the prediction results from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm. The controller execution module generates control signals based on the current environment and prediction results to achieve precise control of the unmanned ship's main engine, steering gear, and load.

[0010] Furthermore, the workflow of the digital twin module is as follows: first, real-time data is collected through the perception information fusion module; then, the collected data is used to establish and update the digital twin model; then, the digital twin model is used to simulate the marine environment in the area where the unmanned ship is located; finally, the changes in the marine environment in the future are predicted based on the simulation results.

[0011] Furthermore, the workflow of the deep learning module is as follows: first, prepare the ocean environment data; then, use the ocean environment data to train and learn through the deep learning algorithm; then, extract the key features and patterns in the ocean environment; finally, use the features and patterns to optimize the path planning and navigation control of the unmanned ship.

[0012] Furthermore, the digital twin module can be regularly updated and optimized by collecting more real-time data, introducing more accurate ocean dynamics models, or using more advanced algorithms and technologies.

[0013] Furthermore, the deep learning module can be continuously trained and improved by using more marine environment data, improving the deep learning algorithm, or adjusting training parameters.

[0014] Furthermore, the real-time data collected by the perception information fusion module includes: heading, speed, engine speed, hydrology, and meteorology, and the data is transmitted to the digital twin module and deep learning module through the data interface.

[0015] Furthermore, the perception information fusion module is regularly inspected and calibrated, including testing and maintaining the performance of the sensor and adjusting its installation position.

[0016] A method for controlling an unmanned ship based on digital twin and deep learning technology, characterized in that it comprises the following steps:

[0017] Step 1) Collect real-time data, including unmanned ship status data and marine environment data, through the perception information fusion module, and transmit the real-time data to the digital twin module;

[0018] Step 2) Predict changes in the ocean environment. The digital twin module builds and updates the digital twin model based on the collected real-time data, simulates the ocean environment in the area where the unmanned ship is located, and uses historical data, real-time data and ocean dynamics models to predict changes in the ocean environment in the future. The prediction results provide a basis for subsequent path planning and navigation control;

[0019] Step 3) extracting feature recognition results. The deep learning module establishes a marine environment feature recognition model through training and learning. The module uses marine environment data and performs training and learning through a deep learning algorithm to extract feature recognition results in the marine environment. The feature recognition results are used to optimize the path planning and navigation control of the unmanned ship.

[0020] Step 4) To control the unmanned ship, the controller execution module receives the prediction results from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm. The controller execution module generates control signals based on the current environment and prediction results to achieve precise control of the unmanned ship's main engine, steering gear, and load.

[0021] Furthermore, in step 2), when predicting changes in the ocean environment, more real-time data is collected, more accurate ocean dynamics models are introduced, or more advanced algorithms and technologies are used for regular updates and optimizations; specifically, the following are included:

[0022] 2-1) Data collection and integration;

[0023] Real-time data collection: collect various data of the ocean environment in real time through the perception information fusion module, including wind speed, wind direction, wave height, ocean temperature, salinity, current speed, tidal information, etc.; Historical data integration: retrieve and integrate historical ocean environment data from the data warehouse for comparison and analysis with real-time data;

[0024] 2-2) Establish digital model;

[0025] Marine environment model: Based on real-time data and historical data, a marine environment model of the area is established, which can simulate and predict the dynamic changes of the marine environment; Unmanned ship digital model: including navigation trajectory model, control system model, communication model, path planning model, fault diagnosis model, energy management model and human-computer interaction model; these models can simulate the behavior and response of unmanned ships in the marine environment;

[0026] 2-3) Ocean dynamics model;

[0027] Constructing ocean dynamics models: Using knowledge of ocean physics and fluid mechanics, constructing ocean dynamics models to simulate physical processes in the ocean environment, such as the generation and propagation of waves, the evolution of ocean currents, etc.;

[0028] 2-4) Predictive Analysis

[0029] Data preprocessing: Perform preprocessing operations such as cleaning, interpolation and normalization on the collected real-time data to improve the accuracy of the prediction; Model training and verification: Use historical data to train and verify the marine environment model and the unmanned ship digital model to ensure that the model can accurately simulate and predict changes in the marine environment; Predict future marine environmental changes: Input real-time data into the marine environment model to simulate the changing trend of the marine environment in the future; Consider the output results of the ocean dynamics model, such as wave height, changes in the ocean flow field, etc., to correct and optimize the marine environment model; Unmanned ship navigation control deduction: Based on the predicted changes in the marine environment, perform navigation control deduction on the unmanned ship digital model; Simulate the response and performance of the unmanned ship in the marine environment under different navigation strategies; Evaluate the feasibility and effectiveness of various navigation strategies to provide reference for subsequent path planning and navigation control;

[0030] 2-5) Result output and interpretation;

[0031] Prediction result output: output the predicted marine environment change results in the form of visualization or data tables to facilitate user understanding and use; Result interpretation: interpret and analyze the prediction results, including the trend, cause and possible impact of marine environment change;

[0032] 2-6) Feedback and optimization;

[0033] Practical application feedback: Apply the prediction results to the actual unmanned ship navigation and collect actual data and feedback during the navigation process; Model optimization: Optimize and improve the marine environment model and unmanned ship digital model based on actual application feedback and data analysis results to improve the accuracy and reliability of the prediction.

[0034] Furthermore, when extracting the feature recognition results in step 3), continuous training and improvement are performed by using more marine environment data, improving the deep learning algorithm, or adjusting the training parameters, including:

[0035] 3-1) The deep learning module mainly maps the bottom-level features to the high-level features through the deep neural network, and abstracts the features through the high-level features; deep learning adopts a layered structure similar to that of neural networks. The multi-layer network structure consists of an input layer, a hidden layer, and an output layer, and each connection between neurons has a weight; the deep learning model constructs a multi-layer network, and the input of the next layer is the output of the previous layer. Through multi-layer nonlinear operations, the extracted bottom-level features of the samples are combined into more abstract high-level features to achieve visualization of high-level features; the deep network contains multi-layer nonlinear mapping and has multiple hidden layers. Each layer can extract corresponding features. The model obtains high-level features that are conducive to classification through extraction and combination;

[0036] 3-2) The deep learning neural network models used in the marine environment include convolutional neural network CNN and recurrent neural network RNN;

[0037] 3-2-1) Convolutional neural network CNN consists of input layer, convolution layer, pooling layer, fully connected layer and output layer; the convolution layer uses convolution kernel to learn the features of input data. The size and number of convolution kernels are set manually, and the weights in the convolution kernels are shared; the pooling layer downsamples the data from the convolution layer, and the fully connected layer connects the data features and outputs the data in the required dimensional form;

[0038] 3-2-2) Recurrent Neural Network (RNN) mines the characteristics of the time dimension through the order of events. RNN introduces directional loops to connect hidden nodes into loops. It connects different neurons in the same layer and the network layers at the same time can share weight parameters. The input and output of RNN are both vector sequences. The input data X is connected to the hidden layer containing weights and bias terms after linear operations, and the output value is obtained after linear operations. W is the weight of the hidden layer. The hidden layers are connected through nodes and share weights in the sequence data. The weight W will be updated after each hour to mine the characteristics of sequence changes. The final output will be affected by the previous multiple input layers. The specific calculation formula of RNN is as follows:

[0039] hk=fh(Wihxk+Whhxhk–l+bh)

[0040] Among them, Wih is the connection matrix from the input layer to the hidden layer; Whh is the connection matrix between the hidden layer at adjacent moments; Who is the connection matrix from the hidden layer to the output layer; hk is the state vector of the RNN hidden layer at time k; bh is the bias vector; fh is the nonlinear activation function; in RNN, fh is usually a sigmoid or tanh function; the process of calculating ok through hk is:

[0041] ok=fo(Whohk+bo),

[0042] Among them, bo is the bias vector of the output layer; fo is the nonlinear activation function; the gating algorithm in RNN can be divided into LSTM and GRU;

[0043] 3-3) Prediction and extraction of marine environmental information, using long short-term memory network LSTM and gated recurrent neural unit network GRU;

[0044] 3-3-1) Long Short-Term Memory Network (LSTM) adds gate structure and cell state to RNN. The gate structure is used to control the amount of information retained at the previous moment, while the cell state is used to record the characteristics of the time series. It uses gate functions to control the input and output of information in neurons. The main parameter formula of LSTM is as follows

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] Among them, i, f, o represent the input gate, forget gate and output gate respectively; x represents the network input; W represents the network parameter, b represents the bias; h represents the hidden state; represents the sigmoid function; the weight is placed in the convolution kernel and the loop kernel of the loop layer; the convolution kernel W i , W f , W o The convolution result is obtained by sliding the window of the two-dimensional matrix in space, and then inputting, updating the cell state, forgetting, and outputting according to the gate structure. represents the Hadamard product;

[0051] 3-3-2) The gated recurrent neural unit network GRU has two gates: one update gate and one reset gate; the GRU model is a spatial model based on convolution, and the formula is as follows:

[0052] Zt=σ(Wz·[ht-1,xt]),

[0053] rt=σ(Wr·[ht-1,xt],

[0054]

[0055]

[0056] 3-4) Ocean environment characteristics, patterns, wave and current characteristics: Deep learning algorithms analyze the height, period, direction of waves and the speed and direction parameters of currents to perform precise navigation control.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows: the unmanned ship control system based on digital twin and deep learning technology and its application method provided by the present invention can realize high-precision and adaptive control of unmanned ships, and have broad application prospects and market value; through real-time simulation and prediction of the marine environment by digital twin technology, and identification of environmental characteristics by deep learning algorithm, the accuracy and stability of unmanned ship navigation can be improved, and adaptive control of unmanned ships can be realized at the same time, and the environmental adaptability of unmanned ships can be improved; by continuously optimizing and improving digital twin models, deep learning algorithms and control strategies, the navigation performance and efficiency of unmanned ships can be further improved; therefore, the control system of the present invention not only improves the navigation accuracy and stability of unmanned ships, but also provides strong technical support for marine exploration, environmental monitoring, cargo transportation and other fields, and has broad application prospects and market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0059] Figure 1 This is the control system structure of the unmanned ship of the present invention.

[0060] Figure 2 This is a schematic diagram of the connection between the digital twin module and the perception information fusion module in the present invention.

[0061] Figure 3 Schematic diagram of the connection between the deep learning module and the controller execution module in the present invention.

[0062] Figure 4 This is a workflow diagram of the deep learning module in the present invention.

[0063] Figure 5 This is the workflow diagram of the digital twin module in the present invention.

[0064] Figure 6 This is a flowchart of the controller execution module in the present invention.

[0065] Figure 7 This is a workflow diagram of the perception information module in the present invention.

[0066] Figure 8Schematic diagram of the convolutional neural network CNN in the present invention.

[0067] Fig. 9 Schematic diagram of the recurrent neural network RNN ​​in the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] Example 1

[0070] In this embodiment, an unmanned ship control system based on digital twin and deep learning technology includes a digital twin module, a deep learning module, a perception information fusion module and a controller execution module.

[0071] The digital twin module establishes and updates the digital twin model through the real-time data collected by the perception information fusion module to simulate the marine environment in the area where the unmanned ship is located; the digital twin module uses historical data, real-time data and ocean dynamics models to predict changes in the marine environment in the future; the prediction results provide a basis for subsequent path planning and navigation control.

[0072] The deep learning module builds a model for identifying marine environment features through training and learning. The module uses a large amount of marine environment data to train and learn through deep learning algorithms to extract key features and patterns in the marine environment. These features and patterns are used to optimize the path planning and navigation control of the unmanned ship.

[0073] The perception information fusion module is installed on the unmanned ship to collect environmental data (such as hydrology, meteorology, etc.) in real time and transmit the data to the digital twin module. The selection of the perception information fusion module should be based on the actual application requirements to ensure the accuracy and real-time nature of the data.

[0074] The controller execution module receives the prediction data from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm. The control algorithm can be traditional PID control, fuzzy control, etc., or a deep learning algorithm. The controller execution module generates control signals based on the current environment and prediction results to achieve precise control of the unmanned ship.

[0075] In this embodiment, the digital twin technology is used to simulate and predict the marine environment in real time, which improves the accuracy and stability of the unmanned ship's navigation; the deep learning algorithm is used to identify environmental features, realize the adaptive control of the unmanned ship, and improve the environmental adaptability of the unmanned ship. It can be widely used in the fields of marine exploration, environmental monitoring, cargo transportation, etc., and has broad application prospects and market value.

[0076] Example 2

[0077] This embodiment is an unmanned ship control system based on digital twin and deep learning technology, which is improved on the basis of embodiment 1; on the basis of embodiment 1, regular updating and optimization of the digital twin module is added to improve its accuracy in simulating the marine environment and predicting future environmental changes. This can be achieved by collecting more real-time data, introducing more accurate ocean dynamics models, or using more advanced algorithms and technologies. At the same time, the deep learning module is continuously trained and improved to adapt to the changing environmental characteristics and patterns. This can be achieved by using more marine environmental data, improving the deep learning algorithm, or adjusting the training parameters.

[0078] In addition, the inspection and calibration of the perception information fusion module has been added to ensure the accuracy and real-time nature of its collected data. This includes testing and maintaining the performance of the sensor, as well as adjusting its installation position to obtain the best measurement effect. At the same time, according to the actual application requirements, the control algorithm is continuously adjusted and optimized to improve the accuracy and stability of the unmanned ship's path planning and navigation control. This can be done through experiments and tests to find the best control parameters and strategies.

[0079] In this embodiment, by adding updates, training, calibration and adjustments to the digital twin module, deep learning module, perception information fusion module and controller execution module, the navigation performance and efficiency of the unmanned ship are further improved. The system has a wider application prospect and market value, and is expected to become an important development direction in the field of unmanned ship navigation and control in the future.

[0080] Example 3

[0081] A method for controlling an unmanned ship based on digital twin and deep learning technology comprises the following steps:

[0082] Step 1: Use the unmanned ship control system of Example 1 or Example 2 ( Figure 1 ), its main functional modules, including the digital twin module ( Figure 5 ), Deep Learning Module( Figure 4 ), perception information fusion module ( Figure 7 ) and the controller execution module ( Figure 6 ).

[0083] Step 2: The digital twin module receives the information from the perception information fusion module ( Figure 2 ) to build a digital model of the unmanned ship (ocean environment model of the area, navigation trajectory model, control system model, communication model, path planning model, fault diagnosis model, energy management model, human-computer interaction model, etc.). The digital twin module uses historical data, real-time data, unmanned ship digital models and ocean dynamics models to predict changes in the ocean environment in the future and the navigation control deduction of the unmanned ship caused by such changes; the prediction results provide a basis for subsequent path planning and navigation control; predicting changes in the ocean environment is a comprehensive process involving data collection, integration, modeling and simulation analysis. The specific steps of the process are as follows:

[0084] 2-1) Data collection and integration;

[0085] Real-time data collection: through the perception information fusion module ( Figure 2 ) collects various data on the ocean environment in real time, including wind speed, wind direction, wave height, ocean temperature, salinity, current speed, tidal information, etc.

[0086] Historical Data Integration: Retrieve and integrate historical marine environmental data from the data warehouse for comparison and analysis with real-time data.

[0087] 2-2) Establish digital model;

[0088] Marine environment model: Based on real-time and historical data, a marine environment model of the area is established, which can simulate and predict the dynamic changes of the marine environment.

[0089] Unmanned ship digital model: including navigation trajectory model, control system model, communication model, path planning model, fault diagnosis model, energy management model and human-computer interaction model, etc. These models can simulate the behavior and response of unmanned ships in the marine environment.

[0090] 2-3) Ocean dynamics model;

[0091] Constructing ocean dynamics models: Using the knowledge of ocean physics and fluid mechanics, constructing ocean dynamics models to simulate the physical processes of the ocean environment, such as the generation and propagation of waves, the evolution of ocean flow fields, etc.

[0092] 2-4) Predictive Analysis

[0093] Data preprocessing: Perform preprocessing operations such as cleaning, interpolation, and normalization on the collected real-time data to improve the accuracy of the prediction.

[0094] Model training and validation: Use historical data to train and validate the marine environment model and unmanned vessel digital model to ensure that the model can accurately simulate and predict changes in the marine environment.

[0095] Predict future changes in the ocean environment: Input real-time data into the ocean environment model to simulate the changing trend of the ocean environment in the future. Consider the output results of the ocean dynamics model, such as wave height, changes in ocean flow fields, etc., to correct and optimize the ocean environment model.

[0096] Unmanned ship navigation control simulation: Based on the predicted changes in the ocean environment, the navigation control simulation of the unmanned ship digital model is carried out; the response and performance of the unmanned ship in the ocean environment under different navigation strategies are simulated; the feasibility and effectiveness of various navigation strategies are evaluated to provide a reference for subsequent path planning and navigation control.

[0097] 2-5) Result output and interpretation;

[0098] Forecast result output: The predicted marine environment change results are output in the form of visualization or data tables to facilitate user understanding and use.

[0099] Result interpretation: Interpret and analyze the forecast results, including the trends, causes and possible impacts of changes in the marine environment.

[0100] 2-6) Feedback and optimization;

[0101] Practical application feedback: Apply the prediction results to the actual unmanned ship navigation and collect actual data and feedback during the navigation process.

[0102] Model optimization: Based on actual application feedback and data analysis results, the marine environment model and unmanned ship digital model are optimized and improved to improve the accuracy and reliability of the prediction.

[0103] Through the above steps, the digital twin module can accurately predict changes in the ocean environment and provide a strong basis for subsequent path planning and navigation control.

[0104] Step 3: Connect the deep learning module to the controller execution module ( Figure 3 ) Through training and learning, a model for identifying marine environmental features is established. This module uses a large amount of marine environmental data to train and learn through deep learning algorithms, predict marine environmental changes, and extract key features and patterns in the marine environment. These features and patterns are used to optimize the path planning and navigation control of the unmanned ship.

[0105] 3-1) The deep learning module mainly maps the bottom-level features to the top-level through the deep neural network (DNN), and abstracts the features through the top-level. Deep learning uses a layered structure similar to that of neural networks. The multi-layer network structure consists of an input layer, a hidden layer, and an output layer. Each connection between neurons has a certain weight. The deep learning model constructs a complex multi-layer network. The input of the next layer is the output of the previous layer. Through multi-layer nonlinear operations, the extracted bottom-level features of the samples are combined into more abstract high-level features to achieve visualization of high-level features. The deep network contains multi-layer nonlinear mapping and has multiple hidden layers. Each layer can extract corresponding features. The model obtains high-level features that are conducive to classification through extraction and combination. Its characteristics can be summarized as deep, nonlinear, and layer-by-layer feature extraction.

[0106] 3-2) Deep learning neural network models used in marine environments include convolutional neural networks (CNN) and recurrent neural networks (RNN).

[0107] 3-2-1) If Figure 8 As shown in the figure, the convolutional neural network (CNN), as a multi-layer feedforward neural network with a convolution structure, is a deep learning model commonly used for marine target recognition and detection. CNN is composed of multiple layers of neural networks, and each layer of the neural network is composed of multiple two-dimensional planes, with multiple independent neurons in each plane. CNN is composed of an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. The convolution layer uses the convolution kernel to learn the features of the input data. The size and number of the convolution kernel are set manually, and the weights in the convolution kernel are shared. The pooling layer downsamples the data from the convolution layer. The advantage of this method is that the receptive field becomes larger, the amount of data is continuously compressed, and the number of parameters is significantly reduced. The main function of the fully connected layer is to connect the data features and output the data in the required dimensional form.

[0108] 3-2-2) If Fig. 9As shown in the figure, the recurrent neural network (RNN) is a commonly used deep learning model for predicting marine environmental information factors. It mines the characteristics of the time dimension through the order of events. RNN introduces a directional loop to connect hidden nodes in a directional manner, which is more conducive to information transmission. It connects different neurons in the same layer and the network layers at the same time can share weight parameters. The input of RNN is a vector sequence, and the output is also a vector sequence, which reflects the time series of the data (see the figure below). After linear operation, the input data X is connected to the hidden layer containing weights and bias terms, and the output value is output after linear operation. Among them, W is the weight of the hidden layer. The hidden layers are connected by nodes and share weights in the sequence data. The weight W will be updated after each hour (h) to mine the characteristics of the sequence changes. The final output will be affected by the previous multiple input layers. The specific calculation formula of RNN is as follows:

[0109] hk=fh(Wihxk+Whhxhk–l+bh)

[0110] Among them, Wih is the connection matrix from the input layer to the hidden layer; Whh is the connection matrix between the hidden layer at adjacent moments; Who is the connection matrix from the hidden layer to the output layer; hk is the state vector of the RNN hidden layer at moment k; bh is the bias vector; fh is the nonlinear activation function; in RNN, fh is usually a sigmoid or tanh function. The process of calculating ok through hk is:

[0111] ok=fo(Whohk+bo),

[0112] Among them, bo is the bias vector of the output layer; fo is the nonlinear activation function. The gating algorithms in RNN can be divided into LSTM and GRU.

[0113] 3-3) Prediction and extraction of marine environmental information;

[0114] 3-3-1) Long Short-Term Memory Network LSTM;

[0115] LSTM adds gate structure and cell state to RNN. The gate structure is used to control the amount of information retained at the previous moment, while the cell state is used to record time series features. It uses gate functions to control the input and output of information in neurons, overcoming the problems of RNN being difficult to train and gradient disappearance.

[0116] According to the main parameter formula of LSTM, stands for Hadamard product.

[0117]

[0118]

[0119]

[0120]

[0121]

[0122] Among them, i, f, o represent input gate, forget gate and output gate respectively; x represents network input; W represents network parameter, b represents bias; h represents hidden state. represents sigmoid function. The weight is placed in the convolution kernel and the loop kernel of the loop layer. The convolution kernels Wi, Wf, and Wo slide in the two-dimensional matrix of space, and the convolution results are input, cell state update, forget, and output according to the gate structure.

[0123] 3-3-2) Gated Recurrent Neural Unit Network GRU

[0124] The gated recurrent neural unit network GRU has 2 gates: 1 update gate and 1 reset gate. Compared with the 3 gates of LSTM, GRU requires fewer and simpler parameters, but can maintain comparable performance to LSTM, and has a faster training convergence time. The GRU model is a spatial model based on convolution, and the formula is as follows:

[0125] Zt=σ(Wz·[ht-1,xt]),

[0126] rt = σ(Wr · [ht-1, xt],

[0127]

[0128]

[0129] 3-4) Marine environment characteristics and patterns

[0130] Characteristics of waves and currents: Deep learning algorithms can analyze the height, period, direction of waves, as well as the speed, direction and other parameters of currents, helping unmanned boats better adapt to the dynamic environment of the ocean and perform precise navigation control.

[0131] Deep learning algorithms can deeply analyze the complexity and dynamics of the ocean environment and extract the key features of waves and currents. Parameters such as the height, period and direction of waves, as well as the speed and direction of currents, are crucial for the navigation control of unmanned boats. Through deep learning algorithms, unmanned boats can sense the changes in these parameters in real time and adjust their navigation attitude and speed accordingly to better adapt to the dynamic environment of the ocean. For example, when encountering large waves or strong currents, unmanned boats can automatically adjust their course and speed to avoid navigation deviations or instability caused by the impact of waves and currents.

[0132] In addition, deep learning algorithms can also sense and predict marine meteorological characteristics in real time, such as wind speed, wind direction, temperature and humidity. These meteorological characteristics have an important impact on the navigation stability, energy consumption and safety of unmanned boats. Through deep learning algorithms, unmanned boats can give early warnings and respond to potential severe weather conditions, such as strong winds and heavy rains, to ensure safe and stable navigation. For example, when strong winds are predicted to be coming, unmanned boats can automatically adjust their routes or find a safe harbor to avoid navigation accidents caused by weather reasons.

[0133] Marine meteorological characteristics: including wind speed, wind direction, temperature, humidity, etc. These characteristics have an important impact on the navigation stability, energy consumption and safety of unmanned boats. Deep learning algorithms can sense and predict these meteorological changes in real time, and provide timely navigation adjustment suggestions for unmanned boats.

[0134] Deep learning algorithms also demonstrate excellent performance in target detection and recognition. On the sea, obstacles such as ships, buoys, and reefs are targets that need to be focused on during the navigation of unmanned boats. Through deep learning algorithms, unmanned boats can efficiently identify these obstacles and accurately measure their position, speed, heading and other parameters. This information is crucial for the obstacle avoidance decision-making of unmanned boats. For example, when encountering a ship in front, the unmanned boat can automatically adjust its heading and speed to avoid collision with it.

[0135] In addition, in some special application scenarios, deep learning algorithms can also identify marine life such as dolphins and turtles. These organisms are of great significance to the balance and stability of the marine ecosystem. Through deep learning algorithms, unmanned boats can avoid disturbing or harming these organisms during navigation, thereby protecting the health and sustainable development of the marine ecosystem.

[0136] In summary, the application of deep learning algorithms in the navigation control of unmanned boats, especially in the aspects of marine environment perception and target detection and recognition, provides strong technical support and decision-making basis for unmanned boats. The improvement of these capabilities helps unmanned boats to achieve safe, efficient and autonomous navigation in complex marine environments.

[0137] Step 4: The perception information fusion module is installed on the unmanned ship to collect status data (heading, speed, engine speed, etc.) and environmental data (such as hydrology, meteorology, etc.) in real time, and transmit the data to the digital twin module and deep learning module through a specific data interface. The selection of the perception information fusion module should be based on actual application requirements to ensure the accuracy and real-time nature of the data.

[0138] Step 5: The controller execution module receives the prediction data from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm; the control algorithm adopts traditional PID control, fuzzy control, etc., and can also adopt deep learning algorithm; the controller execution module generates control signals according to the current environment and prediction results to achieve precise control of the unmanned ship.

[0139] Step 6: In actual applications, the unmanned ship control system based on digital twin and deep learning technology of the present invention can perform adaptive path planning and navigation control according to the real-time environment and ship status; by continuously optimizing and improving the digital twin model, deep learning algorithm and control strategy, the navigation accuracy, stability and environmental adaptability of the unmanned ship can be further improved.

[0140] Step 7: Regularly update and optimize the digital twin module to improve its accuracy in simulating the ocean environment and predicting future environmental changes; this can be achieved by collecting more real-time data, introducing more accurate ocean dynamics models, or using more advanced algorithms and technologies.

[0141] Step 8: Continuously train and improve the deep learning module to adapt to changing environmental characteristics and patterns. This can be achieved by using more marine environmental data, improving the deep learning algorithm, or adjusting training parameters.

[0142] Step 9: Regularly check and calibrate the perception information fusion module to ensure the accuracy and real-time nature of the data it collects. This includes testing and maintaining the performance of the sensor and adjusting its installation position to obtain the best measurement results.

[0143] Step 10: According to the actual application requirements, continuously adjust and optimize the control algorithm to improve the accuracy and stability of the unmanned ship's path planning and navigation control. This can be done through experiments and tests to find the best control parameters and strategies.

[0144] Step 11: According to the actual application requirements, select the appropriate perception information fusion module and controller execution module, and connect them with the digital twin module and deep learning module.

[0145] Step 12: According to the navigation requirements of the unmanned ship, set the parameters of the digital twin module and deep learning module and initialize them.

[0146] Step 13: Start the control system, the digital twin module begins to simulate and predict the ocean environment, and the deep learning module begins to learn and identify environmental characteristics.

[0147] Step 14: The controller execution module receives the prediction data of the digital twin module and the feature recognition results of the deep learning module, and performs path planning and navigation control according to the preset control algorithm.

[0148] Step 15: Send the control signal to the unmanned boat actuator to achieve precise control of the unmanned boat.

[0149] Step 16: During the application process, according to the actual navigation results and environmental changes, continuously adjust and optimize the parameters of the digital twin module, deep learning module and controller execution module to improve the navigation performance and efficiency of the unmanned ship.

[0150] The above embodiments are only used to help understand the method and core idea of ​​the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An unmanned ship control system based on digital twin and deep learning technology, characterized in that: include: The perception information fusion module is installed on the unmanned ship to collect real-time data, including the status data of the unmanned ship and the marine environment data, and transmit the real-time data to the digital twin module. The selection of the perception information fusion module depends on the actual application requirements; The digital twin module builds and updates the digital twin model through the real-time data collected by the perception information fusion module to simulate the marine environment in the area where the unmanned ship is located. The digital twin module uses historical data, real-time data and ocean dynamics models to predict changes in the marine environment in the future. The prediction results provide a basis for subsequent path planning and navigation control; A deep learning module, which establishes a marine environment feature recognition model through training and learning. This module uses marine environment data to train and learn through a deep learning algorithm to extract feature recognition results in the marine environment. The feature recognition results are used to optimize the path planning and navigation control of the unmanned ship; The controller execution module receives the prediction results from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm. The controller execution module generates control signals based on the current environment and prediction results to achieve precise control of the unmanned ship's main engine, steering gear, and load.

2. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 is characterized in that: The workflow of the digital twin module is as follows: first, real-time data is collected through the perception information fusion module; then, the collected data is used to establish and update the digital twin model; then, the digital twin model is used to simulate the marine environment in the area where the unmanned ship is located; finally, the changes in the marine environment in the future are predicted based on the simulation results.

3. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 or 2, characterized in that: The deep learning module workflow is as follows: first, prepare marine environment data; then, use the marine environment data to train and learn through a deep learning algorithm; then, extract key features and patterns in the marine environment; Finally, the features and patterns are used to optimize the path planning and navigation control of the unmanned ship.

4. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 or 2, characterized in that: The digital twin module can be regularly updated and optimized by collecting more real-time data, introducing more accurate ocean dynamics models, or using more advanced algorithms and technologies.

5. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 or 2, characterized in that: The deep learning module can be continuously trained and improved by using more ocean environment data, improving the deep learning algorithm, or adjusting training parameters.

6. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 or 2, characterized in that: The real-time data collected by the perception information fusion module includes heading, speed, engine speed, hydrology, and meteorology, and the data is transmitted to the digital twin module and deep learning module through the data interface.

7. The unmanned ship control system based on digital twin and deep learning technology according to claim 1 or 2, characterized in that: The perception information fusion module is regularly inspected and calibrated, including testing and maintaining the performance of the sensor and adjusting its installation position.

8. An unmanned ship control method based on digital twin and deep learning technology, characterized in that: The following steps are involved: Step 1) Collect real-time data, including unmanned ship status data and marine environment data, through the perception information fusion module, and transmit the real-time data to the digital twin module; Step 2) Predict changes in the ocean environment. The digital twin module builds and updates the digital twin model based on the collected real-time data, simulates the ocean environment in the area where the unmanned ship is located, and uses historical data, real-time data and ocean dynamics models to predict changes in the ocean environment in the future. The prediction results provide a basis for subsequent path planning and navigation control; Step 3) extracting feature recognition results. The deep learning module establishes a marine environment feature recognition model through training and learning. The module uses marine environment data and performs training and learning through a deep learning algorithm to extract feature recognition results in the marine environment. The feature recognition results are used to optimize the path planning and navigation control of the unmanned ship. Step 4) To control the unmanned ship, the controller execution module receives the prediction results from the digital twin module and the feature recognition results from the deep learning module, and performs path planning and navigation control according to the preset control algorithm. The controller execution module generates control signals based on the current environment and prediction results to achieve precise control of the unmanned ship's main engine, steering gear, and load.

9. The unmanned ship control method based on digital twin and deep learning technology according to claim 8 is characterized in that: Step 2) When predicting changes in the ocean environment, more real-time data can be collected, more accurate ocean dynamics models can be introduced, or more advanced algorithms and technologies can be used for regular updates and optimization; specifically: 2-1) Data collection and integration; Real-time data collection: collect various data of the ocean environment in real time through the perception information fusion module, including wind speed, wind direction, wave height, ocean temperature, salinity, current speed, tidal information, etc.; Historical data integration: retrieve and integrate historical ocean environment data from the data warehouse for comparison and analysis with real-time data; 2-2) Establish digital model; Marine environment model: Based on real-time data and historical data, a marine environment model of the area is established, which can simulate and predict the dynamic changes of the marine environment; Unmanned ship digital model: including navigation trajectory model, control system model, communication model, path planning model, fault diagnosis model, energy management model and human-computer interaction model; these models can simulate the behavior and response of unmanned ships in the marine environment; 2-3) Ocean dynamics model; Constructing ocean dynamics models: Using knowledge of ocean physics and fluid mechanics, constructing ocean dynamics models to simulate physical processes in the ocean environment, such as the generation and propagation of waves, the evolution of ocean currents, etc.; 2-4) Predictive Analysis Data preprocessing: Perform preprocessing operations such as cleaning, interpolation and normalization on the collected real-time data to improve the accuracy of the prediction; Model training and verification: Use historical data to train and verify the marine environment model and the unmanned ship digital model to ensure that the model can accurately simulate and predict changes in the marine environment; Predict future marine environmental changes: Input real-time data into the marine environment model to simulate the changing trend of the marine environment in the future; Consider the output results of the ocean dynamics model, such as wave height, changes in the ocean flow field, etc., to correct and optimize the marine environment model; Unmanned ship navigation control deduction: Based on the predicted changes in the marine environment, perform navigation control deduction on the unmanned ship digital model; Simulate the response and performance of the unmanned ship in the marine environment under different navigation strategies; Evaluate the feasibility and effectiveness of various navigation strategies to provide reference for subsequent path planning and navigation control; 2-5) Result output and interpretation; Prediction result output: output the predicted marine environment change results in the form of visualization or data tables to facilitate user understanding and use; Result interpretation: interpret and analyze the prediction results, including the trend, cause and possible impact of marine environment change; 2-6) Feedback and optimization; Practical application feedback: Apply the prediction results to the actual unmanned ship navigation and collect actual data and feedback during the navigation process; Model optimization: Optimize and improve the marine environment model and unmanned ship digital model based on actual application feedback and data analysis results to improve the accuracy and reliability of the prediction.

10. The unmanned ship control method based on digital twin and deep learning technology according to claim 8, characterized in that: Step 3) When extracting feature recognition results, continuous training and improvement is performed by using more marine environment data, improving the deep learning algorithm, or adjusting training parameters, including: 3-1) The deep learning module mainly maps the bottom-level features to the high-level features through the deep neural network, and abstracts the features through the high-level features; deep learning adopts a layered structure similar to that of neural networks. The multi-layer network structure consists of an input layer, a hidden layer, and an output layer, and each connection between neurons has a weight; the deep learning model constructs a multi-layer network, and the input of the next layer is the output of the previous layer. Through multi-layer nonlinear operations, the extracted bottom-level features of the samples are combined into more abstract high-level features to achieve visualization of high-level features; the deep network contains multi-layer nonlinear mapping and has multiple hidden layers. Each layer can extract corresponding features. The model obtains high-level features that are conducive to classification through extraction and combination; 3-2) The deep learning neural network models used in the marine environment include convolutional neural network CNN and recurrent neural network RNN; 3-2-1) Convolutional neural network CNN consists of input layer, convolution layer, pooling layer, fully connected layer and output layer; the convolution layer uses convolution kernel to learn the features of input data. The size and number of convolution kernels are set manually, and the weights in the convolution kernels are shared; the pooling layer downsamples the data from the convolution layer, and the fully connected layer connects the data features and outputs the data in the required dimensional form; 3-2-2) Recurrent Neural Network (RNN) mines the characteristics of the time dimension through the order of events. RNN introduces directional loops to connect hidden nodes into loops. It connects different neurons in the same layer and the network layers at the same time can share weight parameters. The input and output of RNN are both vector sequences. The input data X is connected to the hidden layer containing weights and bias terms after linear operations, and the output value is obtained after linear operations. W is the weight of the hidden layer. The hidden layers are connected through nodes and share weights in the sequence data. The weight W will be updated after each hour to mine the characteristics of sequence changes. The final output will be affected by the previous multiple input layers. The specific calculation formula of RNN is as follows: hk=fh(Wihxk+Whhxhk–l+bh) Among them, Wih is the connection matrix from the input layer to the hidden layer; Whh is the connection matrix between the hidden layer at adjacent moments; Who is the connection matrix from the hidden layer to the output layer; hk is the state vector of the RNN hidden layer at time k; bh is the bias vector; fh is the nonlinear activation function; in RNN, fh is usually a sigmoid or tanh function; the process of calculating ok through hk is: ok=fo(Whohk+bo), Among them, bo is the bias vector of the output layer; fo is the nonlinear activation function; the gating algorithm in RNN can be divided into LSTM and GRU; 3-3) Prediction and extraction of marine environmental information, using long short-term memory network LSTM and gated recurrent neural unit network GRU; 3-3-1) Long Short-Term Memory Network (LSTM) adds gate structure and cell state to RNN. The gate structure is used to control the amount of information retained at the previous moment, while the cell state is used to record the characteristics of the time series. It uses gate functions to control the input and output of information in neurons. The main parameter formula of LSTM is as follows Among them, i, f, o represent the input gate, forget gate and output gate respectively; x represents the network input; W represents the network parameter, b represents the bias; h represents the hidden state; represents the sigmoid function; the weight is placed in the convolution kernel and the loop kernel of the loop layer; the convolution kernel W i , W f , W o The convolution result is obtained by sliding the window of the two-dimensional matrix in space, and then inputting, updating the cell state, forgetting, and outputting according to the gate structure. represents the Hadamard product; 3-3-2) The gated recurrent neural unit network GRU has two gates: one update gate and one reset gate; the GRU model is a spatial model based on convolution, and the formula is as follows: Z t =σ(W z ·[h t-1 ,x t ]), r t =σ(W r ·[h t-1 ,x t ], 3-4) Ocean environment characteristics, patterns, wave and current characteristics: Deep learning algorithms analyze the height, period, direction of waves and the speed and direction parameters of currents to perform precise navigation control.

Images