Ship berthing safety auxiliary control system based on multi-data fusion
Through multi-data fusion technology and machine learning algorithms, the shortcomings of ship mooring systems based on image processing in complex environments are solved, and more efficient and safe ship mooring control is achieved, reducing costs and operational complexity, and is suitable for small and medium-sized ships. .
Patent Information
- Application Number
- CN202410971566.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing ship mooring safety auxiliary control system based on image processing faces problems such as reduced image quality, insufficient processing speed, strong dependence on environmental markers, high operator skills and high cost in complex marine environments, which limits its wide application. and the popularization of small and medium-sized ships.
Using multi-data fusion technology, collects multiple types of data through integrated sensors, optimizes data quality using machine learning and deep learning methods, combines Kalman filters and Bayesian networks for data fusion, establishes a ship dynamic behavior model, and utilizes reinforcement learning The algorithm generates real-time control instructions to adjust the speed and heading.
It improves the safety and decision-making support capabilities during ship mooring, enhances the dynamic response capabilities of the system, reduces dependence on environmental markers, reduces operational complexity and cost, and improves the application potential of small and medium-sized ships.
Smart Images

Figure CN118915747A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a ship mooring safety auxiliary control system, in particular to a ship mooring safety auxiliary control system based on multi-data fusion. Background Art
[0002] Although the current auxiliary control systems for ship mooring safety have made significant progress in technology development and practical application, there are still some shortcomings and drawbacks. The image processing-based ship mooring control system and control method described in the Chinese invention patent, patent number: 201710026020X, although demonstrating the potential of image processing technology in improving the accuracy and automation of ship mooring, these systems still face several challenges in terms of wider application and operational performance in different environments. First, image processing-based systems rely on high-quality image data, which requires good visibility under all navigation conditions. However, the actual marine environment is often affected by dense fog, heavy rain, strong light or night conditions, all of which seriously reduce the clarity and accuracy of the image. For example, at night or in low visibility, even advanced image processing systems have difficulty accurately capturing markers or identifying the exact position and heading of the ship. In addition, reflections on the sea surface, direct sunlight or background light from offshore facilities may also cause overexposure of the image, which in turn affects the system's judgment and decision-making ability.
[0003] Secondly, the image processing system requires extremely high processing speed and real-time response capabilities. During the ship's berthing process, the rapid changes in the environment and ship situation require the system to process and respond to image data instantly. However, image processing and analysis are usually computationally intensive, requiring powerful hardware support and efficient algorithms to meet real-time requirements. When there are delays, failures or processing bottlenecks in the hardware or software, the system will not respond in time, increasing operational risks. In addition, for image-based systems, correctly identifying and interpreting objects in the image (such as other ships, shore facilities, surface obstacles, etc.) is a major challenge. This not only depends on the quality of the image, but also requires highly accurate algorithms to distinguish and identify various elements in the image. In complex marine environments, such as ports or busy waters, multi-target recognition and tracking will be disturbed, and the increase in misrecognition rate will lead to navigation errors. Another key limitation of the image processing system is its dependence on environmental markers. In the system mentioned in the comparative document, markers need to be set up on the ship and the shore to provide positioning and navigation information. This dependence means that the system's functionality will be limited in waters without appropriate markers. In addition, the maintenance and updating of markers is also a problem, especially in bad weather and sea conditions, the markers will be damaged or displaced, further affecting the reliability and accuracy of the system. In terms of user acceptance, although image processing provides a way to reduce human operation and improve accuracy, it requires relatively high skills from operators. Operators need to have a deep understanding and operating experience of image processing systems so that they can intervene effectively when problems arise or system prompts. In addition, the high reliance on automated systems makes it difficult for operators to manually take over control when necessary, increasing the risk of the system in responding to emergencies. Economic cost is also an issue that cannot be ignored. The high cost of advanced image processing hardware and software limits the popularity of the system on small and medium-sized ships. The high initial investment and maintenance costs make this technology less cost-effective than traditional navigation and mooring systems. Summary of the invention
[0004] The purpose of the present invention is to provide a ship berthing safety auxiliary control system based on multi-data fusion, so as to solve some of the drawbacks and shortcomings pointed out in the background technology.
[0005] The present invention solves the above-mentioned technical problems by adopting the following technical solutions: firstly, various types of data required during the ship's mooring process are collected through various sensors and data acquisition interfaces, including GPS positioning data, meteorological data including wind speed, wind direction, humidity, and temperature, hydrological data including tide, flow rate, and water depth, ship dynamic data including speed, heading, acceleration, and surrounding environment data;
[0006] Secondly, the collected data is cleaned, formatted and synchronized, machine learning methods are used to optimize data quality, and data fusion algorithms such as Kalman filters are used to integrate data from different sources;
[0007] Then, deep learning techniques including convolutional neural network (CNN) and recurrent neural network (RNN) are used to establish a dynamic behavior model of the ship based on the fused data, and combined with Bayesian network for real-time risk assessment.
[0008] Finally, based on the model prediction and risk assessment results, a control decision algorithm based on reinforcement learning was developed and implemented to generate control instructions in real time to adjust ship operations including adjusting speed and changing course.
[0009] Furthermore, the method for collecting various types of data required during the ship's berthing process adopts:
[0010] S1. First develop the integrated sensor module through high-order calculus formula:
[0011]
[0012] Synchronously collect GPS positioning, meteorological data, hydrological data and ship dynamic data, where X represents the sensor data vector, T represents the collection period, and a i ,μ i ,σ i ,λ i They are adjustment parameters respectively;
[0013] S2, then through the multivariate response adjustment formula:
[0014]
[0015] Dynamic optimization processing is performed on the collected data, where D represents the original data vector obtained from the sensor, t represents the collection time, and b j ,c j ,α j It is the adjustment coefficient, which is used to dynamically adjust the frequency and priority of data collection;
[0016] S3. Finally, the fusion analysis formula is used:
[0017]
[0018] Among them, U, V represent various environmental perception data that have been preliminarily processed, including sonar and radar data, and s represents the environmental noise variable to integrate environmental information.
[0019] Furthermore, the data cleaning, formatting and synchronization process adopts a method including:
[0020] S1, first through the adaptive data cleaning algorithm:
[0021]
[0022] Processing collected multi-source data, where D i represents the data collected by different sensors including GPS, meteorological sensors, and hydrological sensors, μ i Represents the expected value of the i-th data type, which is used to locate the center of data cleaning; σ i Represents the standard deviation of the i-th data type, which is used to quantify the dispersion of the data; α i represents the adjustment factor, which is used to adjust the cleaning intensity of the i-th data type; β i Represents the power exponent, which is used to control the influence and processing rate of data outliers;
[0023] S2, then use the time synchronization algorithm:
[0024]
[0025] Synchronize the data from various ship systems, where λ adjusts the accuracy of synchronization to control data consistency during berthing operations; t represents the actual time point of data acquisition; Δt represents the expected synchronization time difference, which is used to correct the data time axis; T represents the total acquisition period, which defines the operating window of the time synchronization algorithm;
[0026] S3. Through feature engineering algorithm:
[0027]
[0028] Extract the features that have the greatest impact on berthing safety from the data, among which X k represents the kth feature extracted from the original data; γ k Represents the importance weight of the kth feature, which determines the influence of the feature in the model;
[0029] S4. Simultaneous data quality assessment model:
[0030]
[0031] Real-time monitoring of data quality, where D represents the evaluated data set, which contains multiple data points; v j represents the quality impact factor of the jth data point, which is used to weigh the importance of the data point in the overall evaluation; δ j Represents the adjustment coefficient, which is used to adjust the influence of the jth data point;
[0032] S5. Finally, the improved Kalman filter algorithm is used:
[0033]
[0034] Among them, U, V represent the processing results of different data sources, which represent the data vector obtained through preliminary processing; ψ represents the data fusion function, which is used to combine the information of U and V, as well as the environmental noise s; s represents the state estimation variable, which is used to simulate the uncertainty and noise in the environment; ξ i ,η i Represents adjustment parameters, which are used to optimize the effect of data fusion.
[0035] Furthermore, the method for constructing the dynamic behavior model of the ship includes:
[0036] Firstly, adaptive filtering technology is used to fuse multi-source data from GPS, meteorological, hydrological and ship dynamic sensors and eliminate data noise and inconsistency.
[0037] Then, convolutional neural network (CNN) is used to process the fused data, especially the image and sequence data, and the convolutional layer structure and activation function are designed to extract the spatial characteristics of ship behavior and the patterns in the time series;
[0038] Then, a recurrent neural network (RNN), specifically a long short-term memory (LSTM) or a gated recurrent unit (GRU), is applied to capture the dynamic characteristics that change over time based on past ship status and environmental data, and to predict future ship behavior and risk situations.
[0039] Finally, the dynamic Bayesian network is combined with the data output from CNN and RNN to update the risk prediction, and a real-time risk assessment model is constructed taking into account the influence of historical data and observation data.
[0040] Furthermore, the multi-source data is fused by:
[0041] First, the adaptive filtering algorithm is used:
[0042]
[0043] Where x(t) represents the data collected from GPS, meteorological, hydrological and ship dynamic sensors, t is the acquisition time, θ(s) represents the environmental parameters of wind speed and water flow speed at time s, α, β, γ are dynamically adjusted parameters used to adjust the filter strength and period according to environmental changes;
[0044] Then use the data preprocessing and synchronization formula:
[0045]
[0046] Where x(t) and y(t) represent the time series data of different sensors respectively. It is the average value of the y data in period T, which is used to synchronize the data of different sensors and calculate the absolute value integral of the deviation to measure and correct the synchronization error of the time series;
[0047] Finally, implement the machine learning-assisted data fusion model:
[0048]
[0049] where x={x1(t),x2(t),…,x n (t)} represents the preprocessed multi-sensor data vector, w i It is the weights determined by deep learning process optimization.
[0050] Furthermore, the convolutional neural network CNN processes the fused data from GPS, meteorological, hydrological and ship dynamic sensors and is designed with a 1D convolutional layer:
[0051]
[0052] Processing time series data, where x(t) represents the time series of sensor output, T is the time window considered, and α is the decay factor, adjusted to adapt to different environmental change rates, while using 2D convolutional layers:
[0053]
[0054] Processing spatial data, including images or radar scans, x(i,j) represents the pixel value in the image, w ij is the weight of the convolution kernel, k is the size of the convolution kernel, which is used to extract spatial features in the image;
[0055] The network then passes through the parameterized ReLU activation function:
[0056] h((x)=max(0,x)+λmin(0,x)
[0057] where λ is a learnable parameter that is adjusted to enhance the model’s sensitivity to changes in the ship’s dynamic behavior;
[0058] Finally, the deep learning algorithm is used through the formula:
[0059]
[0060] Perform data fusion, where x m represents the features passed down from the previous network layer, w nm and θ n are optimized weight parameters used to extract and fuse data from multiple sensors in a deep network, controlling the prediction of the ship’s behavior under different berthing conditions.
[0061] Furthermore, the recurrent neural network RNN structure adopts a long short-term memory network LSTM or a gated recurrent unit GRU structure for real-time processing and analysis of fused data from GPS, meteorological, hydrological and ship dynamic sensors; through high-level functions:
[0062]
[0063] To update the hidden state of the network, σ represents the sigmoid activation function, which captures the time-dependent characteristics of the ship's behavior and the changes in environmental factors;
[0064] Then, using the learned time series features, the risk assessment module calculates the future risk level using the formula:
[0065]
[0066] To integrate the output of the model at different time points, where β t It is the risk weight corresponding to the time point, which assesses the risks that the ship may face in real time, including collision or grounding;
[0067] Finally, by continuously receiving new input data and using the update algorithm:
[0068]
[0069] in is the loss function and η is the learning rate to optimize the network parameters.
[0070] Furthermore, the architecture adopted by the data update risk prediction includes:
[0071] S1. First, use the convolutional neural network CNN and the recurrent neural network RNN to process the data collected from various sensors respectively; through the double-layer convolution process, CNN uses the formula:
[0072]
[0073] Processing image and spatial data, where d(x,y) represents the image data matrix, α controls the spatial decay rate, and β adjusts the frequency response, so that the network extracts spatial distribution features about the ship's surroundings and other ships;
[0074] S2. Then use the long short-term memory network LSTM to process time series data, using the formula:
[0075]
[0076] where h t represents the hidden state at time t, h t-1is the hidden state at the previous moment, x t is the input data collected from GPS and weather sensors at the current moment, and b (k) are the weight and bias parameters of the kth gating unit in the LSTM network.
[0077] Furthermore, the control decision algorithm layout method adopts:
[0078] First, a comprehensive environmental model is used to integrate multi-source data from GPS, meteorological, hydrological and ship dynamic sensors, using a mathematical model:
[0079]
[0080] To simulate the dynamic state of the ship and its surroundings in real time, where v(τ) and d(τ) represent the speed and heading respectively, and α, β, and ω are model parameters that control and predict the behavior of the ship under different environmental conditions;
[0081] Reinforcement learning-based control decisions are then used to optimize ship operations, especially regulating speed and changing course, by:
[0082]
[0083] Calculate the optimal operation, where μ represents the policy network and θ μ are the network parameters, ∈ and σ 2 Control the scope of exploration of operations;
[0084] Finally, use the reward function:
[0085] R(s t ,a t )=r t +γ∫ a′ Q(s t+1 ,a′;θ Q )exp(-λa′ 2 )da′
[0086] To learn and update, where r t is the immediate reward, representing the effect of the current action, γ is the discount rate of future rewards, Q represents the action value function, and θ Q and λ are model parameters.
[0087] The present invention has the following beneficial effects:
[0088] 1. Improve safety: By integrating data from GPS, meteorological, hydrological and ship dynamic sensors, the system is able to provide a more comprehensive and accurate environmental perception. This all-round data integration helps to identify potential risks and obstacles in real time, thus significantly improving the safety of ships during berthing.
[0089] 2. Enhanced decision support: Using advanced machine learning techniques, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), the system can effectively analyze and predict ship behavior and environmental changes. This predictive ability makes decision-making more scientific and accurate, providing powerful decision support for captains, pilots, ship drivers or automatic navigation systems.
[0090] 3. Dynamic response capability: By applying reinforcement learning algorithms, the control system not only responds based on current data, but also learns from historical behaviors and continuously optimizes the ship's control strategy. This learning and adaptation mechanism enables the system to dynamically respond to complex and changing marine environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 This is a flow chart of the ship berthing safety auxiliary control system based on multi-data fusion of the present invention.
[0092] Figure 2 The present invention is a flow chart of a method for collecting various types of data required during the ship's berthing process.
[0093] Figure 3 A flow chart of the method used for the cleaning, formatting and synchronization process of data of the present invention.
[0094] Figure 4 The present invention is a flow chart of a method for constructing a dynamic behavior model of a ship.
[0095] Figure 5 This is a schematic flow chart of the core driver blocks of the multi-source data fusion method of the present invention. DETAILED DESCRIPTION
[0096] The specific implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings.
[0097] Combined with Figure 1:Improve the accuracy and safety of ship berthing by integrating multi-source data. The system first uses various sensors and data acquisition interfaces to collect multiple types of data required by ships during berthing to ensure comprehensive environmental perception and dynamic monitoring. These data include: GPS positioning data, which provides precise location information of the ship, which is essential for determining the specific location of the ship in the port or specific waters; meteorological data, including wind speed, wind direction, humidity and temperature, which are important for evaluating how weather conditions affect ship berthing operations, such as strong winds will affect the course and stability of the ship; hydrological data, including tides, flow rate and water depth, which are key to assessing whether the ship can safely enter or leave the port and whether there is enough excess water depth for safe berthing; ship dynamic data, such as speed, heading and acceleration, which help the system monitor and adjust the movement of the ship to ensure that the ship travels safely according to the scheduled route; and surrounding environment data, including the location of nearby ships, obstacles on the water surface, etc., which are essential for avoiding collisions and other potential dangers.
[0098] After collecting various data about the ship and its environment, the next step is to clean, format and synchronize the data. This step is crucial because the quality of the data directly affects the efficiency and accuracy of the subsequent decision-making system. During the data cleaning process, the system will eliminate incomplete, erroneous or irrelevant data, such as deleting outliers collected due to sensor failures, or filtering out data generated by noise interference. Formatting ensures that all data follows a unified format and standard, so that data from different sources and sensors can be effectively interpreted and processed by the system. Data synchronization is to ensure the timeliness and consistency of data, especially for time-sensitive operations such as ship navigation and berthing control. Synchronization ensures that all dynamic data reflects the actual situation at the same point in time, supporting the system to respond in real time.
[0099] In addition, the system uses machine learning methods to further optimize data quality. By training models to identify patterns and trends in the data, machine learning can automatically adjust the data processing process and improve the algorithms for data cleaning and synchronization to make them more intelligent and adaptive. For example, a machine learning model can learn how the data behaves under specific weather conditions and automatically adjust the corresponding data cleaning standards and synchronization parameters. After the data quality is optimized, the system uses data fusion algorithms such as Kalman filters to integrate data from different sources. The Kalman filter is an effective linear dynamic system estimation method that can predict and adjust state variables in the data to reduce the impact of measurement noise and uncertainty. The Kalman filter is particularly effective in multi-sensor data fusion by updating its estimated state and covariance in real time and incorporating new measurement observations. This approach allows the system to fuse various sensors and time series data, such as GPS and meteorological monitoring data, to provide a consistent and accurate information basis for ship berthing.
[0100] After data cleaning, formatting and synchronization, deep learning techniques, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), are then used to build dynamic behavior models of ships. These models are able to learn the behavior patterns and dynamic characteristics of ships under various environmental conditions from the fused data. CNNs are mainly used here to process and analyze image data, such as extracting visual features of ships from cameras or satellite images, which reflect the physical location of the ship and the surrounding environment. This network is widely used in image recognition tasks due to its powerful spatial hierarchical feature extraction capabilities. On the other hand, RNNs are particularly suitable for processing time series data, such as ship speed, heading, and acceleration data collected by GPS and other sensors. RNNs are able to handle the temporal dependencies in these data and predict the future state and behavior of the ship. In particular, two RNN variants, long short-term memory networks (LSTMs) and gated recurrent units (GRUs), solve the training difficulties of traditional RNNs on long sequences through specialized gating mechanisms and can capture dependencies over a longer time range.
[0101] In addition, the system also combines Bayesian networks for real-time risk assessment. Bayesian networks are graphical models that use probabilistic reasoning to deal with uncertainty problems and can calculate posterior probabilities based on prior knowledge and real-time observation data. In this system, the Bayesian network uses the output from CNN and RNN as input to assess the various risks faced by ships, such as the risk of collision, grounding or deviation from the route. By modeling the probabilities of these risk factors, the network can predict the likelihood of various adverse events under current navigation conditions and ship behavior. This feature enables ship operators to take preventive measures before risks occur, thereby significantly improving the safety of ships.
[0102] After building the behavioral model and conducting the risk assessment, the final step is to use the model predictions and risk assessment results to develop and implement a control decision algorithm based on reinforcement learning. The core of this algorithm is to generate control commands in real time to adjust the ship's operations, including speed adjustment and heading changes, to respond to and manage identified risks and optimize the ship's berthing process. Reinforcement learning is a machine learning method that enables a system to learn how to take actions in a given environment to maximize a certain cumulative reward through trial and error. In this system, the reinforcement learning algorithm uses inputs obtained from multiple data fusion and prediction models, such as the ship's current position, speed, expected path, and potential collision or deviation risks. This information is used to form a state representation of the system, and the algorithm then determines the best operating action based on these states, such as acceleration, deceleration, turning left or right, etc. Specifically, the reinforcement learning model receives immediate feedback on the effectiveness of its decisions by constantly interacting with the environment. This feedback is usually based on a predetermined reward function that considers multiple factors, such as navigation safety, efficiency, energy consumption, etc. The more reasonably the reward function is designed, the more effective the learned strategy will be in guiding the ship to berth safely. In actual operation, this control decision algorithm must also be able to adjust and optimize in real time, because the ship's berthing environment will change rapidly, such as sudden weather conditions, changes in sea currents or unforeseen movements of other ships. Therefore, the real-time and adaptive capabilities of the system are crucial to ensure that all navigation decisions are based on the latest environmental data and system status.
[0103] Embodiment 1:
[0104] Combined with Figure 2 The following example will explain in detail how to use high-order calculus formulas to simultaneously collect multiple types of data including GPS positioning, meteorological data, hydrological data, and ship dynamic data. This process not only reflects the high technical nature of data collection, but also ensures the real-time and accuracy of the data, providing a solid foundation for subsequent data processing and risk assessment.
[0105] A commercial cargo ship is entering a busy port area for anchoring. The integrated sensor module installed on the ship is responsible for collecting key data of the ship. This data includes the precise position obtained from the GPS system, wind speed and direction obtained from meteorological sensors, tide and current data obtained from hydrological sensors, and dynamic data such as the ship's speed and heading.
[0106] The data collection formula used is:
[0107]
[0108] Where X represents the data vector collected from each sensor, T is the data collection period, and a i ,μ i,σ i ,λ i These are adjustment parameters set for each type of data. These parameters are set as follows:
[0109] a i (Gain factor) The value range is 0.5 to 2.0 and is adjusted according to the importance and sensitivity of the sensor. i (Expected value) is set based on historical data, such as μ of GPS data is set to the average position of the ship under normal sailing conditions. i (Standard deviation) represents the variability of the data. For meteorological data, due to its high uncertainty, the σ value is set to a larger value, such as 5.0. i (Decay factor) is used to adjust the effect of time and is usually between 0.01 and 0.1 to ensure that the data decays effectively over time.
[0110] For example, suppose that within a certain collection period T = 60 seconds, for the GPS position data X of the ship GPS , set a GPS =1.5,μ GPS =50.1234,σ GPS =2.0,λ GPS =0.05. According to the formula, first calculate the integral of the exponential part and get:
[0111]
[0112] The value of this integral can be obtained by numerical integration method, and the result is about 11.46. Next, calculate the entire expression:
[0113]
[0114] Because (X GPS -μ GPS ) 2 =0, the result of this part is 1.5×11.46=17.19.
[0115] Through this approach, a comprehensive data quality score can be calculated for various types of data. This score can then be used to optimize the ship's berthing strategy, such as adjusting speed and changing course, to ensure that the ship berths safely and efficiently in complex port environments.
[0116] The next step of this embodiment is to dynamically optimize the data. In this link, the system uses a multivariable response adjustment formula, which is optimized for different types of sensor data to adjust the data collection frequency and priority to ensure that the system can respond to environmental changes and ship operation requirements in real time and accurately.
[0117] The specific multivariate response adjustment formula is defined as:
[0118]
[0119] In this formula, D represents the raw data vector obtained from various sensors such as GPS, meteorological, and hydrological sensors, t is the time from the start of data collection to the present, and b j ,c j ,α j It is a coefficient adjusted for each data type to dynamically adjust the data processing method and priority.
[0120] To illustrate this process with a specific example, a ship is entering a port with a high traffic density and needs to accurately adjust its speed and course to avoid collisions with other ships. In this case, the ship's dynamic data (such as speed and course) is more critical than meteorological data (such as wind speed and temperature). Therefore, different adjustment coefficients can be set for these data types to reflect their importance in the current situation.
[0121] Assume that for the speed and heading data, set b j =2.0,c j =0.5,α j =2, and for wind speed and temperature data, set b j =1.0,c j =1.0,α j = 1. This means that speed and heading data will be given higher priority and updated more frequently to adapt to rapidly changing navigation conditions.
[0122] In the specific calculation, if the speed value Dj recorded by the speed sensor is 15 knots within 10 minutes (i.e. 600 seconds) after the start of the acquisition, the formula is used to calculate:
[0123]
[0124] This value indicates the importance of processing and updating the speed data at this point in time. Similarly, the processing priority of wind speed or temperature data will be lower due to its lower b j and higher c j and α j The corresponding decrease reflects the relatively low importance in emergency navigation adjustment situations.
[0125] The final fusion analysis of the data in this embodiment is one of the key steps to achieve efficient and safe mooring. The fusion analysis formula involved in this stage makes full use of the environmental perception data collected from various sensors, such as sonar and radar, and integrates environmental noise variables to ensure a comprehensive and accurate understanding of the surrounding environment. The following example will explain in detail how to apply the fusion analysis formula:
[0126]
[0127] Among them, U, V represent various environmental perception data that have been preliminarily processed, and s represents the environmental noise variable.
[0128] In a specific mooring scenario, the data collected from sonar and radar need to be processed to assess the position of obstacles and other ships around the ship. After preliminary formatting and synchronization, these data will be substituted into the formula, where U and V represent the data vectors from sonar and radar, respectively. In order to demonstrate the application of the formula in practice, p = 2 is set, that is, two sets of data are considered, one from sonar and the other from radar.
[0129] Set the specific values as follows:
[0130] For the sonar data, set U1 = 8 meters (the distance between the ship and the nearest obstacle) and V1 = 2 meters (the height of the obstacle).
[0131] For the radar data, let U2 = 15 meters (horizontal distance of the other ship) and V2 = 5 meters (height of the ship).
[0132] For each set of data, first calculate the magnitude of the distance vector:
[0133]
[0134] Then, an integral calculation is required, which involves the ambient noise variable s:
[0135]
[0136] and
[0137]
[0138] because In are all greater than 0. This part of the integral is mainly affected by The impact of
[0139] The final fusion analysis result will be the weighted sum of these two integral values. Since the actual calculation of the integral requires a numerical integration method, the processing is simplified here and the estimated integral values are all very small numbers, thus:
[0140] H(U,V)≈8.25×smallnumber+15.81×smallnumber
[0141] This process shows that by integrating the analysis formula, it is possible to quantitatively analyze the combined impact of sonar and radar data and consider the role of environmental noise to provide a comprehensive environmental assessment result for the ship. This method not only improves data utilization, but also enhances the ability to adapt to complex environments, ensuring that the ship can safely avoid obstacles and other ships during approach and mooring.
[0142] Embodiment 2:
[0143] Combined with Figure 3 ,Data cleaning and synchronization are key steps to ensure efficient and accurate data analysis.,The following example elaborates on how to process the collected multi-source data through,adaptive data cleaning algorithms and time synchronization algorithms,,including the specific calculation process and application of,formulas.
[0144] S1, Adaptive data cleaning algorithm:
[0145]
[0146] It is used to process data from multiple sources including GPS, meteorological sensors, hydrological sensors, etc. In this formula, D i represents the data of the i-th sensor, μ i and σ i They represent the expected value and standard deviation of the data type respectively. These two parameters help locate the center and dispersion of each type of data, thereby performing effective data cleaning.
[0147] The parameters are set as follows:
[0148] α i The adjustment factor can range from 0.5 to 2.0, depending on the importance of the data and its impact on system performance.
[0149] β i The power exponent ranges from 1 to 3 and is used to control the influence of outliers. Higher values help reduce the impact of extreme data points.
[0150] For example, if the expected position of GPS data is set to 50.1234 longitude and the standard deviation is 0.5, if the GPS position obtained from a data collection is 50.1284, then use:
[0151] α GPS =1.5,β GPS =2 to calculate:
[0152]
[0153] The above results show that the weight of this GPS data after data cleaning is small because it is close to the expected value, indicating that its data quality is high and reliable.
[0154] S2, time synchronization algorithm:
[0155]
[0156] Used to synchronize data from different systems. In this algorithm, λ controls the synchronization accuracy, t is the actual time point of data acquisition, Δt is the desired synchronization time difference, and T is the total acquisition period.
[0157] Set λ = 0.1. If we consider data synchronization within the entire acquisition period T = 10 seconds, the actual data acquisition time point t = 5 seconds, and the expected synchronization time difference Δt = 1 second, then the synchronization algorithm is calculated as follows:
[0158]
[0159] The above results indicate that within a given acquisition cycle, the time deviation of the data will be effectively corrected after being processed by the synchronization algorithm, thereby ensuring the consistency and real-time nature of the data.
[0160] S3. Feature Engineering:
[0161] This step is done by the algorithm:
[0162]
[0163] The features that have the greatest impact on parking safety are extracted from a large amount of raw data. These features will then be used to drive decision-making models, such as predicting potential risks or formulating parking strategies. In this algorithm, X k represents the kth feature extracted from the original data, and γ k These weights determine the influence of each feature in the model.
[0164] Set each feature weight γ k The range of is 0.1 to 10, which allows the model to distinguish the importance of different features, from low minor effects to significant effects of key features. In order to explain the implementation of this scheme in detail and prove the feasibility of its steps, a specific example will be used for illustration.
[0165] During a given mooring, the system collects multi-source data from the sensor network, including GPS positioning accuracy, the ship's current speed, the number of nearby ships (detected by radar), and the current wind speed and current conditions. Five key features are extracted from this data:
[0166] X1: GPS positioning accuracy, set its value to 0.95 (close to 1 indicates high accuracy);
[0167] X2: ship speed, set its value to 12 knots;
[0168] X3: number of ships in close range, set its value to 3;
[0169] X4: wind speed, set its value to 15 km / h;
[0170] X5: water flow speed, set its value to 6 km / h;
[0171] Assume that the corresponding importance weights are:
[0172] γ1=10 (GPS positioning accuracy is crucial to parking safety);
[0173] γ2=5 (speed is also a key factor);
[0174] γ3=2 (the number of other ships has little impact on safety);
[0175] γ4=3 (wind speed has a significant impact on ship berthing);
[0176] γ5=4 (water velocity is also important);
[0177] Substituting these data into the feature engineering formula, we can calculate:
[0178]
[0179]
[0180] This result indicates that data points critical to mooring safety are effectively identified and emphasized based on the selected features and weights.
[0181] S4. Data quality assessment model:
[0182]
[0183] Used to monitor the quality of each data point in real time. In this model, D represents the evaluation data set, v j is the quality impact factor of the jth data point, and δ j are the adjustment coefficients, which together determine the influence of each data point in the overall evaluation.
[0184] Set v j The value range of δ is from 0.1 to 1, where 1 means that the data point is extremely important to the overall evaluation. j The value range of is 0.5 to 2, which allows for flexible adjustment of the influence of data points. Let's set up a specific scenario with three data points: the accuracy of GPS positioning, the number of nearby ships detected by radar, and the water depth measurement:
[0185] D = [0.8, 5, 2] (GPS accuracy 0.8, 5 ships nearby, water depth 2 meters);
[0186] v = [1, 0.6, 0.8] (importance of each data point);
[0187] δ = [1.5, 0.7, 1] (adjustment coefficient of each data point);
[0188] Substitute this data into the model:
[0189]
[0190] S45, Improved Kalman filter algorithm:
[0191]
[0192] In this formula, U and V represent the data vectors processed from different data sources, and ψ is the data fusion function used to combine the information of U and V and the environmental noise s.
[0193] Setting i The value range of is 0.5 to 2.0, and η i The value range is 1 to 3. The specific data is as follows:
[0194] U=[0.7,0.9]
[0195] V=[1.1,0.85]
[0196] ξ=[1.5,1.2]
[0197] η=[1,2]
[0198] For the data fusion function ψ, it is assumed to simply take U i and V i The average value of , plus the influence of noise s:
[0199]
[0200] In the calculation, s is set to a simple linear change, that is, s changes from -1 to 1. Calculate ψ and integrate it:
[0201]
[0202] Finally, substitute into the Kalman filter algorithm:
[0203] r(U,V)=1.5·(1.8) 1 +1.2 (1.75) 2
[0204] =1.5·1.8+1.2·3.0625
[0205] =2.7+3.675
[0206] =6.375
[0207] The above results show that by combining different data sources and considering the influence of environmental noise through the improved Kalman filter algorithm, a comprehensive and weighted data fusion result can be obtained.
[0208] Embodiment 3:
[0209] Combined with Figure 5 Adaptive filtering technology is one of the core technologies. It is used to integrate multi-source data from GPS, meteorological, hydrological and ship dynamic sensors, and effectively eliminate data noise and inconsistency. The following example will explain in detail how to use the adaptive filtering algorithm:
[0210]
[0211] To achieve this goal, where x(t) represents the data collected from the sensor, t is the acquisition time, θ(s) represents the environmental parameters (such as wind speed, water flow speed) at time s, and α, β, γ are dynamically adjusted parameters.
[0212] Parameter settings:
[0213] The α adjustment parameter controls the effect of environmental changes on the filter strength, and its value range is set to 0.1 to 1.0 to adapt to different environmental sensitivities. β controls the amplitude of the periodic adjustment, and its value range is set to 0.5 to 2.0 to adjust the filter's response to periodic changes. γ determines the frequency of the periodic function, and its value range is to π, corresponding to a periodic fluctuation from once per hour to once per minute.
[0214] Practical application examples:
[0215] In a typical parking operation, data from multiple sensors needs to be processed synchronously. The specific data is as follows:
[0216] x(t) represents the data set at t = 60 seconds, including a GPS position accuracy of 0.98, a wind speed of 15 km / h, a current speed of 6 km / h, and a ship speed of 12 knots.
[0217] The environmental parameters θ(s), namely wind speed and water flow velocity, vary linearly with time.
[0218] For wind speed and water flow velocity, a simplified model is set, where θ(s) = 0.05s (the wind speed and water flow velocity increase by 0.05km / h per second). Set α = 0.7, β = 1.5,
[0219] First, calculate the integral:
[0220]
[0221] Substituting these values into the filtering formula:
[0222]
[0223] This result shows that under given environmental conditions and parameter settings, the filtering algorithm can effectively reduce noise and data inconsistency, although the weight of the original data is greatly reduced after strong filtering.
[0224] This embodiment then uses a series of advanced data preprocessing and synchronization methods, as well as a machine learning-assisted data fusion model. The following will elaborate on the implementation of these two methods, including specific mathematical expressions, parameter settings, and use a specific example to demonstrate the feasibility and practical application of each step.
[0225] Data preprocessing and synchronization formula:
[0226]
[0227] It is used to synchronize the time series data of different sensors and calculate the absolute value integral of the deviation to measure and correct the synchronization error of the time series. In this formula, x(t) and y(t) represent the time series data of two different sensors, respectively. is the average value of the y data in period T.
[0228] Assume that during a specific ship mooring operation, GPS positioning data x(t) and water flow velocity sensor data y(t) are collected:
[0229] x(t) represents the GPS positioning coordinates of the ship every second.
[0230] y(t) represents the corresponding water flow velocity (m / s).
[0231] If the data collected within 10 seconds is as follows:
[0232] x(t)=[2.0,2.1,1.9,2.0,2.1,2.2,2.1,2.0,1.9,2.0]
[0233] y(t)=[1.0,1.1,1.0,1.2,1.1,1.0,1.1,1.0,1.0,1.1]
[0234] calculate
[0235]
[0236] Calculate g(x,y):
[0237]
[0238] Calculate the specific value and divide it by T=10 seconds.
[0239] Next, implement a machine learning-assisted data fusion model:
[0240]
[0241] where x={x1(t),x2(t),…,x n (t)} represents the preprocessed multi-sensor data vector, w i It is the weights determined by deep learning process optimization.
[0242] Set the weights w determined using deep learning i The range is 0.1 to 5.0. Here x(t) includes the GPS location and water velocity data processed by the above synchronization algorithm. Using these data, calculate tanh(x i (t)), then time-integrate each result and apply weights. Finally, add these integral values and substitute them into the log and exp functions to get the final data fusion result.
[0243] Embodiment 4:
[0244] Using convolutional neural networks (CNNs) to process fused data from GPS, meteorological, hydrological, and ship dynamic sensors is a key technical step. CNNs can effectively extract spatial features of ship behavior and patterns in time series from large amounts of data, thereby improving the safety and accuracy of the mooring process. The following details how to process this data through 1D and 2D convolutional layer structure design, and provides specific application examples and calculations.
[0245] 1D convolution layer design:
[0246] The 1D convolution layer is specifically used to process time series data, which is particularly important in ship dynamic data processing. Formula:
[0247]
[0248] Describe the Gaussian filtering process, where x(t) represents the time series of sensor output, T is the time window considered, and α is the decay factor used to adjust the data processing to adapt to different environmental change rates.
[0249] Set the time series data for the ship's speed, set T = 30 seconds, α = 0.1, and set the speed data within these 30 seconds as follows:
[0250] x(t)=[12,12.5,13,12.8,12.6,12.7,13.1,13.5,13.2,12.9]
[0251] Using Gaussian filtering to process these data, the calculation is as follows:
[0252]
[0253] Here x(t) is sampled from t = -30 to t = 30, and the specific integration needs to be processed by numerical methods, such as using Simpson's rule or numerical integration software. The result of the calculation will reflect the smoothed version of the time series data after the attenuation factor is adjusted, effectively extracting the main trend of time change.
[0254] 2D convolution layer design:
[0255] 2D convolutional layers process spatial data, such as images or radar scans. Formula:
[0256]
[0257] Use the convolution kernel w ij to process the image data x(i,j), where k is the size of the convolution kernel.
[0258] For example, if processing image data from a ship radar, set the convolution kernel size k = 3, which represents a 7×7 convolution kernel, to extract spatial features in the image. The radar image data x(i,j) represents the distance and direction of objects around the ship. The convolution kernel w ij It is set according to the specific priorities of the ship's navigation needs, such as emphasizing the detection of objects approaching the ship. In practice, the convolution kernel weights are set, for example, the center weight is the highest and the edge weight is lower to emphasize the features of the center area, and then the convolution kernel is applied to the image data. This operation will generate a new feature map that can help identify and locate obstacles around the ship, thereby providing decision support for the ship's berthing.
[0259] This embodiment will next elaborate on how to use the parameterized ReLU activation function and deep learning algorithm to process and fuse sensor data, including specific formula applications and examples.
[0260] Parameterized ReLU activation function:
[0261] ReLU (Rectified Linear Unit) is a commonly used activation function in deep learning. Due to its simplicity and high efficiency, it is widely used in many neural network models. In this system, a parameterized version is used:
[0262] h(x)=max(0,x)+λmin(0,x)
[0263] Here, λ is a learnable parameter that allows the network to automatically adjust its response to negative input values, thereby enhancing the model's sensitivity to changes in the dynamic behavior of the ship. Setting the value of λ to -0.1 to 0 controls the degree to which the network suppresses negative features, allowing the model to focus more on positive features that are helpful for mooring decisions.
[0264] The deep learning algorithm performs data fusion through the following formula:
[0265]
[0266] In this formula, x m represents the features passed down from the previous network layer, w nm and θ n It is a weight parameter optimized through training and used to extract and fuse data from multiple sensors in a deep network.
[0267] Assume that the system is processing data from GPS, weather sensors, and hydrological sensors:
[0268] x1, x2, and x3 represent GPS positioning accuracy, wind speed, and water flow speed, respectively.
[0269] Assume that the network has 3 hidden layer nodes (N=3), each node is linked to all sensor features (M=3). Set the weight parameter w nm The value range of is -0.5 to 0.5, which allows the network to flexibly adjust the contribution of each feature. n Set to 0.1 to 1.0 to adjust the impact of each node's output on the final fusion result.
[0270] Take actual data as an example:
[0271] x1=0.95(high-precision GPS positioning)
[0272] x2=15km / h (wind speed)
[0273] x3=6m / s (water flow speed)
[0274] Compute the activation of each hidden layer node:
[0275] Input of node 1 = 0.3 0.95 + 0.2 15 + 0.1 6 = 3.885
[0276] Activation value = tanh(3.885)≈0.998
[0277] Similar calculations are performed for other nodes, and the activation values are substituted into the final output formula of deep learning to calculate the final data fusion output. This process enables the system to comprehensively consider information from different sensors, provide accurate and real-time data support, and ensure that the ship can be safely and effectively moored at the designated location.
[0278] Embodiment 5:
[0279] This embodiment applies recurrent neural networks (RNN), especially long short-term memory networks (LSTM) or gated recurrent units (GRU), which are key technologies for realizing advanced time series data analysis. These network structures can capture time-related dynamic features, such as ship position changes, changes in meteorological conditions, hydrological conditions and other ship dynamics, and predict future ship behavior and potential risks based on this information. The following will explain the application of these network structures in detail, and demonstrate their feasibility in actual data processing through a specific example.
[0280] LSTM / GRU structure and data processing:
[0281] LSTM and GRU are special types of RNNs designed to solve the gradient vanishing or gradient exploding problems encountered by traditional RNNs when processing long sequence data. They effectively manage the long-term dependencies of information by introducing several gating mechanisms (such as forget gate, input gate, and output gate).
[0282] In the ship mooring system, LSTM or GRU can be used to process the fused data from GPS, meteorological, hydrological and ship dynamic sensors in real time. These data usually contain complex time series, whose characteristics and patterns change rapidly over time, which is crucial for predicting the future position and potential risks of the ship.
[0283] Application of advanced functions:
[0284] Advanced function f(h t ,x t ) is shown below, which is used to update the hidden state of the network in LSTM / GRU:
[0285]
[0286] Here, h t is the hidden state at time t, x tis the input feature vector, σ is the sigmoid activation function, which is used to generate output in the range of 0, 1 and control the flow of information; W f ,W i ,W c and b f ,b i ,b c are the weight and bias of the gating mechanism respectively.
[0287] Assume that at a certain time point t, the location data x collected from GPS t is 2.5,3.0, representing the latitude and longitude of the ship; the data collected from the meteorological sensor is [15,80], representing the wind speed km / h and humidity%. These data are combined into the input vector x t .
[0288] In the LSTM network, the weight W and bias b of each gate are randomly initialized, for example, each weight is in the range of [-0.5, 0.5] and the bias is in the range of [-0.1, 0.1]. Through forward propagation, the network uses the above function to update its hidden state, and the calculated new state h t This setting allows the system to dynamically adjust its response to environmental changes and predict future trends in ship behavior through in-depth learning of historical data.
[0289] Next, the key step in this embodiment is to use the time series features learned by the recurrent neural network (RNN) to perform risk assessment. With this method, the system can evaluate and respond to potential risks such as collision or grounding in real time, thereby greatly improving the safety of the ship. In addition, the system also uses a dynamic parameter update algorithm to continuously optimize network performance. The following details how to implement these steps, including the specific calculation process and parameter settings.
[0290] The risk assessment module utilizes the formula:
[0291]
[0292] to integrate the output of the model at different time points. In this formula, h t is the hidden state at time point t, reflecting the ship dynamics and environmental characteristics extracted from the sensor data. The risk weight β at each time point t Adjust according to its importance to the safety assessment. Set β t The value of is usually in the range of 0.1 to 1.0 to ensure that the importance of the time point is appropriately reflected.
[0293] The system is set to process data from the past 30 seconds, collect data once a second, and calculate the risk assessment for each second. For example, if the features extracted from the hidden state show that the ship is approaching an obstacle, β t is set to a higher value to reflect the increased risk.
[0294] In order to further optimize network performance, the system adopts a dynamic parameter update algorithm. This algorithm is based on the loss function The gradient adjusts the network weights W and bias b:
[0295]
[0296] Where η is the learning rate, which controls the speed of parameter update. The learning rate η is usually set in the range of 0.001 to 0.01 to ensure that the network learns stably without gradient explosion or gradient disappearance problems.
[0297] In the specific mooring scenario, the network has identified the risk of collision due to dense fog and multiple nearby ships.
[0298] In the past 30 seconds, the hidden state h per second t and the corresponding risk weight β t As shown below (simplified example):
[0299] h t =[0.5,0.6,0.7,…,0.9]——The hidden state value gradually increases, reflecting the increase in risk.
[0300] β t =[0.1,0.1,0.1,…,1.0]——Risk weight increases as it approaches the current moment.
[0301] Computational risk assessment:
[0302] r t =tanh(0.1×0.5+0.1×0.6+…+1.0×0.9)
[0303] This calculation will determine the immediate actions the ship needs to take, such as adjusting its course or speed. These steps show how to apply advanced data analytics techniques in actual ship operations to assess risks in real time and ensure that the ship conducts berthing operations safely and efficiently through continuous learning and parameter optimization.
[0304] Embodiment 6:
[0305] This embodiment uses advanced neural network models, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), combined with dynamic Bayesian networks to perform real-time risk assessment, which is an efficient technical strategy. This strategy can integrate data collected from multiple sensors and accurately predict the environment around the ship and potential risks. The following is a detailed description of how to use these technologies to achieve risk assessment, including specific implementation plans and data processing procedures.
[0306] In the ship berthing safety auxiliary control system, CNN is mainly used to process images and spatial data to extract the spatial distribution characteristics of the ship's surrounding environment and other ships. The specific data processing uses a double-layer convolution process, and the formula is as follows:
[0307]
[0308] Where d(x,y) represents the image data matrix, α controls the spatial decay rate, and β adjusts the frequency response.
[0309] For example, setting α = 0.1 and β = 1 allows the model to focus on the main features in the image while ignoring high-frequency noise and increasing sensitivity to environmental features. This processing can help the system better identify key information related to ship safety, such as nearby obstacles, ships, and channel markers.
[0310] Applications of Recurrent Neural Networks (RNN):
[0311] RNNs, specifically LSTM or GRU variants, are used to process time series data, such as data collected from GPS, meteorological and hydrological sensors. These networks predict future ship behavior and potential risks by learning the temporal dependencies in the data. Combining the outputs of CNNs and RNNs, the system uses a dynamic Bayesian network for risk assessment. The network is able to combine historical data and real-time observation data to update risk predictions for the current and future states of the ship.
[0312] The dynamic Bayesian network uses the outputs from CNN and RNN, as well as data obtained directly from sensors, to update risk assessments in real time. This integrated approach enables the model to not only react based on current data, but also predict future risk scenarios by learning past behavior patterns. Assuming that the ship is passing through a busy waterway, the system processes image data from the ship's radar through CNN and detects the presence of multiple small ships ahead. At the same time, GPS and hydrological data processed by RNN indicate that the ship is approaching these ships. In this case, the dynamic Bayesian network will combine this data, assess the risk of collision, and adjust the navigation strategy in real time to avoid potential collisions.
[0313] This embodiment uses a long short-term memory network (LSTM) to process time series data, which is a very critical technical link. LSTM is particularly suitable for processing data with long sequence dependencies with time gaps, such as GPS and meteorological data. In this way, the system can effectively predict the conditions and risks that ships will encounter in the future. The following is a detailed example that shows how to process time series data through the LSTM model, including specific calculation formulas and parameter settings.
[0314] The LSTM network controls the retention and forgetting of information through a series of gating mechanisms (forget gate, input gate, output gate), so that the network can learn long-term dependencies from data. In the ship berthing safety auxiliary control system, the specific application of LSTM can be described as the following formula:
[0315]
[0316] Among them, h t represents the hidden state at time t, h t-1 is the hidden state at the previous moment, x t is the input data collected from GPS and weather sensors at the current moment, and b (k) are the weight and bias parameters of the kth gating unit in the LSTM network.
[0317] The system is assumed to be processing data collected from GPS and meteorological sensors, including the ship's position, speed, wind speed and direction. The parameters in the LSTM network are set as follows:
[0318] K: The number of gating units, set to 3.
[0319] and The weights are usually initialized in the range of -0.5, 0.5.
[0320] b (k) The bias is usually initialized in the range of -0.1,0.1.
[0321] ω k : The influence factor of each gating unit weight is set to 1 to simplify the model calculation.
[0322] Consider a specific time t, x t Contains the following data:
[0323] GPS location: 22°55'N, 122°09'E
[0324] Wind speed: 15km / h
[0325] Wind direction: North
[0326] These data are first processed by the LSTM gating mechanism. In LSTM, the forget gate decides to discard part of the state information of the previous moment, the input gate updates the new input information, and the output gate controls the output of the information. The calculation of each gate is as follows:
[0327] For example,
[0328]
[0329] These results are then combined to update h t :
[0330] h t =tanh(forget gate result × input gate result + output gate result)
[0331] Through the constantly updated h t ,LSTM can reflect the state changes of the ship in real time and predict its future behavior.
[0332] Embodiment 7:
[0333] It is crucial to use a comprehensive environmental model to simulate the dynamic state of the ship and its surroundings in real time. This model integrates multi-source data from GPS, meteorological, hydrological and ship dynamic sensors, and applies a complex mathematical model for data processing and state prediction. The following are steps to implement this model, including an explanation of the model's specific formulas and a specific application example.
[0334] Application of mathematical models:
[0335] The mathematical model expression used is:
[0336]
[0337] In this model:
[0338] v(τ) is the speed at time τ.
[0339] d(τ) represents the heading at time τ.
[0340] α, β, and ω are model parameters that control the predicted behavior of the ship under different environmental conditions.
[0341] α controls the decay rate of speed over time, and its value range is 0.01 to 0.1, which is adjusted according to the stability of the environment and the length of the prediction time.
[0342] β adjusts the amplitude of the sinusoidal fluctuation, and its value range can be set to 0.5 to 2.0 to adapt to different meteorological and hydrological conditions. ω represents the frequency of the sinusoidal fluctuation, and its value range is to π, which reflects how frequently environmental conditions change.
[0343] Assuming a specific time window t = 10 seconds, the following data are collected from the ship’s sensor system:
[0344] The function of the speed v(τ) changing with time, for example due to the acceleration process, can be expressed as:
[0345] v(τ)=5+0.5τ (initial velocity is 5 m / s, increasing by 0.5 m / s per second).
[0346] The heading d(τ) is assumed to be relatively stable, for example d(τ) = 090° (east).
[0347] Substitute these data into the model for integral calculation. Set α = 0.05, β = 1.5, Numerical integration methods can be used to approximate the calculation of S(t). The specific calculation steps are as follows:
[0348] 1. For each small time increment Δτ, calculate and d(τ)·log(1+τ 2 ).
[0349] 2. Add up the values of all these small increments to get the sum, which will be an approximation of the integral from τ = 0 to τ = t.
[0350] This calculation provides a real-time simulated dynamic state, allowing the system to predict the ship's behavior and environmental risks in the next few seconds, so that rapid adjustments and responses can be made to ensure the safety and efficiency of the ship.
[0351] This embodiment uses reinforcement learning-based control decisions to optimize ship operations as an advanced method. This method relies on real-time data to dynamically adjust the speed and heading to adapt to changing environmental conditions. The following details how to implement this strategy, including the implementation of specific reinforcement learning algorithms and how to use real data to drive these algorithms.
[0352] First, the system adopts a strategy network μ(s t |θ μ ) to predict the given state s t The best action a under t The policy network is constructed by network parameters θ μParameterized, these parameters are constantly updated during the training process. In order to ensure that the algorithm can explore new strategies and avoid falling into local optimality, an exploration term is introduced where ∈ is a small random scaling factor, σ 2 Control the scope of exploration.
[0353] The value range of ∈ is generally set to 0.01 to 0.1, and σ 2 It can be set according to the sensitivity of ship operation and environmental uncertainty, usually between 0.1 and 1.0.
[0354] Reward function R(s t ,a t ) is designed to:
[0355] r t +γ∫ a′ Q(s t+1 ,a′;θ Q )exp(-λa′ 2 )da′
[0356] Here, r t is an immediate reward that reflects the effect of the current action, such as the improvement in safety or efficiency after the ship adjusts its course or speed. γ is the discount rate for future rewards, usually set between 0.9 and 0.99 to emphasize long-term benefits in the future. The Q function estimates the expected return under a given state and action, while λ adjusts the penalty for future action selection, and the value of λ is usually set between 0.01 and 0.1.
[0357] Assume that a ship is facing a complex navigation environment at a certain moment and needs to decide whether to speed up to pass through a narrow waterway or change course to avoid potential obstacles. The system collects the current GPS position, water flow and wind speed data as status s t , and then use the policy network to calculate the optimal operation a t If the policy network recommends speeding up, but considering the exploration factor, the system randomly chooses to slow down slightly to test the long-term effect of this operation. After the system executes the action, it calculates the immediate reward r based on the smoothness and safety of the navigation. t , and use the reward function and value function to update the network parameters and optimize future decisions.
[0358] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. Ship berthing safety auxiliary control system based on multi-data fusion, Features The following steps are involved: First, various types of data required during the ship's berthing process are collected through a variety of sensors and data acquisition interfaces, including GPS positioning data, meteorological data including wind speed, wind direction, humidity, and temperature, hydrological data including tides, flow rate, and water depth, ship dynamic data including speed, heading, acceleration, and surrounding environment data; Secondly, the collected data is cleaned, formatted and synchronized, machine learning methods are used to optimize data quality, and data fusion algorithms such as Kalman filters are used to integrate data from different sources; Then, deep learning techniques including convolutional neural network (CNN) and recurrent neural network (RNN) are used to establish a dynamic behavior model of the ship based on the fused data, and combined with Bayesian network for real-time risk assessment. Finally, based on the model prediction and risk assessment results, a control decision algorithm based on reinforcement learning was developed and implemented to generate control instructions in real time to adjust ship operations including adjusting speed and changing course.
2. According to the multi-data fusion-based ship berthing safety auxiliary control system of claim 1, Features The method for collecting various types of data required during the ship's berthing process adopts: S1. First develop the integrated sensor module through high-order calculus formula: Synchronously collect GPS positioning, meteorological data, hydrological data and ship dynamic data, where X represents the sensor data vector, T represents the collection period, and a i ,μ i ,σ i ,λ i They are adjustment parameters respectively; S2, then through the multivariate response adjustment formula: Dynamic optimization processing is performed on the collected data, where D represents the original data vector obtained from the sensor, t represents the collection time, and b j ,c j ,α j It is the adjustment coefficient, which is used to dynamically adjust the frequency and priority of data collection; S3. Finally, the fusion analysis formula is used: Among them, U, V represent various environmental perception data that have been preliminarily processed, including sonar and radar data, and s represents the environmental noise variable to integrate environmental information.
3. According to the multi-data fusion-based ship berthing safety auxiliary control system of claim 1, Features The method for cleaning, formatting and synchronizing the data includes: S1, first through the adaptive data cleaning algorithm: Processing collected multi-source data, where D i represents the data collected by different sensors including GPS, meteorological sensors, and hydrological sensors, μ i Represents the expected value of the i-th data type, which is used to locate the center of data cleaning; σ i Represents the standard deviation of the i-th data type, which is used to quantify the dispersion of the data; α i represents the adjustment factor, which is used to adjust the cleaning intensity of the i-th data type; β i Represents the power exponent, which is used to control the influence and processing rate of data outliers; S2, then use the time synchronization algorithm: Synchronize the data from various ship systems, where λ adjusts the accuracy of synchronization to control data consistency during berthing operations; t represents the actual time point of data acquisition; Δt represents the expected synchronization time difference, which is used to correct the data time axis; T represents the total acquisition period, which defines the operating window of the time synchronization algorithm; S3. Through feature engineering algorithm: Extract the features that have the greatest impact on berthing safety from the data, among which X k represents the kth feature extracted from the original data; γ k Represents the importance weight of the kth feature, which determines the influence of the feature in the model; S4. Simultaneous data quality assessment model: Real-time monitoring of data quality, where D represents the evaluated data set, which contains multiple data points; v j represents the quality impact factor of the jth data point, which is used to weigh the importance of the data point in the overall evaluation; δ j Represents the adjustment coefficient, which is used to adjust the influence of the jth data point; S5. Finally, the improved Kalman filter algorithm is used: Among them, U, V represent the processing results of different data sources, which represent the data vector obtained through preliminary processing; ψ represents the data fusion function, which is used to combine the information of U and V, as well as the environmental noise s; s represents the state estimation variable, which is used to simulate the uncertainty and noise in the environment; ξ i ,η i Represents adjustment parameters, which are used to optimize the effect of data fusion.
4. According to the multi-data fusion-based ship berthing safety auxiliary control system of claim 1, Features The method for constructing the dynamic behavior model of the ship comprises: Firstly, adaptive filtering technology is used to fuse multi-source data from GPS, meteorological, hydrological and ship dynamic sensors and eliminate data noise and inconsistency. Then, convolutional neural network (CNN) is used to process the fused data, especially the image and sequence data, and the convolutional layer structure and activation function are designed to extract the spatial characteristics of ship behavior and the patterns in the time series; Then, a recurrent neural network (RNN), specifically a long short-term memory (LSTM) or a gated recurrent unit (GRU), is applied to capture the dynamic characteristics that change over time based on past ship status and environmental data, and to predict future ship behavior and risk situations. Finally, the dynamic Bayesian network is combined with the data output from CNN and RNN to update the risk prediction, and a real-time risk assessment model is constructed taking into account the influence of historical data and observation data.
5. According to claim 4, the ship berthing safety auxiliary control system based on multi-data fusion, Features The multi-source data fusion method: First, the adaptive filtering algorithm is used: Where x(t) represents the data collected from GPS, meteorological, hydrological and ship dynamic sensors, t is the acquisition time, θ(s) represents the environmental parameters of wind speed and water flow speed at time s, α, β, γ are dynamically adjusted parameters used to adjust the filter strength and period according to environmental changes; Then use the data preprocessing and synchronization formula: Where x(t) and y(t) represent the time series data of different sensors respectively. It is the average value of the y data in period T, which is used to synchronize the data of different sensors and calculate the absolute value integral of the deviation to measure and correct the synchronization error of the time series; Finally, implement the machine learning-assisted data fusion model: where x = {x 1 (t),x 2 (t),…,x n (t)} represents the preprocessed multi-sensor data vector, w i It is the weights determined by deep learning process optimization.
6. The ship berthing safety auxiliary control system based on multi-data fusion according to claim 4, Features The convolutional neural network CNN processes the fused data from GPS, meteorological, hydrological and ship dynamic sensors and is designed with 1D convolutional layers: Processing time series data, where x(t) represents the time series of sensor output, T is the time window considered, and α is the decay factor, adjusted to adapt to different environmental change rates, while using 2D convolutional layers: Processing spatial data, including images or radar scans, x(i,j) represents the pixel value in the image, w ij is the weight of the convolution kernel, k is the size of the convolution kernel, which is used to extract spatial features in the image; The network then passes through the parameterized ReLU activation function: h(x)=max(0,x)+λmin(0,x) where λ is a learnable parameter that is adjusted to enhance the model’s sensitivity to changes in the ship’s dynamic behavior; Finally, the deep learning algorithm is used through the formula: Perform data fusion, where x m represents the features passed down from the previous network layer, w nm and θ n are optimized weight parameters used to extract and fuse data from multiple sensors in a deep network, controlling the prediction of the ship’s behavior under different berthing conditions.
7. The ship berthing safety auxiliary control system based on multi-data fusion according to claim 4, Features The RNN structure adopts the LSTM or GRU structure to process and analyze the fused data from GPS, meteorological, hydrological and ship dynamic sensors in real time. Through high-level functions: To update the hidden state of the network, σ represents the sigmoid activation function, which captures the time-dependent characteristics of the ship's behavior and the changes in environmental factors; Then, using the learned time series features, the risk assessment module calculates the future risk level using the formula: To integrate the output of the model at different time points, where β t It is the risk weight corresponding to the time point, which assesses the risks that the ship may face in real time, including collision or grounding; Finally, by continuously receiving new input data and using the update algorithm: in is the loss function and η is the learning rate to optimize the network parameters.
8. The ship berthing safety auxiliary control system based on multi-data fusion according to claim 4, Features The architecture adopted by the data update risk prediction includes: S1. First, use the convolutional neural network CNN and the recurrent neural network RNN to process the data collected from various sensors respectively; through the double-layer convolution process, CNN uses the formula: Processing image and spatial data, where d(x,y) represents the image data matrix, α controls the spatial decay rate, and β adjusts the frequency response, so that the network extracts spatial distribution features about the ship's surroundings and other ships; S2. Then use the long short-term memory network LSTM to process time series data, using the formula: where h t represents the hidden state at time t, h t-1 is the hidden state at the previous moment, x t is the input data collected from GPS and weather sensors at the current moment, and b (k) are the weight and bias parameters of the kth gating unit in the LSTM network.
9. The ship berthing safety auxiliary control system based on multi-data fusion according to claim 1, Features The control decision algorithm layout method adopts: First, a comprehensive environmental model is used to integrate multi-source data from GPS, meteorological, hydrological and ship dynamic sensors, using a mathematical model: To simulate the dynamic state of the ship and its surroundings in real time, where v(τ) and d(τ) represent the speed and heading respectively, and α, β, and ω are model parameters that control and predict the behavior of the ship under different environmental conditions; Reinforcement learning-based control decisions are then used to optimize ship operations, especially regulating speed and changing course, by: Calculate the optimal operation, where μ represents the policy network and θ μ are the network parameters, ∈ and σ 2 Control the scope of exploration of operations; Finally, use the reward function: To learn and update, where r t is the immediate reward, representing the effect of the current action, γ is the discount rate of future rewards, Q represents the action value function, and θ Q and λ are model parameters.
Citation Information
Patent Citations
Unmanned ship path following system and method based on deep reinforcement learning
CN114859910A
Building construction management method based on multi-data fusion
CN117788223A
Vehicle safety decision-making method and device, electronic equipment, storage medium and vehicle
CN117818659A
Ship working condition online identification analysis method and system based on deep learning
CN117932279A
Marine transport multimodal transport path optimization system based on artificial intelligence
CN118228979A
Cited By
High-efficiency ship automatic control system
CN119717529A
Monitoring data acquisition and analysis method
CN119939116A
A method for monitoring data acquisition and analysis
CN119939116B
Ship automatic mooring control strategy optimization method and system based on multi-source data
CN120122424A
Optimization Method and System for Ship Automatic Mooring Control Strategy Based on Multi-Source Data
CN120122424B
