Multi-source data fusion-based ship berthing behavior prediction and violation early warning method and system

Abstract

The invention relates to a ship berthing behavior prediction and violation early warning method and system based on multi-source data fusion. The method comprises the steps that a multi-source data set is acquired; constructing a multi-source feature set based on the multi-source data set; inputting the multi-source feature set into a behavior simulation model; obtaining port scheduling information and port safety area information of the target port, and performing analysis and comparison based on a risk assessment strategy; and when any risk probability in the risk assessment strategy is greater than a corresponding threshold value, generating a corresponding graded early warning signal. The problem that a traditional single-source monitoring system is large in error in a severe environment is solved through multi-source data collaborative perception, the limitation of fixed threshold judgment is overcome by introducing a dynamic risk assessment model, and intelligent collaboration of berthing operation is achieved by fusing port scheduling information; the problems that in the prior art, ship berthing monitoring is not comprehensive, early warning is not accurate, and adaptability is poor are effectively solved, and the effects of improving port safety and operation efficiency are achieved.

Description

Technical Field

[0001] This application relates to the technical field of ship berthing prediction, and in particular to a method and system for ship berthing behavior prediction and violation warning based on multi-source data fusion. Background Technique

[0002] Ship berthing operation is one of the most critical and risky links in port operation, and its safety and efficiency directly affect port throughput and operating costs. With the rapid development of the global shipping industry, the trends of ship enlargement and port congestion are becoming increasingly obvious, and the traditional berthing safety management method is facing challenges.

[0003] Currently, ports mainly rely on crew experience and basic navigation aids for berthing operations. However, due to the complex and changeable port environment, involving multiple factors such as ship dynamics, meteorology and hydrology, and terminal structure, manual judgment and single-source monitoring systems have difficulty meeting the high standards of modern ports for berthing safety. Especially in adverse conditions such as poor visibility, strong winds and rapid currents, the risk of berthing accidents increases significantly. At best, it will cause damage to ships and terminal facilities, and at worst, it will lead to casualties and environmental pollution. Statistics from the International Maritime Organization (IMO) show that about 42% of global port accidents occur during berthing and unberthing, and most of them are due to misjudgment of ship motion states and potential risks.

[0004] Existing technologies usually adopt rule-based single-source data analysis. For example, they only predict ship trajectories through AIS data or rely on static port electronic fences for out-of-bounds judgment. Such methods have obvious technical limitations: First, the monitoring method of single data sources is difficult to comprehensively perceive the ship state in a complex port environment. Especially in adverse weather conditions, AIS signals are easily interfered with and radar has blind spots, resulting in trajectory prediction errors often exceeding 10 meters. Second, existing systems lack the ability to dynamically evaluate the risks throughout the ship berthing process and cannot effectively integrate multi-dimensional information such as port scheduling and real-time hydrometeorology, making the warning accuracy rate generally lower than 70%. In addition, current systems mostly use fixed thresholds for risk judgment, making it difficult to adapt to the personalized safety needs of different berths and different ship types, resulting in either missed reporting of dangerous situations or frequent false alarms in actual applications, seriously affecting port operation efficiency. Summary of the Invention

[0005] To solve the above defects, this application provides a method and system for ship berthing behavior prediction and violation warning based on multi-source data fusion.

[0006] The first invention object of this application is achieved through the following technical solutions:

[0007] A method for ship berthing behavior prediction and violation warning based on multi-source data fusion, including the steps:

[0008] Obtain the ship information associated with the target port, and obtain the multi-source data set associated with the ship information;

[0009] Extract multi-source features based on the obtained multi-source data set and construct a multi-source feature set;

[0010] Input the multi-source feature set into the pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating the ship berthing behavior;

[0011] Obtain the port scheduling information and port safety area information of the target port, and perform risk assessment based on risk items on the prediction result vector, port scheduling information and port safety area information based on the pre-set risk assessment strategy;

[0012] When the risk probability of any risk item in the risk assessment strategy is greater than its corresponding pre-set threshold, generate a corresponding graded early warning signal and send it to the scheduling terminal associated with the target port.

[0013] By adopting the above technical solution, obtain the target ship information associated with the target port, integrate multi-source heterogeneous data, and construct a multi-source feature set reflecting the ship state and port environment through feature extraction and fusion; input the multi-source feature set into the pre-trained behavior simulation model to make it output the simulation result vector of the ship berthing behavior; combine the port scheduling plan and port safety area information, and perform multi-dimensional analysis and comparison based on risk items through the pre-set risk assessment strategy, and automatically generate a graded early warning signal when a risk item exceeding the threshold is detected; this application solves the problem of large errors in traditional single-source monitoring systems in harsh environments through multi-source data collaborative perception, overcomes the limitations of fixed threshold judgment by introducing a dynamic risk assessment model, and realizes intelligent collaboration of berthing operations by integrating port scheduling information; in addition, the graded early warning mechanism can trigger different response strategies according to the risk level, avoiding interference with normal operations caused by over-warning while ensuring safety, and effectively solving the problems of incomplete ship berthing monitoring, inaccurate early warning and poor adaptability in the prior art, and having the effect of significantly improving the safety and operation efficiency of port operations.

[0014] In a preferred example of this application, it can be further configured that: the multi-source data set includes ship static data, AIS data, port radar data, video surveillance data, meteorological and hydrological data, and historical berthing record data. The step of extracting multi-source features based on the obtained multi-source data set and constructing a multi-source feature set includes the steps:

[0015] Extract ship static features based on ship static data, extract dynamic navigation features based on AIS data, extract ship space features based on port radar data and video surveillance data, extract environmental features based on meteorological and hydrological data, and extract berthing features based on historical berthing record data;

[0016] Standardize the extracted features and construct a multi-source feature set.

[0017] By adopting the above technical solution, through the extraction and fusion processing of multi-source features, a comprehensive digital representation of ship berthing behavior characteristics is realized, which has the effect of significantly improving the quality and reliability of the input data of the prediction model. Specifically, static ship features are extracted from the static ship data to establish a benchmark for the physical characteristics of the ship; dynamic navigation features are extracted from the AIS data to capture the ship's motion state; ship space features are extracted by combining port radar and video surveillance data to construct a high-precision spatial relationship network; environmental features are extracted by integrating meteorological and hydrological data to quantify the impact of external conditions on berthing; and finally, berthing features are extracted by analyzing historical berthing records to form knowledge supplementation. Through multi-dimensional feature extraction, this application realizes the digital characterization of multiple elements of ship berthing. After standardization processing to eliminate the dimensional differences of the feature quantities, a unified multi-source feature set is constructed, providing comprehensive, accurate and regular input data for the subsequent behavior simulation model, thereby significantly improving the accuracy and reliability of the prediction results and providing a data basis for berthing safety early warning.

[0018] In a preferred example of this application, it can be further configured that: the behavior simulation model includes a spatio-temporal encoding layer, a feature fusion layer, and a behavior simulation layer. The step of inputting the multi-source feature set into the pre-trained behavior simulation model to make the behavior simulation model output a prediction result vector for simulating ship berthing behavior includes the steps:

[0019] The spatio-temporal encoding layer constructs a ship spatio-temporal graph structure based on the dynamic navigation features, environmental features, and berthing features;

[0020] The feature fusion layer updates the ship spatio-temporal graph structure based on the static ship features and the ship space features to generate an enhanced spatio-temporal graph structure;

[0021] The behavior simulation layer simulates the ship behavior based on the enhanced spatio-temporal graph structure and outputs a prediction result vector.

[0022] By adopting the above technical solution, a behavior simulation model architecture including a spatio-temporal encoding layer, a feature fusion layer, and a behavior simulation layer is constructed to achieve accurate modeling and prediction of ship berthing behavior, which has the effect of significantly improving the prediction accuracy and reliability. Specifically, through the spatio-temporal encoding layer, the dynamic navigation features, environmental features, and berthing features are fused to construct a ship spatio-temporal graph structure, effectively capturing the dynamic interactions of ships and environmental influencing factors. The feature fusion layer integrates the static features of the ship and the ship's spatial features into the ship spatio-temporal graph structure for updating, generating an enhanced spatio-temporal graph structure containing the physical characteristics and precise spatial positions of the ship. The behavior simulation layer performs ship behavior simulation based on the enhanced spatio-temporal graph structure and outputs a prediction result vector. This application realizes end-to-end modeling from raw data to prediction results by constructing a behavior simulation model architecture including a spatio-temporal encoding layer, a feature fusion layer, and a behavior simulation layer. Among them, the spatio-temporal encoding layer solves the alignment problem of heterogeneous spatio-temporal data, the feature fusion layer establishes an interaction mechanism for cross-modal features, and the behavior simulation layer integrates data-driven and physical rules for simulation prediction, improving the accuracy of the prediction results and providing technical support for berthing safety early warning.

[0023] In a preferred example, this application can be further configured as follows: The dynamic navigation features include the real-time position of the ship. The step of the spatio-temporal encoding layer constructing a ship spatio-temporal graph structure based on the dynamic navigation features, environmental features, and berthing features includes the steps of:

[0024] Taking the real-time position of the ship as the central node and using the dynamic navigation features, environmental features, and berthing features as node feature vectors, an initial ship node feature matrix is constructed;

[0025] Based on the Gaussian kernel function, spatial correlation modeling is performed, and the spatial relationship weights between the node feature vectors are calculated;

[0026] Based on the spatial correlation modeling results, a ship spatio-temporal graph structure is constructed, where the edges of the ship spatio-temporal graph structure include spatial edges and time edges.

[0027] By adopting the above technical solution, the spatio-temporal graph structure of the ship is constructed to realize the spatio-temporal modeling of the ship berthing behavior, which has the effect of improving the accuracy and real-time performance of behavior prediction. Specifically, taking the real-time position of the ship as the central node, the dynamic navigation features, environmental features and berthing features are integrated into a multi-dimensional node feature vector to construct an initial ship node feature matrix. The spatial relationship weights between nodes are calculated through an adaptive Gaussian kernel function, comprehensively considering the distance between ships, tonnage differences and environmental interference factors, and accurately quantifying the interaction intensity between ships. And a ship spatio-temporal graph structure is constructed based on the spatial correlation modeling result. By constructing a ship spatio-temporal graph structure with a dual design including spatial edges and temporal edges, this application can not only reflect the real-time spatial interaction between ships, but also capture the temporal sequence law of single-ship movement, providing a standardized data representation containing spatio-temporal information for subsequent behavior prediction, enabling the model to accurately simulate the dynamic behavior of ships in a complex port environment.

[0028] In a preferred example, this application can be further configured as follows: The feature fusion layer includes an embedding layer. The step of updating the ship spatio-temporal graph structure based on the ship static features and the ship spatial features to generate an enhanced spatio-temporal graph structure includes the steps:

[0029] The ship static features are mapped into low-dimensional feature vectors through the embedding layer, and the mapping result is concatenated with the ship spatio-temporal graph structure.

[0030] The ship static features and the ship spatial features are weighted and fused to obtain a weighted fusion feature.

[0031] The nodes and edges of the ship spatio-temporal graph structure are updated based on the weighted fusion feature.

[0032] By adopting the above technical solution, the coupling of the ship static characteristics and the dynamic behavior is realized through a feature fusion mechanism, which has the effect of improving the integrity and accuracy of berthing behavior prediction. Specifically, the ship static features are converted into low-dimensional dense vectors through the embedding layer, and the mapping result is concatenated with the ship spatio-temporal graph structure, retaining the correlation between the inherent attributes of the ship and the real-time state. Through the weighted fusion mechanism, the contribution weights of the static features and the spatial features are dynamically adjusted to generate a weighted fusion feature that includes both the physical characteristics of the ship and reflects the spatial position. The nodes and edges of the spatio-temporal graph structure are updated based on the weighted fusion feature. Through a hierarchical feature fusion strategy, this application realizes the unity of the essential attributes of the ship, the spatial position relationship and the dynamic behavior characteristics, enabling the enhanced spatio-temporal graph structure to simultaneously represent multi-dimensional information and providing more comprehensive and accurate input data for the behavior simulation layer.

[0033] In a preferred example, this application can be further configured as follows: The step of simulating the ship behavior based on the enhanced spatio-temporal graph structure and outputting a prediction result vector by the behavior simulation layer includes the steps:

[0034] Perform spatio-temporal graph convolution operations on the enhanced spatio-temporal graph structure to extract spatio-temporal features based on ship behavior;

[0035] Perform multi-task synchronous prediction based on the extracted spatio-temporal features, where the multi-task synchronous prediction includes ship berthing trajectory prediction and berthing end state prediction;

[0036] Integrate the multi-task synchronous prediction results into a standardized prediction vector, where the standardized prediction vector includes a trajectory prediction sub-vector and a state prediction sub-vector;

[0037] Perform physical rationality correction on the standardized prediction vector based on the ship kinematic model and output it as a prediction result vector.

[0038] By adopting the above technical solutions, the architecture of the behavior simulation layer is designed to achieve multi-dimensional accurate prediction of ship berthing behavior, which has the effect of improving the reliability and practicality of the prediction results; specifically, perform spatio-temporal graph convolution operations on the enhanced spatio-temporal graph structure, and capture spatio-temporal features based on ship behavior through the feature propagation mechanism in the spatial dimension and the time dimension; adopt a multi-task learning framework to synchronously predict the ship berthing trajectory and the berthing end state; integrate the multi-task prediction results into a standardized prediction vector, which includes a trajectory prediction sub-vector and a state prediction sub-vector; perform physical rationality correction on the prediction results based on the ship kinematic model to ensure that the output prediction result vector conforms to the ship dynamics law; through a hybrid modeling method of data-driven and physical constraints, this application not only improves the trajectory prediction accuracy, but also ensures the physical rationality of the prediction results, especially in complex scenarios, significantly improving the reliability of the prediction results.

[0039] In a preferred example of this application, it can be further configured as: the step of obtaining the port scheduling information and the port safety area information of the target port, and performing risk assessment based on risk items on the prediction result vector, the port scheduling information, and the port safety area information based on a pre-set risk assessment strategy, includes the steps of:

[0040] Perform a spatial region comparison between the prediction result vector and the port safety area information;

[0041] Perform a time comparison between the prediction result vector and the port scheduling information;

[0042] Based on the pre-set risk assessment strategy, perform risk feature extraction and corresponding risk coefficient calculation on the spatial region comparison result and the time comparison result.

[0043] By adopting the above technical solution, multi-dimensional risk comparison is carried out with a preset risk assessment strategy to achieve intelligent assessment and early warning of ship berthing risks, which has the effect of improving the accuracy and response efficiency of port safety management. Specifically, spatial region comparison is performed on the prediction result vector and port safety region information. By calculating the real-time distance between the predicted position and the safety boundary, potential over-border collision risks are accurately identified. Temporal comparison is performed on the prediction result vector and port scheduling information to detect whether there are time conflicts or delay risks. Based on the preset risk assessment strategy, risk feature extraction and corresponding risk coefficient calculation are performed on spatial position deviation and temporal deviation. Through the risk quantification method in both spatial and temporal dimensions, this application realizes the technical leap from extensive early warning to refined assessment, and can not only identify obvious spatial over-border risks, but also discover hidden temporal scheduling conflicts, which has the effect of improving the safety margin and scheduling efficiency of port operations.

[0044] The second above-mentioned inventive object of this application is achieved through the following technical solutions:

[0045] A ship berthing behavior prediction and violation early warning system based on multi-source data fusion, comprising:

[0046] A data acquisition module, configured to acquire ship information associated with a target port and acquire a multi-source data set associated with the ship information;

[0047] A feature set construction module, configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set;

[0048] An input module, configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector of the simulated ship berthing behavior;

[0049] A comparison module, configured to acquire port scheduling information and port safety region information of a target port, and perform risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety region information based on a preset risk assessment strategy;

[0050] An early warning module, configured to generate a corresponding hierarchical early warning signal and send it to a scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than its corresponding preset threshold.

[0051] By adopting the above technical solution, a data acquisition module is configured to acquire ship information associated with a target port and acquire a multi-source data set associated with the ship information; a feature set construction module is configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set; an input module is configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating the ship berthing behavior; a comparison module is configured to acquire port scheduling information and port safety area information of the target port, and perform risk assessment based on risk items on the prediction result vector, the port scheduling information, and the port safety area information based on a pre-set risk assessment strategy; an early warning module is configured to generate a corresponding graded early warning signal and send it to a scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than its corresponding pre-set threshold.

[0052] In summary, the present application includes at least one of the following beneficial technical effects:

[0053] 1. The present application solves the problem of large errors in traditional single-source monitoring systems in harsh environments through multi-source data collaborative perception, overcomes the limitations of fixed threshold judgment by introducing a dynamic risk assessment model, and realizes intelligent collaboration of berthing operations by integrating port scheduling information; in addition, the graded early warning mechanism can trigger differentiated response strategies according to the risk level, avoiding interference with normal operations caused by over-early warning while ensuring safety, effectively solving the problems of incomplete monitoring of ship berthing, inaccurate early warning, and poor adaptability in the prior art, and having the effect of significantly improving the safety and operation efficiency of port operations. Description of the Drawings

[0054] Figure 1 is a flowchart of an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to the present application;

[0055] Figure 2 is an implementation flowchart of step S30 in an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to the present application;

[0056] Figure 3 is an implementation flowchart of step S31 in an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to the present application;

[0057] Figure 4 is an implementation flowchart of step S32 in an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to the present application;

[0058] Figure 5 is an implementation flowchart of step S33 in an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to the present application;

[0059] Figure 6 This is a flowchart showing one implementation of step S40 in an embodiment of a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion in this application. Detailed implementation manner

[0060] The following further elaborates on this application in conjunction with the attached Figures 1-6 for a more detailed description.

[0061] In one embodiment, as Figure 1 shown, this application discloses a method for predicting ship berthing behavior and warning against violations based on multi-source data fusion, which specifically includes the following steps:

[0062] S10: Obtain the ship information associated with the target port and obtain the multi-source data set associated with the ship information;

[0063] In this embodiment, the target port is the port where the target ship berths or the port for ship berthing selected manually; the ship information is the identity information corresponding to the ship; the multi-source data set is the ship-related data obtained from different sensors or systems;

[0064] Specifically, obtain the identity information of the target ship associated with the target port and integrate the multi-source heterogeneous data associated with the ship identity information;

[0065] S20: Extract multi-source features based on the obtained multi-source data set and construct a multi-source feature set;

[0066] In this embodiment, the multi-source feature set is a set of features extracted from multi-source data and processed by standardization, which is used to describe ship status, environmental factors, berthing behavior patterns, etc.;

[0067] Specifically, extract the corresponding features from different data sources, perform standardization processing, and form a unified feature matrix, so as to construct a multi-source feature set reflecting ship status and port environment through feature extraction and fusion;

[0068] S30: Input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector of the simulated ship berthing behavior;

[0069] In this embodiment, the behavior simulation model is a prediction model based on machine learning, which is used to simulate ship berthing behavior. After inputting the multi-source feature set, it outputs a prediction result vector of ship berthing behavior (such as future trajectory, berthing attitude, risk probability, etc.); the prediction result vector is the output of the behavior simulation model, which contains the prediction data information of the ship's future berthing behavior, such as trajectory prediction (ship position sequence within a certain period of time in the future), berthing final state prediction (final berthing position, attitude angle, etc.), risk probability (risk values such as collision, crossing the boundary, attitude abnormality, etc.);

[0070] Specifically, input the multi-source feature set into the pre-trained behavior simulation model to output a prediction result vector for simulating the ship berthing behavior.

[0071] S40: Obtain the port scheduling information and port safety area information of the target port, and perform risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety area information based on a pre-set risk assessment strategy.

[0072] In this embodiment, the port scheduling information is port operation plan data, including ship berthing time window, berth allocation, priority, etc.; the port safety area information is port electronic fence data, defining the safe area where ships are allowed to navigate; the risk assessment strategy is a pre-set rule or algorithm for calculating the risk level of risk items for ship berthing behavior, and the risk items are such as: spatial risk (whether the predicted trajectory exceeds the safe area), time risk (whether the berthing time conflicts with the scheduling plan), status risk (whether the berthing attitude is abnormal), etc.

[0073] Specifically, obtain the port operation plan data and port safety area information (such as electronic fence data) of the target port, and analyze and compare the obtained data and the prediction result vector output by the behavior simulation model based on a pre-set risk assessment strategy to calculate the risk level of the ship berthing behavior.

[0074] S50: When the risk probability of any risk item in the risk assessment strategy is greater than its corresponding pre-set threshold, generate a corresponding graded warning signal and send it to the scheduling terminal associated with the target port.

[0075] In this embodiment, the graded warning signal is an alarm message generated according to the risk level (such as high, medium, low), which is used for the decision-making of the scheduling terminal.

[0076] Specifically, when the risk probability of any kind in the risk assessment strategy is greater than its corresponding set threshold, generate a corresponding graded warning signal, that is, an alarm message, and send it to the management and scheduling terminal associated with the target port for the decision-making of the scheduling terminal.

[0077] In one embodiment, the multi-source data set includes ship static data, AIS data, port radar data, video surveillance data, meteorological and hydrological data, and historical berthing record data. Step S20 includes the steps:

[0078] S21: Extract ship static features based on ship static data, extract dynamic navigation features based on AIS data, extract ship spatial features based on port radar data and video surveillance data, extract environmental features based on meteorological and hydrological data, and extract berthing features based on historical berthing record data.

[0079] S22: Standardize the extracted features and construct a multi-source feature set.

[0080] In this embodiment, the ship static data includes the inherent attributes of the ship, such as ship type, tonnage, size, etc.; the AIS data is the dynamic navigation information provided by the Automatic Identification System for Ships, such as position, speed, course, etc.; the port radar data is the ship position and movement data collected by the port radar system; the video surveillance data is the ship visual information collected by the port cameras; the meteorological and hydrological data is the port environmental data, such as wind speed, water flow, wave height, etc.; the historical berthing record data is the ship's past berthing behavior data; the ship static features are the features extracted from the ship static data, including ship type code, tonnage grade, ship length, ship width, the ratio of ship length to ship width, maximum draft depth, etc.; the dynamic navigation features are the features extracted from the AIS data, including real-time position coordinates, instantaneous velocity vector, course change rate, acceleration, etc.; the ship spatial features are the features extracted from the radar and video data, including the exact distance between the ship and the dock edge, berthing angle (the angle between the ship's longitudinal axis and the shoreline), the distribution of surrounding obstacles, etc.; the environmental features are the features extracted from the meteorological and hydrological data, including the influence of wind speed and direction, water flow force, the influence degree of wave height on maneuvering, etc.; the berthing features are the features extracted from the historical records, including typical berthing trajectory patterns, historical berthing deviation statistics, berthing performance features under different environmental conditions, etc.

[0081] Specifically, extract ship static features based on ship static data, extract dynamic navigation features based on AIS data, extract ship spatial features based on port radar data and video surveillance data, extract environmental features based on meteorological and hydrological data, extract berthing features based on historical berthing record data, standardize the extracted features, and construct a multi-source feature set.

[0082] In one embodiment, the behavior simulation model includes a spatio-temporal encoding layer, a feature fusion layer, and a behavior simulation layer, as Figure 2 shown, step S30 includes the steps:

[0083] S31: The spatio-temporal encoding layer constructs a ship spatio-temporal graph structure based on the dynamic navigation features, environmental features, and berthing features;

[0084] S32: The feature fusion layer updates the ship spatio-temporal graph structure based on the ship static features and ship spatial features to generate an enhanced spatio-temporal graph structure;

[0085] S33: The behavior simulation layer simulates the ship behavior based on the enhanced spatio-temporal graph structure and outputs a prediction result vector.

[0086] In this embodiment, the spatio-temporal encoding layer is the first processing module of the behavior simulation model, which is used to encode the dynamic behavior, environmental factors, and historical berthing patterns of the ship into a ship spatio-temporal graph structure; the ship spatio-temporal graph structure is a graph data structure, where the nodes represent ships and contain their feature vectors, and the edges represent the spatio-temporal relationships (spatial proximity, temporal continuity) between ships or between ships and ports; the feature fusion layer is the second processing module of the behavior simulation model, which is used to integrate the static attributes (such as dimensions) and spatial features (such as positional relationships) of the ship into the ship spatio-temporal graph structure to update and generate an enhanced spatio-temporal graph structure; the enhanced spatio-temporal graph structure is a graph structure optimized by the feature fusion layer, which contains various ship representation information integrating the static attributes (such as dimensions) and spatial features (such as positional relationships) of the ship; the behavior simulation layer is the third processing module of the behavior simulation model, which is used to predict the future behavior of the ship based on the enhanced spatio-temporal graph structure.

[0087] Specifically, the dynamic navigation features, environmental features, and berthing features are fused through the spatio-temporal encoding layer to construct a ship spatio-temporal graph structure, effectively capturing the dynamic interactions and environmental influencing factors of the ship; the feature fusion layer integrates the ship static features and ship spatial features into the ship spatio-temporal graph structure for updating, generating an enhanced spatio-temporal graph structure containing the physical characteristics and precise spatial positions of the ship; the behavior simulation layer performs ship behavior simulation based on the enhanced spatio-temporal graph structure and outputs a prediction result vector.

[0088] In one embodiment, the dynamic navigation features include the real-time position of the ship, such as Figure 3 As shown, step S31 includes the steps:

[0089] S311: Using the real-time position of the ship as the central node, and using the dynamic navigation features, environmental features, and berthing features as node feature vectors, construct an initial ship node feature matrix;

[0090] S312: Perform spatial correlation modeling based on the Gaussian kernel function and calculate the spatial relationship weights between the node feature vectors;

[0091] S313: Construct a ship spatio-temporal graph structure based on the spatial correlation modeling results, where the edges of the ship spatio-temporal graph structure include spatial edges and temporal edges.

[0092] In this embodiment, the real-time position of the ship is the current precise coordinates (longitude, latitude) of the ship obtained through AIS or GPS; the central node is the graph node representing the target ship at the current moment in the spatio-temporal graph, which is the core representation of the spatio-temporal state of the ship; the node feature vector is a numerical vector describing the comprehensive state of the ship, and the dimension is usually 64 to 256 dimensions. The node feature vector includes dynamic components, environmental components, historical components, etc. Among them, the dynamic components include position (x, y), speed (vx, vy), course angle, etc.; the environmental components include environmental impact coefficients such as wind speed and water flow; the historical components include the mean berthing deviation and trajectory pattern coding, etc.; the Gaussian kernel function is a mathematical function for calculating the spatial correlation between ships: where ω ij is the spatial relationship weight, that is, the interaction intensity between ship i and ship j; p i is the planar coordinate of ship i (local coordinate system after longitude and latitude conversion); p j is the planar coordinate of ship j; σ is an adaptive bandwidth parameter, which is positively correlated with the ship tonnage; the spatial edge is the edge connecting ship nodes with a spatial distance less than the preset value in nautical miles, and the weight reflects the interaction intensity; the time edge is the edge connecting nodes of the same ship at adjacent time steps (such as an interval of 10 seconds), maintaining motion continuity;

[0093] Specifically, taking the real-time position of the ship as the central node, integrating dynamic navigation features, environmental features, and berthing features into a multi-dimensional node feature vector, constructing an initial ship node feature matrix; calculating the spatial relationship weight between nodes through an adaptive Gaussian kernel function, comprehensively considering the distance, tonnage difference, and environmental interference factors between ships, and accurately quantifying the interaction intensity between ships; and constructing a ship spatio-temporal graph structure based on the spatial correlation modeling results.

[0094] In one embodiment, the feature fusion layer includes an embedding layer, as Figure 4 shown, step S32 includes the steps:

[0095] S321: Map the static features of the ship into a low-dimensional feature vector through the embedding layer, and splice the mapping result with the ship spatio-temporal graph structure;

[0096] S322: Perform weighted fusion on the static features of the ship and the spatial features of the ship to obtain a weighted fusion feature;

[0097] S323: Update the nodes and edges of the ship spatio-temporal graph structure based on the weighted fusion feature.

[0098] In this embodiment, the embedding layer is a trainable parameter matrix of the feature fusion layer, which is used to integrate the static attributes (inherent features) and dynamic spatial features (real-time position relationships) of the ship; the low-dimensional feature vector is a compact feature representation obtained through dimensionality reduction, usually with a dimension of 32 to 128; concatenation is one of the operations of feature fusion, which connects the feature vectors from different sources end to end; weighted fusion dynamically assigns weights to different features through an attention mechanism; updating the nodes and edges of the ship spatio-temporal graph structure is to recalculate the attributes and connection relationships of the graph nodes according to the new features.

[0099] Specifically, the static features of the ship are converted into low-dimensional dense vectors through the embedding layer, and the mapping result is concatenated with the ship spatio-temporal graph structure, retaining the relevance between the inherent attributes and real-time states of the ship; the contribution weights of the static features and spatial features are dynamically adjusted through a weighted fusion mechanism to generate weighted fusion features that include both the physical characteristics of the ship and reflect the spatial position; the nodes and edges of the spatio-temporal graph structure are updated based on the weighted fusion features.

[0100] In one embodiment, as Figure 5 shown, step S33 includes the steps of:

[0101] S331: Perform spatio-temporal graph convolution operations on the enhanced spatio-temporal graph structure to extract spatio-temporal features based on ship behavior;

[0102] S332: Perform multi-task synchronous prediction based on the extracted spatio-temporal features, and the multi-task synchronous prediction includes ship berthing trajectory prediction and berthing end state prediction;

[0103] S333: Integrate the multi-task synchronous prediction results into a standardized prediction vector, and the standardized prediction vector includes a trajectory prediction sub-vector and a state prediction sub-vector;

[0104] S334: Perform physical rationality correction on the standardized prediction vector based on the ship kinematic model and output it as a prediction result vector.

[0105] In this embodiment, the spatio-temporal graph convolution operation is a graph neural network operation that simultaneously processes spatial and temporal dimensions, and updates node features by aggregating neighboring node information; it includes spatial convolution and temporal convolution, where spatial convolution aggregates the features of adjacent ships at the same time point, and temporal convolution aggregates the features of the same ship at historical time steps; multi-task synchronous prediction is to predict multiple related targets in parallel, which includes: berthing trajectory prediction and berthing end state prediction, where the berthing trajectory prediction is a position point every preset second within a preset future time, for example, a position point every 2 seconds within the next 30 seconds, and the berthing end state prediction is the (x, y, θ) coordinates and attitude angle at the final berthing; the standardized prediction vector is an output data structure in a unified format, including: a trajectory prediction sub-vector and a state prediction sub-vector; the ship kinematic model is a physical equation describing the motion of the ship, mainly including: mass-inertia equation, hydrodynamic equation, rudder effect-heading response model, etc.; physical rationality correction is a rationality correction to ensure that the prediction conforms to the maximum turning rate (usually <10° / s for cargo ships), minimum braking distance (positively correlated with tonnage), and hydrodynamic constraints, etc.

[0106] Specifically, perform spatio-temporal graph convolution operation on the enhanced spatio-temporal graph structure, and capture spatio-temporal features based on ship behavior through the feature propagation mechanism in the spatial and temporal dimensions; adopt a multi-task learning framework to synchronously predict the ship berthing trajectory and berthing end state; integrate the multi-task prediction results into a standardized prediction vector, which includes a trajectory prediction sub-vector and a state prediction sub-vector; perform physical rationality correction on the prediction results based on the ship kinematic model to ensure that the output prediction result vector conforms to the laws of ship dynamics.

[0107] In one embodiment, as Figure 6 shown, step S40 includes the steps:

[0108] S41: Perform a spatial region comparison between the prediction result vector and the port safety region information;

[0109] S42: Perform a time comparison between the prediction result vector and the port scheduling information;

[0110] S43: Based on the pre-set risk assessment strategy, perform risk feature extraction and corresponding risk coefficient calculation on the spatial region comparison result and the time comparison result.

[0111] In this embodiment, the spatial region comparison is to analyze the spatial position relationship between the predicted trajectory points and the boundary of the safety region through computational geometry algorithms to quantitatively evaluate the potential spatial violation risks; the time comparison is to compare the predicted berthing time with the scheduling plan time window through time series analysis methods to identify possible scheduling conflicts or delay risks; the risk assessment strategy is a pre-set risk quantification model that defines the weight allocation rules and risk level classification criteria for spatial violations and time conflicts; the risk feature extraction is to screen key risk indicators from the results of spatial region comparison and time comparison, including the spatial intrusion depth, violation duration, time deviation amount, etc.; the risk coefficient calculation is a process of converting the extracted risk features into standardized risk values using a mathematical model, and the risk coefficient comprehensively reflects the overall risk level of the ship berthing behavior;

[0112] Specifically, perform a spatial region comparison between the predicted result vector and the port safety region information. By calculating the real-time distance between the predicted position and the safety boundary, accurately identify potential overboard collision risks; perform a time comparison between the predicted result vector and the port scheduling information to detect whether there are time conflicts or delay risks; based on the preset risk assessment strategy, perform risk feature extraction and corresponding risk coefficient calculation for spatial position deviation and time deviation.

[0113] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0114] In one embodiment, a ship berthing behavior prediction and violation warning system based on multi-source data fusion is provided. This ship berthing behavior prediction and violation warning system based on multi-source data fusion corresponds one-to-one with the above-mentioned ship berthing behavior prediction and violation warning method based on multi-source data fusion. This ship berthing behavior prediction and violation warning system based on multi-source data fusion includes:

[0115] A data acquisition module, configured to acquire ship information associated with the target port and acquire a multi-source data set associated with the ship information;

[0116] A feature set construction module, configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set;

[0117] An input module, configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a predicted result vector of the simulated ship berthing behavior;

[0118] A comparison module, configured to obtain the port scheduling information and port safety area information of a target port, and perform risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety area information according to a preset risk assessment strategy;

[0119] An early warning module, configured to generate a corresponding hierarchical early warning signal and send it to the scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than its corresponding preset threshold.

[0120] For the specific limitations of a ship berthing behavior prediction and violation early warning system based on multi-source data fusion, reference can be made to the limitations of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion in the foregoing text, which will not be elaborated here. Each module in the above-mentioned ship berthing behavior prediction and violation early warning system based on multi-source data fusion can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0121] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for predicting ship berthing behavior and warning against violations based on multi-source data fusion, characterized in that: Including the steps: Obtain the ship information associated with the target port, and obtain the multi-source dataset associated with the ship information; Extract multi-source features based on the obtained multi-source dataset and construct a multi-source feature set; Input the multi-source feature set into the pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating the ship berthing behavior; Obtain the port scheduling information and port safety area information of the target port, and conduct risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety area information based on the pre-set risk assessment strategy; When the risk probability of any risk item in the risk assessment strategy is greater than its corresponding pre-set threshold, generate a corresponding graded warning signal and send it to the scheduling terminal associated with the target port.

2. The method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to claim 1, wherein: The multi-source dataset includes ship static data, AIS data, port radar data, video surveillance data, meteorological and hydrological data, and historical berthing record data. The step of extracting multi-source features based on the obtained multi-source dataset and constructing a multi-source feature set includes the steps: Extract ship static features based on ship static data, extract dynamic navigation features based on AIS data, extract ship space features based on port radar data and video surveillance data, extract environmental features based on meteorological and hydrological data, and extract berthing features based on historical berthing record data; Perform standardization processing on the extracted features and construct a multi-source feature set.

3. A method for predicting ship berthing behavior and early warning of violations based on multi-source data fusion according to claim 2, characterized in that: The behavior simulation model includes a spatio-temporal encoding layer, a feature fusion layer, and a behavior simulation layer. The step of inputting the multi-source feature set into the pre-trained behavior simulation model so that the behavior simulation model outputs a prediction result vector simulating the ship berthing behavior includes the steps: The spatio-temporal encoding layer constructs a ship spatio-temporal graph structure based on dynamic navigation features, environmental features, and berthing features; The feature fusion layer updates the ship spatio-temporal graph structure based on ship static features and ship space features to generate an enhanced spatio-temporal graph structure; The behavior simulation layer simulates the ship behavior based on the enhanced spatio-temporal graph structure and outputs a prediction result vector.

4. A method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to claim 3, characterized in that: The dynamic navigation features include the real-time position of the ship. The step of the spatio-temporal encoding layer constructing a ship spatio-temporal graph structure based on dynamic navigation features, environmental features, and berthing features includes the steps: Use the real-time position of the ship as the central node, and use dynamic navigation features, environmental features, and berthing features as node feature vectors to construct an initial ship node feature matrix; Conduct spatial correlation modeling based on the Gaussian kernel function and calculate the spatial relationship weights between node feature vectors; Construct a ship spatio-temporal graph structure based on the spatial correlation modeling result, where the edges of the ship spatio-temporal graph structure include spatial edges and time edges.

5. The method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to claim 3, characterized in that: The feature fusion layer includes an embedding layer. The step of the feature fusion layer updating the ship spatio-temporal graph structure based on ship static features and ship space features to generate an enhanced spatio-temporal graph structure includes the steps: Map the ship static features to low-dimensional feature vectors through the embedding layer and splice the mapping result with the ship spatio-temporal graph structure; Perform weighted fusion of ship static features and ship space features to obtain weighted fusion features; Update the nodes and edges of the ship spatio-temporal graph structure based on the weighted fusion features.

6. The method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to claim 3, wherein: The step of simulating ship behavior based on the enhanced spatio-temporal graph structure in the behavior simulation layer and outputting a prediction result vector includes the steps of: Performing spatio-temporal graph convolution operation on the enhanced spatio-temporal graph structure to extract spatio-temporal features based on ship behavior; Performing multi-task synchronous prediction based on the extracted spatio-temporal features, where the multi-task synchronous prediction includes ship berthing trajectory prediction and berthing end state prediction; Integrating the multi-task synchronous prediction results into a standardized prediction vector, where the standardized prediction vector includes a trajectory prediction sub-vector and a state prediction sub-vector; Performing physical rationality correction on the standardized prediction vector based on the ship kinematic model and outputting it as a prediction result vector.

7. A method for predicting ship berthing behavior and warning against violations based on multi-source data fusion according to claim 1, characterized in that: The step of obtaining the port scheduling information and port safety area information of the target port, and performing risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety area information based on a pre-set risk assessment strategy includes the steps of: comparing the spatial regions of the prediction result vector and the port safety area information; Performing time comparison between the prediction result vector and the port scheduling information; Extracting risk characteristics and calculating corresponding risk coefficients based on the pre-set risk assessment strategy for the spatial region comparison result and the time comparison result.

8. A ship berthing behavior prediction and violation warning system based on multi-source data fusion, characterized in that: Including: A data acquisition module for acquiring ship information associated with the target port and acquiring a multi-source data set associated with the ship information; A feature set construction module for extracting multi-source features based on the acquired multi-source data set and constructing a multi-source feature set; An input module for inputting the multi-source feature set into a pre-trained behavior simulation model to enable the behavior simulation model to output a prediction result vector simulating ship berthing behavior; A comparison module for acquiring the port scheduling information and port safety area information of the target port and performing risk assessment based on risk items on the prediction result vector, port scheduling information, and port safety area information based on a pre-set risk assessment strategy; An early warning module for generating a corresponding graded early warning signal and sending it to the scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than its corresponding pre-set threshold.

Images