Ship berthing behavior prediction and violation early warning method and system based on multi-source data fusion

Abstract

The application relates to a ship berthing behavior prediction and violation early warning method and system based on multi-source data fusion, which comprises the following steps: acquiring a multi-source data set; constructing a multi-source feature set based on the multi-source data set; inputting the multi-source feature set into a behavior simulation model; acquiring port scheduling information and port safety area information of a target port, and performing analysis and comparison based on a risk assessment strategy; when any risk probability in the risk assessment strategy is greater than a corresponding threshold value, a corresponding graded early warning signal is generated. The application solves the problem of large errors of a traditional single-source monitoring system in a harsh environment through multi-source data collaborative sensing, overcomes the limitations of fixed threshold value judgment by introducing a dynamic risk assessment model, realizes intelligent collaboration of berthing operations by fusing port scheduling information, effectively solves the problems of incomplete ship berthing monitoring, inaccurate early warning and poor adaptability in the prior art, and has the effects of improving port safety and operation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship berthing prediction, and in particular to a ship berthing behavior prediction and violation early warning method and system based on multi-source data fusion. BACKGROUND

[0002] Ship berthing operation is one of the most critical and riskiest links in port operation, and its safety and efficiency directly affect port throughput and operating costs. With the rapid development of global shipping industry, the trend of large-scale ships and port congestion is becoming increasingly apparent, and the traditional berthing safety management method is facing challenges.

[0003] Currently, ports mainly rely on the experience of crew members and basic navigation aids for berthing operations, but due to the complex and variable port environment, involving ship dynamics, meteorological and hydrological conditions, and the interaction of multiple factors such as wharf structure, manual judgment and single-source monitoring systems have been difficult to meet the high standards of modern port berthing safety. Especially in poor visibility, strong wind and other adverse conditions, the risk of berthing accidents increases significantly, which may cause damage to ships and wharf facilities, or even lead to casualties and environmental pollution. According to the statistics of the International Maritime Organization (IMO), about 42% of global port accidents occur during berthing and unberthing, most of which are caused by misjudgment of the ship's motion state and potential risks.

[0004] The existing technology usually adopts rule-based single-source data analysis, such as predicting ship trajectory only through AIS data, or relying on static port electronic fence for crossing judgment. This kind of method has obvious technical limitations: first, the single data source monitoring method is difficult to fully perceive the ship state in complex port environment, especially in adverse weather conditions, AIS signal is easy to be disturbed and radar has blind area, which leads to trajectory prediction error often exceeding 10 meters; second, the existing system lacks the ability of dynamic risk assessment for the whole process of ship berthing, and cannot effectively integrate multi-dimensional information such as port scheduling, real-time hydrology and meteorology, so that the early warning accuracy is generally less than 70%; in addition, the current system uses fixed threshold for risk judgment, which is difficult to adapt to the individual safety needs of different berths and different ship types, resulting in either missing dangerous situations or frequent false alarms, which seriously affects the port operation efficiency. SUMMARY

[0005] In order to solve the above defects, the present application provides a ship berthing behavior prediction and violation early warning method and system based on multi-source data fusion.

[0006] The above invention purpose of the present application is realized by the following technical scheme:

[0007] A ship berthing behavior prediction and violation early warning method based on multi-source data fusion, comprising the steps of:

[0008] Obtain ship information associated with the target port, and obtain a multi-source data set associated with the ship information;

[0009] Extract multi-source features based on the obtained multi-source data set and construct a multi-source feature set;

[0010] Input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating the berthing behavior of the ship;

[0011] Obtain port scheduling information and port safety area information of the target port, and perform risk assessment based on risk items on the prediction result vector, the port scheduling information and the port safety area information based on a pre-set risk assessment strategy;

[0012] When the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold, a corresponding graded early warning signal is generated and sent to the scheduling terminal associated with the target port.

[0013] By adopting the above technical solution, the target ship information associated with the target port is obtained, and multi-source heterogeneous data is integrated. The multi-source feature set reflecting the state of the ship and the environment of the port is constructed through feature extraction and fusion. The multi-source feature set is input into the pre-trained behavior simulation model, so that the simulation result vector of the berthing behavior of the ship is output. The port scheduling plan and the port safety area information are combined, and multi-dimensional analysis and comparison based on risk items are performed through the pre-set risk assessment strategy. When the threshold value risk item is detected, a graded early warning signal is automatically generated. The present application solves the problem of large error of traditional single-source monitoring system in harsh environment through multi-source data collaborative perception. The limitation of fixed threshold value judgment is overcome by introducing a dynamic risk assessment model. The intelligent collaboration of berthing operation is realized by integrating the port scheduling information. In addition, the graded early warning mechanism can trigger a differentiated response strategy according to the risk level, ensuring safety while avoiding excessive early warning interference with normal operation, effectively solving the problems of incomplete berthing monitoring, inaccurate early warning and poor adaptability in the prior art, and having the effect of significantly improving the safety and operation efficiency of the port operation.

[0014] In a preferred example, the present application can be further configured as follows: the multi-source data set includes ship static data, AIS data, port radar data, video monitoring data, meteorological and hydrological data, and historical berthing record data. The step of extracting multi-source features based on the obtained multi-source data set and constructing a multi-source feature set includes the following steps:

[0015] Extracting ship static features based on ship static data, dynamic navigation features based on AIS data, ship spatial features based on port radar data and video monitoring data, environmental features based on meteorological and hydrological data, and berthing features based on historical berthing record data;

[0016] The extracted features are standardized and a multi-source feature set is constructed.

[0017] By adopting the above technical solutions, the extraction and fusion processing of multi-source features are realized, the comprehensive digital representation of ship berthing behavior characteristics is achieved, and the input data quality and reliability of the prediction model are significantly improved. Specifically, ship static features are extracted from ship static data, and a physical characteristic benchmark of the ship is established. Dynamic navigation features are extracted from AIS data to capture the ship's motion state. The ship's spatial features are extracted from port radar and video monitoring data to construct a high-precision spatial relationship network. Environmental features are extracted by integrating meteorological and hydrological data to quantify the influence of external conditions on berthing. Finally, berthing features are extracted by analyzing historical berthing records to form knowledge supplements. The present application realizes the digital description of the multi-element berthing of the ship through multi-dimensional feature extraction, and after standardization processing to eliminate the dimensional difference of the feature quantity, a unified multi-source feature set is constructed, which provides comprehensive, accurate and regular input data for subsequent behavior simulation models, thereby significantly improving the accuracy and reliability of the prediction results, and providing a data basis for berthing safety warning.

[0018] In a preferred example, the application can be further configured to: the behavior simulation model includes a space-time coding layer, a feature fusion layer, and a behavior simulation layer, the step of inputting the multi-source feature set into the pre-trained behavior simulation model to make the behavior simulation model output a prediction result vector of the simulated ship berthing behavior includes the steps of:

[0019] The space-time coding layer constructs a ship space-time graph structure based on dynamic navigation features, environmental features, and berthing features;

[0020] The feature fusion layer updates the ship space-time graph structure based on ship static features and ship spatial features to generate an enhanced space-time graph structure;

[0021] The behavior simulation layer simulates the ship behavior based on the enhanced space-time graph structure and outputs the prediction result vector.

[0022] By adopting the technical scheme, the behavior simulation model architecture including the space-time coding layer, the feature fusion layer and the behavior simulation layer is constructed, precise modeling and prediction of ship berthing behavior are realized, and the effects of significantly improving prediction accuracy and reliability are achieved. Specifically, the dynamic navigation features, the environment features and the berthing features are fused to construct a ship space-time graph structure through the space-time coding layer, the dynamic interaction of the ship and the environmental influence factors are effectively captured; the ship static features and the ship space features are integrated into the ship space-time graph structure for updating through the feature fusion layer, an enhanced space-time graph structure containing the physical characteristics of the ship and the accurate spatial position is generated; the behavior simulation layer performs ship behavior simulation based on the enhanced space-time graph structure, and outputs a prediction result vector; the behavior simulation model architecture including the space-time coding layer, the feature fusion layer and the behavior simulation layer is constructed, end-to-end modeling from original data to prediction result is realized, the space-time coding layer solves the alignment problem of heterogeneous space-time data, the feature fusion layer establishes an interaction mechanism for cross-modal features, and the behavior simulation layer integrates data-driven and physical rules for simulation prediction, thereby improving the accuracy of the prediction result and providing technical support for berthing safety warning.

[0023] In a preferred example, the dynamic navigation features include a real-time position of the ship, and the space-time coding layer constructs a ship space-time graph structure based on the dynamic navigation features, the environment features and the berthing features, including the steps of:

[0024] Taking the real-time position of the ship as a center node and taking the dynamic navigation features, the environment features and the berthing features as node feature vectors, an initial ship node feature matrix is constructed;

[0025] Based on a Gaussian kernel function, spatial correlation modeling is performed, and spatial relationship weights between the node feature vectors are calculated;

[0026] Based on the spatial correlation modeling result, a ship space-time graph structure is constructed, wherein edges of the ship space-time graph structure include spatial edges and time edges.

[0027] By adopting the technical solution, the ship space-time graph structure is constructed, the ship berthing behavior is modeled in space-time, and the behavior prediction accuracy and real-time performance are improved. Specifically, the ship real-time position is taken as a central node, dynamic navigation features, environmental features and berthing features are integrated into a multi-dimensional node feature vector, and an initial ship node feature matrix is constructed. The spatial relationship weight between nodes is calculated by an adaptive Gaussian kernel function, the distance between ships, tonnage difference and environmental interference factors are comprehensively considered, and the interaction strength between ships is accurately quantified. The ship space-time graph structure is constructed based on the spatial correlation modeling result. The ship space-time graph structure with double design of space edges and time edges is constructed, the real-time space interaction between ships is reflected, the single ship motion time sequence rule is captured, the standardized data representation containing space-time information is provided for subsequent behavior prediction, and the model can accurately simulate the dynamic behavior of the ship in the complex port environment.

[0028] In a preferred example, the application can be further configured to: the feature fusion layer includes an embedding layer, the feature fusion layer updates the ship space-time graph structure based on the ship static features and the ship spatial features, and the step of generating an enhanced space-time graph structure includes the steps of:

[0029] The ship static features are mapped into low-dimensional feature vectors by the embedding layer, and the mapping result is spliced with the ship space-time graph structure;

[0030] The ship static features and the ship spatial features are weighted and fused to obtain weighted fusion features;

[0031] The nodes and edges of the ship space-time graph structure are updated based on the weighted fusion features.

[0032] By adopting the technical solution, the ship static characteristics and dynamic behavior are coupled by the feature fusion mechanism, the berthing behavior prediction integrity and accuracy are improved. Specifically, the ship static features are converted into low-dimensional dense vectors by the embedding layer, and the mapping result is spliced with the ship space-time graph structure, so that the correlation between the inherent properties and the real-time state of the ship is retained. The contribution weights of the static features and the spatial features are dynamically adjusted by the weighted fusion mechanism to generate weighted fusion features containing ship physical characteristics and reflecting spatial positions. The nodes and edges of the space-time graph structure are updated based on the weighted fusion features. The hierarchical feature fusion strategy is adopted to unify the ship essential properties, spatial position relationships and dynamic behavior features, so that the enhanced space-time graph structure can represent multi-dimensional information, and more comprehensive and accurate input data are provided for the behavior simulation layer.

[0033] In a preferred example, the application can be further configured to: the behavior simulation layer simulates the ship behavior based on the enhanced space-time graph structure, and outputs a prediction result vector, and the step includes the steps of:

[0034] performing spatio-temporal graph convolution operation on the enhanced spatio-temporal graph structure to extract spatio-temporal features based on the ship behavior;

[0035] performing multi-task synchronous prediction based on the extracted spatio-temporal features, the multi-task synchronous prediction including ship berthing trajectory prediction and berthing terminal state prediction;

[0036] integrating the multi-task synchronous prediction results into a standardized prediction vector, the standardized prediction vector including a trajectory prediction sub-vector and a state prediction sub-vector;

[0037] performing physical rationality correction on the standardized prediction vector based on a ship kinematics model, and outputting the same as a prediction result vector.

[0038] By adopting the above technical solutions, the behavior simulation layer is designed to achieve multi-dimensional accurate prediction of ship berthing behavior, which improves the reliability and practicality of the prediction results. Specifically, spatio-temporal graph convolution operation is performed on the enhanced spatio-temporal graph structure to capture spatio-temporal features based on ship behavior through a feature propagation mechanism in the spatial and temporal dimensions. A multi-task learning framework is used to simultaneously predict ship berthing trajectories and berthing terminal states. The multi-task prediction results are integrated into a standardized prediction vector, which includes a trajectory prediction sub-vector and a state prediction sub-vector. The prediction results are physically corrected based on a ship kinematics model to ensure that the output prediction result vector conforms to the laws of ship dynamics. The hybrid modeling method of data-driven and physical constraints not only improves the trajectory prediction accuracy, but also ensures the physical rationality of the prediction results, especially in complex scenarios, significantly improving the reliability of the prediction results.

[0039] In a preferred example, the application can be further configured to perform risk assessment based on risk items on the prediction result vector, the port scheduling information, and the port safety area information based on a pre-set risk assessment strategy, including the steps of:

[0040] spatial region comparison of the prediction result vector and the port safety area information;

[0041] time comparison of the prediction result vector and the port scheduling information;

[0042] risk feature extraction and corresponding risk coefficient calculation based on the spatial region comparison result and the time comparison result based on the pre-set risk assessment strategy.

[0043] By adopting the technical scheme, multi-dimensional risk comparison is performed on the basis of the pre-set risk assessment strategy, intelligent assessment and early warning of ship berthing risk are realized, and the effect of improving the precision and response efficiency of port safety management is achieved. Specifically, the predicted result vector and the port safety area information are compared in space, the real-time distance between the predicted position and the safety boundary is calculated, and the potential out-of-bound collision risk is accurately identified. The predicted result vector and the port scheduling information are compared in time, and whether there is a time conflict or delay risk is detected. Based on the pre-set risk assessment strategy, the spatial position deviation and the time deviation are extracted for risk characteristics and corresponding risk coefficient calculation. Through the spatial and temporal dual-dimensional risk quantification method, the present application realizes the technical leap from extensive early warning to fine assessment, can not only identify the explicit spatial out-of-bound risk, but also find the implicit time scheduling conflict, and has the effect of improving the safety margin and scheduling efficiency of port operation.

[0044] The second application object of the present application is realized by the following technical scheme:

[0045] A ship berthing behavior prediction and violation early warning system based on multi-source data fusion, comprising:

[0046] A data acquisition module is configured to acquire ship information associated with a target port and acquire a multi-source data set associated with the ship information.

[0047] A feature set construction module is configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set.

[0048] An input module is configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a predicted result vector of simulated ship berthing behavior.

[0049] A comparison module is configured to acquire port scheduling information and port safety area information of a target port, and perform risk assessment on the predicted result vector, the port scheduling information and the port safety area information based on a pre-set risk assessment strategy.

[0050] An early warning module is configured to generate a corresponding graded early warning signal and send it to a scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold.

[0051] By adopting the technical scheme, the data acquisition module is used for acquiring ship information associated with a target port and acquiring a multi-source data set associated with the ship information; the feature set construction module is used for extracting multi-source features based on the acquired multi-source data set and constructing a multi-source feature set; the input module is used for inputting the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector of simulated ship berthing behavior; the comparison module is used for acquiring port scheduling information and port safety area information of the target port, and performing risk assessment based on a risk item based on a pre-set risk assessment strategy on the prediction result vector, the port scheduling information and the port safety area information; and the early warning module is used for generating a corresponding hierarchical early warning signal and sending the signal to a scheduling terminal associated with the target port when a risk probability of any risk item in the risk assessment strategy is greater than a corresponding pre-set threshold.

[0052] To sum up, the present application includes at least one of the following beneficial technical effects:

[0053] 1. The present application solves the problem of large error of traditional single-source monitoring system in harsh environment through multi-source data collaborative perception, overcomes the limitations of fixed threshold judgment by introducing a dynamic risk assessment model, and realizes intelligent collaboration of berthing operation by fusing port scheduling information. In addition, the hierarchical early warning mechanism can trigger differentiated response strategies according to the risk level, ensuring safety while avoiding excessive early warning interference with normal operation, effectively solving the problems of incomplete ship berthing monitoring, inaccurate early warning and poor adaptability in the prior art, and significantly improving the safety and efficiency of port operation. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of an embodiment of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion of the present application;

[0055] Figure 2 is an implementation flowchart of step S30 in an embodiment of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion of the present application;

[0056] Figure 3 is an implementation flowchart of step S31 in an embodiment of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion of the present application;

[0057] Figure 4 is an implementation flowchart of step S32 in an embodiment of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion of the present application;

[0058] Figure 5 is an implementation flowchart of step S33 in an embodiment of a ship berthing behavior prediction and violation early warning method based on multi-source data fusion of the present application;

[0059] Figure 6 This is a flowchart of step S40 in an embodiment of a method for predicting and warning of violations of ship berthing behavior based on multi-source data fusion in this application. Detailed Implementation

[0060] The following is in conjunction with the appendix Figures 1-6 This application will be described in further detail.

[0061] In one embodiment, such as Figure 1 As shown, this application discloses a method for predicting ship berthing behavior and providing early warning of violations based on multi-source data fusion, which specifically includes the following steps:

[0062] S10: Obtain the vessel information associated with the target port, and obtain the multi-source dataset associated with the vessel information;

[0063] In this embodiment, the target port is the port where the target vessel berths or a port selected by the user; the vessel information is the vessel's identity information; and the multi-source dataset is vessel-related data obtained from different sensors or systems.

[0064] Specifically, the identification information of the target vessel associated with the target port is obtained, and the multi-source heterogeneous data associated with the vessel's identification information is integrated;

[0065] S20: Extract multi-source features based on the acquired multi-source dataset and construct a multi-source feature set;

[0066] In this embodiment, the multi-source feature set is a set of features extracted from multi-source data and standardized, used to describe ship status, environmental factors, and berthing behavior patterns, etc.

[0067] Specifically, features are extracted from different data sources and standardized to form a unified feature matrix, so as to construct a multi-source feature set reflecting the ship status and port environment through feature extraction and fusion.

[0068] S30: Input the multi-source feature set into the pre-trained behavior simulation model, so that the behavior simulation model outputs the prediction result vector of the simulated ship berthing behavior;

[0069] In this embodiment, the behavior simulation model is a machine learning-based prediction model used to simulate ship berthing behavior. After inputting a multi-source feature set, it outputs a prediction result vector of ship berthing behavior (such as future trajectory, berthing attitude, risk probability, etc.). The prediction result vector is the output of the behavior simulation model, which contains prediction data information of the ship's future berthing behavior, such as trajectory prediction (ship position sequence within a future period), berthing final state prediction (final berthing position, attitude angle, etc.), risk probability (risk values ​​such as collision, boundary crossing, and attitude abnormality).

[0070] Specifically, the multi-source feature set is input into the pre-trained behavior simulation model, so that a prediction result vector of simulating the ship berthing behavior is output;

[0071] S40: Obtain the port scheduling information and the port safety area information of the target port, and perform risk assessment based on a risk item based on the pre-set risk assessment strategy on the prediction result vector, the port scheduling information and the port safety area information;

[0072] In this embodiment, the port scheduling information is port operation plan data, including ship berthing time window, berth allocation, priority, etc.; the port safety area information is port electronic fence data, defining the safety area allowing the ship to sail; the risk assessment strategy is a pre-set rule or algorithm for calculating the risk level of the ship berthing behavior risk item, such as spatial risk (whether the predicted trajectory exceeds the safety area), time risk (whether the berthing time conflicts with the scheduling plan), state risk (whether the berthing attitude is abnormal), etc.

[0073] Specifically, the port operation plan data and the port safety area information (electronic fence data, etc.) of the target port are obtained, and the obtained data and the prediction result vector output by the behavior simulation model are analyzed and compared based on the pre-set risk assessment strategy, and the risk level of the ship berthing behavior is calculated.

[0074] S50: When the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold, a corresponding graded warning signal is generated and sent to the scheduling terminal associated with the target port.

[0075] In this embodiment, the graded warning signal is alarm information generated according to the risk level (such as high, medium and low), which is used for scheduling terminal decision-making.

[0076] Specifically, when the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold, a corresponding graded warning signal, i.e. alarm information, is generated and sent to the management scheduling terminal associated with the target port, which is used for scheduling terminal decision-making.

[0077] In an embodiment, the multi-source data set includes ship static data, AIS data, port radar data, video monitoring data, meteorological and hydrological data, and historical berthing record data, and step S20 includes steps of:

[0078] S21: extracting ship static features based on ship static data, extracting dynamic sailing features based on AIS data, extracting ship spatial features based on port radar data and video monitoring data, extracting environmental features based on meteorological and hydrological data, and extracting berthing features based on historical berthing record data;

[0079] S22: standardize the extracted features and construct a multi-source feature set.

[0080] In this embodiment, the ship static data includes ship inherent attributes such as ship type, tonnage, size, etc.; the AIS data is dynamic navigation information provided by the ship automatic identification system, including position, speed, heading, etc.; the port radar data is the ship position and motion data collected by the port radar system; the video monitoring data is the ship visual information collected by the port camera; the meteorological and hydrological data is the port environment data such as wind speed, water flow, wave height, etc.; the historical berthing record data is the past berthing behavior data of the ship; the ship static features are the features extracted from the ship static data, including ship type code, tonnage level, ship length, ship width, the ratio of ship length to ship width, maximum draft, etc.; the dynamic navigation features are the features extracted from the AIS data, including real-time position coordinates, instantaneous speed vector, heading rate of change, acceleration, etc.; the ship space features are the features extracted from the radar and video data, including the accurate distance between the ship and the wharf edge, the berthing angle (the angle between the ship longitudinal axis and the shoreline), the distribution of surrounding obstacles, etc.; the environment features are the features extracted from the meteorological and hydrological data, including the influence of wind speed and direction, the water flow force, the influence degree of wave height on maneuvering, etc.; the berthing features are the features extracted from the historical record, including the typical berthing trajectory pattern, the historical berthing deviation statistics, the berthing performance characteristics under different environmental conditions, etc.

[0081] Specifically, the ship static features are extracted based on the ship static data, the dynamic navigation features are extracted based on the AIS data, the ship space features are extracted based on the port radar data and the video monitoring data, the environment features are extracted based on the meteorological and hydrological data, and the berthing features are extracted based on the historical berthing record data. The extracted features are standardized, and a multi-source feature set is constructed.

[0082] In an embodiment, the behavior simulation model includes a space-time coding layer, a feature fusion layer, and a behavior simulation layer, as shown in Figure 2 Step S30 includes the following steps:

[0083] S31: The space-time coding layer constructs a ship space-time graph structure based on the dynamic navigation features, the environment features, and the berthing features;

[0084] S32: The feature fusion layer updates the ship space-time graph structure based on the ship static features and the ship space features, to generate an enhanced space-time graph structure;

[0085] S33: The behavior simulation layer simulates the ship behavior based on the enhanced space-time graph structure, and outputs a prediction result vector.

[0086] In the embodiment, the spatio-temporal coding layer is a first layer processing module of the behavior simulation model, used to code the dynamic behavior of the ship, the environmental factors and the historical berthing mode into a ship spatio-temporal graph structure; the ship spatio-temporal graph structure is a kind of graph data structure, wherein the node represents the ship and contains the feature vector thereof, and the edge represents the spatio-temporal relationship (spatial proximity, time continuity) between the ships or between the ship and the port; the feature fusion layer is a second layer processing module of the behavior simulation model, used to integrate the static attributes (such as size) and the spatial features (such as position relationship) of the ship into the ship spatio-temporal graph structure to update and generate an enhanced spatio-temporal graph structure; the enhanced spatio-temporal graph structure is a graph structure optimized by the feature fusion layer, containing various ship representation information of the integrated static attributes (such as size) and the spatial features (such as position relationship) of the ship; the behavior simulation layer is a third layer processing module of the behavior simulation model, used to predict the future behavior of the ship based on the enhanced spatio-temporal graph structure.

[0087] Specifically, the dynamic navigation features, the environmental features and the berthing features are fused and constructed into the ship spatio-temporal graph structure by the spatio-temporal coding layer, so as to effectively capture the dynamic interaction of the ship and the environmental influence factors; the ship static features and the ship spatial features are integrated into the ship spatio-temporal graph structure by the feature fusion layer to update and generate the enhanced spatio-temporal graph structure containing the physical characteristics and the accurate spatial position of the ship; the behavior simulation layer performs the behavior simulation of the ship based on the enhanced spatio-temporal graph structure to output the prediction result vector.

[0088] In an embodiment, the dynamic navigation features include the real-time position of the ship, such as Figure 3 As shown in the figure, step S31 includes the following steps:

[0089] S311: taking the real-time position of the ship as the center node and taking the dynamic navigation features, the environmental features and the berthing features as the node feature vectors, an initial ship node feature matrix is constructed;

[0090] S312: based on the Gaussian kernel function, the spatial correlation is modeled, and the spatial relationship weight between the node feature vectors is calculated;

[0091] S313: based on the modeling result of the spatial correlation, a ship spatio-temporal graph structure is constructed, wherein the edges of the ship spatio-temporal graph structure include spatial edges and time edges.

[0092] In the embodiment, the real-time position of the ship is the current accurate coordinates (longitude, latitude) of the ship obtained through AIS or GPS; the center node is the graph node representing the target ship at the current time in the space-time graph, which is the core representation of the space-time state of the ship; the node feature vector is a numerical vector describing the comprehensive state of the ship, usually with a dimension of 64 to 256, and the node feature vector includes dynamic components, environmental components, historical components, etc., wherein the dynamic components include position (x, y), speed (vx, vy), heading angle, etc.; the environmental components include wind speed, water flow and other environmental influence coefficients; the historical components include berthing deviation mean, trajectory pattern coding, etc.; and the Gaussian kernel function is a mathematical function for calculating the spatial correlation between ships: wherein ω ij is the spatial relationship weight, i.e., the interaction intensity between ship i and ship j; p i is the planar coordinate of ship i (converted from longitude and latitude to a local coordinate system); p j is the planar coordinate of ship j; and σ is an adaptive bandwidth parameter, which is positively correlated with the tonnage of the ship; the spatial edge is an edge connecting ship nodes with a spatial distance less than a preset value of sea miles, and the weight reflects the interaction intensity; and the time edge is an edge connecting adjacent time step (e.g., 10 seconds apart) nodes of the same ship, which maintains the continuity of motion.

[0093] Specifically, taking the real-time position of the ship as the center node, the dynamic navigation features, environmental features and berthing features are integrated into a multi-dimensional node feature vector to construct an initial ship node feature matrix; the spatial relationship weight between nodes is calculated through an adaptive Gaussian kernel function, the distance, tonnage difference and environmental interference factors between ships are comprehensively considered, and the interaction intensity between ships is accurately quantified; and the ship space-time graph structure is constructed based on the modeling results of the spatial correlation.

[0094] In an embodiment, the feature fusion layer includes an embedding layer, as shown in Figure 4 Step S32 includes the following steps:

[0095] S321: mapping the static features of the ship into a low-dimensional feature vector through the embedding layer, and splicing the mapping result with the ship space-time graph structure;

[0096] S322: weighting and fusing the static features of the ship with the spatial features of the ship to obtain weighted fusion features;

[0097] S323: updating the nodes and edges of the ship space-time graph structure based on the weighted fusion features.

[0098] In the embodiment, the embedding layer is a trainable parameter matrix of the feature fusion layer, used to integrate the static attributes (intrinsic features) and dynamic spatial features (real-time position relationships) of the ship; the low-dimensional feature vector is a compact feature representation obtained by dimension reduction, usually with a dimension of 32 to 128; the splicing is one of the operations of feature fusion, which connects the feature vectors of different sources end to end; the weighted fusion is a dynamic allocation of weights of different features through an attention mechanism; and the nodes and edges of the ship space-time graph structure are updated according to the new features to recalculate the attributes and connection relationships of the graph nodes.

[0099] Specifically, the ship static features are converted into low-dimensional dense vectors through the embedding layer, and the mapping results are spliced with the ship space-time graph structure, thereby retaining the relevance of the inherent attributes and real-time state of the ship; the weighted fusion mechanism is used to dynamically adjust the contribution weights of the static features and the spatial features, thereby generating weighted fusion features containing both the physical characteristics of the ship and reflecting the spatial position; and the nodes and edges of the space-time graph structure are updated based on the weighted fusion features.

[0100] In an embodiment, as shown in FIG. 3, the step S33 includes the steps of: Figure 5

[0101] S331: performing space-time graph convolution operation on the enhanced space-time graph structure to extract space-time features based on ship behavior;

[0102] S332: performing multi-task synchronous prediction based on the extracted space-time features, wherein the multi-task synchronous prediction includes ship berthing trajectory prediction and berthing terminal state prediction;

[0103] S333: integrating the multi-task synchronous prediction results into a standardized prediction vector, wherein the standardized prediction vector includes a trajectory prediction sub-vector and a state prediction sub-vector;

[0104] S334: performing physical rationality correction on the standardized prediction vector based on a ship kinematic model, and outputting the result as a prediction result vector.

[0105] ​In the embodiment, the spatio-temporal graph convolution operation is a graph neural network operation that simultaneously processes spatial and temporal dimensions to update node features by aggregating neighboring node information; the spatio-temporal graph convolution operation includes spatial convolution and temporal convolution, wherein the spatial convolution aggregates features of adjacent ships at the same time point, and the temporal convolution aggregates features of the same ship at different time steps; the multi-task synchronous prediction is a parallel prediction of multiple related targets, which includes: berthing trajectory prediction and berthing terminal state prediction, wherein the berthing trajectory prediction is a position point every preset second in the future, for example, a position point every 2 seconds in the future 30 seconds, and the berthing terminal state prediction is the (x, y, θ) coordinates and the attitude angle at the final berthing; the standardized prediction vector is a uniform format output data structure, which includes: a trajectory prediction sub-vector and a state prediction sub-vector; the ship kinematics model is a physical equation describing the motion of the ship, mainly including: mass-inertia equation, fluid dynamics equation, rudder effect-course response model, etc.; the physical rationality correction is a rationality correction to ensure that the prediction conforms to the maximum turning rate (usually <10° / s for cargo ships), the minimum braking distance (positively correlated with tonnage), and the fluid dynamics constraint;

[0106] Specifically, the spatio-temporal graph structure is subjected to spatio-temporal graph convolution operation to capture spatio-temporal features based on ship behavior through feature propagation mechanisms in spatial and temporal dimensions; a multi-task learning framework is used to synchronously predict the berthing trajectory and the berthing terminal state of the ship; the multi-task prediction results are integrated into a standardized prediction vector, which includes a trajectory prediction sub-vector and a state prediction sub-vector; based on the ship kinematics model, the prediction results are subjected to physical rationality correction to ensure that the output prediction result vector conforms to the ship dynamics law.

[0107] In an embodiment, as shown in Figure 6 step S40 includes the following steps:

[0108] S41: performing spatial region comparison between the prediction result vector and the port safety region information;

[0109] S42: performing time comparison between the prediction result vector and the port scheduling information;

[0110] S43: based on the pre-set risk assessment strategy, performing risk feature extraction and corresponding risk coefficient calculation on the spatial region comparison result and the time comparison result.

[0111] In the embodiment, the spatial region comparison is to analyze the spatial positional relationship between the predicted trajectory points and the safety region boundary through a computational geometry algorithm, to quantitatively evaluate the potential spatial violation risk; the time comparison is to compare the predicted berthing time with the scheduling plan time window through a time sequence analysis method, to identify the possible plan conflict or delay risk; the risk evaluation strategy is a pre-set risk quantification model, which defines the weight distribution rules and risk level classification standards of the spatial violation and time conflict; the risk feature extraction is to screen key risk indicators from the spatial region comparison and time comparison results, including the spatial intrusion depth, violation duration, time deviation amount, etc.; the risk coefficient calculation is a process of converting the extracted risk features into standardized risk values by using a mathematical model, and the risk coefficient comprehensively reflects the overall risk level of the ship berthing behavior;

[0112] Specifically, the spatial region comparison is performed on the predicted result vector and the port safety region information, the real-time distance between the predicted position and the safety boundary is calculated, and the potential boundary collision risk is accurately identified; the time comparison is performed on the predicted result vector and the port scheduling information, to detect whether there is a time conflict or delay risk; based on the pre-set risk evaluation strategy, the spatial position deviation and the time deviation are extracted for risk feature extraction and corresponding risk coefficient calculation.

[0113] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0114] In an embodiment, a ship berthing behavior prediction and violation warning system based on multi-source data fusion is provided, which corresponds one-to-one to the ship berthing behavior prediction and violation warning method based on multi-source data fusion in the above embodiment. The ship berthing behavior prediction and violation warning system based on multi-source data fusion comprises:

[0115] A data acquisition module is configured to acquire ship information associated with a target port and acquire a multi-source data set associated with the ship information;

[0116] A feature set construction module is configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set;

[0117] An input module is configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a predicted result vector of simulated ship berthing behavior;

[0118] The comparison module is configured to obtain the port scheduling information and the port safety area information of the target port, and perform risk assessment on the prediction result vector, the port scheduling information and the port safety area information based on a pre-set risk assessment strategy.

[0119] The early warning module is configured to generate a corresponding graded early warning signal and send it to a scheduling terminal associated with the target port when the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold.

[0120] For specific limitations of the ship berthing behavior prediction and violation early warning system based on multi-source data fusion, refer to the limitations of the ship berthing behavior prediction and violation early warning method based on multi-source data fusion described above, which will not be repeated here. Each module in the ship berthing behavior prediction and violation early warning system based on multi-source data fusion can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0121] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A ship berthing behavior prediction and violation early warning method based on multi-source data fusion, characterized in that: The method comprises the steps of: obtaining ship information associated with a target port and obtaining a plurality of source data sets associated with the ship information; extracting multi-source features based on the obtained plurality of source data sets and constructing a multi-source feature set; inputting the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating the berthing behavior of the ship; obtaining port scheduling information and port safety area information of the target port, and performing risk assessment based on a risk item based on a pre-set risk assessment strategy on the prediction result vector, the port scheduling information and the port safety area information; when the risk probability of any risk item in the risk assessment strategy is greater than the corresponding pre-set threshold, a corresponding graded early warning signal is generated and sent to a scheduling terminal associated with the target port; The behavior simulation model comprises a space-time coding layer, a feature fusion layer and a behavior simulation layer. The step of inputting the multi-source feature set into the pre-trained behavior simulation model so that the behavior simulation model outputs a prediction result vector simulating the berthing behavior of the ship comprises the steps of: The space-time coding layer constructs a ship space-time graph structure based on dynamic navigation features, environmental features and berthing features; The feature fusion layer updates the ship space-time graph structure based on ship static features and ship spatial features to generate an enhanced space-time graph structure; The behavior simulation layer simulates the behavior of the ship based on the enhanced space-time graph structure and outputs a prediction result vector; The feature fusion layer comprises an embedding layer. The step of updating the ship space-time graph structure based on the ship static features and the ship spatial features to generate the enhanced space-time graph structure comprises the steps of: mapping the ship static features into a low-dimensional feature vector through the embedding layer, and splicing the mapping result with the ship space-time graph structure; weighting and fusing the ship static features and the ship spatial features to obtain weighted fusion features; updating the nodes and edges of the ship space-time graph structure based on the weighted fusion features; The step of simulating the behavior of the ship based on the enhanced space-time graph structure and outputting a prediction result vector by the behavior simulation layer comprises the steps of: performing space-time graph convolution operation on the enhanced space-time graph structure to extract space-time features based on the behavior of the ship; performing multi-task synchronous prediction based on the extracted space-time features, wherein the multi-task synchronous prediction comprises ship berthing trajectory prediction and berthing final state prediction; integrating the multi-task synchronous prediction results into a standardized prediction vector, wherein the standardized prediction vector comprises a trajectory prediction sub-vector and a state prediction sub-vector; performing physical rationality correction on the standardized prediction vector based on a ship kinematics model, and outputting a prediction result vector.

2. The method according to claim 1, characterized in that: The plurality of source data sets comprise ship static data, AIS data, port radar data, video monitoring data, meteorological and hydrological data, and historical berthing record data. The step of extracting multi-source features based on the obtained plurality of source data sets and constructing a multi-source feature set comprises the steps of: extracting ship static features based on ship static data, dynamic navigation features based on AIS data, ship spatial features based on port radar data and video monitoring data, environmental features based on meteorological and hydrological data, and berthing features based on historical berthing record data; The extracted features are standardized and a multi-source feature set is constructed.

3. The method according to claim 1, characterized in that: The dynamic navigation features include a real-time position of the ship, and the step of constructing a ship spatio-temporal graph structure based on the dynamic navigation features, the environment features, and the berthing features includes the steps of: taking the real-time position of the ship as a center node, and taking the dynamic navigation features, the environment features, and the berthing features as node feature vectors to construct an initial ship node feature matrix; performing spatial correlation modeling based on a Gaussian kernel function, and calculating spatial relationship weights between the node feature vectors; constructing a ship spatio-temporal graph structure based on the spatial correlation modeling results, wherein edges of the ship spatio-temporal graph structure include spatial edges and temporal edges.

4. The ship berthing behavior prediction and violation early warning method based on multi-source data fusion according to claim 1, characterized in that: The step of obtaining port scheduling information and port safety area information of a target port, and performing risk assessment based on a risk item based on a pre-set risk assessment strategy on the prediction result vector, the port scheduling information, and the port safety area information includes the steps of: performing spatial region comparison on the prediction result vector and the port safety area information; performing time comparison on the prediction result vector and the port scheduling information; performing risk feature extraction and corresponding risk coefficient calculation on the spatial region comparison result and the time comparison result based on the pre-set risk assessment strategy.

5. A ship berthing behavior prediction and violation early warning system based on multi-source data fusion, characterized in that: It includes: a data acquisition module configured to acquire ship information associated with a target port and acquire a multi-source data set associated with the ship information; a feature set construction module configured to extract multi-source features based on the acquired multi-source data set and construct a multi-source feature set; an input module configured to input the multi-source feature set into a pre-trained behavior simulation model, so that the behavior simulation model outputs a prediction result vector simulating berthing behavior of the ship; a comparison module configured to obtain port scheduling information and port safety area information of a target port, and perform risk assessment based on a risk item based on a pre-set risk assessment strategy on the prediction result vector, the port scheduling information, and the port safety area information; a warning module configured to generate a corresponding graded warning signal and send it to a scheduling terminal associated with the target port when a risk probability of any risk item in the risk assessment strategy is greater than a corresponding pre-set threshold value; The behavior simulation model includes a spatio-temporal coding layer, a feature fusion layer, and a behavior simulation layer. The step of inputting the multi-source feature set into the pre-trained behavior simulation model so that the behavior simulation model outputs a prediction result vector simulating berthing behavior of the ship includes: The spatio-temporal coding layer constructs a ship spatio-temporal graph structure based on dynamic navigation features, environment features, and berthing features. The feature fusion layer updates the ship spatio-temporal graph structure based on ship static features and ship spatial features to generate an enhanced spatio-temporal graph structure. The behavior simulation layer simulates the behavior of the ship based on the enhanced spatio-temporal graph structure and outputs the prediction result vector. The feature fusion layer includes an embedding layer. The step of updating the ship spatio-temporal graph structure based on ship static features and ship spatial features to generate an enhanced spatio-temporal graph structure includes: mapping the ship static features into low-dimensional feature vectors through the embedding layer, and concatenating the mapping results with the ship spatio-temporal graph structure. The ship static features are weighted and fused with the ship spatial features to obtain weighted fusion features; The nodes and edges of the ship space-time graph structure are updated based on the weighted fusion features; The step of simulating the ship behavior based on the enhanced space-time graph structure and outputting a prediction result vector, includes: Performing space-time graph convolution operation on the enhanced space-time graph structure to extract space-time features based on the ship behavior; Performing multi-task synchronous prediction based on the extracted space-time features, the multi-task synchronous prediction including ship berthing trajectory prediction and berthing terminal state prediction; Integrating the multi-task synchronous prediction results into a standardized prediction vector, the standardized prediction vector including a trajectory prediction sub-vector and a state prediction sub-vector; Physically correcting the standardized prediction vector based on a ship kinematics model and outputting the same as a prediction result vector.

Images