Tug configuration method based on AIS piloting data

Abstract

The invention discloses a tug configuration method based on AIS piloting data. The tug configuration method comprises the steps of obtaining original AIS data used for tug configuration; performing data analysis on the original AIS data to obtain an initial time sequence ship track; performing trajectory data cleaning processing on the initial time sequence ship trajectory to obtain a ship time sequence continuous trajectory; interpolating the acquired meteorological / ocean multi-source data and fusing the meteorological / ocean multi-source data with a ship time sequence continuous trajectory to obtain a ship trajectory fusion database; constructing a ship space-time matching and speed / course difference judgment mechanism to obtain a historical feature data set of ship-tug configuration according to the ship trajectory fusion database; constructing a tug configuration prediction model of AIS piloting data, and obtaining an optimal tug configuration prediction model based on historical feature data; and setting the optimal tug configuration prediction model in an existing port scheduling system to deploy an online service model for tug configuration so as to realize a tug configuration demand scheme based on AIS piloting data. The problem that the accuracy and the efficiency of tug configuration of AIS piloting data cannot be achieved due to the fact that AIS data is complex and uneven in quality and the temporal-spatial resolution, the coordinate system and the like of meteorological sea condition data are different from those of the AIS at present is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of port shipping big data and artificial intelligence, and particularly relates to a tugboat configuration method based on AIS pilotage data. Background Art

[0002] With the rapid development of the modern shipping industry, the port throughput has been continuously increasing, and the pilotage services required for large ships (such as bulk carriers, oil tankers, container ships) to enter and leave ports have become increasingly complex. In order to perform berthing and unberthing operations safely and efficiently, several tugboats with different powers (horsepower) are usually required to cooperate. On the one hand, the current tugboat configuration schemes often rely on manual experience or simple fixed processes, and it is difficult to adapt to the ever-changing marine environment (wind speed, wind direction, tide, current speed) and ship characteristics (scale, draft, type, etc.) in real time, resulting in overuse or insufficiency of tugboats, affecting the port operation efficiency and even posing potential safety hazards.

[0003] On the other hand, the AIS system has been widely deployed worldwide, making it possible to automatically collect ship positions (latitude and longitude), motion states (SOG, COG), and static information (MMSI, IMO, ship type, etc.). At the same time, with the opening of reanalysis meteorological data (such as ERA5) and ocean current data (such as CMEMS), it has become a feasible means to fuse AIS trajectories with meteorological and ocean conditions information and perform data mining on historical tugboat configuration information. Using these massive multi-source data and combining machine learning models to quantitatively predict tugboat requirements is expected to significantly improve the digital and intelligent levels of ports.

[0004] However, currently, due to the complexity of AIS data, uneven data quality, and differences in spatio-temporal resolution, coordinate systems, etc. between meteorological and ocean conditions data and AIS, it is impossible to achieve the accuracy and efficiency of tugboat configuration based on AIS pilotage data. Summary of the Invention

[0005] The present invention provides a tugboat configuration method based on AIS pilotage data to overcome the above technical problems.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A tugboat configuration method based on AIS pilotage data specifically includes the following steps:

[0008] S1: Obtain the original AIS data for tugboat configuration;

[0009] S2: Parse the original AIS data to obtain the initial time-series ship track;

[0010] S3: Perform trajectory data cleaning on the initial time-series ship track to obtain the continuous ship track in time series;

[0011] S4: Obtain meteorological / ocean multi-source data, interpolate and fuse the meteorological / ocean multi-source data with the continuous ship time-series trajectory to obtain a ship trajectory fusion database for the multi-source data-fused ship time-series continuous trajectory;

[0012] S5: Construct a ship spatio-temporal matching and speed / course difference determination mechanism, and mine the tugboat configuration mining data of the target ship during the pilotage period according to the ship trajectory fusion database;

[0013] And the tugboat configuration mining data at least includes the tugboat quantity category and the tugboat horsepower;

[0014] And obtain a historical feature dataset of ship-tugboat configuration according to the tugboat configuration mining data;

[0015] And the historical features include the target ship attribute features, environmental features and tugboat configuration;

[0016] S6: Construct a tugboat configuration prediction model for AIS pilotage data;

[0017] And the tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model; the historical feature dataset is divided into a training set and a test set by using the cross-validation method;

[0018] And use the target ship attribute features and environmental features as feature inputs, use the tugboat quantity category of the tugboat configuration as the feature output of the random forest classification algorithm model, and use the tugboat horsepower as the feature output of the random forest regression algorithm model;

[0019] Train the tugboat configuration prediction model according to the training set, and optimize the hyperparameters of the tugboat configuration prediction model by using the grid search method to obtain the trained tugboat configuration prediction model;

[0020] Evaluate the trained tugboat configuration prediction model through the test set to obtain the optimal tugboat configuration prediction model;

[0021] S7: Set the optimal tugboat configuration prediction model in the existing port scheduling system to deploy it as an online service model for tugboat configuration, so as to realize the requirement solution for tugboat configuration based on AIS pilotage data.

[0022] Further, the S2 specifically includes the following steps:

[0023] S21: Store the original AIS data into a CSV file;

[0024] S22: Traverse the original AIS data in the CSV file, and batch-parse each AIS message data to obtain AIS parsed data;

[0025] And the method for batch parsing of each AIS message data is as follows:

[0026] Call the AIS message processing library PyAIS to batch parse the AIS message data to obtain AIS dynamic messages and AIS static messages;

[0027] And the AIS dynamic messages at least include longitude and latitude, speed and heading, and the timestamp of the AIS;

[0028] The AIS static messages at least include the ship name, ship type, and ship IMO number;

[0029] Take the AIS dynamic messages and AIS static messages as AIS parsing data;

[0030] S23: Map the AIS parsing data to a predetermined trajectory storage field to obtain the initial ship trajectory;

[0031] And the predetermined trajectory storage field: {MMSI, longitude and latitude, speed, heading, IMO, ship name, timestamp, ship type,...}.

[0032] Furthermore, the trajectory data cleaning process described in S3 specifically includes:

[0033] Abnormal elimination: Eliminate the initial ship trajectories that do not meet the standard digital length of MMSI or whose MMSI is a non-numeric field;

[0034] Data filtering: After performing abnormal elimination, filter out the initial ship trajectories where the longitude and latitude / speed / heading exceed the preset threshold;

[0035] Data thinning: After performing data filtering, when the time interval between adjacent trajectory points of the same ship is less than the given threshold, only keep the previous trajectory point according to the ship trajectory direction;

[0036] Trajectory splitting: After performing data thinning, when the time interval between adjacent trajectory points of the same ship is greater than the preset time threshold or the spatial distance between them exceeds the preset distance threshold, take the trajectory between these adjacent trajectory points as a new segment in the initial ship trajectory;

[0037] Trajectory linear interpolation: After performing trajectory splitting, perform linear interpolation of trajectory points for each ship trajectory to obtain the optimized ship trajectory;

[0038] RDP compression: After performing trajectory linear interpolation, use the slope of the straight line between adjacent trajectory points as the trajectory deviation angle, and retain the trajectory points whose trajectory deviation angle meets the preset deviation distance threshold to constrain the tolerance of ship trajectory points and obtain the ship time-sequential continuous trajectory.

[0039] Furthermore, the specific steps of S4 are as follows:

[0040] S41: Obtain meteorological / oceanic multi-source data stored in NetCDF / HDF5 files;

[0041] And the meteorological / oceanic multi-source data at least includes wind field data, flow field data, and tidal data under set time data;

[0042] S42: Perform dimensional correction on the obtained meteorological / oceanic multi-source data to obtain corrected data;

[0043] S43: Divide the longitude and latitude grid of the target sea condition area to obtain the longitude and latitude grid of multi-source data;

[0044] Based on the longitude and latitude grid of multi-source data, obtain the grid data of each longitude and latitude grid point according to the corrected data, and the grid data includes wind field speed / direction, flow field speed / direction, and tidal speed / direction under the corresponding timestamp;

[0045] S44: Obtain the longitude and latitude of a given trajectory point in the sequential continuous trajectory of the ship, and based on the nearest neighbor interpolation method, obtain the grid data of the longitude and latitude grid point with the closest longitude and latitude distance to the given trajectory point according to the longitude and latitude grid of multi-source data, and assign the corresponding grid data to the given trajectory point;

[0046] To fuse the meteorological / oceanic multi-source data into the sequential continuous trajectory of the ship to obtain a ship trajectory fusion database.

[0047] Further, in S5, the mechanism for constructing ship spatio-temporal matching and speed / course difference determination is described, and the tugboat configuration mining data of the target ship during the pilotage period is mined according to the ship trajectory fusion database, which specifically includes the following steps:

[0048] S51: Based on the ship pilotage period determination rule, confirm the start time and end time of the ship's pilotage to obtain the pilotage period of the target ship;

[0049] And the ship pilotage period determination rule:

[0050] Obtain the track point speed of the sequential continuous trajectory of the ship in the ship trajectory fusion database;

[0051] When the track point speed meets the first preset speed threshold, confirm this moment as the start time of pilotage;

[0052] When the track point speed meets the second preset speed threshold, confirm this moment as the end time of pilotage;

[0053] S52: Preset several tugboats, and encode each tugboat to obtain a set of encoded tugboats;

[0054] Traverse each tugboat in the coded tugboat set, and based on the constructed ship spatio-temporal matching and speed / heading difference determination mechanism, obtain the matching assisting tugboats according to the pilotage period of the target ship;

[0055] And the ship spatio-temporal matching and speed / heading difference determination mechanism includes a ship spatio-temporal matching rule and a speed / heading difference determination rule;

[0056] The ship spatio-temporal matching rule: Combine the tugboats whose time periods overlap with the pilotage period of the target ship or meet the preset time threshold, and the shortest distance between the real-time obtained tugboat position and the target ship's trajectory segment, and combine the tugboats corresponding to the shortest distance meeting the preset shortest distance threshold as the potential cooperative tugboats of the target ship;

[0057] The speed / heading difference determination rule: Obtain the deviation value between the speed / heading of the potential cooperative tugboat and the speed / heading of the target ship at the corresponding moment;

[0058] And confirm the potential cooperative tugboat corresponding to the deviation value meeting the preset deviation threshold as the matching assisting tugboat for finally assisting the target ship's pilotage, and then obtain the tugboat configuration mining data of the target ship during the pilotage period according to the matching assisting tugboat.

[0059] Further, step S6 specifically includes the following steps:

[0060] S61: Construct a tugboat configuration prediction model for AIS pilotage data;

[0061] The tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model;

[0062] S62: Use the cross-validation method to divide the historical feature dataset into K subsets;

[0063] And randomly use any one of the K subsets as the test set, and use the remaining subsets as the training set;

[0064] S63: Use the target ship attribute features and environmental features as feature inputs, use the tugboat quantity category of the tugboat configuration as the feature output of the random forest classification algorithm model, and use the tugboat horsepower as the feature output of the random forest regression algorithm model;

[0065] S64: Train the tugboat configuration prediction model according to the training set, and use the grid search method to optimize the hyperparameters of the tugboat configuration prediction model to obtain the trained tugboat configuration prediction model;

[0066] S65: Evaluate the trained tugboat configuration prediction model through the test set to obtain the optimal tugboat configuration prediction model;

[0067] And the metrics for model evaluation include classification accuracy, confusion matrix, and regression error RMSE;

[0068] A method for evaluating a trained tugboat configuration prediction model:

[0069] Obtain the current feature error between the predicted feature output and the true feature of the trained tugboat configuration prediction model, and determine whether the current feature error satisfies the preset index threshold;

[0070] If the current feature errors all satisfy the preset index threshold, confirm that the trained tugboat configuration prediction model at this time is the optimal tugboat configuration prediction model;

[0071] Otherwise, use the cross-validation method to re-partition the historical feature dataset into a training set and a test set, and repeat steps S63 to S64.

[0072] The present invention provides a tugboat configuration method based on AIS pilotage data, and the beneficial effects are as follows:

[0073] (1) Automation and intelligence: The whole process consists of data parsing, cleaning, interpolation, mining, and machine learning models, which can operate efficiently under large-scale AIS data, automatically obtain tugboat configuration suggestions, reduce the over-reliance on manual experience, and greatly improve the accuracy and efficiency of tugboat configuration based on AIS pilotage data in the current situation where AIS data is complex, of uneven quality, and there are differences in the spatio-temporal resolution, coordinate system, etc. between meteorological and sea condition data and AIS;

[0074] (2) Improvement of safety and efficiency: Use meteorological and sea condition information for more refined tugboat demand assessment, so that the pilotage operation can improve the ship berthing and unberthing efficiency on the premise of greater safety and achieve the optimal scheduling of tugboat resources.

[0075] (3) Strong versatility: The present invention does not depend on a specific port or specific AIS / meteorological data source, and can be implemented as long as the corresponding grid meteorological data and ship AIS data are obtained, and it supports further expansion to other features or more machine learning algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0077] Figure 1 It is a flowchart of the tugboat configuration method based on AIS pilotage data of the present invention;

[0078] Figure 2 This is the core block diagram of the tugboat configuration method based on AIS pilotage data in this embodiment;

[0079] Figure 3 This is the schematic diagram of the initial sequential ship track obtained in this embodiment;

[0080] Figure 4 This is the schematic diagram of the tugboat track matched with the target ship in this embodiment;

[0081] Figure 5 This is the schematic diagram of the tugboat track that has not matched the target ship in this embodiment;

[0082] Figure 6 This is the program diagram of the tugboat configuration predicted by the tugboat configuration method based on AIS pilotage data in this embodiment;

[0083] Figure 7 This is the flowchart of data parsing for the original AIS data in this embodiment;

[0084] Figure 8 This is the flowchart of trajectory data cleaning for the initial sequential ship track in this embodiment;

[0085] Figure 9 This is the flowchart of interpolating and fusing meteorological / ocean multi-source data with the sequential continuous track of the ship in this embodiment;

[0086] Figure 10 This is the flowchart of obtaining the historical feature dataset of ship-tugboat configuration in this embodiment;

[0087] Figure 11 This is the flowchart of obtaining the optimal tugboat configuration prediction model to achieve online service recommendation for tugboat configuration in this embodiment. Detailed implementation manners

[0088] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0089] This embodiment provides a tugboat configuration method based on AIS pilotage data, as Figures 1 to 2 shown, which specifically includes the following steps:

[0090] S1: Obtain the original AIS data for tugboat configuration;

[0091] S2: Parse the original AIS data to obtain the initial time-series ship track;

[0092] In this embodiment, by parsing the AIS original data of the Automatic Identification System, accurate and complete ship basic information is obtained. The specific implementation methods include extracting dynamic and static fields from NMEA or CSV format data, using the MMSI prefix to map the flag state of the ship, and integrating ship information using a static cache mechanism, etc. Finally, structured AIS data for subsequent processing is generated;

[0093] As Figure 7 shown, it specifically includes the following steps:

[0094] S21: Store the original AIS data into a CSV file;

[0095] S22: Traverse the original AIS data in the CSV file, and batch parse each AIS message data to obtain AIS parse data; in this embodiment, the AIS original message data generally presents in NMEA format, and the typical start string is!AIVDM or!AIVDO, as shown in Table 1;

[0096] Table 1. AIS original message NMEA format table

[0097]

[0098]

[0099] And the method for batch parsing each AIS message data is:

[0100] Call the AIS message processing library PyAIS to batch parse the AIS message data to obtain AIS dynamic messages and AIS static messages; in this embodiment, the decode tool in the AIS message processing library PyAIS is used to decode the NMEA statements of the AIS message data, and the message type can be automatically identified (for example, 1 / 2 / 3 are AIS dynamic information, 5 / 24 are AIS static information);

[0101] And the AIS dynamic messages at least include AIS information such as latitude and longitude, speed and heading, and the timestamp of the AIS that changes frequently; the AIS static messages at least include AIS information such as ship name, ship type, and ship IMO number that changes infrequently;

[0102] Regard the AIS dynamic messages and AIS static messages as AIS parse data;

[0103] In addition, in order to avoid repeated parsing or loss of fields of AIS static messages in AIS dynamic messages in this embodiment, a static_cache dictionary is set during parsing: when an AIS static message is received, values such as shipname (vessel name), callsign (call sign), imo (vessel IMO number), vesseltype (vessel type), etc. will be stored in the cache, so that subsequent AIS dynamic messages can be automatically supplemented when missing, thereby improving data integrity;

[0104] S23: Map the AIS parsed data to the predetermined trajectory storage fields to obtain the initial vessel track;

[0105] And the predetermined trajectory storage fields: {MMSI, longitude and latitude, speed, course, IMO, vessel name, timestamp, vessel type,...}. In this embodiment, AIS data is often stored in daily / hourly sub-files. For efficient processing, this embodiment specifies the input path through command-line parameters, and can batch parse all CSV files in all sub-folders at one time. After parsing, if the "merge" mode is enabled, all records can be summarized and output to a large CSV file, otherwise they will be output separately by individual files; the specific mapping of the MMSI prefix to the flag country is as follows: according to the relationship between the first 3 digits of the MMSI (Maritime Mobile Service Identity of the vessel) and the country abbreviation, the flag_ctry of the vessel can be quickly matched through the MMSI_COUNTRY_PREFIX dictionary, and its expression is: flag_ctry = MMSI_COUNTRY_PREFIX(MMSI0:3). When the MMSI is successfully matched, the vessel flag country information can be recorded for assisting subsequent data mining or country statistics;

[0106] In this embodiment, the main implementation code for batch parsing of each AIS message data to obtain AIS parsed data is as follows: # Omitted part of import and variable declarations

[0107]

[0108]

[0109] main function: Provide a batch processing entry, configure the logging system, traverse the directory and parse all CSV files, and finally merge and output the results as needed. Table 2 shows the AIS information format after parsing:

[0110] Table 2. AIS Parsed Data

[0111]

[0112]

[0113]

[0114]

[0115] In this embodiment, through the precise parsing and intelligent caching of AIS raw data, the pre-workload required for subsequent processes such as track data cleaning, meteorological interpolation, and tugboat mining is greatly simplified, laying a foundation for the application of port pilotage big data. This embodiment uses static_cache to merge information of different types of AIS messages, greatly improving the integrity and accuracy of the data. At the same time, it supports automatic parsing of large-scale data and single parsing of a small number of test messages, with high adaptability and easy connection to subsequent data processing stages;

[0116] S3: Perform track data cleaning on the initial sequential ship track to obtain the sequential continuous track of the ship;

[0117] After this embodiment completes the parsing of AIS raw data, it is necessary to perform track data cleaning on the obtained initial sequential ship track to ensure the quality and coherence of the data, so as to reduce the amount of redundant data while retaining the core information of the track as much as possible, providing reliable data support for subsequent ship pilotage analysis or tugboat configuration decision-making;

[0118] In a specific embodiment, as Figure 8 shown, the track data cleaning specifically includes:

[0119] Abnormal elimination: Eliminate the initial ship tracks that do not meet the standard digital length of MMSI or whose MMSI is a non-digital field; specifically, in this embodiment, MMSI is usually 9 digits. If the MMSI has less than 9 digits or is a non-digital field, it is regarded as abnormal and eliminated;

[0120] Data filtering: After performing abnormal elimination, filter out the initial ship tracks whose longitude / latitude / speed / heading exceed the preset threshold; for the track data generated by the parsing of AIS raw data in this embodiment, abnormal marks are made and eliminated in cases such as "longitude / latitude out of bounds, speed / heading exceeding the limit, or the number of track points in the initial sequential ship track is insufficient"; corresponding judgment thresholds are set for longitude / latitude out of bounds, speed / heading exceeding the limit, and the number of track points insufficient, and then data filtering operations are implemented;

[0121] Data thinning: After performing data filtering, when the time interval between adjacent track points of the same ship is less than the given threshold, only retain the previous track point according to the ship track direction; preferably, the given threshold is 1 minute;

[0122] Trajectory splitting: After performing data thinning, for the time interval between adjacent trajectory points of the same ship being greater than the preset time threshold or being spatially separated by more than the preset distance threshold, the trajectory between these adjacent trajectory points is taken as a new segment in the initial ship track; preferably, the preset time threshold is 3 hours;

[0123] Trajectory linear interpolation: After performing trajectory splitting, linear interpolation of trajectory points is performed on each ship track to obtain an optimized ship track; specifically, in this embodiment, to ensure the smoothness and integrity of the ship trajectory, linear interpolation is performed between trajectory points with a time difference within a certain range (such as 10 minutes) between adjacent trajectory points,

[0124] and the expression for performing linear interpolation is

[0125]

[0126] In the formula: X interp represents the data of the trajectory point to be interpolated, which includes longitude and latitude or motion parameters, namely speed, course, or heading; X i represents the AIS parsing data at the i-th trajectory point; X i+1 represents the AIS parsing data at the (i + 1)-th trajectory point; α represents the interpolation time ratio coefficient, and when α satisfies the preset time coefficient threshold, interpolation processing is performed; otherwise, that is, when the set "long time gap" is exceeded, interpolation is skipped; t interp represents the timestamp of the trajectory point to be interpolated; t i represents the timestamp of the i-th trajectory point; t i+1 represents the timestamp of the (i + 1)-th trajectory point;

[0127] RDP compression: After performing trajectory linear interpolation, the slope of the straight line between adjacent trajectory points is used as the trajectory deviation angle, and the trajectory points whose trajectory deviation angles satisfy the preset deviation distance threshold are retained to constrain the tolerance of ship trajectory points and obtain a temporally continuous ship trajectory.

[0128] In this embodiment, the RDP (Ramer-Douglas-Peucker) algorithm is an algorithm for simplifying a set of points of a polyline or polygon. It can simplify the geometric shape by reducing the number of points while trying to maintain its overall shape. This algorithm is commonly used in fields such as geographic information systems (GIS), computer graphics, and data compression. In this embodiment, the specific compression of the trajectory points in the ship trajectory is

[0129] S001: Given a set of trajectory points P = {p1, p2, …, p i , …, p n} of a ship trajectory and the preset deviation distance threshold ε, select the trajectory point p1 and the trajectory point p n as the initial key trajectory points;

[0130] S002: Obtain all intermediate points p i to the perpendicular distance of the line segment p1p n and determine the trajectory point p with the maximum perpendicular distance max and the corresponding maximum perpendicular distance d max ;

[0131] S003: Recursively simplify the trajectory points of the ship trajectory:

[0132] That is, judge the relationship between the maximum perpendicular distance d max and the preset offset distance threshold ε;

[0133] If the maximum perpendicular distance d max is greater than the preset offset distance threshold ε, then use the trajectory point p max as the new key trajectory point, and obtain the line segment p1p max and the line segment p max p n ;

[0134] And repeatedly execute step S002 to recursively simplify the trajectory points from trajectory point p1 to trajectory point p max and the trajectory points between trajectory point p max and trajectory point p n ;

[0135] If the maximum perpendicular distance d max is less than or equal to the preset offset distance threshold ε, then discard all intermediate points p i and only retain trajectory point p1 and trajectory point p n , thereby realizing the compression of the trajectory points in the ship trajectory to obtain the RDP-compressed ship trajectory;

[0136] The trajectory data cleaning process of this embodiment also includes trajectory visualization: By calling the folium map drawing tool, draw a trajectory line on the HTML map for the RDP-compressed ship trajectory, and can selectively display or hide interpolation points, which is convenient for manual review and verification of the processing quality;

[0137] The main implementation code for cleaning the trajectory data of the initial time-series ship track in this embodiment is as follows: class TrajectoryProcessor:

[0138] @staticmethod

[0139] def haversine_nm(lat1, lon1, lat2, lon2):

[0140] """

[0141] Calculate the spherical distance (unit: nautical miles, NM). Use the Haversine formula with the Earth's radius of 3440.065 NM.

[0142] """

[0143] # Code omitted...

[0144] returnd_nm

[0145] def clean_data(self, df):

[0146] """

[0147] Remove anomalies in MMSI, out-of-bounds latitude and longitude, over-limit speed and course, etc.; Remove vessels with too few trajectory points.

[0148] """

[0149] # 1) MMSI length 9-digit check

[0150] # 2) lat / lon range

[0151] # 3) sog / cog range

[0152] # 4) Trajectory points < min_trajectory_points are removed

[0153] # 5) fill_missing_values for forward / backward filling

[0154] returndf

[0155] def thin_data(self, df):

[0156] """

[0157] Thinning: If the time difference between adjacent points < g_threshold, skip the latter.

[0158] """

[0159] # 1) Sort by mmsi and time

[0160] # 2) Calculate the time difference from the previous record

[0161] # 3) Delete short-interval points

[0162] returndf_thinned

[0163] def split_trajectories(self, df):

[0164] """

[0165] Trajectory splitting: If the time difference > split_threshold or the spatial difference > distance_split_threshold, a new segment is created.

[0166] """

[0167] #1) Record time_diff_prev

[0168] #2) Mark'split' = True for large intervals

[0169] #3) Accumulate to generate traj_segment number

[0170] return df

[0171] def interpolate_data(self, df):

[0172] """

[0173] Linear interpolation: Interpolate at minute intervals between points where the adjacent time difference <= large_gap_threshold.

[0174] """

[0175] #1) Group by (mmsi, traj_segment)

[0176] #2) Perform linear interpolation for each pair of adjacent points

[0177] #3) Mark is_interpolated = True

[0178] return df_interp

[0179] def compress_trajectories(self, df):

[0180] """

[0181] Use LineString.simplify(epsilon) of shapely to compress trajectories based on RDP.

[0182] """

[0183] #1) Process each (mmsi, traj_segment) independently

[0184] #2) Extract coordinates -> LineString ->.simplify()

[0185] #3) Use cKDTree to match and retain point indices

[0186] returndf_compressed

[0187] def process_file(self, file_path):

[0188] """

[0189] Main process: load_data -> filter_data -> clean_data -> thin_data ->

[0190] split_trajectories -> interpolate_data -> compress_trajectories -> remove_small_trajectories

[0191] """

[0192] #1) Read CSV data (only mmsi, postime, lon, lat, sog, cog, hdg)

[0193] #2) Clean -> thin -> split -> interpolate -> RDP compression -> remove small segments

[0194] returndf

[0195] def generate_map(self, df, output_path):

[0196] """

[0197] Visualize the trajectories using folium, supporting the option to show or hide interpolation points.

[0198] """

[0199] #1) Build the Map

[0200] #2) Draw the trajectory polyline for each (mmsi, seg) in sequence

[0201] #3) Save to html

[0202] def run(self):

[0203] """

[0204] Execute process_file on all CSV files in the input directory, and generate maps and processed CSVs.

[0205] """

[0206] #1) Traverse all files

[0207] #2) Aggregate abnormal data and output anomalies_collected.csv

[0208] logging.info("All files have been processed.")

[0209] In this embodiment, through anomaly rejection and data filtering, noise and outliers can be effectively removed to obtain a more coherent ship trajectory in terms of time series and spatial distribution, significantly improving the data quality; through thinning and RDP compression, the number of points can be greatly reduced, which is extremely valuable in the massive AIS scenario, and the segmented ship trajectory is more in line with the actual operation scenario, making the subsequent pilotage analysis or tugboat matching more accurate; in addition, by generating a map file with trajectory lines (.html), it is convenient for manual inspection to timely detect any data anomalies and ensure the transparency and controllability of the processing process. For example, Figure 3 the example diagram shown is the port pilotage trajectory map for a certain day after data processing;

[0210] S4: Obtain meteorological / oceanic multi-source data, interpolate and fuse the meteorological / oceanic multi-source data with the continuous ship time series trajectory to obtain a ship trajectory fusion database for the multi-source data fusion continuous ship time series trajectory, as Figure 9 shown, which specifically includes the following steps:

[0211] S41: Obtain the meteorological / oceanic multi-source data stored in NetCDF / HDF5 files;

[0212] And the meteorological / oceanic multi-source data at least includes wind field data, flow field data, and tidal data under the set time data;

[0213] S42: Perform dimension correction on the obtained meteorological / oceanic multi-source data to obtain corrected data;

[0214] Common variables of wind field data in this embodiment: u10 (east-west wind speed), v10 (north-south wind speed). The corresponding data sets are accessed using the Python h5py library, and the possible high-dimensional data is reduced in dimension (such as (time, 1, lat, lon) → (time, lat, lon)), where time represents the sampling time; lat represents longitude; lon represents latitude. In this embodiment, the time data time in the NetCDF / HDF5 file is usually the number of seconds since a certain preset reference time (such as starting to count seconds from "1970-01-01" or "1993-01-01"). After reading the meteorological / oceanic multi-source data in this embodiment, a pandas.Timestamp array corresponding to AISpostime is obtained by adding the corresponding reference time through pd.to_timedelta;

[0215] Common variables of flow field data: uo (east-west velocity), vo (north-south velocity), which may be four-dimensional (with an additional dimension of depth). In this embodiment, after taking the surface layer (depth = 0), the redundant dimension is removed, that is, the four-dimensional (time, depth, lat, lon) is converted to three-dimensional (time, lat, lon);

[0216] Tidal data, that is, tide / sea level height: The total sea level total_sea_level is obtained from ocean_tide + sea_surface_height, and it is also obtained based on the three-dimensional (time, lat, lon) data structure;

[0217] Among them, the method for dimension correction of meteorological / oceanographic multi-source data is a well-known existing technology and will not be elaborated here;

[0218] S43: Divide the longitude and latitude grid of the target sea condition area to obtain the longitude and latitude grid of multi-source data;

[0219] Based on the longitude and latitude grid of multi-source data, grid data of each longitude and latitude grid point is obtained according to the corrected data, and the grid data includes wind field velocity / direction, flow field velocity / direction, and tidal velocity / direction at the corresponding timestamp; The method of longitude and latitude grid division is a well-known existing technical means and will not be elaborated here;

[0220] S44: Obtain the longitude and latitude of a given trajectory point in the sequential continuous trajectory of the ship, and based on the nearest neighbor interpolation method, according to the longitude and latitude grid of multi-source data, obtain the grid data of the longitude and latitude grid point with the closest longitude and latitude distance to the given trajectory point;

[0221] Among them, the nearest neighbor interpolation method is: Given a ship trajectory point (t s , lon_s, lat_s), find the longitude and latitude grid point with the closest longitude and latitude distance to the given trajectory point in the longitude and latitude grid of multi-source data, and the expression is

[0222]

[0223] In the formula: lon[i] represents the longitude of the i-th longitude and latitude grid point; lon[j] represents the latitude of the j-th longitude and latitude grid point; time[k] represents the timestamp of the corresponding longitude and latitude grid point; lon_s represents the longitude of the given ship trajectory point; lat_s represents the latitude of the given ship trajectory point; t s represents the timestamp of the given ship trajectory point;

[0224] Assign the wind speed data / flow velocity data / tidal data at the (i, j, k) longitude-latitude grid point position to the corresponding grid data and assign it to the given trajectory point, so as to integrate meteorological / ocean multi-source data into the continuous time series trajectory of the ship and obtain the ship trajectory fusion database.

[0225] In this embodiment, by reading the ship trajectory CSV file one by one and saving the results of interpolating the wind field data, flow field data, and tidal data as a new ship trajectory CSV file, new wind field data, namely wind speed wind_speed and wind direction wind_dir; flow field data, namely flow velocity flow_speed and flow direction flow_dir, and tidal data total_sea_level are added to the fields of the predetermined trajectory storage field, which helps to simultaneously consider the characteristics of the ship's own movement and external environmental interference in the subsequent tugboat configuration analysis;

[0226] The main implementation code for interpolating and integrating the meteorological / ocean multi-source data into the continuous time series trajectory of the ship in this embodiment is as follows:

[0227] The file loading, parsing, and interpolation are implemented through the following Python function:

[0228] #1) Read the wind file (ERA5)

[0229] def load_wind_data_with_h5py(wind_file):

[0230] # Open with h5py.File(wind_file, 'r')

[0231] # Read the u10, v10 arrays and squeeze()

[0232] # Parse time_raw -> parsed_time

[0233] return (u10_arr, v10_arr, lon, lat, parsed_time)

[0234] #2) Read the current file (CMEMS flow field)

[0235] def load_current_data_with_h5py(current_file):

[0236] # Read uo, vo; if 4D, take depth = 0

[0237] # Parse time_raw -> parsed_time

[0238] return(uo_arr,vo_arr,lon,lat,parsed_time)

[0239] #3) Read the tide file (tide_file)

[0240] def load_sea_level_data_with_h5py(tide_file):

[0241] #ocean_tide + sea_surface_height = total_sea_level

[0242] # Similarly, perform squeeze() and parse time

[0243] return(total_sea_level,lon,lat,parsed_time)

[0244] After completion of loading, perform the following interpolation operations on each trajectory CSV, example:

[0245] def interpolate_wind(ship_data,wind_data):

[0246] # Unpack wind_data

[0247] u10_arr,v10_arr,wind_lon,wind_lat,wind_time = wind_data

[0248] # Calculate wind_speed_arr,wind_dir_arr

[0249] # Find the nearest indices (time_idx,lat_idx,lon_idx) and assign values

[0250] ship_data['wind_speed'] =...

[0251] ship_data['wind_dir'] =...

[0252] return ship_data

[0253] Similarly, interpolate_current and interpolate_tide perform the same logic on the flow field and tide data respectively;

[0254] def main():

[0255] #1) Specify wind_file, current_file, tide_file

[0256] #2) Load wind_data = load_wind_data_with_h5py(...)

[0257] #current_data =...

[0258] #tide_data =...

[0259] #3) Traverse the CSV files in the input_folder

[0260] # Read ship_data

[0261] #interpolate_wind(ship_data,wind_data)

[0262] #interpolate_current(ship_data,current_data)

[0263] #interpolate_tide(ship_data,tide_data)

[0264] # Write back to the CSV file

[0265] In this embodiment, by interpolating meteorological / ocean multi-source data for each trajectory point of the ship trajectory, a complete data file containing fields such as wind_speed, wind_dir, flow_speed, flow_dir, total_sea_level, etc. is obtained. Table 3 shows an example after inserting meteorological / ocean multi-source data:

[0266] Table 3. Ship Trajectory Fusion Database

[0267]

[0268]

[0269]

[0270] In this embodiment, by simultaneously combining wind field (ERA5), flow field (CMEMS), and tidal sea level height data, subsequent pilotage operations and tugboat demand analysis can consider real and complex sea conditions. If a more refined effect is required, the nearest neighbor interpolation method can be changed to the bilinear interpolation method or the three-dimensional spatial interpolation method. This embodiment can batch traverse the target folder and interpolate and update a large number of CSV trajectories at one time, significantly reducing the manual operation cost and improving the processing efficiency. By inserting wind field, flow field, and tidal information into the cleaned track data, this embodiment constructs a complete "ship movement - environmental state" fusion database, providing more accurate and real-time environmental information support for subsequent tugboat configuration models or other maritime applications (such as route planning, risk warning).

[0271] S5: Construct a ship spatio-temporal matching and speed / heading difference determination mechanism, and mine the tugboat configuration mining data of the target ship during the pilotage period based on the ship trajectory fusion database;

[0272] And the tugboat configuration at least includes the tugboat quantity category and the tugboat horsepower;

[0273] And obtain the historical feature dataset of ship-tugboat configuration according to the tugboat configuration mining data;

[0274] And the historical features include the target ship attribute features, environmental features, and tugboat configuration;

[0275] After this embodiment completes the cleaning of ship trajectories and the interpolation and fusion of multi-source meteorological / ocean data, it further mines the spatio-temporal relationship between the target ship and the tugboat during the pilotage process, so as to identify the number and specific information (such as tugboat MMSI, horsepower, etc.) of the tugboats assisting the target ship in actual pilotage operations. This step can be regarded as the key connection from "qualified clean track data" to "tugboat configuration history record", which is helpful for subsequent model training or statistical analysis, as Figure 4 shown;

[0276] such as Figure 10 shown, specifically including the following steps:

[0277] S51: Based on the ship pilotage period determination rule, confirm the start time and end time of the ship's pilotage to obtain the target ship's pilotage period;

[0278] And the ship pilotage period determination rule:

[0279] Obtain the track point speed of the ship's time-series continuous track in the ship trajectory fusion database;

[0280] When the track point speed meets the first preset speed threshold, confirm this moment as the start time of the pilotage;

[0281] When the speed of the trajectory point satisfies the second preset speed threshold, confirm this moment as the pilotage end time;

[0282] S52: Preset a number of tugboats, and encode each tugboat to obtain a set of encoded tugboats;

[0283] Traverse each tugboat in the set of encoded tugboats, and based on the constructed ship spatio-temporal matching and speed / course difference determination mechanism, obtain the matching assisting tugboats according to the pilotage period of the target ship;

[0284] And the ship spatio-temporal matching and speed / course difference determination mechanism includes a ship spatio-temporal matching rule and a speed / course difference determination rule;

[0285] The ship spatio-temporal matching rule: Use the tugboats that overlap with the pilotage period of the target ship or meet the preset time threshold as the potential cooperative tugboats of the target ship;

[0286] Obtain the shortest distance between the tugboat position and the target ship's trajectory line segment in real time, and use the tugboat corresponding to the shortest distance meeting the preset shortest distance threshold as the potential cooperative tugboat of the target ship;

[0287] And the formula for obtaining the distance between the tugboat position and the target ship's trajectory line segment is

[0288]

[0289] In the formula: d nm represents the distance between the tugboat position and the target ship's trajectory line segment; R nm represents the radius of the earth and R nm ≈ 3440.065 nautical miles; and Δλ respectively represent the latitude and longitude radian differences between the tugboat position point and the trajectory point of the target ship's trajectory; represents the tugboat position point; represents the trajectory point of the target ship's trajectory;

[0290] Among them, in this embodiment, the preset shortest distance threshold is preferably 250 meters or 0.2 nautical miles;

[0291] The speed / course difference determination rule: Obtain the deviation value between the speed / course of the potential cooperative tugboat and the speed / course of the target ship at the corresponding moment;

[0292] And use the potential cooperative tugboat corresponding to the deviation value meeting the preset deviation threshold as the matching assisting tugboat for finally cooperating with the target ship's pilotage, and then obtain the tugboat configuration mining data of the target ship during the pilotage period according to the matching assisting tugboat;

[0293] Among them, in this embodiment, the preset deviation threshold preferably has a speed difference within 6 knots and a course difference within 180°. This embodiment also includes performing the same segmented matching on the "unmatched" tugboats again to avoid missed detection due to improper data sampling or threshold setting. In the map visualization, the running tracks of the "unmatched" tugboats are drawn in a unified color (such as black) as Figure 5 shown, which is convenient for manual verification. Finally, a table is generated by summarizing fields such as "target ship - MMSI, assisting tugboat list, pilotage period, average wind and current speed, tide level", and all processed file results can be merged into a single summary table, that is, the ship - tugboat configuration table, and it is used as the tugboat configuration mining data; as shown in Table 4;

[0294] Table 4. Tugboat Configuration Mining Data Table

[0295]

[0296]

[0297]

[0298] The main implementation code for obtaining the tugboat configuration mining data in this embodiment is as follows:

[0299] #(A) Basic functions

[0300] def haversine_nm(lat1, lon1, lat2, lon2):

[0301] # Calculate the distance in nautical miles

[0302] return d

[0303] def angle_diff(a, b):

[0304] # Return the minimum difference between two angles (considering 360 degrees)

[0305] return diff

[0306] def remove_outliers_by_speed(df, max_speed_knots = 25.0):

[0307] """

[0308] Remove outliers based on the speed threshold of adjacent points

[0309] """

[0310] #1) Group by mmsi

[0311] #2) Calculate the speed of adjacent points in groups, and determine as abnormal if it exceeds max_speed_knots

[0312] returndf_cleaned

[0313] #(B) Pilot time identification

[0314] def get_pilot_time_window(df, mmsi, start_sog = 1.0, end_sog = 0.1, end_buffer_minutes = 10.0):

[0315] """

[0316] Identify the start and end times based on the SOG threshold

[0317] """

[0318] #1) Find this mmsi from df

[0319] #2) SOG > start_sog => pilot_start_time

[0320] #3) SOG < end_sog => pilot_end_time + buffer

[0321] return (start_time, end_time)

[0322] #(C) Tugboat matching

[0323] def filter_tugboats_projection(...):

[0324] """

[0325] Match tugboats under multiple constraints such as given distance, speed, and heading

[0326] """

[0327] #1) Resample the target ship's trajectory within the specified time period

[0328] #2) Also resample the tugboat data according to resample_interval

[0329] #3) Take seg_a and seg_b point by point (adjacent points) to construct a bounding box (add bbox_margin_deg)

[0330] #4) Calculate the shortest line segment distance from each point in sub_tug to seg_a and seg_b

[0331] #5) Speed / Heading Difference Judgment

[0332] return matched_tugs

[0333] #(D) Visualization: plot_piloted_and_tugs()

[0334] # Use Plotly to plot the trajectories of the target ship and tugs, and support the drawing of segmented matching for "unmatched tugs" as well

[0335] #(E) Main Process: main()

[0336] #1) Batch read CSV, first do remove_outliers_by_speed

[0337] #2) Smooth processing (smooth_trajectory)

[0338] #3) Filter large ships (such as ship_length > 50m)

[0339] #4) Get the pilot time window according to the SOG threshold

[0340] #5) Call filter_tugboats_projection to find matching tugs

[0341] #6) Construct the result table and visualize

[0342] #7) Finally, merge and output.

[0343] Among them, plot_piloted_and_tugs() uses plotly.graph_objects or plotly.express for scatter or line drawing, and can display the trajectories of the target ship and multiple tugs on the map background (Mapbox) at the same time, and can support interactive mouse hovering to display detailed information;

[0344] This embodiment can automatically identify the actual number of tugs participating in each pilotage task for entering and leaving the port and the specific target ships based on a large amount of AIS track and meteorological information, so as to systematically construct a "historical feature dataset of ship-tug configuration", provide real and reliable training samples for the subsequent "tug configuration model", and can also be directly used for tug dispatching statistics or efficiency evaluation;

[0345] S6: Construct a tug configuration prediction model for AIS pilotage data;

[0346] The tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model; the historical feature dataset is divided into a training set and a test set using the cross-validation method;

[0347] The target ship attributes and environmental features are used as feature inputs, the number category of tugboats in the tugboat configuration is used as the feature output of the random forest classification algorithm model, and the tugboat horsepower is used as the feature output of the random forest regression algorithm model;

[0348] The tugboat configuration prediction model is trained according to the training set, and the hyperparameters of the tugboat configuration prediction model are optimized using the grid search method to obtain the trained tugboat configuration prediction model;

[0349] The trained tugboat configuration prediction model is evaluated using the test set to obtain the optimal tugboat configuration prediction model;

[0350] In this embodiment, after completing the fusion of ship trajectory and environmental data and the mining of historical tugboat assistance relationships, based on the large-scale sample data constructed from "target ship features, environmental features, and tugboat configuration results", a machine learning model (such as random forest) is used to classify and regressively predict the tugboat demand: the former outputs the number of tugboats required (tug_count), the latter estimates the total horsepower of the corresponding tugboats (total_horsepower), and through the grid search and cross-validation methods, the optimal model hyperparameters can be found according to the historical feature dataset, and intelligent tugboat configuration suggestions can be quickly given in the actual pilotage scenario; in addition, the machine learning model in this embodiment also includes but is not limited to any one of XGBoost, LightGBM, and deep learning models, etc.;

[0351] As Figure 11 shown, it specifically includes the following steps:

[0352] S61: Construct a tugboat configuration prediction model for AIS pilotage data;

[0353] The tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model;

[0354] S62: Divide the historical feature dataset into K subsets using the cross-validation method;

[0355] And randomly take any one of the K subsets as the test set, and the remaining subsets as the training set;

[0356] This embodiment also includes numerical normalization processing of the historical feature dataset, and the numerical normalization processing is a well-known technical means in the art and will not be elaborated here;

[0357] S63: Take the target ship attribute features and environmental features as feature inputs, take the number category of tugs configured by the tug as the feature output of the random forest classification algorithm model, and take the horsepower of the tug as the feature output of the random forest regression algorithm model;

[0358] Among them, the target ship attribute features at least include the length, width, draft of the target ship, and the ship type;

[0359] S64: Train the tug configuration prediction model according to the training set, and use the grid search method to optimize the hyperparameters of the tug configuration prediction model to obtain the trained tug configuration prediction model;

[0360] S65: Evaluate the trained tug configuration prediction model through the test set to obtain the optimal tug configuration prediction model;

[0361] And the indicators of the model evaluation include classification accuracy, confusion matrix, and regression error RMSE;

[0362] Method for evaluating the trained tug configuration prediction model:

[0363] Obtain the current feature error between the predicted feature output and the true feature of the trained tug configuration prediction model, and judge whether the current feature error meets the preset index threshold;

[0364] If the current feature error meets the preset index threshold, confirm that the trained tug configuration prediction model at this time is the optimal tug configuration prediction model;

[0365] Otherwise, use the cross-validation method to re-divide the historical feature dataset into a training set and a test set, and repeat steps S63 to S64;

[0366] The code for obtaining the optimal tug configuration prediction model in this embodiment is as follows:

[0367] #(A) Load data

[0368] def load_data(csv_path):

[0369] #1) Judge the file existence

[0370] #2) Read csv with pandas

[0371] return df

[0372] #(B) Extract tug horsepower & count the quantity

[0373] def extract_horsepower(tug_info:str):

[0374] # Use regular expression to find numbers

[0375] returnhp_val

[0376] defcompute_tug_info(df,tug_columns):

[0377] #Use extract_horsepower for multiple tugboats

[0378] #sum-->total_horsepower

[0379] #count-->tug_count

[0380] #(C) Analyze pilotage time & whether it is daytime

[0381] defparse_time_period(period_str:str):

[0382] #"YYYY-mm-ddHH:MM:SS~YYYY-mm-ddHH:MM:SS"

[0383] return(start_dt,end_dt)

[0384] defis_daytime(dt:datetime):

[0385] return 1 if 6 <= dt.hour < 19 else 0

[0386] defcompute_daytime_feature(df):

[0387] #Add start_time, end_time, is_daytime

[0388] #(D) Feature Engineering:

[0389] #Including ship type coding, angle conversion to sin / cos, NaN filling, etc.

[0390] defencode_ship_type(df,col="Ship Type"):

[0391] encoder = LabelEncoder()

[0392] df["ship_type_encoded"]=encoder.fit_transform(df[col])

[0393] return df,encoder

[0394] def add_sin_cos_features(df, angle_col):

[0395] # angle -> (sin, cos)

[0396] # (E) Model training / evaluation

[0397] def train_classifier_with_cv(X_train, y_train):

[0398] # RandomForestClassifier + Pipeline + GridSearchCV

[0399] def train_regressor_with_cv(X_train, y_train):

[0400] # RandomForestRegressor + Pipeline + GridSearchCV

[0401] def evaluate_models(best_clf, best_reg, X_test, y_test_cls, y_test_reg, label_map):

[0402] # Classification: accuracy, confusion_matrix, classification_report

[0403] # Regression: mse, rmse, r2

[0404] # (F) Prediction example

[0405] def predict_for_new_ship(best_clf, best_reg, new_ship_data, ship_type_encoder, label_map):

[0406] # 1) First, convert new_ship_data into feature row X_new

[0407] # 2) tug_count_factor = best_clf.predict(X_new)

[0408] # 3) total_hp = best_reg.predict(X_new)

[0409] #(G) Main Process

[0410] #1) Read csv, compute_tug_info, compute_daytime_feature

[0411] #2) encode_ship_type, add_sin_cos_features

[0412] #3) factorizetug_count --> y_cls_factor, total_horsepower --> y_reg

[0413] #4) train_test_split

[0414] #5) train_classifier_with_cv + train_regressor_with_cv

[0415] #6) evaluate_models

[0416] #7) predict_for_new_ship (example)

[0417] This embodiment can automatically train a tug configuration prediction model through a large amount of historical "ship characteristics - environmental conditions - tug configuration" data recorded in the historical feature dataset, and then obtain an optimal tug configuration prediction model with the ability to recommend the number of tugs and total horsepower. As Figure 6 shown, when used in an actual port, inputting the ship characteristic parameters of a new target ship can quickly generate a corresponding tug configuration plan;

[0418] S7: Set the optimal tug configuration prediction model in the existing port scheduling system to deploy it as an online service model for tug configuration, and then implement the requirement plan for tug configuration based on AIS pilotage data.

[0419] This embodiment can easily deploy the trained optimal tug configuration prediction model to the port scheduling system whether through a preset local script, API service, or big data platform, providing data support for real-time pilotage. Through the model training and output of the tug configuration plan in this embodiment, the closed-loop process from the acquisition of AIS data to the decision output of the tug configuration plan is improved, enabling the port to intelligently recommend the most suitable number of tugs and horsepower for newly arriving / departing ships with relatively high accuracy based on historical facts and the current environment. By implementing dynamic iterative updates to the tug configuration prediction model, the scheduling efficiency and navigation safety of tugs can be continuously optimized.

[0420] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A tugboat configuration method based on AIS pilotage data, characterized in that Specifically, it includes the following steps: S1: Obtain the original AIS data for tugboat configuration; S2: Parse the original AIS data to obtain the initial sequential ship track; S3: Perform track data cleaning on the initial sequential ship track to obtain the sequential continuous ship track; S4: Obtain meteorological / ocean multi-source data, interpolate and fuse the meteorological / ocean multi-source data with the sequential continuous ship track to obtain a ship track fusion database for the multi-source data fused sequential continuous ship track; S5: Construct a ship spatio-temporal matching and speed / course difference determination mechanism, and mine the tugboat configuration mining data of the target ship during the pilotage period according to the ship track fusion database; And the tugboat configuration mining data at least includes the tugboat quantity category and the tugboat horsepower; And obtain the historical feature dataset of the ship-tugboat configuration according to the tugboat configuration mining data; And the historical features include the target ship attribute features, environmental features, and tugboat configuration; S6: Construct a tugboat configuration prediction model for AIS pilotage data; And the tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model; Use the cross-validation method to divide the historical feature dataset into a training set and a test set; And use the target ship attribute features and environmental features as feature inputs, use the tugboat quantity category of the tugboat configuration as the feature output of the random forest classification algorithm model, and use the tugboat horsepower as the feature output of the random forest regression algorithm model; Train the tugboat configuration prediction model according to the training set, and use the grid search method to optimize the hyperparameters of the tugboat configuration prediction model to obtain the trained tugboat configuration prediction model; Evaluate the trained tugboat configuration prediction model through the test set to obtain the optimal tugboat configuration prediction model; S7: Set the optimal tugboat configuration prediction model in the existing port scheduling system to deploy it as an online service model for tugboat configuration, and then implement the requirement solution for tugboat configuration based on AIS pilotage data.

2. The tugboat configuration method based on AIS pilotage data according to claim 1, wherein The specific steps of S2 include the following: S21: Store the original AIS data in a CSV file; S22: Traverse the original AIS data in the CSV file, and batch parse each AIS message data to obtain AIS parsed data; And the method for batch parsing each AIS message data is: Batch parse the AIS message data by calling the AIS message processing library PyAIS to obtain AIS dynamic messages and AIS static messages; And the AIS dynamic messages at least include longitude and latitude, speed and course, and the timestamp of the AIS; The AIS static messages at least include the ship name, ship type, and ship IMO number; Use the AIS dynamic messages and AIS static messages as the AIS parsed data; S23: Map the AIS parsed data to the predetermined track storage fields to obtain the initial ship track; And the predetermined track storage fields: {MMSI, longitude and latitude, speed, course, IMO, ship name, timestamp, ship type,......} 3. The tugboat configuration method based on AIS pilotage data according to claim 2, wherein The track data cleaning process in S3 specifically includes: Abnormal elimination: Eliminate the initial ship tracks that do not meet the standard digital length of MMSI or whose MMSI is a non-digital field; Data filtering: After performing abnormal elimination, filter out the initial ship tracks whose longitude / latitude / speed / course exceed the preset threshold; Data thinning: After performing data filtering, when the time interval between adjacent track points of the same ship is less than the given threshold, only retain the previous track point according to the ship track direction; Track splitting: After performing data thinning, when the time interval between adjacent track points of the same ship is greater than the preset time threshold or the spatial separation exceeds the preset distance threshold, the track between these adjacent track points is used as a new segment in the initial ship track; Track linear interpolation: After performing track splitting, perform linear interpolation of track points for each ship track to obtain an optimized ship track; RDP compression: After performing track linear interpolation, use the slope of the straight line between adjacent track points as the track deviation angle, and retain the track points whose track deviation angle meets the preset deviation distance threshold to constrain the ship track point tolerance, and obtain the ship time-series continuous track.

4. A tugboat configuration method based on AIS pilotage data according to claim 3, wherein, The specific steps of S4 are as follows: S41: Obtain meteorological / ocean multi-source data stored in NetCDF / HDF5 files; And the meteorological / ocean multi-source data at least includes wind field data, flow field data, and tidal data under the set time data; S42: Perform dimension correction on the obtained meteorological / ocean multi-source data to obtain corrected data; S43: Divide the longitude and latitude grid of the target sea condition area to obtain the multi-source data longitude and latitude grid; Based on the multi-source data longitude and latitude grid, obtain the grid data of each longitude and latitude grid point according to the corrected data, and the grid data includes the wind field speed / direction, flow field speed / direction, and tidal speed / direction at the corresponding timestamp; S44: Obtain the longitude and latitude of the given track point in the ship time-series continuous track, and based on the nearest neighbor interpolation method, obtain the grid data of the longitude and latitude grid point with the closest longitude and latitude distance to the given track point according to the multi-source data longitude and latitude grid, and assign the corresponding grid data to the given track point; To fuse the meteorological / ocean multi-source data into the ship time-series continuous track to obtain a ship track fusion database.

5. A tugboat configuration method based on AIS pilotage data according to claim 4, characterized in that, The specific steps of constructing a ship spatio-temporal matching and speed / course difference determination mechanism in S5 and mining the tugboat configuration mining data of the target ship during the pilotage period according to the ship track fusion database are as follows: S51: Based on the ship pilotage period determination rule, confirm the start time and end time of the ship's pilotage to obtain the target ship pilotage period; And the ship pilotage period determination rule: Obtain the track point speed of the ship time-series continuous track in the ship track fusion database; When the track point speed meets the first preset speed threshold, confirm this moment as the start time of pilotage; When the track point speed meets the second preset speed threshold, confirm this moment as the end time of pilotage; S52: Preset several tugboats and encode each tugboat to obtain an encoded tugboat set; Traverse each tugboat in the encoded tugboat set, and based on the constructed ship spatio-temporal matching and speed / course difference determination mechanism, obtain the matching assisting tugboat according to the target ship pilotage period; The ship spatio-temporal matching and speed / course difference determination mechanism includes a ship spatio-temporal matching rule and a speed / course difference determination rule; The ship spatio-temporal matching rule: Combine the tugboats whose periods overlap with the target ship's pilotage period or meet the preset time threshold, and the shortest distance between the real-time obtained tugboat positions and the target ship's trajectory segments, and combine the tugboats corresponding to the shortest distance meeting the preset shortest distance threshold as the potential cooperative tugboats of the target ship; The speed / course difference determination rule: Obtain the deviation values between the speeds / courses of the potential cooperative tugboats and the speeds / courses of the target ship at the corresponding moments; And confirm the potential cooperative tugboats corresponding to the deviation values meeting the preset deviation threshold as the matching assistance tugboats for finally assisting in the pilotage of the target ship, and then obtain the tugboat configuration mining data of the target ship during the pilotage period according to the matching assistance tugboats.

6. The tugboat configuration method based on AIS pilotage data according to claim 5, wherein, The specific steps of S6 are as follows: S61: Construct a tugboat configuration prediction model for AIS pilotage data; The tugboat configuration prediction model includes a random forest classification algorithm model and a random forest regression algorithm model; S62: Use the cross-validation method to divide the historical feature dataset into K subsets; And randomly use any one of the K subsets as the test set, and use the remaining subsets as the training set; S63: Use the target ship attribute features and environmental features as feature inputs, use the tugboat quantity category of the tugboat configuration as the feature output of the random forest classification algorithm model, and use the tugboat horsepower as the feature output of the random forest regression algorithm model; S64: Train the tugboat configuration prediction model according to the training set, and use the grid search method to optimize the hyperparameters of the tugboat configuration prediction model to obtain the trained tugboat configuration prediction model; S65: Evaluate the trained tugboat configuration prediction model through the test set to obtain the optimal tugboat configuration prediction model; And the indicators of the model evaluation include classification accuracy, confusion matrix, and regression error RMSE; The method for evaluating the trained tugboat configuration prediction model: Obtain the current feature error between the predicted feature output and the real feature of the trained tugboat configuration prediction model, and judge whether the current feature errors all meet the preset index thresholds; If the current feature errors all meet the preset index thresholds, confirm that the trained tugboat configuration prediction model at this time is the optimal tugboat configuration prediction model; Otherwise, use the cross-validation method to re-divide the historical feature dataset into a training set and a test set, and repeat steps S63 to S64.

Images