A maritime data rapid checking and completing method and system based on AIS multi-modal data fusion

Abstract

This invention provides a method and system for rapid verification and completion of maritime data based on AIS multimodal data fusion, relating to the field of ship AIS data processing. This invention establishes a ship specification dictionary by parsing Lloyd's Register records, performs time, trajectory, and attribute verification and preprocessing on AIS dynamic data, adds specification information to trajectory points and associates them with draft data, and combines value range checks and nearest neighbor completion to generate a continuous and reliable dataset. This achieves automated verification and completion of key attributes, trajectory information, and draft data, significantly improving the accuracy and completeness of maritime data.

Description

Technical Field

[0001] This invention relates to the field of ship AIS data processing, and in particular to a method and system for rapid verification and completion of maritime data based on AIS multimodal data fusion. Background Technology

[0002] Automatic Identification System (AIS) is one of the fundamental data sources for modern maritime monitoring and shipping data analysis. It provides a wealth of ship navigation information and is widely used in important fields such as navigation safety, ship behavior identification, environmental impact assessment, Arctic navigation research, global trade flow analysis, and ship and port operation management. Typically, AIS data includes three parts: static data (ship name, IMO number, type, call sign, etc.), dynamic data (latitude and longitude, speed, heading, UTC timestamp, etc.), and voyage-related information (destination port, estimated time of arrival, draft, etc.).

[0003] Although the quality and completeness of AIS data have significantly improved in recent years due to advancements in satellite reception and mandatory AIS equipment usage by relevant organizations and departments, many problems still exist. Furthermore, the rapid application of big data and artificial intelligence in the shipping industry has placed higher demands on the accuracy, continuity, and attribute completeness of AIS data. AIS data that has not undergone systematic preprocessing and multi-source fusion is no longer sufficient to meet the needs of these cutting-edge applications.

[0004] First, some important fields in AIS data (especially draft, some static and voyage information) often rely on manual input by crew members, which frequently results in false alarms, incorrect entries, or long periods without updates. Furthermore, even dynamic data automatically generated by sensors may exhibit anomalies, missing information, and duplication due to equipment malfunctions or environmental interference. Current technologies often employ trajectory cleaning, anomaly detection, or machine learning methods to correct dynamic trajectories, but in low-frequency sampling and large jump points, a single trajectory cleaning method is insufficient to completely eliminate errors. Additionally, research on processing static and voyage information such as draft data and ship attributes is relatively lacking.

[0005] Meanwhile, the raw static data fields in AIS are limited and of varying quality, while classification society archives contain a large amount of ship specification information, such as ship main engine specifications, deadweight, maximum design draft, year of construction, flag state, shipping company, length, beam, fuel type, and design speed. This information is crucial for ship energy consumption and emissions accounting and ship behavior modeling.

[0006] Furthermore, dynamic data, static data, and voyage information from different data sources in AIS exhibit significant differences in storage format, temporal resolution, and sampling density. For instance, dynamic trajectory point data is stored in second- or minute-level datasets, while draft records may be sparse and timestamps inaccurate (the ratio of dynamic trajectory data to draft data is approximately 80:1). Static information is stored by vessel entry and in various formats. Directly merging via a button often results in a large number of missing draft fields, affecting the usability of the merged data. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for rapid verification and completion of maritime data based on AIS multimodal data fusion. By fusing Lloyd's Register records with AIS multimodal data, it can achieve automated verification and continuous completion of key ship attributes, trajectory information, and draft data, thereby generating a complete and reliable integrated dataset and significantly improving the accuracy and completeness of maritime data.

[0008] To achieve the above objectives, the present invention provides the following solution:

[0009] A method for rapid verification and completion of maritime data based on AIS multimodal data fusion includes:

[0010] The Lloyd's Register records are analyzed, and key ship attribute information is standardized and verified to establish a specification dictionary with the ship's unique identifier as the key.

[0011] The original AIS dynamic data is formatted with a unified timestamp format. The trajectory points are sorted and deduplicated according to the ship's unique identifier and time. Jump points and outliers are cleaned and corrected according to geographical, speed and time constraints. Longitude, latitude, speed and heading are smoothed to obtain preprocessed AIS trajectory data. The corresponding attributes in the specification dictionary are added to each trajectory point.

[0012] The draft data is associated with the preprocessed AIS trajectory data according to the ship's unique identifier and time. The draft value range is set to not exceed the design maximum draft. Bidirectional nearest neighbor completion is used within the effective navigation time series. When there are no observations on both sides, the average value of this navigation is used. The source is marked, and the verified and completed dataset is output.

[0013] Preferably, the key attribute information includes the year of construction, main engine type, fuel type, maximum rated engine power, tonnage class, load capacity, design speed, maximum design draft, main engine speed, shipping company, and ship size.

[0014] Preferably, the Lloyd's Register is parsed, and key ship attribute information is standardized and verified to establish a specification dictionary with the ship's unique identifier as the key, including:

[0015] Import the original data from Lloyd's Register, standardize field names, data types, and units of measurement, and delete records that lack the ship's unique identifier.

[0016] The unique identifier of the ship is formatted and uniquely verified. If duplicates exist, the record is retained for a period of less than a preset time threshold and the key attribute information is complete.

[0017] The reasonableness of the values ​​for the year of construction, ship size, maximum design draft, maximum rated engine power, design speed, tonnage, load capacity, and main engine speed is checked, and records that are zero or less than zero or exceed the valid range of the records are removed.

[0018] Ship types are standardized according to a pre-defined mapping relationship to form a unified ship type category;

[0019] Based on the ship's tonnage and type, statistical fitting was performed to complete the missing maximum rated power and design speed of the engine.

[0020] Based on the original fuel field 1, fuel field 2, and main engine type, the fuel type is determined according to the preset judgment rules: if both fuel fields are distilled fuel, it is determined to be distilled fuel; if either field is liquefied natural gas or the main engine is steam, it is determined to be liquefied natural gas; if the record is residual fuel oil, it is determined to be heavy oil.

[0021] The key attribute information, after verification and completion, is written into the specification dictionary.

[0022] Preferably, the AIS trajectory data includes the ship's unique identifier, time, longitude, latitude, speed, and heading.

[0023] Preferably, the original AIS dynamic data is formatted with a unified timestamp format, the trajectory points are sorted and deduplicated according to the ship's unique identifier and time, and jump points and outliers are cleaned and corrected based on geographical, speed, and time constraints. Longitude, latitude, speed, and heading are smoothed to obtain preprocessed AIS trajectory data, including:

[0024] Import the original AIS dynamic data, retain the fields of ship unique identifier, time, longitude, latitude, speed, and heading, and delete records that are missing any field;

[0025] Standardize the time field to a standard time format;

[0026] Sort by the ship's unique identifier and time, and retain only one duplicate record for the same ship's unique identifier and the same time.

[0027] Implement geographical constraints to eliminate trajectory points with invalid longitude and latitude values, as well as trajectory points that fall within land areas;

[0028] Execute speed and time constraints, calculate implicit speed based on the ground distance and time interval between adjacent trajectory points, and mark trajectory points that exceed the reasonable range as abnormal points or jump points;

[0029] Anomalies and jump points are corrected according to the principles of temporal and geographical continuity, and adjacent valid trajectory points are used for transition.

[0030] Longitude, latitude, speed and heading are smoothed. The heading is expanded by angle before smoothing to eliminate backtracking. Median filtering with a window size of three is applied to each field.

[0031] Using the ship's unique identifier as the key, the corresponding key attribute information is searched in the specification dictionary. At each trajectory point, the following information is appended: year of construction, main engine type, fuel type, maximum rated power of the engine, tonnage class, load capacity, design speed, maximum design draft, main engine speed, shipping company, and ship size. The preprocessed AIS trajectory data is then output.

[0032] Preferably, the draft data is associated with the preprocessed AIS trajectory data according to the ship's unique identifier and time, the draft value range is set to not exceed the design maximum draft, bidirectional nearest neighbor completion is used within the effective navigation time series, and the average value of this voyage is used when there are no observations on both sides, and the source is marked. The verified and completed dataset is output, including:

[0033] Import draft data, standardize the time to a standard time format, delete records that lack the ship's unique identifier or time, and retain only one duplicate record for the same ship and the same time.

[0034] The draft data is associated with the preprocessed AIS trajectory data according to the ship's unique identifier and time, and those that are not matched are marked as pending completion.

[0035] Based on the maximum design draft in the specification dictionary, the associated and supplementary draft values ​​are checked for their range. Records exceeding the maximum design draft are marked as invalid and are not included in the supplementation and mean calculation.

[0036] The vessels are grouped and the effective time series of navigation are defined in the preprocessed AIS trajectory data;

[0037] For trajectory points with missing draft within the effective navigation time series, the nearest effective draft observations are searched both forward and backward, and the side with the closer time distance is used as the completion value;

[0038] When there are no valid observations on either side, the average draft within the valid navigation time series is used as the complete value.

[0039] Each draft record is marked as either original or complete, and a verified and complete dataset is output. The verified and complete dataset includes the preprocessed AIS trajectory data, key attribute information from the specification dictionary, and draft data and source markers.

[0040] A rapid verification and completion system for maritime data based on AIS multimodal data fusion includes:

[0041] The ship specification information fusion module is used to parse Lloyd's Register records, standardize and verify key ship attribute information, and establish a specification dictionary with the ship's unique identifier as the key.

[0042] The AIS dynamic data preprocessing module is used to unify the timestamp format of the original AIS dynamic data, sort the trajectory points according to the ship's unique identifier and time and remove duplicates, clean and correct jump points and outliers according to geographical, speed and time constraints, smooth longitude, latitude, speed and heading to obtain preprocessed AIS trajectory data, and add the corresponding attributes from the specification dictionary to each trajectory point.

[0043] The draft data fusion and completion module is used to associate draft data with the preprocessed AIS trajectory data according to the ship's unique identifier and time, set the draft value range to not exceed the design maximum draft, use bidirectional nearest neighbor completion within the effective navigation time series, use the average value of this navigation when there are no observations on both sides, mark the source, and output the verified and completed dataset.

[0044] The present invention discloses the following technical effects:

[0045] This invention enriches the original static fields of AIS by fusing ship Lloyd's Register data and AIS data from multiple sources, forming more complete ship specification information. Through joint verification and completion of dynamic and draft data, it significantly improves the continuity and accuracy of draft data, solving the problem of draft information being commonly neglected in existing AIS data processing research. It accelerates the processing of massive amounts of data through GPU acceleration, partitioned multi-process parallel processing, and binary search algorithms. Through multi-source data fusion and systematic processing, it forms a high-precision AIS data database covering dynamic, static, and voyage information, achieving standardized data structure, precise data accuracy, and complete information attributes, meeting the higher data quality requirements of modern applications driven by big data and artificial intelligence. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the technical route provided in the embodiments of the present invention;

[0049] Figure 3 This is a schematic diagram of the system structure provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides a method for rapid verification and completion of maritime data based on AIS multimodal data fusion, including:

[0053] Step 100: Parse Lloyd's Register records, standardize and verify key ship attribute information to establish a specification dictionary with the ship's unique identifier as the key;

[0054] Step 200: Unify the timestamp format of the original AIS dynamic data, sort the trajectory points by ship unique identifier and time and remove duplicates, clean and correct jump points and outliers according to geographical, speed and time constraints, smooth longitude, latitude, speed and heading to obtain preprocessed AIS trajectory data, and attach the corresponding attributes from the specification dictionary to each trajectory point.

[0055] Step 300: Associate the draft data with the preprocessed AIS trajectory data by ship unique identifier and time, set the draft value range to not exceed the design maximum draft, use bidirectional nearest neighbor completion within the valid navigation time series, use the average value of this navigation when there are no observations on both sides, mark the source, and output the verified and completed dataset.

[0056] This invention proposes a multi-source fusion processing method for AIS data covering dynamic, static, and flight range data. The aim is to improve the continuity, accuracy, and completeness of AIS data, thereby providing reliable data support for refined applications. Simultaneously, it utilizes algorithms such as GPU acceleration, partitioned multi-process parallelism, and binary search to accelerate the processing of massive amounts of data. Figure 2 As shown, this method mainly consists of three parts:

[0057] Step S1: Multi-source fusion of ship specification information:

[0058] The Lloyd's Register data is parsed and standardized to extract and correct key ship attribute information. Multi-source data fusion of Lloyd's Register data and dynamic trajectory data is then performed to supplement key ship attribute information in the AIS data, such as ship construction year, main engine type, fuel type, maximum engine rated power, tonnage, deadweight, design speed, maximum design draft, main engine speed, shipping company, and ship size. The ship information processed in this step not only meets the needs for more refined data but also supports subsequent data processing.

[0059] Step S2: AIS Dynamic Data Preprocessing

[0060] The module performs trajectory cleaning and smoothing on AIS dynamic data, removing outliers and duplicates, correcting trajectory jumps, and ensuring the continuity and accuracy of the trajectory. After processing, a large number of high-quality AIS ship trajectory points can be obtained, with each trajectory point including the ship's unique identifier MMSI, timestamp, latitude and longitude, instantaneous speed, and heading.

[0061] Step S3: Draft data fusion and completion

[0062] The manually entered draft data is integrated with the processed AIS dynamic-static data, and the draft field is validated and interpolated to improve the continuity and reliability of the draft data.

[0063] This method enables the systematic verification and processing of AIS data, resulting in high-quality AIS fusion data covering dynamic, static, and flight range information.

[0064] In another embodiment of the present invention, a multi-source fusion preprocessing method for AIS data covering dynamic, static, and flight range data includes the following steps:

[0065] Step S1: Multi-source fusion of ship specification information

[0066] This step integrates Lloyd's Register data with dynamic trajectories from multiple sources, supplementing key ship attribute information. It can process 160,000 ships within 1 minute with O(N) complexity, not only meeting more refined data requirements but also supporting subsequent draft data processing.

[0067] Step S11: Lloyd's Records Data Standardization and Verification

[0068] First, read the Lloyd's Register data from the CSV file, using the ship's unique identifier MMSI as the primary key for interfacing with AIS data. Null records are not entered into the database, and the remaining MMSIs are parsed into integer MMSIs to ensure key consistency.

[0069] In accordance with the International Maritime Organization (IMO) ship type classification standards, a custom ship type mapper is used to standardize the ship_type_level3 and ship_type_level4 recorded in the Lloyd's Register data into a unified ship type ship_type required for calculation. Ship types include dry bulk carriers, chemical tankers, container ships, general cargo ships, liquefied gas carriers, oil tankers, other liquid carriers, ferries, roll-on / roll-off ferries, passenger ships, refrigerated ships, roll-on / roll-off ships, car carriers, yachts, fishing vessels, coastal vessels, tugboats, other service vessels and other miscellaneous vessels.

[0070] For the missing maximum rated power and design speed of main engines in the Lloyd's Register, the missing values ​​were filled in by linear regression fitting based on ship tonnage and ship type, in accordance with IMO statistical standards.

[0071]

[0072]

[0073] in, For the design speed, Loa is the maximum rated power of the main engine, Loa is the total length of the vessel, and Deadweight is the deadweight of the vessel. These are the fitted parameter values ​​obtained after fitting.

[0074] Fuel type plays a crucial role in fuel consumption accounting and defining empirical emission factors. The Lloyd's Register provides each ship's FuelType1First (describing the lightest fuel), FuelType2First (describing the heaviest fuel), and main engine type. This invention uses these fields, along with the processed ship type field, to determine the ship's fuel according to IMO standards. The criteria for determining the primary fuel for each ship are as follows:

[0075] (1) Heavy oil (HFO / residual fuel oil): Fuel Type 1 First or Fuel Type 2 First is residual fuel oil; except for liquefied gas ships with steam turbine main engines, whose fuel type is liquefied natural gas (LNG).

[0076] (2) Distilled fuel (MDO): FuelType1First and FuelType2First are both distilled fuels; either FuelType1First or FuelType2First is a distilled fuel and the other is "NA"; FuelType1First is coal and FuelType2First is a distilled fuel; FuelType1First is methanol and FuelType2First is distilled fuel, except that its fuel type is methanol.

[0077] (3) Liquefied natural gas (LNG): All LNG carriers with steam as the main engine type; Fuel Type 1 First is gas fuel and Fuel Type 2 First is distilled fuel; Fuel Type 1 First is LNG;

[0078] (4) Nuclear fuel: Fuel Type 1 First is nuclear fuel;

[0079] (5) Coal: FuelType1First is coal;

[0080] (6) Methanol: FuelType1First is methanol;

[0081] (7) Ammonia: FuelType1First is ammonia;

[0082] (8) Hydrogen: FuelType1First is hydrogen;

[0083] (9) Biofuels: FuelType1First is biofuel;

[0084] (10) Liquefied petroleum gas (LPG): FuelType1First is LPG;

[0085] In addition, the Lloyd's Register records are used to verify and convert the following fields: maximum engine speed, deadweight (deadweight tons, gross tonnage, TEU, number of loaded vehicles, the values ​​vary for different types of ships), engine type, maximum design draft, year of construction, flag state, shipping company, length, beam, fuel type, maximum rated engine power, and design speed.

[0086] The normalized fields are encapsulated into a dictionary indexed by MMSI, forming a Lloyd's Register database with MMSI as the key. This structure facilitates subsequent O(1) time access and supports efficient association with AIS.

[0087] Step S12: Integrating AIS dynamic data with Lloyd's Register data

[0088] The standardized ship specification dictionary has the ship's MMSI as the key and the ship's specification attribute fields and corresponding values ​​as the values:

[0089]

[0090] in The name representing the ship's attributes. Each ship corresponds to one of the above dictionaries, representing the attribute values.

[0091] This step involves querying the corresponding MMSI vessel attribute dictionary in batches and in parallel on the AIS trajectory. If a match is found, the Lloyd's Register specification field is appended to the AIS record, forming a dynamic-static combined record.

[0092]

[0093] in This represents the AIS dynamic trajectory corresponding to the ship with MMSI value i. This represents the dictionary of specifications for ships with MMSI value i.

[0094] By using a dictionary indexed by MMSI to perform key-value concatenation with AIS data, the complexity of dictionary lookup is O(1), and the complexity of fusing all ship trajectory data is O(N). Therefore, the overall complexity of this step is O(N), which can achieve fast matching at the millisecond level.

[0095] Step S2: AIS Dynamic Data Preprocessing

[0096] This step includes two parts: trajectory cleaning and smoothing of AIS dynamic data. It removes outliers and duplicates from the trajectory and corrects trajectory jump points. The complexity is O(N). Each trajectory point after processing includes the ship's unique identifier MMSI, timestamp, latitude and longitude, instantaneous speed, and heading.

[0097] Step S21: Trajectory Cleaning

[0098] First, the original records are initially screened according to the following constraints:

[0099] (1) Latitude ;

[0100] (2) Longitude ;

[0101] (3) Instantaneous velocity Festival;

[0102] (4) Heading Spend;

[0103] Convert the timestamp postime field from UTC to standard time in the format YYYY-MM-DD hh-mm:ss. Sort the trajectory points by MMSI and time, remove duplicates by (MMSI, postime), and discard points with empty or invalid times.

[0104] Then, by combining ship specification information and geographic information, anomaly points are identified and corrected for the trajectory points:

[0105] (1) The instantaneous speed is greater than the ship's maximum design speed;

[0106] (2) The ship is located in a land area;

[0107] (3) Let the surface distance d (in km) between two adjacent points of the same ship be calculated using the Haversine formula:

[0108]

[0109] Where R is the Earth's radius, which is taken as 6371 km in this invention, and c is calculated as follows:

[0110]

[0111]

[0112]

[0113]

[0114] Let the time interval between two adjacent points on the same ship be... The implicit velocity is calculated from the surface distance and time interval between two adjacent points, and its unit is converted from km / h to knot.

[0115]

[0116] Where 1.852 is the ratio of kilometers to nautical miles; finally, extreme outliers are identified according to the following conditions (any condition being true is considered an anomaly):

[0117] (a) ;

[0118] (b) ;

[0119] And remove trajectory points marked as abnormal.

[0120] Step S12: Smoothing

[0121] Three-point median filtering and heading deployment are employed to minimize noise and avoid false signals caused by heading circling.

[0122] First, perform an angle expansion on the COG to ensure that the heading is monotonically continuous over time, avoiding pseudo-jumps caused by numerical reversals at 0 / 360 degrees: assuming the original heading sequence... ,in The unfolded continuous angular sequence is denoted as :

[0123]

[0124]

[0125]

[0126] The `round()` function takes the nearest integer.

[0127] Then, for the same ship, lat, lon, sog, and cog are smoothed using a window size of 3 and a median filter to suppress sharp noise and single-point anomalies without excessively smoothing out the edges of steering and speed changes.

[0128] Step S3: Draft data fusion and completion

[0129] This step performs point-in-time fusion and time-series completion of draft data and processed ship dynamic and static data, with a complexity of O(logN). It can complete the processing of 4.4 billion data points within 30 minutes, improve the continuity and accuracy of the draft field, and meet the higher requirements of modern applications driven by big data and artificial intelligence for draft data.

[0130] Step S31: Fusion of AIS dynamic and static data with draft data

[0131] First, the postime field of the draft data is uniformly converted from UTC to a type and format consistent with AIS dynamic data. Then, using MMSI and postime as key pairs, AIS trajectory data and draft data are quickly fused by primary key matching. Aligned records have a new draft field added, while mismatched draft values ​​are left blank. This process supports the rapid fusion of massive trajectories through multi-partition parallelism and GPU acceleration, with an algorithm complexity of O(N), and can complete the fusion of 4 billion data points within 10 minutes.

[0132] Step S32: Complete draft data

[0133] For each trajectory point with missing draft data, a nearest neighbor filling algorithm with time constraints is used. A binary search is performed within the valid time series of this voyage, centered on the point at time t, and the nearest valid draft observation is taken from both the preceding and following points. To avoid completion distortion, the following value range constraints are applied and their reasonableness is verified:

[0134]

[0135] in The maximum draft was designed for the ship (from Lloyd's Register). Each ship was grouped and processed in ascending chronological order. For each missing point i in the valid set... Perform bidirectional nearest neighbor search within a limited time window. :

[0136] (1) Let , .

[0137] (2) If and If both conditions are met, the value of the closer side is taken as the completion value; if one side is satisfied, that side is taken as the completion value; if neither side is satisfied or the condition does not exist, the value is reverted to the mean value within the current voyage sequence group. .

[0138] The original draft and the completed draft are marked with a `draught_src` field, where 1 represents the original value and 0 represents the completed value, to support downstream weighting and uncertainty propagation. This strategy has a time complexity of O(logN) and, with parallel technology, can complete the draft field completion for 4 billion trajectory data points within 20 minutes, making it suitable for efficient completion of massive trajectory data.

[0139] Corresponding to the above methods, such as Figure 3 As shown, this embodiment also provides a rapid verification and completion system for maritime data based on AIS multimodal data fusion, including:

[0140] The ship specification information fusion module is used to parse Lloyd's Register records, standardize and verify key ship attribute information, and establish a specification dictionary with the ship's unique identifier as the key.

[0141] The AIS dynamic data preprocessing module is used to unify the timestamp format of the original AIS dynamic data, sort the trajectory points according to the ship's unique identifier and time and remove duplicates, clean and correct jump points and outliers according to geographical, speed and time constraints, smooth longitude, latitude, speed and heading to obtain preprocessed AIS trajectory data, and add the corresponding attributes from the specification dictionary to each trajectory point.

[0142] The draft data fusion and completion module is used to associate draft data with the preprocessed AIS trajectory data according to the ship's unique identifier and time, set the draft value range to not exceed the design maximum draft, use bidirectional nearest neighbor completion within the effective navigation time series, use the average value of this navigation when there are no observations on both sides, mark the source, and output the verified and completed dataset.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0144] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A maritime data rapid verification and completion method based on AIS multi-modal data fusion, characterized in that, include: The Lloyd's Register records are analyzed, and key ship attribute information is standardized and verified to establish a specification dictionary with the ship's unique identifier as the key. The original AIS dynamic data is formatted with a unified timestamp format. The trajectory points are sorted and deduplicated according to the ship's unique identifier and time. Jump points and outliers are cleaned and corrected according to geographical, speed and time constraints. Longitude, latitude, speed and heading are smoothed to obtain preprocessed AIS trajectory data. The corresponding attributes in the specification dictionary are added to each trajectory point. The draft data is associated with the preprocessed AIS trajectory data according to the ship's unique identifier and time. The draft value range is set to not exceed the design maximum draft. Bidirectional nearest neighbor completion is used within the effective navigation time series. When there are no observations on both sides, the average value of this navigation is used. The source is marked, and the verified and completed dataset is output. The AIS trajectory data includes the ship's unique identifier, time, longitude, latitude, speed, and heading; The raw AIS dynamic data is formatted with a unified timestamp format. Track points are sorted and deduplicated according to the ship's unique identifier and time. Jump points and outliers are cleaned and corrected based on geographical, speed, and time constraints. Longitude, latitude, speed, and heading are smoothed to obtain preprocessed AIS track data, including: Import the original AIS dynamic data, retain the fields of ship unique identifier, time, longitude, latitude, speed, and heading, and delete records that are missing any field; Standardize the time field to a standard time format; Sort by the ship's unique identifier and time, and retain only one duplicate record for the same ship's unique identifier and the same time. Implement geographical constraints to eliminate trajectory points with invalid longitude and latitude values, as well as trajectory points that fall within land areas; Execute speed and time constraints, calculate implicit speed based on the ground distance and time interval between adjacent trajectory points, and mark trajectory points that exceed the reasonable range as abnormal points or jump points; Anomalies and jump points are corrected according to the principles of temporal and geographical continuity, and adjacent valid trajectory points are used for transition. Longitude, latitude, speed and heading are smoothed. The heading is expanded by angle before smoothing to eliminate backtracking. Median filtering with a window size of three is applied to each field. Using the ship's unique identifier as the key, the corresponding key attribute information is searched in the specification dictionary. At each trajectory point, the following information is appended: year of construction, main engine type, fuel type, maximum rated power of the engine, tonnage class, load capacity, design speed, maximum design draft, main engine speed, shipping company, and ship size. The preprocessed AIS trajectory data is then output.

2. The maritime data quick checking and completing method based on AIS multi-modal data fusion according to claim 1, characterized in that, The key attribute information includes the year of construction, main engine type, fuel type, maximum rated engine power, tonnage, load capacity, design speed, maximum design draft, main engine speed, shipping company, and ship size.

3. The maritime data quick checking and completing method based on AIS multi-modal data fusion according to claim 2, characterized in that, The Lloyd's Register records are parsed to standardize and verify key ship attribute information, in order to establish a specification dictionary with the ship's unique identifier as the key, including: Import the original data from Lloyd's Register, standardize field names, data types, and units of measurement, and delete records that lack the ship's unique identifier. The unique identifier of the ship is formatted and uniquely verified. If duplicates exist, the record is retained for a period of less than a preset time threshold and the key attribute information is complete. The reasonableness of the values ​​for the year of construction, ship size, maximum design draft, maximum rated engine power, design speed, tonnage, load capacity, and main engine speed is checked, and records that are zero or less than zero or exceed the valid range of the records are removed. Ship types are standardized according to a pre-defined mapping relationship to form a unified ship type category; Based on the ship's tonnage and type, statistical fitting was performed to complete the missing maximum rated power and design speed of the engine. Based on the original fuel field 1, fuel field 2, and main engine type, the fuel type is determined according to the preset judgment rules: if both fuel fields are distilled fuel, it is judged as distilled fuel; if either field is liquefied natural gas or the main engine is steam, it is judged as liquefied natural gas; if the record is residual fuel oil, it is judged as heavy oil. The key attribute information, after verification and completion, is written into the specification dictionary.

4. The method for rapid verification and completion of maritime data based on AIS multimodal data fusion according to claim 1, characterized in that, The draft data is associated with the preprocessed AIS trajectory data using the ship's unique identifier and time. The draft value range is set to not exceed the design maximum draft. Bidirectional nearest neighbor completion is used within the valid navigation time series. When there are no observations on either side, the average value of this voyage is used. The source is marked, and the verified and completed dataset is output, including: Import draft data, standardize the time to a standard time format, delete records that lack the ship's unique identifier or time, and retain only one duplicate record for the same ship and the same time. The draft data is associated with the preprocessed AIS trajectory data according to the ship's unique identifier and time, and those that are not matched are marked as pending completion. Based on the maximum design draft in the specification dictionary, the associated and supplementary draft values ​​are checked for their range. Records exceeding the maximum design draft are marked as invalid and are not included in the supplementation and mean calculation. The vessels are grouped and the effective time series of navigation are defined in the preprocessed AIS trajectory data; For trajectory points with missing draft within the effective navigation time series, the nearest effective draft observations are searched both forward and backward, and the side with the closer time distance is used as the completion value; When there are no valid observations on either side, the average draft within the valid navigation time series is used as the complete value. Each draft record is marked as either original or complete, and a verified and complete dataset is output. The verified and complete dataset includes the preprocessed AIS trajectory data, key attribute information from the specification dictionary, and draft data and source markers.

5. A rapid verification and completion system for maritime data based on AIS multimodal data fusion, characterized in that, The system is used to implement the method for rapid verification and completion of maritime data based on AIS multimodal data fusion as described in any one of claims 1 to 4, the system comprising: The ship specification information fusion module is used to parse Lloyd's Register records, standardize and verify key ship attribute information, and establish a specification dictionary with the ship's unique identifier as the key. The AIS dynamic data preprocessing module is used to unify the timestamp format of the original AIS dynamic data, sort the trajectory points according to the ship's unique identifier and time and remove duplicates, clean and correct jump points and outliers according to geographical, speed and time constraints, smooth longitude, latitude, speed and heading to obtain preprocessed AIS trajectory data, and add the corresponding attributes from the specification dictionary to each trajectory point. The draft data fusion and completion module is used to associate draft data with the preprocessed AIS trajectory data according to the ship's unique identifier and time, set the draft value range to not exceed the design maximum draft, use bidirectional nearest neighbor completion within the effective navigation time series, use the average value of this voyage when there are no observations on both sides, mark the source, and output the verified and completed dataset.

Images