A method for integrating ship transportation management data

Abstract

The application provides a ship transportation management data integration method for an un-intelligentized operation ship. The method comprises the following steps: S1: obtaining AIS dynamic data from an AIS database of a local data center and removing AIS latitude and longitude drift points to obtain an adjusted AIS dynamic data set; S2: aligning the adjusted AIS dynamic data set with post-report weather data to form an AIS data set with post-report weather data; and S3: aligning the AIS data set with post-report weather data with MRV message data, generating a virtual point by using a daily report time, and dividing an AIS point sequence by using the virtual point to form a collection data set with AIS dynamic data, post-report weather data and MRV message data information. The method finds the connection between different data sets according to the logical relationship between the data field contents contained in the data sets, and constructs a new convenient and efficient collection data set, thereby laying a foundation for subsequent data mining.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of processing of ship transportation management data, and particularly relates to a ship transportation management data integration method. BACKGROUND

[0002] AIS data, post-weather data, MRV noon report data, MRV voyage report data and HIS ship registration basic data are several major data sources commonly used in ship transportation management. With the development of the shipping industry and the increase in data volume, the relationship between various data is becoming more and more complex, and the query of multiple data sources is not conducive to efficient work.

[0003] If the relationship between various data sources is sorted according to business logic, and a data set containing more and more comprehensive data attributes is spliced, it will be beneficial to related enterprises, departments and government management analysis of the shipping industry. Therefore, the present application invents a ship message and AIS data automatic comparison method and system based on the above-mentioned needs, constructs a more comprehensive and more detailed ship data set, without repeatedly querying between different data sets, and can more detailedly and more quickly understand the multiple data materials of the ship, improve the management and operation efficiency, and facilitate further analysis and prediction work. SUMMARY

[0004] In view of the problem that the current shipping industry separately queries between multiple data sets when performing data analysis, and the data analysis operation is complex, the present application provides a ship transportation management data integration method, uses PostgreSQL and Python language, is based on AIS data, post-weather data, MRV noon report data and MRV voyage report data, finds the relationship between different data sets according to the logical relationship between the data field contents contained in the data sets, and constructs a new convenient and efficient collection data set, thereby laying a foundation for subsequent data mining.

[0005] A ship transportation management data integration method, comprising the following steps:

[0006] S1: obtaining AIS dynamic data from an AIS database of a local data center and removing AIS latitude and longitude drift points to obtain an adjusted AIS dynamic data set, the AIS dynamic data set comprising AIS points of the ship, the AIS points being latitude and longitude and reporting time of the position of the ship, and the AIS points being connected to form a voyage track of the ship;

[0007] S2: Align the adjusted AIS dynamic dataset with the post-weather data: Obtain the post-weather data from the post-weather database of the local data center, search for the closest grid point of the post-weather data in latitude and longitude and the refresh time according to the reporting time and position of the AIS point in the adjusted AIS dataset, assign the weather data in the closest grid point to the AIS point, and form an AIS dataset with post-weather data;

[0008] S3: Align the AIS dataset with post-weather data with MRV message data: Obtain MRV message data, perform time zone conversion on the daily report time in the MRV message data, generate a virtual point using the daily report time, and divide the AIS point sequence using the virtual point to form a collection dataset with AIS dynamic data, post-weather data, and MRV message data information.

[0009] The specific steps for removing AIS latitude and longitude drift points are as follows:

[0010] S1: Obtain the AIS dynamic data field from the AIS database of the local data center, including the ship and the longitude lon i , latitude lat i , and reporting time t i ;

[0011] S2: Confirm the ship AIS navigation track points: S = {P1, P2, …, P i , …, P N}, where point P i = (lon i , lat i , t i );

[0012] S3: Calculate the distance between points P i and P i+1 : distance (i,i+1) = ΔD (i,i+1) and time difference time_diff (i,i+1) = ΔT (i,i+1) ;

[0013] S4: Record the designed speed of the ship in the IHS ship registration information of the local data center as V max , when V max · ΔT (i,i+1) > ΔD (i,i+1) , consider that a drift point is generated, and remove P i+1 ;

[0014] S5: Calculate P i and P i+2 , and execute steps S3-S4 in a loop until Pi and P i+k No longer produce drift, no longer produce drift, the current initial detection point set to P i+k Recycle S3-S4 step until the completion of the AIS trajectory point of ship A in S2, get the adjusted AIS dynamic data set of AIS latitude and longitude drift point.

[0015] The AIS dynamic data includes: the mobile communication service identification code mmsi for uniquely identifying the ship, the time postime of the AIS point collection, the time pre_postime of the previous AIS point collection, the time_diff of postime and pre_postime, the longitude lon of the ship when receiving the AIS data point, the longitude pre_lon of the previous AIS point of the received AIS point, the latitude lat of the ship when receiving the AIS data point, and the latitude prelat of the previous AIS point of the received AIS point.

[0016] The data in the post-weather database includes wind_s, wind_d, wave_h, wave_d, stream_s, stream_d, and weather data refresh time slice T.

[0017] The specific steps of aligning the AIS data with the post-weather data are:

[0018] S1: Obtain post-weather data from the post-weather database of the local data center;

[0019] S2: According to the AIS point reporting time and position, calculate the nearest latitude and longitude grid point in the 1*1 latitude and longitude grid in the weather database according to the Haversine formula, and the position includes longitude and latitude lon and lat;

[0020] S3: Set the ship AIS point P i The reporting time is t i =(YY-MM-DD hh:mm:ss), extract the date information YY-MM-DD, locate to the day, extract the hour information hh, locate to the refresh time sequence slice, extract the minute and second information mm:ss, calculate the time interval between the previous and next refresh time, and select the refresh time with the shortest time interval:

[0021] S4: Assign the weather data in the nearest latitude and longitude grid point to the AIS point to form an AIS data set with post-weather data.

[0022] The wind and wave data in the weather database are refreshed every 4 hours, at 0, 4, 8, 12, 16, 20 o'clock, and the flow data are refreshed every 24 hours, at 0 o'clock.

[0023] The Haversine formula for calculating the distance between the nearest latitude and longitude grid points in the 1*1 latitude and longitude grid is:

[0024]

[0025] where θ=d / R, d is the distance between two places, and R is the radius of the earth, so the secant formula is:

[0026]

[0027] where, is the latitude of the two points, and λ1, λ2 is the longitude of the two points, and the simplified formula is:

[0028]

[0029] The alignment steps of the AIS data and the MRV message data are as follows:

[0030] S1: Obtain the noon report and voyage report in the MRV message data, and first convert the time of the noon report and the voyage report, the AIS reporting time from local time to Beijing time according to the time zone for comparison;

[0031] S2: According to the converted Beijing time, the reporting time of the previous day's daily report is used as the starting time of today's daily report status statistics, and the mmsi data of the AIS point is used as the ship index. The AIS points with AIS reporting time between the daily report starting time and the daily report time of the day are marked with the daily report status label of the day;

[0032] S3: Use the reporting time of the noon report and the voyage report to generate a virtual point D i =(t i ,time_diff i ,distance i ), and use the time ratio to divide the AIS points closest to the virtual point to calculate the distance i in the virtual point, and form a collection data set with AIS dynamic data, weather data and MRV message data information.

[0033] The method for generating a virtual point is,

[0034] Let the virtual point D i =(t i ,time_diff i ,distance i), between AIS points P1=(t1, time_diff1, distance1) and P2=(t2, time_diff2, distance2), then:

[0035] time_diff i =t i -t1

[0036]

[0037] time_diff2′=t2-t i

[0038]

[0039] The distance ′ 2 is the distance from the virtual point to P2, that is, after the virtual point is generated, P2 is changed to

[0040] A computer device comprising a storage, a processor and a computer program stored on the storage and executable on the processor, characterized in that the processor executes the program using PostgreSQL and Python language to realize the steps of the above-mentioned ship transportation management data integration method.

[0041] The beneficial effects of the present application are directed to ships that have not been intelligently transformed, and a ship transportation management data integration method is proposed. Based on AIS data, post-weather data, MRV noon report data and MRV voyage report data, the logical relationship between the data fields contained in the data sets is found to find the relationship between different data sets. By eliminating AIS latitude and longitude drift points, aligning AIS data with post-weather data, and aligning AIS data with MRV message data, a new convenient and efficient set data set is constructed, laying a foundation for subsequent data mining.

[0042] The present application realizes the statistics of the daily voyage mileage by generating a virtual point to segment the AIS sequence, avoiding the voyage time and mileage error caused by AIS data loss.

[0043] At the same time, by comparing and matching the AIS data with the post-weather data and the MRV daily report data, the efficiency of data consultation is increased, and the difficulties and obstacles brought by reading different data are reduced. And it can more accurately and accurately master the corresponding weather data, fuel consumption data and voyage data of each AIS point, and more effectively evaluate the cost, benefit and carbon emission of the route. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1A flow chart of a method for integrating ship transportation management data.

[0045] Figure 2 An effect picture after AIS sequence segmentation in embodiment 4. DETAILED DESCRIPTION

[0046] The application will be further described below in conjunction with the drawings and embodiments.

[0047] Embodiment 1: A method for integrating ship transportation management data.

[0048] As shown in the drawings, Figure 1 A method for integrating ship transportation management data, comprising the following steps:

[0049] S1: Obtain AIS dynamic data from an AIS database of a local data center and eliminate AIS latitude and longitude drift points to obtain an adjusted AIS dynamic data set, the AIS dynamic data set comprising AIS points of the ship, the AIS points being the latitude and longitude of the position of the ship and the reporting time, and the AIS points connecting to form a navigation track of the ship;

[0050] S2: Align the adjusted AIS dynamic data set with post-report weather data: obtain post-report weather data from a post-report weather database of the local data center, search for the latitude and longitude grid points of the post-report weather data closest to the AIS points according to the reporting time and position of the AIS points in the adjusted AIS data set, and calculate the refresh time, assign the weather data in the closest latitude and longitude grid points to the AIS points to form an AIS data set with post-report weather data;

[0051] S3: Align the AIS data set with post-report weather data with MRV message data: obtain MRV message data, perform time zone conversion on the daily report time in the MRV message data, generate a virtual point using the daily report time, and segment the AIS point sequence using the virtual point to form a collection data set with AIS dynamic data, post-report weather data and MRV message data information.

[0052] Embodiment 2: AIS latitude and longitude drift point elimination.

[0053] The latitude and longitude coordinate information in the AIS data comes from the GPS positioning device, and may have drift, missing, etc. For the case of latitude and longitude coordinate point drift, the abnormal point detection processing is performed to detect and eliminate the abnormal data points. The ship appears obvious drift in the visualization interface, which needs to be detected and eliminated by the following algorithm:

[0054] Let the ship AIS navigation track point S = {P1, P2, …, P i ,…,P N}, where point Pi = (lon i , lat i , t i ), lon i is the longitude of the point, lat i is the latitude, and t i is the reporting time. The distance distance i = ΔD i+1 between the points P (i,i+1) and P (i,i+1) and the time difference time_diff (i,i+1) = ΔT (i,i+1) can be calculated. The maximum speed that the ship can reach is denoted as V max . When V max · ΔT (i,i+1) > ΔD (i,i+1) , it is considered that a drift point has been generated, and the elimination operation should be performed.

[0055] Example 3: Alignment of AIS dynamic data and post-report weather data.

[0056] First, the closest grid point in terms of longitude and latitude is searched in the weather database according to the AIS point reporting time. Let the ship AIS point P i = (lon i , lat i ) be given, where lon i and lat i are the longitude and latitude, respectively. The shortest distance to the grid point is calculated using the Haversine formula, and the information of the grid point with the shortest distance is recorded.

[0057] Second, the wind and wave data are refreshed every 4 hours, at 0, 4, 8, 12, 16, 20 o'clock, and the flow data are refreshed every 24 hours, at 0 o'clock. Let the ship AIS point P i reporting time be t i = (YY-MM-DD hh:mm:ss). First, the date information YY-MM-DD is extracted, and the day is located. Second, the hour information hh is extracted, and the refresh time sequence slice is located. Then, the minute and second information mm:ss is extracted, the time interval between the previous and next refresh times is calculated, and the refresh time with the shortest time interval is selected:

[0058] Δt i = min(t i -t 前 , t 后 -t i )

[0059] The latest refresh time is obtained, and the weather data of the latest latitude and longitude grid point is assigned to the AIS point, and the process is completed.

[0060] Embodiment 4: AIS data is aligned with MRV message data.

[0061] Firstly, the MRV daily report time is converted by time zone. Assuming that there is a ship daily report D i =(t i ,time_zone i ), the report time is local time t i , and the time zone is time_zone i , and time_zone i W represents replacing the symbol, and E represents replacing the positive sign. Then the daily report Beijing time t E8i is:

[0062]

[0063] According to the converted Beijing time, the previous day daily report time is used as the starting time of today's daily report status statistics, and the AIS point mmsi is used as the ship index. The AIS points whose report time is between the daily report starting time and the daily report time are marked with the daily report status label of the day.

[0064] Secondly, a virtual point is generated by using the daily report time to divide the AIS point sequence. The virtual point is generated at the daily report time to divide the AIS point time across the daily report time, and the AIS point distance is divided according to the time proportion:

[0065] Assuming that the virtual point D i =(t i ,time_diff i ,distance i ), is between AIS points P1=(t1,time_diff1,distance1) and P2=(t2,time_diff2,distance2), then:

[0066] time_diff i =t i -t1

[0067]

[0068] time_diff2′=t2-t i

[0069]

[0070] As Figure 2As shown, it represents the change of sailing distance after generating the virtual point by using the above formula at 2022-01-23 time, and the distance = 2.51 represents the sailing distance from P1 to P2. When part of AIS data is missing, the ship's distance in this period can be calculated by the distance difference between two adjacent virtual points.

[0071] The above embodiments are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. It is impossible to enumerate all the embodiments here. Any obvious changes or variations derived from the technical solutions of the present application are still within the scope of the present application.

Claims

1. A method of integrating ship transportation management data, characterized by, It comprises the following steps: S1: obtaining AIS dynamic data from an AIS database of a local data center and eliminating AIS latitude and longitude drift points to obtain an adjusted AIS dynamic data set, the AIS dynamic data set comprising AIS points of the ship, the AIS points being the latitude and longitude of the location of the ship and the reporting time, and the AIS points connecting to form a navigation track of the ship; S2: aligning the adjusted AIS dynamic data set with post-weather data: obtaining post-weather data from a post-weather database of the local data center, calculating the nearest latitude and longitude grid point in a 1*1 latitude and longitude grid in the weather database according to the Haversine formula according to the AIS point reporting time and location, the location comprising longitude lon and latitude lat; setting AIS point P of a ship i The reporting time is t i =(YY-MM-DD hh:mm:ss), extract the date information YY-MM-DD, locate to the day, extract the hour information hh, locate to the refresh time sequence slice, extract the minute and second information mm:ss, calculate the time interval between the two refresh times, and select the refresh time with the shortest time interval: assign the weather data in the closest latitude and longitude grid point to the AIS point, and form an AIS data set with the reported weather data. S3: Align AIS data set with post-weather data with MRV message data: Obtain the noon report and voyage report in the MRV message data, first convert the time of the noon report and the voyage report, the AIS report time from local time according to the time zone to Beijing time for comparison; according to the converted Beijing time, take the report time of the previous day's daily report as the start time of today's daily report status statistics, use the mmsi data of the AIS point as the ship index, and label the AIS point with the daily report status label if the AIS report time is between the daily report start time and the daily report time of the day; use the report time of the noon report and the voyage report to generate a virtual point D i =(t i ,time_diff i ,distance i ), calculate the distance i in the virtual point using the time ratio segmentation and the AIS point closest to the virtual point distance, and form a collection data set with AIS dynamic data, post-weather data and MRV message data information.

2. The method of claim 1, wherein the data integration method is used for a ship transportation management system, and the data integration method comprises the steps of: The specific steps for eliminating AIS latitude and longitude drift points are as follows: S1: Obtain AIS dynamic data fields including the ship and the longitude lon and latitude lat and the reporting time t of the ship through the AIS database of the local data middle platform i i i ;​​ S2: Confirm the vessel's AIS navigation track points: S = {P1, P2, ..., P} i ,…,P N }, where point P i =(lon) i ,lat i ,t i ); S3: Calculate point P i Distance between P i+1 and P (i,i+1) = ΔD (i,i+1) and time difference time_diff (i,i+1) = ΔT (i,i+1) ; S4: record the designed speed of the ship in the IHS ship registration information of the local data center as V max When V max · ΔT (i,i+1) > ΔD (i,i+1) , it is considered that a drift point is generated, and P i+1 is eliminated; S5: Calculate P i and P i+2 , the steps S3-S4 are executed in a loop until P i and P i+k no longer produces drift, when the current initial detection point is set as P i+k , the steps S3-S4 are executed in a loop again until all the AIS trajectory points of the ship A in S2 are calculated, and the adjusted AIS dynamic dataset with the AIS latitude and longitude drift points removed is obtained.

3. The method of claim 2, wherein the data is integrated by the ship transport management system. The AIS dynamic data comprises: a mobile communication service identification code mmsi for uniquely identifying the ship, a time postime collected by the AIS point, a time pre_postime collected by the previous AIS point, a time difference time_diff between postime and pre_postime, a longitude lon of the ship when the AIS data point is received, a longitude pre_lon of the previous AIS point of the received AIS point, a latitude lat of the ship when the AIS data point is received, and a latitude pre_lat of the previous AIS point of the received AIS point.

4. The method of claim 1, wherein the data integration method is used for a ship transportation management system, and The data in the post-weather database comprises wind size wind_s, wind direction wind_d, wave height wave_h, wave direction wave_d, stream size stream_s, stream direction stream_d, and weather data refresh time slice T.

5. The method of claim 1, wherein the data integration method is used for a ship transportation management system, and The wind and wave data in the post-weather database are refreshed every 4 hours, at 0, 4, 8, 12, 16, 20, and the stream data are refreshed every 24 hours, at 0.

6. The method of claim 1, wherein the data integration method is used for a ship transportation management system. The Haversine formula for calculating the nearest latitude and longitude grid point in a 1*1 latitude and longitude grid is as follows: Then there is the secant formula: wherein are the latitudes of the two points, and λ1, λ2are the longitudes of the two points, and the simplification uses: Where θ=d / R, d is the distance between two places, and R is the radius of the earth.

7. The method of claim 6, wherein the data is integrated by the vessel management system. The method for generating virtual points is as follows, Let virtual point D i = (t i , time_diff i , distance i ), between AIS points P1 = (t1, time_diff1, distance1) and P2 = (t2, time_diff2, distance2), then: time_diff i = t i - t1 time_diff2' = t2 - t i The distance'2 is the distance from the virtual point to P2, i.e. after the virtual point is generated, P2 is transformed to 8. A computer device comprising a storage, a processor, and a computer program stored on the storage and executable on the processor, characterized in that, The processor executes the program using PostgreSQL and Python language to realize the steps of the ship transportation management data integration method according to any one of claims 1-7.

Images