Method and device for processing ship schedule data, electronic device and readable storage medium

Abstract

The present invention proposes a method and device for processing ship schedule data, an electronic device, and a readable storage medium. The ship schedule data processing method includes: obtaining historical ship schedule data and actual arrival and departure information; correlating the historical ship schedule data with the actual arrival and departure information; based on the correlated historical ship schedule data and the actual arrival and departure information, cleaning and deduplicating the historical ship schedule data to obtain first valid ship schedule data; aggregating the first valid ship schedule data to obtain a ship schedule data statistical table; generating virtual arrival and departure information based on the ship schedule data statistical table; correlating the virtual arrival and departure information with the current ship schedule data; based on the correlated virtual arrival and departure information and the current ship schedule data, cleaning and deduplicating the current ship schedule data to obtain second valid ship schedule data. In this way, virtual arrival and departure information is generated based on the historical ship schedule data and the actual arrival and departure information, and the current ship schedule data is cleaned and deduplicated, thereby improving the credibility of the ship schedule data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for processing ship schedule data, an electronic device, and a readable storage medium. Background Art

[0002] For shippers, accurate shipping schedule data plays a vital role in inventory management and production. Relatively stable shipping schedule data can reduce the amount of surplus cargo that needs to be prepared in advance to cope with fluctuations in the transportation cycle, thereby significantly reducing costs and improving capital utilization.

[0003] However, users currently obtain sailing schedule data directly from shipping companies. However, each company provides inconsistent descriptions of departure times and port calls for the same vessel, making it difficult for users to accurately determine which data is valid and which is misleading. Historically, it has been found that a large number of ports listed in sailing schedules do not correspond to actual port routes, and the departure times can vary by more than two weeks, making the sailing schedule data less reliable. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0005] To this end, the first aspect of the present invention is to propose a method for processing ship schedule data.

[0006] The second aspect of the present invention is to provide a device for processing ship schedule data.

[0007] A third aspect of the present invention is to provide an electronic device.

[0008] A fourth aspect of the present invention is to provide a readable storage medium.

[0009] In view of this, according to one aspect of the present invention, a method for processing ship schedule data is proposed, which includes: obtaining historical ship schedule data and real arrival and departure information; associating the historical ship schedule data with the real arrival and departure information; cleaning and deduplicating the historical ship schedule data based on the associated historical ship schedule data and the real arrival and departure information to obtain first valid ship schedule data; aggregating the first valid ship schedule data to obtain a ship schedule data statistical table; generating virtual arrival and departure information based on the ship schedule data statistical table; associating the virtual arrival and departure information with current ship schedule data; cleaning and deduplicating the current ship schedule data based on the associated virtual arrival and departure information and current ship schedule data to obtain second valid ship schedule data.

[0010] Specifically, in the method for processing ship schedule data provided by the present invention, when cleaning current ship schedule data, historical ship schedule data that has occurred and its corresponding real arrival and departure information are obtained, and the obtained historical ship schedule data and the real arrival and departure information are associated. Furthermore, based on the associated historical ship schedule data and the real arrival and departure information, the historical ship schedule data is cleaned and deduplicated, and invalid data such as abnormal data and duplicate data are removed to obtain first valid ship schedule data. Furthermore, the first valid ship schedule data is aggregated to obtain a ship schedule data statistical table, and then virtual arrival and departure information that has not occurred is estimated and generated based on the ship schedule data statistical table. Furthermore, the generated virtual arrival and departure information is associated with the current ship schedule data that has not yet occurred, and based on the associated virtual arrival and departure information and the current ship schedule data, the current ship schedule data is cleaned and deduplicated, and invalid data such as abnormal data and duplicate data in the current ship schedule data is removed to obtain second valid ship schedule data. In this way, by summarizing and counting the historical ship schedule data of actual arrivals and departures, recursively inferring the virtual arrival and departure information to be occurred, and then cleaning the current ship schedule data that has not occurred, specifically, based on the historical ship schedule data and the actual arrival and departure information, a ship schedule data statistical table is determined, and then based on the ship schedule data statistical table, virtual arrival and departure information is determined, and the current ship schedule data is cleaned, useful and reliable ship schedule data can be retained, thereby improving the credibility of the ship schedule data.

[0011] The method for processing the ship schedule data according to the present invention may also have the following additional technical features:

[0012] In some technical solutions, historical ship schedule data and actual arrival and departure information are optionally associated, including: standardizing the historical ship schedule data; sorting the actual arrival and departure information; and associating the standardized historical ship schedule data with the sorted actual arrival and departure information.

[0013] In this technical solution, when correlating historical schedule data with actual arrival and departure information, the historical schedule data is standardized and sorted in ascending order based on the actual arrival and departure times indicated by the actual arrival and departure information. Based on this, the standardized historical schedule data is then correlated with the sorted actual arrival and departure information. This ensures the accuracy of the data correlation and, by extension, the accuracy of subsequent data cleaning.

[0014] In some technical solutions, historical sailing schedule data may be optionally standardized, including: retaining the first type of characters and the second type of characters in the ship name information in the historical sailing schedule data, and unifying the first type of characters into a target writing format; retaining the ship company identification information in the ship name information; obtaining the longitude and latitude information corresponding to the port of call place name in the historical sailing schedule data; and determining the port of call name corresponding to the port of call place name based on the longitude and latitude information.

[0015] In this technical solution, during the standardization process of historical sailing schedule data, specifically, the ship name information and the port names in the historical sailing schedule data are standardized separately. Specifically, for the ship name information in the historical sailing schedule data, only the first type of characters and the second type of characters in the ship name information are retained, and the first type of characters are unified into the target writing format. On this basis, the ship operator keywords in the ship name information are standardized, and only the ship operator identification information in the ship name information is retained. Furthermore, for the port names in the historical sailing schedule data, a map search engine is called to obtain the latitude and longitude information corresponding to the port names, and then the name of the port closest to the port name is determined based on the latitude and longitude information. In this way, the ship name information and the port names in the historical sailing schedule data are standardized, ensuring the accuracy of subsequent data association, and thus ensuring the accuracy of subsequent data cleaning.

[0016] In some technical solutions, the standardized historical ship schedule data and the sorted actual arrival and departure information are optionally associated, including: according to the ship arrival and departure time indicated by the historical ship schedule data, the actual arrival and departure time indicated by the actual arrival and departure information, the standardized ship name information and the name of the port of call, the actual arrival and departure information is associated with the port of call that is closest in time sequence in the historical ship schedule data.

[0017] In this technical solution, when associating standardized historical schedule data with sorted actual arrival and departure information, the actual arrival and departure times are specifically linked to the port of call that is closest in time sequence to the historical schedule data, based on the arrival and departure times indicated by the historical schedule data, the actual arrival and departure times indicated by the actual arrival and departure information, and the standardized vessel name and port of call. This ensures the accuracy of data association and, by extension, subsequent data cleaning.

[0018] In some technical solutions, optionally, the historical ship schedule data is cleaned and deduplicated based on the associated historical ship schedule data and the actual arrival and departure information, including: determining the departure time deviation based on the difference between the actual departure time indicated by the actual arrival and departure information and the ship arrival time indicated by the historical ship schedule data; determining the arrival time deviation based on the difference between the actual arrival time indicated by the actual arrival and departure information and the ship arrival time; removing historical ship schedule data with a departure time deviation or an arrival time deviation greater than a preset time length; removing historical ship schedule data with an associated actual arrival time of the destination port that is earlier than the actual departure time of the departure port; and deduplicating historical ship schedule data with the same associated actual arrival and departure information.

[0019] In this technical solution, during the process of cleaning and deduplicating the historical ship schedule data based on the associated historical ship schedule data and the actual arrival and departure information, specifically, the departure time deviation is determined based on the difference between the actual departure time indicated by the actual arrival and departure information and the ship arrival time indicated by the historical ship schedule data, and the arrival time deviation is determined based on the difference between the actual arrival time indicated by the actual arrival and departure information and the ship arrival time. Furthermore, abnormal ship schedule data and duplicate ship schedule data in the historical ship schedule data are marked. Specifically, historical ship schedule data with a departure time deviation or arrival time deviation greater than a preset time length is marked as abnormal ship schedule data, and historical ship schedule data with an actual arrival time associated with the destination port earlier than the actual departure time associated with the departure port is marked as abnormal ship schedule data. Furthermore, historical ship schedule data with the same associated actual arrival and departure information is marked as duplicate ship schedule data. On this basis, abnormal schedule data in the historical schedule data is deleted. In addition, for duplicate schedule data in the historical schedule data, only the historical schedule data with the earliest release time is retained, thus obtaining the first valid schedule data. In this way, the historical schedule data is cleaned and deduplicated before subsequent processing, ensuring the accuracy of the subsequently generated virtual arrival and departure information, and thus ensuring the accuracy of the subsequent processing of the current schedule data.

[0020] In some technical solutions, the first valid ship schedule data is optionally aggregated to obtain a ship schedule data statistical table, including: performing a first aggregation process on the first valid ship schedule data according to the ship name information, ship number information and route code in the first valid ship schedule data to obtain multiple port string information; performing a second aggregation process on the multiple port string information according to the route code to obtain a ship schedule data statistical table.

[0021] In this technical solution, when aggregating the first valid sailing schedule data to generate a sailing schedule data statistical table, the first valid sailing schedule data is aggregated based on the vessel name, vessel number, and route code contained in the first valid sailing schedule data to generate multiple port clusters. Each port cluster includes multiple sub-ports within the cluster and their corresponding sailing times. Furthermore, a second aggregation process is performed on the multiple port clusters based on the route code to generate a sailing schedule data statistical table. This ensures the accuracy of the summary statistics of historical sailing schedule data that has actually arrived and departed from ports, and also ensures the accuracy of subsequent cleaning of current sailing schedule data based on the sailing schedule data statistical table.

[0022] In some technical solutions, optionally, virtual arrival and departure information is generated based on the ship schedule data statistics table, including: generating a port string corresponding to each route code based on the ship schedule data statistics table; determining the virtual port of call and its virtual arrival and departure time based on the current ship position, port string and sailing time in the ship schedule data statistics table to obtain virtual arrival and departure information.

[0023] In this technical solution, when generating virtual arrival and departure information from a vessel schedule data statistical table, a port string corresponding to each route code is generated from the vessel schedule data statistical table. Then, based on the generated port string, the current vessel position, and the sailing time corresponding to the sub-ports in each port string in the vessel schedule data statistical table, a virtual port of call and its virtual arrival and departure time are determined to obtain the virtual arrival and departure information. This ensures the accuracy of the virtual arrival and departure information, thereby ensuring the accuracy of subsequent cleaning of the current vessel schedule data based on the virtual arrival and departure information.

[0024] According to a second aspect of the present invention, a device for processing ship schedule data is proposed, which includes: an acquisition unit for acquiring historical ship schedule data and real arrival and departure information; a processing unit for associating the historical ship schedule data with the real arrival and departure information; the processing unit is further used to clean and de-duplicate the historical ship schedule data based on the associated historical ship schedule data and the real arrival and departure information, to obtain first valid ship schedule data; the processing unit is further used to aggregate the first valid ship schedule data to obtain a ship schedule data statistical table; the processing unit is further used to generate virtual arrival and departure information based on the ship schedule data statistical table; the processing unit is further used to associate the virtual arrival and departure information with current ship schedule data; the processing unit is further used to clean and de-duplicate the current ship schedule data based on the associated virtual arrival and departure information and current ship schedule data, to obtain second valid ship schedule data.

[0025] The present invention provides a device for processing ship schedule data, comprising an acquisition unit and a processing unit. When cleaning current ship schedule data, the acquisition unit acquires historical ship schedule data that has occurred and its corresponding actual arrival and departure information, and the processing unit associates the acquired historical ship schedule data with the actual arrival and departure information. Furthermore, the processing unit cleans and dedupes the historical ship schedule data based on the associated historical ship schedule data and actual arrival and departure information, removing invalid data such as abnormal data and duplicate data, thereby obtaining first valid ship schedule data. Furthermore, the processing unit aggregates the first valid ship schedule data to obtain a ship schedule data statistics table, and then estimates and generates virtual arrival and departure information that has not yet occurred based on the ship schedule data statistics table. Furthermore, the processing unit associates the generated virtual arrival and departure information with current ship schedule data that has not yet occurred, and, based on the associated virtual arrival and departure information and current ship schedule data, cleans and dedupes the current ship schedule data, removing invalid data such as abnormal data and duplicate data, thereby obtaining second valid ship schedule data. In this way, by summarizing and counting the historical ship schedule data of actual arrivals and departures, recursively inferring the virtual arrival and departure information to be occurred, and then cleaning the current ship schedule data that has not occurred, specifically, based on the historical ship schedule data and the actual arrival and departure information, a ship schedule data statistical table is determined, and then based on the ship schedule data statistical table, virtual arrival and departure information is determined, and the current ship schedule data is cleaned, useful and reliable ship schedule data can be retained, thereby improving the credibility of the ship schedule data.

[0026] According to a third aspect of the present invention, an electronic device is provided, comprising a processor and a memory. The memory stores a program or instruction executable by the processor. When executed by the processor, the program or instruction implements the steps of the method for processing sailing schedule data described in any of the aforementioned technical solutions. Therefore, the electronic device provided in the third aspect of the present invention possesses all the beneficial effects of the method for processing sailing schedule data described in any of the aforementioned technical solutions in the first aspect, and will not be further elaborated upon here.

[0027] According to a fourth aspect of the present invention, a readable storage medium is provided. The readable storage medium stores a program or instructions. When executed by a processor, the program or instructions implement the method for processing sailing schedule data described in any of the aforementioned technical solutions. Therefore, the readable storage medium provided in the fourth aspect of the present invention possesses all the beneficial effects of the method for processing sailing schedule data described in any of the aforementioned technical solutions in the first aspect, and will not be further elaborated upon here.

[0028] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0030] Figure 1 FIG1 shows a flow chart of a method for processing ship schedule data according to an embodiment of the present invention;

[0031] Figure 2 FIG2 is a second flow chart showing a method for processing ship schedule data according to an embodiment of the present invention;

[0032] Figure 3 FIG3 shows a flow chart of a method for processing ship schedule data according to an embodiment of the present invention;

[0033] Figure 4 FIG4 is a flow chart showing a method for processing ship schedule data according to an embodiment of the present invention;

[0034] Figure 5 FIG5 is a flowchart of a method for processing ship schedule data according to an embodiment of the present invention;

[0035] Figure 6 A structural block diagram of a device for processing ship schedule data according to an embodiment of the present invention is shown;

[0036] Figure 7 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0037] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features therein can be combined with each other without conflict.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0039] The following combination Figures 1 to 7 , through specific embodiments and their application scenarios, the method and device for processing ship schedule data, electronic equipment and readable storage medium provided in the embodiments of the present application are described in detail.

[0040] In one embodiment of the present invention, Figure 1 As shown, the method for processing the ship schedule data may specifically include the following steps 102 to 114:

[0041] Step 102: Obtain historical ship schedule data and actual arrival and departure information;

[0042] Step 104: Correlate historical ship schedule data with actual arrival and departure information;

[0043] Step 106: Based on the associated historical sailing schedule data and the actual arrival and departure information, the historical sailing schedule data is cleaned and deduplicated to obtain first valid sailing schedule data.

[0044] Step 108: Aggregate the first valid sailing schedule data to obtain a sailing schedule data statistics table;

[0045] Step 110, generating virtual arrival and departure information according to the ship schedule data statistics table;

[0046] Step 112, associating the virtual arrival and departure information with the current ship schedule data;

[0047] Step 114 : Based on the associated virtual arrival and departure information and the current sailing schedule data, the current sailing schedule data is cleaned and deduplicated to obtain second valid sailing schedule data.

[0048] When cleaning the current schedule data, historical schedule data that has occurred and its corresponding real arrival and departure information are obtained, and the obtained historical schedule data and the real arrival and departure information are associated. Furthermore, based on the associated historical schedule data and the real arrival and departure information, the historical schedule data is cleaned and deduplicated, and invalid data such as abnormal data and duplicate data are removed to obtain the first valid schedule data. Furthermore, the first valid schedule data is aggregated to obtain a schedule data statistics table, and then virtual arrival and departure information that has not occurred is estimated and generated based on the schedule data statistics table. Furthermore, the generated virtual arrival and departure information is associated with the current schedule data that has not yet occurred, and based on the associated virtual arrival and departure information and the current schedule data, the current schedule data is cleaned and deduplicated, and invalid data such as abnormal data and duplicate data in the current schedule data is removed to obtain the second valid schedule data. In this way, by summarizing and counting the historical ship schedule data of actual arrivals and departures, recursively inferring the virtual arrival and departure information to be occurred, and then cleaning the current ship schedule data that has not occurred, specifically, based on the historical ship schedule data and the actual arrival and departure information, a ship schedule data statistical table is determined, and then based on the ship schedule data statistical table, virtual arrival and departure information is determined, and the current ship schedule data is cleaned, useful and reliable ship schedule data can be retained, thereby improving the credibility of the ship schedule data.

[0049] In some embodiments of the present invention, optionally, as Figure 2 As shown, the above step 104 may specifically include the following steps 104a to 104c:

[0050] Step 104a, standardizing the historical ship schedule data;

[0051] Step 104b, sorting the actual arrival and departure information;

[0052] Step 104c: associate the standardized historical ship schedule data with the sorted actual arrival and departure information.

[0053] In this embodiment, the process of associating historical schedule data with actual arrival and departure information specifically involves normalizing the historical schedule data and sorting the actual arrival and departure information in ascending order based on the order of the actual arrival and departure times indicated by the actual arrival and departure information. On this basis, the standardized historical schedule data is then associated with the sorted actual arrival and departure information. This ensures the accuracy of data association, and thus the accuracy of subsequent data cleaning.

[0054] In some embodiments of the present invention, optionally, the above step 104a may specifically include the following steps 116 to 122:

[0055] Step 116, retaining the first type of characters and the second type of characters in the ship name information in the historical sailing schedule data, and unifying the first type of characters into the target writing format;

[0056] Step 118, retaining the ship company identification information in the ship name information;

[0057] Step 120, obtaining the latitude and longitude information corresponding to the port of call in the historical ship schedule data;

[0058] Step 122: Determine the name of the port of call corresponding to the port of call place name according to the latitude and longitude information.

[0059] In this embodiment, during the standardization process of historical sailing schedule data, the vessel name information and port names within the historical sailing schedule data are specifically standardized separately. Specifically, for the vessel name information within the historical sailing schedule data, only the first and second type characters within the vessel name information are retained, and the first type characters are unified into the target writing format. Furthermore, the ship operator keywords within the vessel name information are standardized, retaining only the ship operator identification information within the vessel name information. Furthermore, for the port names within the historical sailing schedule data, a map search engine is invoked to obtain the corresponding latitude and longitude information. Based on this latitude and longitude information, the name of the port closest to the port name is determined. In this way, the standardized processing of the vessel name information and port names within the historical sailing schedule data ensures the accuracy of subsequent data association and, consequently, the accuracy of subsequent data cleaning.

[0060] The first type of characters are English characters, the second type of characters are numeric characters, and the target writing format is uppercase English characters.

[0061] Furthermore, the ship company identification information is information in the ship name information that can effectively indicate the name of the ship company. For example, "COSCO" and "COSCO SHIPPING" are both unified into "COSCO".

[0062] In some embodiments of the present invention, optionally, the above step 104c may specifically include the following step 124:

[0063] Step 124 , based on the arrival and departure times of the ship indicated by the historical ship schedule data, the actual arrival and departure times indicated by the actual arrival and departure information, the standardized ship name information and the name of the port of call, the actual arrival and departure information is associated with the port of call that is closest in time sequence in the historical ship schedule data.

[0064] In this embodiment, when associating the standardized historical schedule data with the sorted actual arrival and departure information, the actual arrival and departure information is associated with the port of call that is closest in time sequence to the historical schedule data, based on the arrival and departure times indicated by the historical schedule data, the actual arrival and departure times indicated by the actual arrival and departure information, the standardized vessel name, and the port of call. This ensures the accuracy of data association, and thus the accuracy of subsequent data cleaning.

[0065] In practical applications, the Pandas library's time-series fuzzy association function can be used to perform time-series fuzzy association on standardized historical sailing schedule data and sorted actual arrival and departure information. Specifically, the Pandas library's "on" parameter is set to the scheduled arrival and departure times and actual arrival and departure times, respectively; the "by" parameter is set to the standardized vessel name and the standardized port of call; and the "direction" parameter is set to "nearest" to associate the data with the port of call that is closest in time series.

[0066] Among them, the Pandas library is a powerful open source data analysis and processing library.

[0067] Furthermore, in the Pandas library, the "on" parameter is used to specify the column or index level name used to merge two data frames; the "by" parameter is an important parameter in the groupby function, which is used to group data according to one or more columns for further data processing and analysis. The "by" parameter determines how to group the data; the "direction" parameter is used to specify the direction that should be considered when deleting missing values.

[0068] Similarly, when associating the virtual arrival and departure information with the current sailing schedule data, the "on" parameter of the Pandas library can be set to the sailing arrival and departure time and the virtual arrival and departure time, respectively, the "by" parameter can be set to the standardized vessel name and the standardized port of call, and the "direction" parameter can be set to "nearest." In other words, based on the sailing arrival and departure time indicated by the current sailing schedule data, the virtual arrival and departure time indicated by the virtual arrival and departure information, and the standardized vessel name and port of call in the current sailing schedule data, the virtual arrival and departure information is fuzzy-linked to the port of call with the closest time sequence in the current sailing schedule data.

[0069] In some embodiments of the present invention, optionally, the step of cleaning and deduplicating the historical sailing schedule data based on the associated historical sailing schedule data and the actual arrival and departure information may specifically include the following steps 126 to 134:

[0070] Step 126, determining the departure time deviation based on the difference between the actual departure time indicated by the actual arrival and departure information and the scheduled arrival time indicated by the historical ship schedule data;

[0071] Step 128, determining the arrival time deviation based on the difference between the actual arrival time indicated by the actual arrival and departure information and the scheduled arrival time of the vessel;

[0072] Step 130: remove historical ship schedule data with departure time deviation or arrival time deviation greater than a preset time period;

[0073] Step 132, remove the historical ship schedule data where the actual arrival time of the associated destination port is earlier than the actual departure time of the origin port;

[0074] Step 134 , deduplication processing is performed on the historical ship schedule data with the same associated real arrival and departure information.

[0075] In this embodiment, during the process of cleaning and deduplicating the historical schedule data based on the associated historical schedule data and actual arrival and departure information, specifically, a departure time deviation is determined based on the difference between the actual departure time indicated by the actual arrival and departure information and the scheduled arrival time indicated by the historical schedule data, and an arrival time deviation is determined based on the difference between the actual arrival time indicated by the actual arrival and departure information and the scheduled arrival time. Furthermore, abnormal and duplicate schedule data within the historical schedule data are marked. Specifically, historical schedule data with a departure time deviation or arrival time deviation greater than a preset duration is marked as abnormal schedule data. Furthermore, historical schedule data with an actual arrival time associated with the destination port earlier than the actual departure time associated with the origin port is marked as abnormal schedule data. Furthermore, historical schedule data with the same associated actual arrival and departure information is marked as duplicate schedule data. Based on this, the abnormal schedule data within the historical schedule data is deleted, and for duplicate historical schedule data, only the historical schedule with the earliest publication time is retained, thereby obtaining the first valid schedule data. In this way, the historical ship schedule data is cleaned and deduplicated before subsequent processing, which ensures the accuracy of the subsequently generated virtual arrival and departure information, and further ensures the accuracy of the subsequent processing of the current ship schedule data.

[0076] In actual application, the above-mentioned preset duration can be specifically 2 weeks, 15 days, etc. As for the specific value of the above-mentioned preset duration, those skilled in the art can set it according to actual conditions, and no specific limitation is made here.

[0077] Similarly, when cleaning and deduplicating the current sailing schedule data based on the associated virtual arrival and departure information and the current sailing schedule data to obtain the second valid sailing schedule data, the departure time deviation is determined based on the difference between the virtual departure time indicated by the virtual arrival and departure information and the arrival time of the ship indicated by the current sailing schedule data, and the arrival time deviation is determined based on the difference between the virtual arrival time indicated by the virtual arrival and departure information and the arrival time of the ship. Furthermore, current sailing schedule data with a departure time deviation or arrival time deviation greater than a preset duration is marked as abnormal sailing schedule data, and current sailing schedule data with a virtual arrival time associated with the destination port earlier than the virtual departure time associated with the departure port is marked as abnormal sailing schedule data. Furthermore, current sailing schedule data with the same associated virtual arrival and departure information is marked as duplicate sailing schedule data. Based on this, the abnormal sailing schedule data in the current sailing schedule data is deleted, and for duplicate sailing schedule data in the current sailing schedule data, only the current sailing schedule data with the earliest publication time is retained, thereby obtaining the second valid sailing schedule data.

[0078] In some embodiments of the present invention, optionally, as Figure 3 As shown, the above step 108 may specifically include the following steps 108a and 108b:

[0079] Step 108a, performing a first aggregation process on the first valid sailing schedule data according to the vessel name information, vessel number information, and route code in the first valid sailing schedule data to obtain a plurality of port string information;

[0080] Step 108b: performing a second aggregation process on the multiple port string information according to the route code to obtain a ship schedule data statistics table.

[0081] In this embodiment, when aggregating the first valid sailing schedule data to generate a sailing schedule data statistical table, the first valid sailing schedule data is aggregated based on the vessel name, vessel number, and route code contained in the first valid sailing schedule data to generate multiple port clusters. Each port cluster includes multiple sub-ports within the cluster and their corresponding sailing times. Furthermore, a second aggregation process is performed on the multiple port clusters based on the route code to generate the sailing schedule data statistical table. This ensures the accuracy of the summary statistics of historical sailing schedule data that has actually called or departed ports, and also ensures the accuracy of subsequent cleaning of current sailing schedule data based on the sailing schedule data statistical table.

[0082] The above-mentioned sailing schedule statistics table includes the frequency statistics of each port string within different route codes, the median sailing time corresponding to multiple sub-ports in the port string, and a set of deciles. Thus, by inputting a port code and a port string, you can obtain the frequency of the port string and the median and deciles range of the sailing time corresponding to the sub-ports in the port string.

[0083] In some embodiments of the present invention, optionally, as Figure 4 As shown, the above step 110 may specifically include the following steps 110a and 110b:

[0084] Step 110a, generating a port string corresponding to each route code according to the ship schedule data statistics table;

[0085] Step 110b, according to the current ship position, port string and sailing time in the ship schedule data statistics table, determine the virtual port of call and its virtual arrival and departure time, and obtain virtual arrival and departure information.

[0086] In this embodiment, when generating virtual arrival and departure information based on the vessel schedule data statistics table, specifically, a port string corresponding to each route code is generated based on the vessel schedule data statistics table. Then, based on the generated port string, the current vessel position, and the sailing time corresponding to the sub-ports in each port string in the vessel schedule data statistics table, a virtual port of call and its virtual arrival and departure time are determined to obtain the virtual arrival and departure information. This ensures the accuracy of the virtual arrival and departure information, thereby ensuring the accuracy of the subsequent cleaning of the current vessel schedule data based on the virtual arrival and departure information.

[0087] In summary, if Figure 5 As shown, the method for processing the ship schedule data provided by the present invention may specifically include the following steps 202 to 226:

[0088] Step 202: standardize the vessel name information in the historical sailing schedule data;

[0089] Step 204: standardize the port names in the historical ship schedule data;

[0090] Step 206: Perform time-series fuzzy correlation on the historical ship schedule data and the actual arrival and departure information;

[0091] Step 208, calculating the departure time deviation and the arrival time deviation;

[0092] Step 210, marking abnormal data in the historical sailing schedule data;

[0093] Step 212, marking duplicate data in the historical sailing schedule data;

[0094] Step 214: Clean and remove duplicates from the historical sailing schedule data to generate first valid sailing schedule data;

[0095] Step 216: performing an aggregation operation on the first valid sailing schedule data to obtain a sailing schedule data statistics table;

[0096] Step 218: Generate a virtual port of call and its virtual arrival and departure time according to the shipping schedule data statistics table;

[0097] Step 220, performing time series fuzzy association on the current ship schedule data and the virtual arrival and departure information;

[0098] Step 222, marking abnormal data in the current sailing schedule data;

[0099] Step 224, marking duplicate data in the current sailing schedule data;

[0100] Step 226: Clean and remove duplicates from the current sailing schedule data to generate second valid sailing schedule data.

[0101] Based on this, the method for processing ship schedule data provided by the present invention realizes the generation of reliable ship schedule data statistics tables by looking back at history, determines future virtual arrival and departure information based on the statistical results of historical ship schedule data that has occurred, and detects current repeated ship schedules and abnormal ship schedules based on virtual arrival and departure behaviors, thereby improving the credibility of ship schedule data.

[0102] In one embodiment of the present invention, a device for processing ship schedule data is also provided. Figure 6 As shown, Figure 6The structure block diagram of the vessel schedule data processing device 300 according to an embodiment of the present invention is shown. Specifically, the vessel schedule data processing device 300 may include the following acquisition unit 302 and processing unit 304:

[0103] An acquisition unit 302 is used to acquire historical ship schedule data and actual arrival and departure information;

[0104] Processing unit 304, used to associate historical ship schedule data with actual arrival and departure information;

[0105] The processing unit 304 is further configured to clean and remove duplicates from the historical sailing schedule data based on the associated historical sailing schedule data and the actual arrival and departure information to obtain first valid sailing schedule data;

[0106] The processing unit 304 is further configured to aggregate the first valid sailing schedule data to obtain a sailing schedule data statistics table;

[0107] The processing unit 304 is further configured to generate virtual arrival and departure information according to the ship schedule data statistics table;

[0108] The processing unit 304 is further configured to associate the virtual arrival and departure information with the current ship schedule data;

[0109] The processing unit 304 is further configured to clean and remove duplicates from the current sailing schedule data based on the associated virtual arrival and departure information and the current sailing schedule data to obtain second valid sailing schedule data.

[0110] The apparatus 300 for processing ship schedule data provided in an embodiment of the present invention includes an acquisition unit 302 and a processing unit 304. When cleaning current ship schedule data, the acquisition unit 302 acquires historical ship schedule data and its corresponding actual arrival and departure information, and the processing unit 304 correlates the acquired historical ship schedule data with the actual arrival and departure information. Furthermore, the processing unit 304 cleans and deduplicates the historical ship schedule data based on the correlated historical ship schedule data and the actual arrival and departure information, removing invalid data such as abnormal and duplicate data to obtain first valid ship schedule data. Furthermore, the processing unit 304 aggregates the first valid ship schedule data to obtain a ship schedule data statistical table, and then generates virtual arrival and departure information that has not yet occurred based on the ship schedule data statistical table. Furthermore, processing unit 304 associates the generated virtual arrival and departure information with the current vessel schedule data that has not yet occurred, and based on the associated virtual arrival and departure information and the current vessel schedule data, cleans and deduplicates the current vessel schedule data, removing invalid data such as abnormal data and duplicate data from the current vessel schedule data to obtain second valid vessel schedule data. In this way, by summarizing and statistically analyzing the historical vessel schedule data of actual arrivals and departures, recursively inferring the virtual arrival and departure information that is about to occur, and then cleaning the current vessel schedule data that has not yet occurred, specifically, a vessel schedule data statistical table is determined based on the historical vessel schedule data and the actual arrival and departure information, and then the virtual arrival and departure information is determined based on the vessel schedule data statistical table, and the current vessel schedule data is cleaned. This allows the retention of useful and reliable vessel schedule data, thereby improving the credibility of the vessel schedule data.

[0111] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: standardize the historical ship schedule data; sort the actual arrival and departure information; and associate the standardized historical ship schedule data with the sorted actual arrival and departure information.

[0112] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: retain the first type of characters and the second type of characters in the ship name information in the historical sailing schedule data, and unify the first type of characters into a target writing format; retain the ship company identification information in the ship name information; obtain the longitude and latitude information corresponding to the port of call place name in the historical sailing schedule data; determine the port of call name corresponding to the port of call place name based on the longitude and latitude information.

[0113] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: associate the actual arrival and departure information with the port of call that is closest in time sequence in the historical ship schedule data based on the ship arrival and departure time indicated by the historical ship schedule data, the actual arrival and departure time indicated by the actual arrival and departure information, the standardized ship name information and the name of the port of call.

[0114] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: determine the departure time deviation based on the difference between the actual departure time indicated by the actual arrival and departure information and the ship arrival time indicated by the historical ship schedule data; determine the arrival time deviation based on the difference between the actual arrival time indicated by the actual arrival and departure information and the ship arrival time; remove historical ship schedule data with a departure time deviation or an arrival time deviation greater than a preset time length; remove historical ship schedule data with an associated actual arrival time of the destination port that is earlier than the actual departure time of the departure port; and deduplicate historical ship schedule data with the same associated actual arrival and departure information.

[0115] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: perform a first aggregation processing on the first valid shipping schedule data according to the ship name information, ship number information and route code in the first valid shipping schedule data to obtain multiple port string information; perform a second aggregation processing on the multiple port string information according to the route code to obtain a shipping schedule data statistical table.

[0116] In some embodiments of the present invention, optionally, the processing unit 304 is specifically used to: generate a port string corresponding to each route code according to the ship schedule data statistics table; determine the virtual port of call and its virtual arrival and departure time based on the current ship position, port string and sailing time in the ship schedule data statistics table, and obtain virtual arrival and departure information.

[0117] In one embodiment of the present invention, an electronic device is also provided. Figure 7 As shown, Figure 7 FIG. 4 is a block diagram of an electronic device 400 according to an embodiment of the present invention. The electronic device 400 includes:

[0118] Memory 402 , where programs or instructions are stored;

[0119] The processor 404 implements the steps of the method for processing the ship schedule data in any of the above embodiments when executing the above program or instruction.

[0120] The electronic device 400 provided in this embodiment includes a memory 402 and a processor 404. When the program or instructions in the memory 402 are executed by the processor 404, the steps of the method for processing ship schedule data in any of the above embodiments are implemented. Therefore, the electronic device 400 has all the beneficial effects of the method for processing ship schedule data in any of the above embodiments, which will not be repeated here.

[0121] Specifically, the memory 402 and the processor 404 may be connected via a bus or other means. The processor 404 may include one or more processing units, and the processor 404 may be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other chips.

[0122] In one embodiment of the present invention, a readable storage medium is further provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method for processing the ship schedule data in any of the above embodiments are implemented.

[0123] The readable storage medium provided in an embodiment of the present invention, when executed by a processor, stores a program or instructions that implement the steps of the method for processing sailing schedule data described in any of the aforementioned embodiments. Therefore, the readable storage medium possesses all the beneficial effects of the method for processing sailing schedule data described in any of the aforementioned embodiments, and further description thereof will not be given here.

[0124] Specifically, the readable storage medium may include any medium capable of storing or transmitting information. Examples of readable storage media include electronic circuits, semiconductor memory devices, read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), flash memory, erasable ROM (EROM), magnetic tape, floppy disk, optical disc, hard disk, optical fiber, radio frequency (RF) link, optical data storage device, etc. The code segments may be downloaded via a computer network such as the Internet or an intranet.

[0125] In this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified or limited. Terms such as "connect," "install," and "fix" should be interpreted broadly. For example, "connect" can refer to a fixed connection, a detachable connection, or an integral connection; and can be directly connected or indirectly connected through an intermediary. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0126] Throughout this specification, terms such as "one embodiment," "some embodiments," and "specific embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0127] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0128] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for processing ship schedule data, characterized in that: include: Obtain historical ship schedule data and actual arrival and departure information; Associating the historical ship schedule data with the actual arrival and departure information; According to the associated historical sailing schedule data and the actual arrival and departure information, the historical sailing schedule data is cleaned and deduplicated to obtain first valid sailing schedule data; Aggregating the first valid sailing schedule data to obtain a sailing schedule data statistics table; Generate virtual arrival and departure information according to the ship schedule data statistics table; Associating the virtual arrival and departure information with current ship schedule data; According to the associated virtual arrival and departure information and the current sailing schedule data, the current sailing schedule data is cleaned and deduplicated to obtain second valid sailing schedule data.

2. The method for processing ship schedule data according to claim 1, characterized in that: The associating the historical ship schedule data with the actual arrival and departure information includes: Standardizing the historical ship schedule data; sorting the actual arrival and departure information; The standardized historical ship schedule data and the sorted actual arrival and departure information are associated.

3. The method for processing ship schedule data according to claim 2, characterized in that: The standardization processing of the historical ship schedule data includes: retaining the first type of characters and the second type of characters in the ship name information in the historical sailing schedule data, and unifying the first type of characters into a target writing format; Retain the ship company identification information in the ship name information; Obtaining the longitude and latitude information corresponding to the port of call in the historical ship schedule data; The name of the port of call corresponding to the port of call place name is determined according to the latitude and longitude information.

4. The method for processing ship schedule data according to claim 3, characterized in that: The associating the standardized historical ship schedule data with the sorted actual arrival and departure information includes: According to the arrival and departure time of the ship indicated by the historical ship schedule data, the actual arrival and departure time indicated by the actual arrival and departure information, the standardized ship name information and the name of the port of call, the actual arrival and departure information is associated with the port of call that is closest in time sequence in the historical ship schedule data.

5. The method for processing ship schedule data according to any one of claims 1 to 4, characterized in that: The step of cleaning and removing duplicates from the historical ship schedule data based on the associated historical ship schedule data and the actual arrival and departure information includes: Determining a departure time deviation based on a difference between the actual departure time indicated by the actual arrival and departure information and the ship arrival time indicated by the historical ship schedule data; Determining the arrival time deviation according to the difference between the actual arrival time indicated by the actual arrival and departure information and the scheduled arrival time of the vessel; Remove the historical ship schedule data whose departure time deviation or arrival time deviation is greater than a preset time period; Remove the historical sailing schedule data where the actual arrival time of the associated destination port is earlier than the actual departure time of the origin port; Deduplication processing is performed on the historical ship schedule data with the same associated real arrival and departure information.

6. The method for processing ship schedule data according to any one of claims 1 to 4, characterized in that: The aggregation processing of the first valid sailing schedule data to obtain a sailing schedule data statistical table includes: performing first aggregation processing on the first valid sailing schedule data according to the vessel name information, the vessel number information, and the route code in the first valid sailing schedule data to obtain a plurality of port string information; A second aggregation process is performed on the plurality of port string information according to the route code to obtain the shipping schedule data statistics table.

7. The method for processing ship schedule data according to any one of claims 1 to 4, characterized in that: The generating of virtual arrival and departure information according to the ship schedule data statistics table includes: Generate a port string corresponding to each route code according to the shipping schedule data statistics table; According to the current ship position, the port string and the sailing time in the ship schedule data statistics table, a virtual port of call and its virtual arrival and departure time are determined to obtain the virtual arrival and departure information.

8. A device for processing ship schedule data, characterized in that: include: Acquisition unit, used to obtain historical ship schedule data and actual arrival and departure information; A processing unit, configured to associate the historical ship schedule data with the actual arrival and departure information; The processing unit is further configured to clean and remove duplicates from the historical sailing schedule data based on the associated historical sailing schedule data and the actual arrival and departure information to obtain first valid sailing schedule data; The processing unit is further configured to aggregate the first valid sailing schedule data to obtain a sailing schedule data statistical table; The processing unit is further configured to generate virtual arrival and departure information according to the ship schedule data statistics table; The processing unit is further configured to associate the virtual arrival and departure information with current ship schedule data; The processing unit is further configured to clean and remove duplicates from the current sailing schedule data based on the associated virtual arrival and departure information and the current sailing schedule data to obtain second valid sailing schedule data.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method for processing ship schedule data according to any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method for processing ship schedule data according to any one of claims 1 to 7 are implemented.

Images