Key liner shipping routes identification method based on word2vec model

Abstract

The present invention discloses a method for identifying key routes of liner transportation based on a word2vec model, which belongs to the field of water transportation and includes the following steps: step one: obtaining AIS data and preprocessing; step two: obtaining a sequence of container liner voyage ports; step three: generating port vectors based on the word2vec model; step four: obtaining container liner transportation voyage vectors; step five: identifying key routes of container liner transportation. The present invention uses port data to simplify the calculation content of key route identification, is more suitable for a large-scale international liner transportation network, and deeply explores the potential associations between ports, thereby realizing the identification of key routes of container liner transportation. The present invention helps traffic management departments and shipping companies to accurately grasp the key routes of container liner transportation, and provides strong support for traffic management departments to formulate management policies and shipping companies to plan and manage sea routes.

Description

Technical Field

[0001] The invention belongs to the field of water transport, and in particular relates to a method for identifying key routes of liner transport based on a word2vec model. Background Art

[0002] Thanks to the gradual advancement of global economic integration and the advantages of container transportation, which are safe and convenient, container liner transportation is becoming increasingly important in international trade. Although the routes of container liner transportation are often fixed, the international container liner network composed of different liner routes operated by different companies around the world is still huge and complex. Therefore, how to identify key routes in the complex international container liner network is of great reference value for route management and the operation of liner companies.

[0003] Current research on the identification of key container liner routes mainly starts from a microscopic perspective, clustering ship trajectory data based on trajectory similarity and ultimately obtaining key routes. However, this method has high requirements for ship trajectory data and is only applicable to the identification of key routes in a small range. In the international container liner network, the magnitude of liner ship trajectory data increases exponentially, and clustering trajectory data over a large range of the world requires a lot of computing resources. At the same time, the identification method based on trajectory similarity mainly uses the density and frequency of ship trajectories in the region for clustering, but fails to take into account the correlation between the ports of call before and after the liner, resulting in a gap between the identification results and the actual key container liner shipping routes.

[0004] In order to identify the key routes that consider the correlation between ports of call in the container liner shipping network, the word2vec model is used to train the port data of ships in the complex network, and the segment vectors generated after training are obtained, and the key routes of container liner shipping are identified by combining the clustering method. This method uses the port of call data, which greatly simplifies the calculation content and is more suitable for the large-scale container liner shipping network. At the same time, this method uses the word2vec model to convert the potential connection between ports before and after the call into the correlation between vectors, so that the clustering results are not only the overlapping routes in a certain area, but also the global routes that consider the overall correlation in the network. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the deficiencies of the above-mentioned prior art and propose a method for identifying key routes of container liner transportation.

[0006] The technical solution of the present invention is:

[0007] A method for identifying key liner shipping routes based on a word2vec model comprises the following steps:

[0008] Step 1: Obtain AIS data and preprocess it

[0009] The AIS data in this step is obtained from the actual voyage of the container liner. The original AIS data is first decoded using the AIS decoding algorithm, and then the required AIS data is extracted. The AIS data includes the ship name, IMO number, longitude, latitude, time, sailing speed, etc. Data preprocessing includes data cleaning, data completion and port identification.

[0010] Step 2: Get the container liner voyage port sequence

[0011] Arrange the ports of call of each container liner by voyage and call sequence, and convert all ports into one-hot encoding to obtain the port sequence of the container liner. One-hot encoding is a vector of length N, where N is the total number of ports that appear in all sequences, expressed as v = {0,0,…,1,…,0,0}. In the one-hot encoding, only the position corresponding to the port is 1, and the rest are all 0. Each port corresponds to a non-repeating one-hot encoding. Convert the ports of call of all voyages into one-hot encoding in order, and you will get the port sequence of each voyage: V i = {v i1 ,v i2 ,…,v ij}, where V i represents the port sequence of the i-th voyage, v ij Represents the one-hot encoding of the jth port of call in the i-th voyage.

[0012] Step 3: Generate port vectors based on word2vec model

[0013] In this step, the sequence of container liner voyage ports obtained in step 2 is input into the word2vec model, and then the trained port vector is obtained. The model is divided into three layers: input layer, projection layer and output layer. In the input layer, each port in the port sequence is taken as the central port in turn, and the model input is the unique hot encoding of the K ports before and after each central port, where K is the window size. The projection layer accumulates and sums the 2K input unique hot encodings. The connection edge from the projection layer to the output layer is an N*M weight matrix, where M represents the length of the port vector to be obtained. The vector accumulated by the projection layer is multiplied by the weight matrix, and the result is input into the output layer. The output layer of the model is a softmax regression function, which is used to calculate the probability P of 2K input ports generating a central port. The calculation formula is as follows:

[0014]

[0015] In the formula, x ijrepresents the jth port of call in the i-th voyage, i.e., the central port, x k represents the kth port among the ports input to the model, v ij For port x ij One-hot encoding of k For port x k The unique hot encoding of , D is the length of the port sequence of the voyage, that is, the total number of times the liner calls at the port in the voyage.

[0016] The goal of the model is to maximize the objective function, which is usually taken as the following log-likelihood function:

[0017]

[0018] Where C is the number of all voyages in the data.

[0019] After the model training is completed, the final weight matrix is ​​obtained. The port’s one-hot encoding is multiplied by the weight matrix to obtain a port vector t with a length of M.

[0020] Step 4: Obtain container liner shipping voyage vector

[0021] The port vector is converted into a voyage vector using the TFIDF (term frequency-Inverse Document Frequency) weighting method. The TFIDF weighting method uses the frequency of a port's travel in the container liner network to represent the importance of the port. The higher the importance of the port, the higher the weight of the port vector in the voyage vector. The TFIDF weighting method can further strengthen the important influence of hub ports in container liner voyages. The formula for the TFIDF weighting method is as follows:

[0022]

[0023] Where TF ij is the frequency of the jth port of call in the i-th voyage, f ij is the frequency of the jth port of call in the i-th voyage in the data, D i is the length of the port sequence of the i-th voyage.

[0024]

[0025] In the formula, IDF ij is the reverse frequency of the jth port of call in the i-th voyage, h ij is the number of voyages in the sequence that contain this port.

[0026] w ij =TF ij *IDF ij

[0027] In the formula, w ij is the TFIDF weight of the jth port of call in the i-th voyage.

[0028] The port vectors t in each voyage are accumulated using the TFIDF weighted method to obtain the voyage vector u of container liner transportation:

[0029]

[0030] In the formula, u i is the voyage vector of the i-th voyage, t ij The port vector representing the jth port of call in the i-th voyage.

[0031] Step 5: Identify key container liner shipping routes

[0032] The voyage vectors obtained in step 4 are clustered to obtain the final key container liner shipping routes. The clustering steps are as follows:

[0033] 1) The N-dimensional vector of the voyage vector is regarded as a point in the N-dimensional space, and each dimension of the N-dimensional space is divided equally, and finally the entire space is divided into a number of mutually non-intersecting grids, and the initial state of each grid is "unmarked";

[0034] 2) Calculate the density of each grid, and determine whether the grid is a dense grid according to the given density threshold δ. First calculate the dense grid in the low dimension. If the grid in the n-1 dimensional space is dense, then the grid corresponding to the data in the n-dimensional space may be dense. Conversely, if the grid in the n-1 dimension is not dense, then it is definitely not dense in the n-dimensional space. Then calculate the possible dense grids in the higher dimension until the last dimension.

[0035] 3) Traverse all grids. If the grid is in the "unmarked" state, go to 4). If the grid is in the "marked" state, continue to process the next grid. If all grids are "marked", go to 8);

[0036] 4) Change the grid state to "labeled". If the grid is a non-dense grid, return to 3). If the grid is a dense grid, give the grid a new cluster label, create a set, and put the dense grid into the set.

[0037] 5) Determine whether all grids in the set have been checked. If there are unchecked grids, go to 6); if all grids have been checked, go to 7);

[0038] 6) Check all adjacent and "unmarked" grids of the grid, change the status of the dense grids to "marked", assign the current cluster mark, add them to the set, and go to 5);

[0039] 7) After the inspection is completed, the dense grids in the set are grouped into target clusters, the cluster marks are modified, and the process returns to 3);

[0040] 8) After traversing all grids, output the target cluster.

[0041] The clusters obtained after clustering are the key routes of container liner transportation.

[0042] The technical effects of the present invention are:

[0043] The present invention uses the port data to simplify the calculation content of key route identification, is more suitable for the vast international liner shipping network, and deeply explores the potential relationship between ports, thereby realizing the identification of key routes for container liner shipping. The present invention helps traffic management departments and shipping companies to accurately grasp the key routes for container liner shipping, and provides strong support for traffic management departments to formulate management policies and shipping companies to plan and manage sea routes. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is an overall flow chart of the identification method provided by the present invention.

[0045] Figure 2 This is a flowchart of the present invention for clustering voyage vectors. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0047] Example 1.

[0048] like Figure 1 , 2 As shown, a method for identifying key liner shipping routes based on a word2vec model comprises the following steps:

[0049] Step 1: Obtain AIS data and preprocess it

[0050] The AIS data in this step is obtained from the actual voyage of the container liner. The original AIS data is first decoded using the AIS decoding algorithm, and then the required AIS data is extracted. The AIS data includes the ship name, IMO number, longitude, latitude, time, sailing speed, etc. Data preprocessing includes data cleaning, data completion and port identification.

[0051] First, the AIS data is cleaned, the date, number and other data formats are unified, duplicate information is deleted, unreasonable values ​​are removed, and obviously erroneous data are corrected.

[0052] Secondly, the AIS data is supplemented by inserting and filling in missing data based on other data sources such as shore-based and satellite-based data as well as the ship's previous and subsequent related data.

[0053] Finally, the ship is identified at the port of call. Using the position, time and speed information of AIS and the port position information, when the ship's stay time within the port's call range and the ship's speed meet the call constraints, the ship is identified as being called at the port.

[0054] Step 2: Get the container liner voyage port sequence

[0055] Arrange the ports of call of each container liner by voyage and call sequence, and convert all ports into one-hot encoding to obtain the port sequence of the container liner. One-hot encoding is a vector of length N, where N is the total number of ports that appear in all sequences, expressed as v = {0,0,…,1,…,0,0}. In the one-hot encoding, only the position corresponding to the port is 1, and the rest are all 0. Each port corresponds to a non-repeating one-hot encoding. Convert the ports of call of all voyages into one-hot encoding in order, and you will get the port sequence of each voyage: V i = {v i1 ,v i2 ,…,v ij}, where V i represents the port sequence of the i-th voyage, v ij Represents the one-hot encoding of the jth port of call in the i-th voyage.

[0056] Step 3: Generate port vectors based on word2vec model

[0057] In this step, the sequence of container liner voyage ports obtained in step 2 is input into the word2vec model, and then the trained port vector is obtained. The model is divided into three layers: input layer, projection layer and output layer. In the input layer, each port in the port sequence is taken as the central port in turn, and the model input is the unique hot encoding of the K ports before and after each central port, where K is the window size. The projection layer accumulates and sums the 2K input unique hot encodings. The connection edge from the projection layer to the output layer is an N*M weight matrix, where M represents the length of the port vector to be obtained. The vector accumulated by the projection layer is multiplied by the weight matrix, and the result is input into the output layer. The output layer of the model is a softmax regression function, which is used to calculate the probability P of 2K input ports generating a central port. The calculation formula is as follows:

[0058]

[0059] In the formula, x ij represents the jth port of call in the i-th voyage, i.e., the central port, x k represents the kth port among the ports input to the model, v ij For port x ij One-hot encoding of k For port x k The unique hot encoding of , D is the length of the port sequence of the voyage, that is, the total number of times the liner calls at the port in the voyage.

[0060] The goal of the model is to maximize the objective function, which is usually taken as the following log-likelihood function:

[0061]

[0062] Where C is the number of all voyages in the data.

[0063] After the model training is completed, the final weight matrix is ​​obtained. The port's one-hot encoding is multiplied by the weight matrix to obtain a port vector with a length of M.

[0064] Step 4: Obtain container liner shipping voyage vector

[0065] The port vector is converted into a voyage vector using the TFIDF (term frequency-Inverse Document Frequency) weighting method. The TFIDF weighting method uses the frequency of a port's travel in the container liner network to represent the importance of the port. The higher the importance of the port, the higher the weight of the port vector in the voyage vector. The TFIDF weighting method can further strengthen the important influence of hub ports in container liner voyages. The formula for the TFIDF weighting method is as follows:

[0066]

[0067] Where TF ij is the frequency of the jth port of call in the i-th voyage, f ij is the frequency of the jth port of call in the i-th voyage in the data, D i is the length of the port sequence of the i-th voyage.

[0068]

[0069] In the formula, IDF ij is the reverse frequency of the jth port of call in the i-th voyage, h ij is the number of voyages in the sequence that contain this port.

[0070] w ij =TF ij *IDF ij

[0071] In the formula, w ij is the TFIDF weight of the jth port of call in the i-th voyage.

[0072] The port vector t in each voyage is weighted and accumulated to obtain the voyage vector u of container liner transportation:

[0073]

[0074] In the formula, u i is the voyage vector of the i-th voyage, t ij The port vector representing the jth port of call in the i-th voyage.

[0075] Step 5: Identify key container liner shipping routes

[0076] The voyage vectors obtained in step 4 are clustered to obtain the final key container liner shipping routes. The clustering steps are as follows:

[0077] 1) The N-dimensional vector of the voyage vector is regarded as a point in the N-dimensional space, and each dimension of the N-dimensional space is divided equally, and finally the entire space is divided into a number of mutually non-intersecting grids, and the initial state of each grid is "unmarked";

[0078] 2) Calculate the density of each grid, and determine whether the grid is a dense grid according to the given density threshold δ. First calculate the dense grid in the low dimension. If the grid in the n-1 dimensional space is dense, then the grid corresponding to the data in the n-dimensional space may be dense. Conversely, if the grid in the n-1 dimension is not dense, then it is definitely not dense in the n-dimensional space. Then calculate the possible dense grids in the higher dimension until the last dimension.

[0079] 3) Traverse all grids. If the grid is in the "unmarked" state, go to 4). If the grid is in the "marked" state, continue to process the next grid. If all grids are "marked", go to 8);

[0080] 4) Change the grid state to "labeled". If the grid is a non-dense grid, return to 3). If the grid is a dense grid, give the grid a new cluster label, create a set, and put the dense grid into the set.

[0081] 5) Determine whether all grids in the set have been checked. If there are unchecked grids, go to 6); if all grids have been checked, go to 7);

[0082] 6) Check all adjacent and "unmarked" grids of the grid, change the status of the dense grids to "marked", assign the current cluster mark, add them to the set, and go to 5);

[0083] 7) After the inspection is completed, the dense grids in the set are grouped into target clusters, the cluster marks are modified, and the process returns to 3);

[0084] 8) After traversing all grids, output the target cluster.

[0085] Each cluster obtained after clustering constitutes a key route for container liner transportation.

[0086] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying key liner shipping routes based on word2vec model, characterized in that: The following steps are involved: Step 1: Obtain AIS data and preprocess it; Step 2: Get the port sequence of container liner voyages; Step 3: Generate port vector based on word2vec model; Step 4: Obtain container liner shipping voyage vector; Step 5: Identify key routes for container liner transportation: Cluster the voyage vectors obtained in step 4, and the clusters obtained after clustering are the key routes for container liner transportation; The specific steps of obtaining the container liner voyage port sequence described in step 2 are as follows: Arrange the ports of call of each container liner by voyage and call sequence, and convert all ports into one-hot encoding to obtain the port sequence of the container liner. The one-hot encoding is a vector of length N, where N is the total number of ports appearing in all sequences, expressed as v = {0, 0, ..., 1, ..., 0, 0}. In the one-hot encoding, only the position corresponding to the port is 1, and the rest are all 0. Each port corresponds to a non-repeating one-hot encoding. Convert all the ports of call of the voyage into one-hot encoding in order, and obtain the port sequence of each voyage: V i = {v i1 ,v i2 ,…,v ij }, where V i represents the port sequence of the i-th voyage, v ij represents the one-hot encoding of the jth port of call in the i-th voyage; The specific steps of generating port vectors based on the word2vec model in step 3 are as follows: The container liner voyage port sequence obtained in step 2 is input into the word2vec model, and then the trained port vector is obtained; the model is divided into three layers: input layer, projection layer and output layer; in the input layer, each port in the port sequence is taken as the central port in turn, and the model input is the unique hot encoding of the K ports before and after each central port, K ​​is the window size; the projection layer accumulates and sums the 2K input unique hot encodings; the connection edge from the projection layer to the output layer is an N*M weight matrix, M represents the length of the port vector to be obtained, the vector accumulated by the projection layer is multiplied by the weight matrix, and the result is input into the output layer; the output layer of the model is a softmax regression function, which is used to calculate the probability P of the 2K input ports generating the central port; The specific steps for obtaining the container liner shipping voyage vector described in step 4 are as follows: The TFIDF weighted method is used to convert the port vector into a voyage vector. The TFIDF weighted method uses the frequency of a port's travel in the container liner network to represent the importance of the port. The higher the importance of the port, the higher the weight of the port vector in the voyage vector.

2. The method for identifying key liner shipping routes based on the word2vec model according to claim 1, characterized in that: The AIS data described in step 1 is obtained from the actual voyage of the container liner. The original AIS data is first decoded using the AIS decoding algorithm, and then the required AIS data is extracted. The AIS data includes the ship name, IMO number, longitude, latitude, time, and sailing speed. Data preprocessing includes data cleaning, data completion, and port identification.

3. The method for identifying key liner shipping routes based on the word2vec model according to claim 2 is characterized in that: The probability P of generating a central port from 2K input ports is calculated as follows: In the formula, x ij represents the jth port of call in the i-th voyage, i.e., the central port, x k represents the kth port among the ports input to the model, v ij For port x ij One-hot encoding of k For port x k The unique hot encoding of , D is the length of the port sequence of the voyage, that is, the total number of port calls of the liner in the voyage; The goal of the model is to maximize the objective function, which is the following log-likelihood function: Where C is the number of all voyages in the data; After the model training is completed, the final weight matrix is ​​obtained. The port’s one-hot encoding is multiplied by the weight matrix to obtain a port vector t with a length of M.

4. The method for identifying key liner shipping routes based on the word2vec model according to claim 3 is characterized in that: The TFIDF weighted method formula is as follows: Where TF ij is the frequency of the jth port of call in the i-th voyage, f ij is the frequency of the jth port of call in the i-th voyage in the data, D i is the length of the port sequence of the i-th voyage; In the formula, IDF ij is the reverse frequency of the jth port of call in the i-th voyage, h ij is the number of voyages that include the port in the sequence; w ij =TF ij *IDF ij In the formula, w ij is the TFIDF weight of the jth port of call in the i-th voyage; The port vectors t in each voyage are accumulated using the TFIDF weighted method to obtain the voyage vector u of container liner transportation: In the formula, u i is the voyage vector of the i-th voyage, t ij The port vector representing the jth port of call in the i-th voyage.

5. The method for identifying key liner shipping routes based on the word2vec model according to claim 4 is characterized in that: Step 5 clusters the voyage vectors obtained in step 4. The clustering steps are as follows: 1) The N-dimensional vector of the voyage vector is regarded as a point in the N-dimensional space, and each dimension of the N-dimensional space is divided equally, and finally the entire space is divided into a number of mutually non-intersecting grids, and the initial state of each grid is "unmarked"; 2) Calculate the density of each grid, and determine whether the grid is a dense grid according to the given density threshold δ. First calculate the dense grid in the low dimension. If the grid in the n-1 dimensional space is dense, then the grid corresponding to the data in the n-dimensional space may be dense. Conversely, if the grid in the n-1 dimension is not dense, then it is definitely not dense in the n-dimensional space. Then calculate the possible dense grids in the higher dimension until the last dimension. 3) Traverse all grids. If the grid is in the "unmarked" state, go to 4). If the grid is in the "marked" state, continue to process the next grid. If all grids are "marked", go to 8); 4) Change the grid state to "labeled". If the grid is a non-dense grid, return to 3). If the grid is a dense grid, assign a new cluster label to the grid, create a set, and put the dense grid into the set. 5) Determine whether all grids in the set have been checked. If there are unchecked grids, go to 6); if all grids have been checked, go to 7); 6) Check all adjacent and "unmarked" grids of the grid, change the status of the dense grids to "marked", assign the current cluster mark, add them to the set, and go to 5); 7) After the inspection is completed, the dense grids in the set are grouped into target clusters, the cluster marks are modified, and the process returns to 3); 8) After traversing all grids, output the target cluster.

Images