Transform-based ship water transport volume prediction method

Abstract

The invention discloses a ship water transport volume prediction method based on a Transform. The method comprises the following steps: acquiring historical data related to predicted ship water transport volume; extracting multi-scale time features from historical data by using a pre-trained adaptive multi-scale time convolutional network; a pre-trained graph neural network is utilized to extract spatial dimension features from the historical data, and the spatial dimension features are used for representing the spatial dependency relationship between the wharf and the port; performing feature fusion processing on the extracted multi-scale time features and spatial dimension features to obtain multi-scale spatial-temporal features; and according to the multi-scale spatial-temporal characteristics, a pre-trained Transform model is utilized to carry out ship water transport volume prediction. The method can achieve the precise prediction of the ship water transport volume, and is good in accuracy and robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of ship water transport volume prediction, and in particular relates to a ship water transport volume prediction method based on Transformer. Background Art

[0002] Ship water transport volume forecast plays an important role in port scheduling, route optimization, carbon emission control, and government policy making. First, by predicting ship water transport volume, shipping companies can reasonably arrange ship scheduling plans, optimize route design, avoid idle or overused ships, improve transportation efficiency, and reduce operating risks. Secondly, water transport volume forecast can provide a scientific basis for formulating transportation policies and planning infrastructure construction, and promote the sustainable development of the shipping industry. Thirdly, with the increasing awareness of global environmental protection, the International Maritime Organization (IMO) has formulated strict emission standards, requiring the shipping industry to significantly reduce greenhouse gas emissions, and the shipping industry is facing tremendous pressure to reduce emissions. Therefore, the shipping industry is actively seeking green power solutions, optimizing the configuration of green power systems for ships, and predicting ship flow can optimize speed and fuel consumption, reduce idling and energy consumption, and thus achieve the goal of energy conservation and emission reduction.

[0003] At present, the research on ship water transport volume prediction technology is in a blank stage. There is an urgent need for a technical solution that can achieve accurate prediction of ship water transport volume, thereby promoting the structural optimization of the water transport industry, promoting the development of green and low-carbon transportation, reducing environmental pollution, and achieving sustainable development. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method, device and electronic equipment for predicting ship water transport volume based on Transformer.

[0005] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] In a first aspect, the present invention provides a method for predicting ship water transport volume based on Transformer, comprising:

[0007] Obtain historical data relevant to forecasting vessel water traffic;

[0008] Extracting multi-scale temporal features from the historical data using a pre-trained adaptive multi-scale temporal convolutional network;

[0009] Using a pre-trained graph neural network, extracting spatial dimension features from the historical data; the spatial dimension features are used to characterize the spatial dependency relationship between docks and ports;

[0010] Performing feature fusion processing on the multi-scale time feature and the spatial dimension feature to obtain a multi-scale spatiotemporal feature;

[0011] According to the multi-scale spatio-temporal features, a pre-trained Transformer model is used for predicting the water transportation volume of ships.

[0012] Optionally, the historical data includes: historical water transportation volume data of ships, economic indicators, and terminal port attributes.

[0013] Optionally, the feature fusion processing of the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features includes:

[0014] Adopt a cross-modal attention mechanism to perform feature fusion processing on the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features.

[0015] Optionally, the adaptive multi-scale time convolutional network, the graph neural network, and the Transformer model are pre-trained based on the constructed data set;

[0016] The construction method of the data set includes:

[0017] Obtain the historical data of the ship water transportation scenario;

[0018] Use the Pearson correlation coefficient to analyze the correlation between each item of data in the historical data of the ship water transportation scenario and the water transportation volume of ships, and remove the data with low correlation to obtain the historical data related to predicting the water transportation volume of ships;

[0019] Construct the data set according to the historical data related to predicting the water transportation volume of ships.

[0020] Optionally, the use of the pre-trained graph neural network to extract spatial dimension features from the historical data includes:

[0021] Construct an initial spatial graph according to the historical data; the nodes of the initial spatial graph are terminal ports, the features of the nodes include historical water transportation volume data of ships and terminal port attributes, and the edges of the initial spatial graph are shipping routes;

[0022] Use the pre-trained graph neural network to extract the spatial dimension features of the initial spatial graph.

[0023] Optionally, the pre-trained graph neural network is a pre-trained graph attention network.

[0024] Optionally, the method further includes:

[0025] Before extracting multi-scale time features and spatial dimension features from the historical data, perform data cleaning and preprocessing on the obtained historical data;

[0026] The data cleaning and preprocessing of the historical data of the ship water transportation scenario include: correcting incorrect data and missing data, and formatting the time information in the historical data.

[0027] In a second aspect, the present invention provides a ship water transportation volume prediction device based on a Transformer, including:

[0028] An acquisition module, configured to acquire historical data related to the predicted ship water transportation volume;

[0029] A first extraction module, configured to extract multi-scale time features from the historical data by using a pre-trained adaptive multi-scale time convolutional network;

[0030] A second extraction module, configured to extract spatial dimension features from the historical data by using a pre-trained graph neural network; the spatial dimension features are used to characterize the spatial dependence relationship between wharves and ports;

[0031] A fusion module, configured to perform feature fusion processing on the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features;

[0032] A prediction module, configured to predict the ship water transportation volume by using a pre-trained Transformer model according to the multi-scale spatio-temporal features.

[0033] Optionally, the historical data includes: historical data of ship water transportation volume, economic indicators, and wharf and port attributes.

[0034] Optionally, the fusion module is specifically configured to:

[0035] Adopt a cross-modal attention mechanism to perform feature fusion processing on the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features.

[0036] Optionally, the adaptive multi-scale time convolutional network, the graph neural network, and the Transformer model are pre-trained based on the constructed data set;

[0037] The construction method of the data set includes:

[0038] Acquire historical data of the ship water transportation scenario;

[0039] Use the Pearson correlation coefficient to analyze the correlation between each item of data in the historical data of the ship water transportation scenario and the ship water transportation volume, and remove the data with low correlation to obtain historical data related to the predicted ship water transportation volume;

[0040] Construct the data set according to the historical data related to the predicted ship water transportation volume.

[0041] Optionally, the second extraction module includes a construction sub-module and an extraction sub-module:

[0042] The construction sub-module is used to construct an initial spatial graph based on the historical data; the nodes of the initial spatial graph are terminal ports, the features of the nodes include historical data of ship water transportation volume and terminal port attributes, and the edges of the initial spatial graph are shipping lines;

[0043] The extraction sub-module is used to extract the spatial dimension features of the initial spatial graph by using a pre-trained graph neural network.

[0044] Optionally, the pre-trained graph neural network is a pre-trained graph attention network.

[0045] Optionally, the device further includes: a preprocessing module;

[0046] The preprocessing module is used to perform data cleaning and preprocessing on the obtained historical data before extracting multi-scale time features and spatial dimension features from the historical data;

[0047] The preprocessing module performs data cleaning and preprocessing on the historical data of the ship water transportation scenario, including: correcting error data and missing data, and formatting the time information in the historical data.

[0048] In a third aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0049] The memory is used to store a computer program;

[0050] The processor is used to implement the steps of any of the above-mentioned ship water transportation volume prediction methods based on Transformer when executing the computer program stored on the memory.

[0051] The ship water transportation volume prediction method based on Transformer provided by the present invention uses a pre-trained adaptive multi-scale time convolutional network to extract multi-scale time features from historical data related to predicting the ship water transportation volume; uses a pre-trained graph neural network to extract spatial dimension features from the historical data; then performs feature fusion processing on the multi-scale time features and spatial dimension features to obtain multi-scale spatio-temporal features, so as to predict the ship water transportation volume according to the multi-scale spatio-temporal features by using a pre-trained Transformer model, filling the technical gap in the field of ship water transportation volume prediction.

[0052] Moreover, the present invention extracts multi-scale spatio-temporal features through a spatio-temporal multi-scale convolutional network (ST-MS-TCN) composed of an adaptive multi-scale temporal convolutional network, a graph neural network, and a cross-modal attention module, enhancing the model's ability to capture various time-scale patterns and spatial features of time series data, thereby improving the accuracy of ship water transportation volume prediction. By accurately predicting the water transportation volume under different waters and different ship types, it can provide data support for optimizing the configuration of ship green power systems, help decision-makers reasonably plan the number, tonnage, and routes of ships, reduce resource waste, and improve operating efficiency. Through the precise prediction of the water transportation volume, it can promote the structural optimization of the water transportation industry, drive the development of green and low-carbon transportation, and achieve sustainable development.

[0053] The following will further elaborate on the present invention in conjunction with the accompanying drawings. Brief Description of the Drawings

[0054] Figure 1 is a schematic flowchart of the ship water transportation volume prediction method based on Transformer provided by an embodiment of the present invention;

[0055] Figure 2 is an overall framework diagram of the ship water transportation volume prediction method based on Transformer provided by an embodiment of the present invention;

[0056] Figure 3 is a schematic structural diagram of a Transformer model provided by an embodiment of the present invention;

[0057] Figure 4 is a schematic pre-training flowchart of the ship water transportation volume prediction method based on Transformer provided by an embodiment of the present invention;

[0058] Figure 5 is a schematic structural diagram of the ship water transportation volume prediction device based on Transformer provided by an embodiment of the present invention;

[0059] Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0060] The following further describes the present invention in detail with specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0061] To achieve effective prediction of ship water transportation volume, an embodiment of the present invention provides a method, device, and electronic device for predicting ship water transportation volume based on Transformer. Among them, the execution subject of the method for predicting ship water transportation volume based on Transformer provided by the embodiment of the present invention is the device for predicting ship water transportation volume based on Transformer provided by the embodiment of the present invention; this device is applied to the electronic device provided by the embodiment of the present invention. In specific applications, the electronic device may include: desktop computers, portable computers, intelligent mobile terminals, servers, etc., which are not limited herein. Any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0062] First, the method for predicting ship water transportation volume based on Transformer provided by the embodiment of the present invention will be described in detail. Refer to Figure 1 and Figure 2 , the method includes the following steps:

[0063] S10. Obtain historical data related to predicting ship water transportation volume.

[0064] Here, there are various types of historical data related to predicting ship water transportation volume. Exemplarily, the historical data related to predicting ship water transportation volume may include: historical data of ship water transportation volume, economic indicators, and terminal port attributes. Among them, the historical data of ship water transportation volume may specifically include inland water transportation volume, inland waterway mileage, waterway freight turnover volume, sea water transportation volume (i.e., cargo throughput of major coastal ports), average haul of waterway cargo transportation, economic indicators may specifically include GDP, and terminal port attributes specifically include locations of major inland scale port terminals, locations of major coastal scale port terminals, and route information. It should be noted that the inland water transportation volume is not a directly measured value but is obtained by calculation. The calculation method is:

[0065]

[0066] Among them, W represents the inland water transportation volume, F represents the waterway freight turnover volume, and R represents the inland waterway mileage.

[0067] Details of the above example data can be seen in the following table.

[0068]

[0069] In this step S10, when predicting the water transportation volume of the two different waters of inland rivers and the sea, the required relevant data are different and need to be determined in combination with the actual situation.

[0070] In this step S10, the original historical data related to predicting ship water transportation volume is obtained and integrated into the form of a time series segment. Specifically, this time series segment can be expressed as T = [T1, T2, T3,... Tn , where T n represents the historical data vector at the nth time point, including time, inland waterway transportation volume, inland waterway mileage, waterway freight turnover volume, sea transportation volume, etc.

[0071] Optionally, in one implementation, the obtained historical data can also be subjected to data cleaning and preprocessing.

[0072] Specifically, performing data cleaning and preprocessing on the obtained historical data may include: correcting the error data and missing data in the historical data, and formatting the time information in the historical data.

[0073] Among them, correcting the error data and missing data in the historical data includes steps such as data screening, denoising, normalization, etc., and filtering out most of the outliers, including extreme values and error records, etc., and performing data fusion on data from different sources to obtain more pure and representative valid data. Formatting the time information in the historical data includes creating time features and lag features for the historical data. Creating time features means extracting the valid information in the time stamp, such as year, month, day, etc., which can help the model understand the time pattern and capture seasonal and periodic changes. Creating lag features means using past ship water transportation volume data points as new features to help the model capture the trends and periodicity in the time series.

[0074] S20. Use the pre-trained adaptive multi-scale temporal convolutional network to extract multi-scale temporal features from the historical data.

[0075] Among them, the adaptive multi-scale temporal convolutional network is the AMS-TCN (Adaptive Multi-Scale Temporal Convolutional Network). The AMS-TCN captures the dependencies at different time scales through multi-scale temporal convolutional layers, applies the attention mechanism to assign different weights to features at different scales, and performs weighted summation on features at different scales to obtain multi-scale temporal features.

[0076] Specifically, AMS-TCN introduces a mechanism for dynamically selecting the convolutional kernel size. It has multiple parallel convolutional branches internally, and each branch uses a different convolutional kernel size. The time series data obtained in step S10 is input into AMS-TCN, and the time series data is divided into multiple input samples. Each input sample is usually a time series vector of a fixed length. For each input sample, the system dynamically determines the convolutional branch most suitable for the current sample according to its local statistical characteristics. For example, when short-term fluctuations are detected, the small convolutional kernel branch is activated, and when long-term trends are detected, the large convolutional kernel branch is activated. After each convolutional branch obtains the output, the attention mechanism is applied to assign weights to features of different scales, and these features of different scales are weighted and integrated to obtain multi-scale time features.

[0077] S30. Use the pre-trained graph neural network to extract spatial dimension features from historical data; the spatial dimension features are used to characterize the spatial dependence relationship between wharves and ports.

[0078] In this embodiment, the graph neural network adopted can be a graph attention network, namely GAT (Graph Attention Networks), but it is not limited thereto.

[0079] In this step S30, using the pre-trained graph neural network to extract spatial dimension features from historical data includes:

[0080] (1) Construct an initial spatial graph according to historical data.

[0081] Here, the initial spatial graph is a directed graph. The initial spatial graph is constructed according to data such as wharf and port locations, historical ship water transportation volume data, and route information. The nodes of the initial spatial graph are wharves and ports, and the features of the nodes include historical ship water transportation volume data and wharf and port attributes. The edges of the initial spatial graph are routes, and the edge features include route distances, etc.

[0082] (2) Use the pre-trained graph neural network to extract the spatial dimension features of the initial spatial graph.

[0083] Specifically, input the initial spatial graph into the pre-trained graph neural network. The graph neural network calculates the node embeddings of the initial spatial graph through forward propagation, updates the node features in the initial spatial graph using the parameters optimized in the previous backward propagation stage, and outputs the updated spatial graph as the spatial dimension features. The spatial dimension features extracted in this way not only contain the historical water transportation volume information of themselves but also integrate the spatial correlation information of adjacent nodes.

[0084] In the embodiments of the present invention, the graph neural network preferably adopts a graph attention network, namely GAT (Graph Attention Network), which is a graph neural network based on the attention mechanism and has significant advantages in processing graph-structured data. GAT assigns different weights to the neighbors of each node through the attention mechanism, can adaptively capture the importance between nodes, and GAT supports the multi-head attention mechanism, can capture different types of neighbor information, and GAT has the advantage of low computational complexity.

[0085] S40. Perform feature fusion processing on the multi-scale time features and spatial dimension features to obtain multi-scale spatio-temporal features.

[0086] Among them, the multi-scale time features are obtained from step S20, and the spatial dimension features are obtained from step S30. Based on the above multi-scale time features and spatial dimension features, a cross-modal attention mechanism is used to perform feature fusion processing on the multi-scale time features and spatial dimension features to obtain multi-scale spatio-temporal features. These multi-scale spatio-temporal features not only consider the multi-scale features in the time dimension but also integrate the spatial information, thus providing a more comprehensive data representation.

[0087] Here, the method of using the cross-modal attention mechanism to perform fusion processing on different modal features can refer to relevant existing technologies and will not be elaborated here.

[0088] S50. According to the multi-scale spatio-temporal features, use the pre-trained Transformer model to predict the water transportation volume of ships.

[0089] Specifically, the multi-scale spatio-temporal features obtained in step S40 are input into the Transformer layer by layer, so that each layer of the Transformer can directly access and operate on the original multi-scale spatio-temporal feature information, and use its powerful global dependence modeling ability to further refine the features. After being processed by the Transformer, the prediction result of the water transportation volume of ships can be obtained.

[0090] Specifically, the input obtained by the k-th layer encoder in the Transformer is:

[0091]

[0092] Among them, is the multi-scale spatio-temporal feature obtained in step S40, X (k+1) is the input of the (k + 1)-th layer encoder in the Transformer, and MSA(X (k+1) ) is the processing result of the multi-head attention mechanism on X (k+1) in the (k + 1)-th layer encoder of the Transformer.

[0093] In this step, the Transformer model is selected for predicting the water transportation volume of ships. The reasons are as follows: The water transportation volume of ships has an obvious trend, and it may be necessary to make predictions with very little data (data related to the water transportation volume in recent years). The common ARIMA model requires the time series to be stationary, that is, the statistical characteristics (mean, variance, etc.) of the series do not change with time. Although the data can be transformed into stationary data through means such as differencing, it increases the complexity of the model. The traditional LSTM can handle long time series, but it may encounter problems of gradient disappearance or gradient explosion in very long time series, and its computational complexity is relatively high. In contrast, the self-attention mechanism of the Transformer model can capture the relationship between any two elements in the sequence, is not restricted by distance, and dynamically allocates weights. The multi-head attention mechanism can enhance feature representation and model robustness, making it more suitable for the scenario of predicting the water transportation volume of ships.

[0094] In this embodiment, the structure of the Transformer model is as Figure 3 shown. Based on the existing Transformer model, 4 encoder layers, 4 decoder layers, 4 heads of attention, and a hidden layer dimension of 128 are set.

[0095] The method for predicting the water transportation volume of ships based on Transformer provided by the present invention extracts multi-scale time features from historical data related to predicting the water transportation volume of ships by using a pre-trained adaptive multi-scale time convolutional network; extracts spatial dimension features from the historical data by using a pre-trained graph neural network; then performs feature fusion processing on the multi-scale time features and spatial dimension features to obtain multi-scale spatio-temporal features, so as to predict the water transportation volume of ships by using the pre-trained Transformer model based on the multi-scale spatio-temporal features, filling the technical gap in the field of predicting the water transportation volume of ships.

[0096] Moreover, the present invention extracts multi-scale spatio-temporal features through a spatio-temporal multi-scale convolutional network (ST-MS-TCN) composed of an adaptive multi-scale time convolutional network, a graph neural network, and a cross-modal attention module, enhancing the model's ability to capture various time-scale patterns and spatial features of time series data, thereby improving the accuracy of predicting the water transportation volume of ships. By accurately predicting the water transportation volume under different waters and different ship types, it can provide data support for optimizing the configuration of the ship's green power system, help decision-makers reasonably plan the number, tonnage, and routes of ships, reduce resource waste, and improve operational efficiency. Through the accurate prediction of the water transportation volume, it can promote the structural optimization of the water transportation industry, drive the development of green and low-carbon transportation, and achieve sustainable development.

[0097] Optionally, in one implementation, the above-mentioned adaptive multi-scale temporal convolutional network, the graph neural network, and the Transformer model are pre-trained based on the constructed dataset;

[0098] The construction method of this dataset includes:

[0099] (1) Obtain historical data of the ship water transportation scenario;

[0100] Here, the historical data of the ship water transportation scenario can be any data generated in the ship water transportation scenario.

[0101] Preferably, according to the current situation and experience of social development, select multiple data items that may be related to predicting the ship water transportation volume (such as the mileage of inland waterways, the turnover volume of waterway freight, etc.) as the historical data of the ship water transportation scenario.

[0102] In addition, to ensure the accuracy of the prediction of the finally trained model, the historical data obtained here is preferably able to cover multiple economic cycles, so that the finally trained model can effectively predict the ship water transportation volume at different economic development stages.

[0103] (2) Use the Pearson correlation coefficient to analyze the correlation between each item of data in the historical data of the ship water transportation scenario and the ship water transportation volume, and remove the data with low correlation to obtain the historical data related to predicting the ship water transportation volume;

[0104] Here, the Pearson correlation coefficient is used to calculate the correlation between each item of data in the historical data of the ship water transportation scenario and the ship water transportation volume. If the correlation coefficient is close to 0, it is considered that this item of data has nothing to do with predicting the ship water transportation volume and is no longer included in the data acquisition range.

[0105] (3) Construct a dataset according to the historical data related to predicting the ship water transportation volume.

[0106] Specifically, multiple training samples are constructed according to the historical data related to predicting the ship water transportation volume. Each training sample contains a segment of historical data in the form of a time series, and in addition, it also contains the true value information of this sample, and this true value information is the true ship water transportation volume in the future time period corresponding to this historical data.

[0107] It can be understood that there is a lot of historical data in the ship water transportation scenario, some of which are related to the ship water transportation volume and some are not. Therefore, by using the Pearson correlation coefficient to analyze the correlation between each item of data and the ship water transportation volume, the historical data related to predicting the ship water transportation volume is selected to construct the dataset.

[0108] After constructing the dataset, the three models can be trained using the dataset. The specific training process is similar to steps S10 - S40, seeFigure 4 , the training samples in the dataset are divided into a training set, a test set, and a validation set. The historical data in the training set is input into the model, enabling the model to output the predicted water transportation volume of ships. The loss is calculated based on the difference between the predicted water transportation volume of ships by the model and the true value information in the training samples. The parameters of the model are adjusted according to the loss. Through continuous iteration until the loss converges or reaches a predetermined number of iterations, and the validation set is used to test and adjust the model to ensure the accuracy and robustness of the model. At this time, three trained models can be obtained. In addition, the prediction accuracy of the model can be measured by evaluation indicators such as the mean absolute error (MAE), the root mean square error (RMSE), and the coefficient of determination R2. Thus, accurate prediction of the water transportation volume of ships can be carried out using these three trained models.

[0109] Corresponding to the above-mentioned method for predicting the water transportation volume of ships based on Transformer, an embodiment of the present invention further provides a device for predicting the water transportation volume of ships based on Transformer; as Figure 5 shown, the device may include:

[0110] An acquisition module 501, configured to acquire historical data related to predicting the water transportation volume of ships;

[0111] A first extraction module 502, configured to extract multi-scale time features from the historical data by using a pre-trained adaptive multi-scale time convolutional network;

[0112] A second extraction module 503, configured to extract spatial dimension features from the historical data by using a pre-trained graph neural network; the spatial dimension features are used to characterize the spatial dependence relationship between wharves and ports;

[0113] A fusion module 504, configured to perform feature fusion processing on the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features;

[0114] A prediction module 505, configured to predict the water transportation volume of ships by using a pre-trained Transformer model according to the multi-scale spatio-temporal features.

[0115] Optionally, the historical data includes: historical data of the water transportation volume of ships, economic indicators, and wharf and port attributes.

[0116] Optionally, the fusion module 504 is specifically configured to:

[0117] Adopt a cross-modal attention mechanism to perform feature fusion processing on the multi-scale time features and the spatial dimension features to obtain multi-scale spatio-temporal features.

[0118] Optionally, the adaptive multi-scale temporal convolutional network, graph neural network, and Transformer model are pre-trained based on the constructed dataset;

[0119] The construction method of the dataset includes:

[0120] Obtain historical data of the ship water transportation scenario;

[0121] Use the Pearson correlation coefficient to analyze the correlation between each item of data in the historical data of the ship water transportation scenario and the ship water transportation volume, remove the data with low correlation, and obtain the historical data related to predicting the ship water transportation volume;

[0122] Construct the dataset according to the historical data related to predicting the ship water transportation volume.

[0123] Optionally, the second extraction module 503 includes a construction sub-module and an extraction sub-module:

[0124] The construction sub-module is used to construct an initial spatial graph according to historical data; the nodes of the initial spatial graph are terminal ports, the features of the nodes include historical data of ship water transportation volume and terminal port attributes, and the edges of the initial spatial graph are shipping routes;

[0125] The extraction sub-module is used to extract the spatial dimension features of the initial spatial graph by using a pre-trained graph neural network.

[0126] Optionally, the pre-trained graph neural network is a pre-trained graph attention network.

[0127] Optionally, the device further includes: a preprocessing module;

[0128] The preprocessing module is used to perform data cleaning and preprocessing on the obtained historical data before extracting multi-scale temporal features and spatial dimension features from the historical data;

[0129] The preprocessing module performs data cleaning and preprocessing on the historical data of the ship water transportation scenario, including: correcting error data and missing data, and formatting the time information in the historical data.

[0130] The ship water transportation volume prediction device based on Transformer provided by the present invention uses a pre-trained adaptive multi-scale temporal convolutional network to extract multi-scale temporal features from historical data related to predicting the ship water transportation volume; uses a pre-trained graph neural network to extract spatial dimension features from the historical data; then performs feature fusion processing on the multi-scale temporal features and spatial dimension features to obtain multi-scale spatio-temporal features, so as to predict the ship water transportation volume by using a pre-trained Transformer model according to the multi-scale spatio-temporal features, filling the technical gap in the field of ship water transportation volume prediction.

[0131] Moreover, the present invention extracts multi-scale spatio-temporal features through a spatio-temporal multi-scale convolutional network (ST-MS-TCN) composed of an adaptive multi-scale temporal convolutional network, a graph neural network, and a cross-modal attention module, enhancing the model's ability to capture various time-scale patterns and spatial features of time series data, thereby improving the accuracy of ship water transportation volume prediction. By accurately predicting the water transportation volume under different waters and different ship types, it can provide data support for optimizing the configuration of ship green power systems, help decision-makers reasonably plan the number, tonnage, and routes of ships, reduce resource waste, and improve operational efficiency. Through the accurate prediction of water transportation volume, it can promote the structural optimization of the water transportation industry, drive the development of green and low-carbon transportation, and achieve sustainable development.

[0132] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, as Figure 6 shown, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604. Among them, the processor 601, the communication interface 602, and the memory 603 complete communication with each other through the communication bus 604.

[0133] The memory 603 is used to store a computer program.

[0134] The processor 601, when executing the program stored on the memory 603, implements the method steps described in any of the above background fraud detection methods.

[0135] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0136] The communication interface is used for communication between the above electronic device and other devices.

[0137] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0138] The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0139] It should be noted that for the device / electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments.

[0140] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention.

[0141] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification.

[0142] Although the present invention has been described in connection with various embodiments herein, however, in the process of implementing the claimed invention, those skilled in the art can understand and achieve other variations of the disclosed embodiments by viewing the accompanying drawings and the disclosure. In the description of the present invention, the term "including" does not exclude other components or steps, the term "a" or "one" does not exclude a plurality of cases, and the meaning of "a plurality" is two or more, unless otherwise specifically defined. In addition, certain measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0143] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A Transformer-based method for predicting ship water transport volume, characterized in that, Including: Obtain historical data related to predicting the water transportation volume of ships; Use a pre-trained adaptive multi-scale temporal convolutional network to extract multi-scale temporal features from the historical data; Use a pre-trained graph neural network to extract spatial dimension features from the historical data; the spatial dimension features are used to characterize the spatial dependence relationship between wharves and ports; Perform feature fusion processing on the multi-scale temporal features and the spatial dimension features to obtain multi-scale spatio-temporal features; According to the multi-scale spatio-temporal features, use a pre-trained Transformer model to predict the water transportation volume of ships.

2. The method for predicting the water transportation volume of ships based on Transformer according to claim 1, wherein, The historical data includes: historical data of ship water transportation volume, economic indicators, and wharf and port attributes.

3. The method for predicting the water transportation volume of a ship based on Transformer according to claim 1, wherein, The performing feature fusion processing on the multi-scale temporal features and the spatial dimension features to obtain multi-scale spatio-temporal features includes: Adopt a cross-modal attention mechanism to perform feature fusion processing on the multi-scale temporal features and the spatial dimension features to obtain multi-scale spatio-temporal features.

4. The method for predicting the water transportation volume of ships based on Transformer according to claim 1, wherein The adaptive multi-scale temporal convolutional network, the graph neural network, and the Transformer model are pre-trained based on the constructed dataset; The construction method of the dataset includes: Obtain historical data of ship water transportation scenarios; Use the Pearson correlation coefficient to analyze the correlation between each data in the historical data of ship water transportation scenarios and the ship water transportation volume, and remove the data with low correlation to obtain historical data related to predicting the ship water transportation volume; Construct the dataset according to the historical data related to predicting the ship water transportation volume.

5. The method for predicting the water transportation volume of ships based on Transformer according to claim 2, wherein, The using a pre-trained graph neural network to extract spatial dimension features from the historical data includes: Construct an initial spatial graph according to the historical data; the nodes of the initial spatial graph are wharves and ports, the features of the nodes include historical data of ship water transportation volume and wharf and port attributes, and the edges of the initial spatial graph are shipping routes; Use a pre-trained graph neural network to extract the spatial dimension features of the initial spatial graph.

6. The method for predicting the water transportation volume of a ship based on Transformer according to claim 5, wherein The pre-trained graph neural network is a pre-trained graph attention network.

7. The method for predicting the water transportation volume of ships based on Transformer according to claim 1, characterized in that, The method further includes: Before extracting multi-scale temporal features and spatial dimension features from the historical data, perform data cleaning and preprocessing on the obtained historical data; The performing data cleaning and preprocessing on the obtained historical data includes: correcting the error data and missing data in the historical data, and formatting the time information in the historical data.

8. A ship water transport volume prediction device based on Transformer, characterized in that, Including: An acquisition module for obtaining historical data related to predicting the water transportation volume of ships; A first extraction module for using a pre-trained adaptive multi-scale temporal convolutional network to extract multi-scale temporal features from the historical data; A second extraction module for using a pre-trained graph neural network to extract spatial dimension features from the historical data; the spatial dimension features are used to characterize the spatial dependence relationship between wharves and ports; A fusion module for performing feature fusion processing on the multi-scale temporal features and the spatial dimension features to obtain multi-scale spatio-temporal features; A prediction module for predicting the water transportation volume of ships using a pre-trained Transformer model according to the multi-scale spatio-temporal features.

9. The Transformer-based device for predicting ship water transportation volume according to claim 8, wherein, The historical data includes: historical data of ship water transportation volume, economic indicators, and terminal port attributes.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; When the processor is used to execute the computer programs stored on the memory, it realizes the steps of the Transformer-based ship water transportation volume prediction method according to any one of claims 1 to 7.

Images