Shore power load prediction method and system based on self-attention mechanism

Abstract

The invention discloses a shore power load prediction method and system based on a self-attention mechanism, and the method comprises the steps: building a high-precision load prediction model through combining a historical load time sequence, an environment feature and a ship feature as input features, and employing a multi-head self-attention mechanism of a Transform model; multi-modal feature representation is generated through feature embedding and position coding, a global dependency relationship between a time sequence and multi-modal features is captured by using a multi-head self-attention mechanism, deep features are further extracted in combination with a feedforward neural network, and an accurate load prediction result can be generated after the model is subjected to multiple rounds of training and verification. According to the method, the load prediction precision, the prediction efficiency and the robustness and adaptability of the model are improved, and energy optimization of a green port is supported.

Description

Technical Field

[0001] This invention relates to the field of smart grid load forecasting and scheduling optimization, specifically to a shore power load forecasting method and system based on a self-attention mechanism. Background Technology

[0002] With the continuous improvement of port electrification and intelligence, shore power systems play a crucial role in reducing pollutant emissions and energy consumption during ship berthing. Especially against the backdrop of green port construction, shore power load forecasting, as a vital component of dynamic power dispatch and optimized management, has gradually become a focus of technological research. However, due to the complexity and dynamic nature of shore power system loads, the accuracy of load forecasting faces numerous challenges.

[0003] Currently, shore power system load forecasting is typically achieved by analyzing multi-dimensional features such as historical load data, ship berthing information, and environmental data. These features reflect the changing patterns of shore power load and its dynamic relationship with the environment. Traditional load forecasting methods mainly rely on empirical models and time series analysis methods, such as the Autoregressive Moving Average (ARIMA) model and Long Short-Term Memory (LSTM) networks. Although these methods can achieve certain results in some scenarios, their prediction accuracy and generalization ability are often limited when dealing with high-dimensional, multimodal features and complex nonlinear relationships. Especially under conditions of frequent port load fluctuations and diverse environmental factors, the robustness and stability of traditional methods are insufficient to meet practical needs, and large prediction errors are prone to occur.

[0004] With the rapid development of deep learning technology, models based on self-attention mechanisms have shown great potential in the field of time series forecasting due to their powerful global dependency modeling capabilities and parallel computing efficiency. Through self-attention, the Transformer (a deep learning model based on attention mechanisms) can effectively capture long-term and short-term dependencies in load data, while flexibly handling multimodal data features, significantly improving prediction accuracy and robustness. However, although existing research has attempted to apply deep learning models to power load forecasting, optimizing the hyperparameter configuration of the Transformer model to suit the characteristics of port shore power systems and further improving its prediction performance in complex dynamic environments remains a current research challenge.

[0005] Therefore, how to efficiently integrate the self-attention mechanism with the multidimensional features of shore power load forecasting, optimize the parameter training process of the model, improve the accuracy and stability of the forecast, and solve the limitations of traditional methods in dealing with complex nonlinear relationships remains an important technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide a shore power load forecasting method and system based on a self-attention mechanism, aiming to solve the problems of low load forecasting accuracy and poor adaptability to complex scenarios in the prior art.

[0007] To achieve the above objectives, this invention proposes a shore power load forecasting method based on a self-attention mechanism, comprising:

[0008] S1, Construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system;

[0009] S2. Based on a self-attention mechanism, a shore power load prediction model is constructed. The model includes an input layer, a feature embedding and location encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives the multimodal data and processes it to form a multimodal feature matrix. The feature embedding and location encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds location encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention outputs of the multiple heads. The feedforward neural network layer further performs a nonlinear transformation on the self-attention outputs of the multiple heads. The residual connection and layer normalization layer adds the self-attention outputs of the multiple heads to the output features of the feedforward neural network layer to form a residual output and performs layer normalization on the residual output. The output layer maps the output features of the residual connection and layer normalization layer to the target dimension.

[0010] S3, use the training samples in S1 to train the load prediction model in S2;

[0011] S4. Input the training samples into the trained load prediction model, and output the load prediction value for the future time step after processing by the load prediction model.

[0012] In a preferred embodiment, S1 includes:

[0013] S11, Collect multimodal data of the port shore power system, including historical load data, environmental characteristics and ship information of the port shore power system;

[0014] S12, preprocess the multimodal data to remove missing and outlier values, sort the historical load data according to timestamps to construct time series samples, and fuse the time series samples with environmental features and ship information to form the training samples.

[0015] In a preferred embodiment, in S2, the calculation process of each self-attention head includes calculating the self-attention output of the current head based on the feature representation matrix; and / or, the feedforward neural network layer consists of two fully connected network layers with an activation function introduced in between; and / or, it further includes comparing the result of the output layer with the actual load value and measuring the prediction error through a loss function.

[0016] In a preferred embodiment, the process of calculating the self-attention output of the current head based on the feature representation matrix includes: performing a linear mapping on the feature representation matrix to generate a query vector, a key vector, and a value vector; converting the dot product of the query vector and the key vector into an attention weight matrix through normalization and the Softmax function; and using the attention weight matrix to perform a weighted summation of the value vector to obtain the self-attention output of the current head.

[0017] In a preferred embodiment, the calculation process of the feedforward neural network layer is as follows: the self-attention output of the multi-head layer is projected onto the high-dimensional space of the hidden layer, and complex features are extracted through the activation function to obtain the output features of the first layer; the output features of the first layer are then mapped back to the original embedding dimension to obtain the output features of the second layer.

[0018] In a preferred embodiment, S3 includes:

[0019] S31 defines the initial hyperparameters of the shore power load prediction model, and sets the training rounds and maximum number of iterations for the model;

[0020] S32, input the training samples into the trained shore power load prediction model, and after passing through each layer of the model, generate preliminary prediction results;

[0021] S33, Calculate the loss function based on the preliminary prediction results, evaluate the difference between the predicted value and the actual value, and adjust the model parameters and update the learning state of the network.

[0022] S34. Determine whether the current training round has reached the maximum number of iterations. If it has not reached the maximum number of iterations, proceed to step S35; otherwise, proceed to step S36.

[0023] S35, after each training round, increment the training round number by 1 and return to step S32;

[0024] S36. When the training reaches the maximum number of iterations or the verification error converges to a stable range, stop the training process, save the finally trained shore power load prediction model, and test and evaluate its performance.

[0025] On the other hand, this invention proposes a shore power load forecasting system based on a self-attention mechanism, comprising:

[0026] The training sample construction unit is used to construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system;

[0027] A shore power load forecasting model construction unit is used to construct a shore power load forecasting model based on a self-attention mechanism. The model includes an input layer, a feature embedding and position encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives and processes the multimodal data to form a multimodal feature matrix. The feature embedding and position encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds position encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention outputs of the multiple heads. The feedforward neural network layer further performs nonlinear transformations on the self-attention outputs of the multiple heads. The residual connection and layer normalization layer adds the self-attention outputs of the multiple heads to the output features of the feedforward neural network layer to form residual outputs and performs layer normalization on the residual outputs. The output layer maps the output features of the residual connection and layer normalization layer to the target dimension.

[0028] The model training unit is used to train the load prediction model constructed by the shore power load prediction model construction unit using the training samples constructed by the training sample construction unit.

[0029] The load forecasting unit is used to input training samples into the trained load forecasting model, and output the load forecast value for the future time step after processing by the load forecasting model.

[0030] In a preferred embodiment, the training sample construction unit includes a data acquisition unit and a data preprocessing unit. The data acquisition unit is used to collect multimodal data of the port shore power system, including historical load data, environmental features, and ship information of the port shore power system. The data preprocessing unit is used to preprocess the multimodal data, remove missing values ​​and outliers, sort the historical load data according to timestamps to construct time series samples, and fuse the time series samples with environmental features and ship information to form the training samples.

[0031] In a preferred embodiment, the calculation process of each self-attention head in the multi-head self-attention mechanism layer includes calculating the self-attention output of the current head based on the feature representation matrix; and / or, the feedforward neural network layer consists of two fully connected network layers with an activation function introduced in between; and / or, the shore power load prediction model further includes a comparison unit for comparing the results of the output layer with the actual load value and measuring the prediction error through a loss function.

[0032] In a preferred embodiment, the model training unit includes a parameter definition unit, a prediction unit, a loss function calculation unit, a training epoch determination unit, a training epoch update unit, and a performance evaluation unit. The parameter definition unit defines the initial hyperparameters of the shore power load prediction model and sets the training epochs and maximum number of iterations. The prediction unit inputs training samples into the trained shore power load prediction model, processes them through each layer of the model, and generates preliminary prediction results. The loss function calculation unit calculates the loss function based on the preliminary prediction results, evaluates the difference between the predicted and actual values, and adjusts the model parameters and updates the learning state of the network. The training epoch judgment unit determines whether the current training epoch has reached the maximum number of iterations. If it has not reached the maximum number of iterations, it proceeds to the training epoch update unit; otherwise, it proceeds to the performance evaluation unit. The training epoch update unit increments the training epoch by 1 after each training epoch and returns to the prediction unit. The performance evaluation unit stops the training process when the training reaches the maximum number of iterations or the verification error converges to a stable range, saves the finally trained shore power load prediction model, and tests and evaluates its performance.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. This invention applies the Transformer model to shore power load forecasting, leveraging its powerful multi-head self-attention mechanism to comprehensively model the global dependencies between short- and long-term load time series. Compared to traditional methods (such as LSTM and ARIMA), the Transformer model has significant advantages in handling complex nonlinear data, overcoming the limitations of traditional models in capturing long-term dependencies. Simultaneously, the parallel computing characteristics of the Transformer model significantly improve prediction efficiency, enabling it to adapt to the real-time forecasting needs of large-scale port load data. This meets the dynamic scheduling requirements of modern port shore power systems while significantly reducing computational complexity, thus adapting to the high-efficiency requirements of dynamic power management in port shore power systems.

[0035] 2. This invention innovatively combines time-series load characteristics, environmental characteristics, and ship characteristics into a joint model, generating multimodal feature representations through feature embedding and location encoding. The introduction of multimodal features enables the model to fully capture the influencing factors of load changes, thereby improving the adaptability and accuracy of predictions. It performs particularly well in complex port environments where multiple factors interact, providing strong technical support for dynamic load forecasting.

[0036] 3. This invention improves the structural design of the Transformer by introducing modules such as multi-head self-attention, feedforward neural networks, and residual connections to enhance the model's predictive capabilities. The multi-head self-attention mechanism dynamically extracts global dependencies between time series and multimodal features, identifying the relative importance of features at different time steps. The feedforward neural network further mines deep feature interactions through nonlinear mapping, while residual connections and layer normalization effectively improve the model's training stability and convergence efficiency. These structural designs optimize the model's robustness and generalization ability in complex scenarios.

[0037] 4. This invention trains the Transformer model using a round-by-round optimization training strategy, gradually improving the model's predictive performance. Combining multi-head self-attention mechanisms and the parallel processing capabilities of feedforward neural networks, the model can efficiently handle large-scale port load data, ensuring real-time prediction and high accuracy. Compared to traditional methods, the model significantly improves prediction accuracy while substantially reducing computational complexity, making it suitable for dynamic scheduling and rapid response scenarios in practical shore power systems.

[0038] 5. This invention designs a high-precision prediction model for the practical application needs of port shore power load, applicable to complex load fluctuation scenarios. The prediction results can be used for dynamic load scheduling, optimizing power supply schemes based on future load changes to avoid power overload or underload. Simultaneously, combined with the application of green energy, the model can support the optimized allocation of renewable energy, reducing port energy costs and carbon emissions. Attached Figure Description

[0039] Figure 1 This is a flowchart of a shore power load prediction method based on a self-attention mechanism according to the present invention.

[0040] Figure 2 This is a structural diagram of the Transformer-based load forecasting model;

[0041] Figure 3 Here is a flowchart of the training process for the transformer-based load prediction model;

[0042] Figure 4 This is a load forecast curve for the shore power system. Detailed Implementation

[0043] The specific embodiments of the present invention will be described in detail below, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0044] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprising" shall be understood to include the stated elements or components without excluding other elements or other components.

[0045] like Figure 1 As shown, the shore power load forecasting method based on a self-attention mechanism disclosed in this invention mainly includes the following steps:

[0046] S1, Construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system.

[0047] S1 specifically includes:

[0048] S11, collect multimodal data of the port shore power system, including historical load data, environmental characteristics and ship information of the port shore power system.

[0049] In one specific embodiment, the data primarily originates from the port shore power system's operational records and the real-time monitoring system of related equipment. Specifically, the core data includes historical load data of the shore power system, ship operation information (i.e., ship information), and external environmental factors (i.e., environmental characteristics). In this embodiment, all data is collected within the same time period to ensure consistency in timeliness among the data.

[0050] To create a comprehensive historical load database, this embodiment selected typical vessel berthing examples from multiple ports, covering load data for different seasons, berthing times, and vessel types. This data effectively reflects the variation patterns of shore power load and its relationship with the environment and vessel characteristics. The equipment used during data collection included a high-precision power load monitoring instrument and environmental sensors with storage and real-time transmission capabilities. These devices enable non-intrusive data collection of port load and environmental data and possess strong anti-interference capabilities.

[0051] Specifically, the power load monitor has a resolution of 0.01kW and a measurement accuracy of ±0.5%, supporting real-time power curve recording. Environmental sensors can record air temperature, humidity, and wind speed, with a frequency range of 0.1Hz to 10Hz and an accuracy of ±0.1%. During data acquisition, these devices are installed at shore power stations and ship connection points to ensure the timeliness and accuracy of the collected data.

[0052] After the collected data is transmitted to the data storage system, it needs to undergo preliminary processing, mainly including initial cleaning and organization, focusing on the basic usability of the data. In this step, the data is first cleaned, including removing missing values ​​and obvious outliers, such as power surge anomalies and atypical environmental data. Secondly, the data is sorted according to timestamps to construct a complete time-series data sample, and the load data is matched with environmental and ship characteristic data.

[0053] This step resulted in a historical dataset covering multiple seasons, ship types, and environmental conditions, providing high-quality foundational data for subsequent multimodal feature extraction and model training. This process ensured the stability, consistency, and efficiency of the data, laying a crucial foundation for accurate load forecasting of shore power systems.

[0054] S12. The multimodal data is preprocessed to remove missing and outlier values. Historical load data are sorted according to timestamps to construct time series samples. The time series samples are then fused with environmental features and ship information to form training samples.

[0055] Specifically, in this step, based on the historical load data, ship information, and environmental characteristics of the port shore power system collected in step S11, multimodal training samples are constructed to provide comprehensive and accurate input data for model training. The construction of training samples requires detailed design and operation in multiple stages, from data cleaning and time series processing to feature fusion.

[0056] First, the data undergoes in-depth processing and fusion, focusing on sample standardization and feature adaptation. Specifically, to ensure the quality of the input data, outliers in the load data need to be removed. Statistical methods are used to detect anomalies in the power data, and illogical mooring duration records are deleted. For missing data, linear interpolation or mean imputation based on contextual features can be used to fill in the gaps. Furthermore, correlation analysis is used to screen features closely related to shore power load in the environmental data, removing weakly correlated features and simplifying feature dimensions. In high-dimensional environmental data, dimensionality reduction techniques such as principal component analysis can be used to retain core information.

[0057] Next, time series samples are constructed using the sliding window method. Specifically, historical load data is divided into fixed-length input-output sample sets. Assuming the time window length is T hours and the target prediction length is L hours, the input data includes the historical load data of the most recent T hours, the corresponding ship information, and environmental characteristics. The output is the load prediction value for the next L hours. For example, when T = 24 hours and L = 6 hours, the input of the training samples will include the historical load values ​​of the most recent 24 hours, along with the corresponding environmental and ship characteristics, while the output will be the load value for the next 6 hours. This slicing method of time series data can capture the pattern of load changes over time, providing sufficient time-dependent information for the model.

[0058] Then, multimodal feature fusion is performed on the time series samples, environmental features, and ship information. The time series samples are used as the primary time-dependent features, while environmental features (such as temperature, humidity, and tides) and ship information (such as ship type, berthing duration, and power rating) are also incorporated. To facilitate model training and prediction, the original data (here, the original data is the multimodal intermediate data set used for feature fusion and normalization in S12) needs to be normalized. Load power values ​​are normalized to the [0,1] interval to eliminate the influence of numerical differences on model training. Environmental features are normalized independently for each feature. Ship information is normalized and then encoded as discrete features. The normalized time series data is organized into a complete sample set, and an input matrix containing the following features is constructed:

[0059] Input = {X load ,X ship ,X env}

[0060] In the formula: X load X ship X env These represent time-series samples of load, ship information, and environmental characteristics, respectively.

[0061] Finally, the input data is constructed in matrix form, with time-series samples as the main axis and environmental features and ship information embedded as supplementary information. Through these steps, a high-quality, multimodal training sample set is formed, covering time-series data, environmental features, and ship features, providing comprehensive and consistent input data support for subsequent model training. This step also ensures the stability, consistency, and efficiency of the data, laying an important foundation for the accuracy of shore power system load forecasting.

[0062] S2. Based on a self-attention mechanism, a shore power load prediction model is constructed. The model includes an input layer, a feature embedding and location encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives the multimodal data and processes it to form a multimodal feature matrix. The feature embedding and location encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds location encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention output of the multiple heads. The feedforward neural network layer further performs a nonlinear transformation on the self-attention output of the multiple heads. The residual connection and layer normalization layer adds the self-attention output of the multiple heads to the output features of the feedforward neural network layer to form a residual output and performs layer normalization on the residual output. The output layer maps the output features of the residual connection and layer normalization layer to the target dimension.

[0063] Specifically, in this step, a Transformer-based load forecasting model is constructed to accurately predict future load values ​​for the shore power system. For example... Figure 2 As shown, the model includes an input layer, a feature embedding and location encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer.

[0064] The input layer's task is to receive the multimodal data constructed from step S1 above, and process it to form a multimodal feature matrix. The multimodal data includes: time-series features: historical load data, represented as... Where T is the time step. Environmental characteristics: temperature, humidity, wind speed, tides, etc., are represented as... Where F cnv The quantity of environmental characteristics. Ship characteristics: including ship type, berthing duration, and power rating, expressed as... Where F ship This represents the number of ship features. The input features are concatenated to form the multimodal feature matrix described above:

[0065]

[0066] In the formula: F = 1 + F env +F ship X load X ship X env These represent load time series characteristics, ship characteristics, and environmental characteristics, respectively.

[0067] Ultimately, the model's input data is constructed in matrix form, with time-series samples as the main axis and environmental features and ship information embedded as supplementary information. Through these steps, a high-quality, multimodal training sample set is formed, covering time-series data, environmental features, and ship features, providing comprehensive and consistent input data support for subsequent model training.

[0068] The feature embedding and positional encoding layers are used to embed the aforementioned multimodal feature matrices into embedded feature matrices, and then add positional encoding to these embedded feature matrices to generate feature representation matrices. Specifically, the multimodal data is first processed by feature embedding, mapping the aforementioned time-series samples, environmental features, and ship information to a unified embedded feature dimension D. Assume the input matrix (i.e., the aforementioned multimodal feature matrix)... Where T is the number of time steps and F is the feature dimension, the embedding operation is achieved through a linear transformation, as shown in the following formula:

[0069] E = XW E +b E ;

[0070] In the formula, It is the weight matrix of the embedding layer, used to map F-dimensional input features to D-dimensional space. It is a bias vector. It is the embedded feature matrix.

[0071] Since the Transformer model itself lacks the ability to handle time series order, positional encoding (PE) is introduced to enhance its ability to model temporal information. Positional encoding explicitly adds temporal order information to the features by mathematically representing the position of each time step. In this embodiment, the calculation formula for positional encoding is as follows:

[0072]

[0073] In the formula, pos represents the position of the time step, i is the index of the embedding dimension, and D is the total embedding dimension.

[0074] The use of sine and cosine functions ensures that the relative distances between different positions are preserved, thus helping the model capture the sequential relationships between time steps. After adding the positional encoding PE to the embedded feature matrix E, a feature representation matrix containing temporal sequence information is generated.

[0075] H0 = E + PE;

[0076] In the formula: This is the feature representation matrix after adding positional encoding. Through feature embedding and positional encoding, the model can not only represent the input data in high dimension, but also perceive the sequential characteristics of the time series, laying the foundation for subsequent multi-head self-attention mechanisms.

[0077] The multi-head self-attention mechanism layer, located in the middle layer of the model, is used to capture the complex correlation between shore power load time-series features and multimodal information (such as environmental features and ship information). Compared to traditional methods that can only handle local dependencies, the multi-head self-attention mechanism can dynamically model the mutual influence between different time steps and the relationship between load data and other external features through global weighting. Specifically, the multi-head self-attention mechanism layer is used to compute multiple self-attention heads in parallel based on the aforementioned feature representation matrix, outputting the self-attention output of multiple heads. The computation process of each self-attention head includes calculating the self-attention output of the current head based on the aforementioned feature representation matrix. The specific computation process is as follows:

[0078] To calculate the attention weights at each time step of the time series, we first need to perform a linear mapping on the feature representation matrix H0 to generate the query vector Q, key vector K, and value vector V:

[0079] Q = H0W Q

[0080] K = H0W K

[0081] V = H0W V ;

[0082] In the formula, These are the query vector, key vector, and value vector, respectively. These are the weight matrices of their linear mappings; d k The dimensions of the query and key vectors are typically set to D / h, where h is the number of attention heads.

[0083] The dot product of query Q and key K represents the similarity between the current time step and other time steps. The dot product result is converted into attention weights through normalization and the Softmax function.

[0084]

[0085] In the formula, This represents the attention weight matrix, where the i-th row indicates the degree of attention paid by time step i to other time steps in the entire sequence; scaling factor. To prevent the gradient update from becoming unstable due to excessively large dot product values.

[0086] By using the attention weight matrix A to perform a weighted summation on the value vector V, we obtain the weighted feature representation of the time series: Z = AV;

[0087] In the formula, It is the self-attention output of the current head.

[0088] To enhance the feature representation capability of the model, Transformer introduces a multi-head attention mechanism. This mechanism computes multiple self-attention heads in parallel, each focusing on different aspects of the features in a different subspace. Assuming there are h attention heads, each with the same computation process and outputs Z1, Z2, ..., Z... h The final output of the multi-head attention mechanism is:

[0089] H attn =Concat(Z1,Z2,…,Z) h W O ;

[0090] In the formula, Concat represents the concatenation operation. It is a linear mapping weight matrix. It is the self-attention output of multiple heads.

[0091] The role of the feedforward neural network layer is to further perform a nonlinear transformation on the self-attention output of the multi-headed neural network to uncover deeper relationships between time-series samples and environmental features. The feedforward neural network (FFN) layer consists of two fully connected layers (defined as the first and second fully connected layers for ease of description), with an activation function (ReLU) introduced in between. Its specific calculation process is as follows:

[0092] The above-mentioned multi-head self-attention output Projected onto the higher-dimensional space of the hidden layer (the hidden layer is of size D) hidden And extract complex features through activation functions (such as non-linear activation functions):

[0093] H ffn,1 =ReLU(H attn W1+b1);

[0094] in, It is the weight matrix of the first layer. It is the bias vector of the first layer; It is the output feature of the first layer.

[0095] The output features of the first layer are then mapped back to the original embedding dimension D to ensure consistency with the feature dimensions of other modules in the model.

[0096] H ffn,2 =H ffn,1 W2+b2;

[0097] in, It is the weight matrix of the second layer. It is the bias vector of the second layer. It is the output feature of the second layer.

[0098] To mitigate the vanishing gradient problem that may occur in deep networks and to prevent the loss of original information in feature representations during multiple nonlinear transformations, the Transformer introduces residual connections after each layer. These residual connections, along with the layer normalization layer and the output layer, are used to add the self-attention output of the multi-headed neural network to the output features of the feedforward neural network layer, forming a residual output, which is then normalized. Specifically, the residual connection adds the self-attention output of the multi-headed neural network layer to the output features of the feedforward neural network layer to form a residual output, as shown in the following formula:

[0099] H residual =H attn +H ffn,2 ;

[0100] In the formula, H attn The output feature of the aforementioned multi-head self-attention mechanism, H ffn,2 It is the output feature of the feedforward neural network layer.

[0101] After adding residual connections, the Transformer model further performs layer normalization on the residual output to improve the model's convergence speed and training efficiency. The formula for layer normalization is as follows:

[0102] H output =LayerNorm(H residual ).

[0103] Layer normalization normalizes the features at each time step, making the mean of the features 0 and the variance 1. Normalization can suppress gradient explosion and gradient vanishing problems, while improving the robustness of the model under different input distributions.

[0104] The output layer maps the output features from the residual connections and layer normalization layers to the target dimension. Specifically, the feature matrix after processing by the residual connections and layer normalization layers... It incorporates comprehensive information from time-series load features and multimodal input features. To obtain load predictions for future time steps, the output layer maps this feature matrix to the target dimension, i.e., predicts load values ​​for the next L time steps. This mapping process is achieved through a linear transformation, as shown in the following formula:

[0105] Y = H output W out +b out

[0106] in, It is the output of the residual connection and the layer normalization layer. It is the weight matrix of the output layer. It is the bias vector of the output layer. It is the predicted future load value, where L represents the prediction time step.

[0107] To optimize the model's predictive performance, the output layer's results (i.e., predicted values) need to be compared with the actual load values, and the prediction error is measured using a loss function. The root mean square error formula is:

[0108]

[0109] Among them, y i This is the actual load value. is the predicted value predicted by the model, and n is the number of signal samples.

[0110] S3, the load prediction model in S2 is trained using the training samples in S1.

[0111] Specifically, such as Figure 3 The diagram shown is a flowchart of the training process for the load forecasting model in this invention. The specific training steps for the load forecasting model are as follows:

[0112] S31: Define the initial hyperparameters of the shore power load prediction model, and set the training rounds and maximum number of iterations for the model.

[0113] Specifically, define the initial hyperparameters of the shore power load prediction model (i.e., the Transformer model), such as the number of attention heads, the size of the hidden layer, and the learning rate; and define the training epochs k=1 and the maximum number of iterations K.

[0114] S32: Input the training samples into the shore power load prediction model, and after processing through each layer of the model, generate preliminary prediction results.

[0115] Specifically, in this step, the training samples (i.e., historical time series samples, environmental features, and ship features) are input into the shore power load prediction model (i.e., the Transformer model). After passing through the model's input layer, feature embedding and location encoding layer, multi-head self-attention mechanism layer, feedforward neural network layer, residual connection and layer normalization layer and output layer, preliminary prediction results are generated.

[0116] S33: Calculate the loss function based on the preliminary prediction results, evaluate the difference between the predicted value and the actual value, and adjust the model parameters and update the learning state of the network.

[0117] Specifically, in this step, the loss function is calculated based on the training results (i.e., the preliminary prediction results), the difference between the predicted value and the actual value is evaluated, and then the parameters of the Transformer model (such as weights and biases) are adjusted and the learning state of the network is updated based on the feedback from the calculated loss function, so as to prepare for the next round of training.

[0118] S34: Determine whether the current training round has reached the maximum number of iterations K. If it has not reached the maximum number of iterations, proceed to step S35; otherwise, proceed to step S36.

[0119] S35: After each training round, update the training round k to k+1 and return to step S32 to continue optimizing the model parameters and structure.

[0120] S36: When the training reaches the maximum number of iterations K or the verification error converges to a stable range, stop the training process, save the final trained shore power load prediction model, and test and evaluate its performance to ensure that the model meets the actual application requirements.

[0121] S4. Input the training samples into the trained load prediction model, and output the load prediction value for the future time step after processing by the load prediction model.

[0122] Specifically, the shore power load prediction model, after final training, takes as input training samples (time series samples, environmental features, and ship information), and combines multi-head self-attention mechanism and deep feature mapping to output load prediction values ​​for future time steps, providing support for dynamic scheduling and power resource optimization of the shore power system.

[0123] This invention establishes a high-precision load prediction model by combining historical load time series, environmental features, and ship characteristics as input features and utilizing the multi-head self-attention mechanism of the Transformer model. The invention generates multimodal feature representations through feature embedding and positional encoding, and uses the multi-head self-attention mechanism to capture the global dependencies between time series and multimodal features, further extracting deep features using a feedforward neural network. After multiple rounds of training and validation, the model can generate accurate load prediction results. Furthermore, this invention employs standardization and nonlinear mapping methods to process multimodal input data, and enhances the model's robustness and training efficiency through residual connections and layer normalization techniques, significantly improving prediction accuracy and adaptability. This invention achieves accurate prediction of dynamic fluctuations in port shore power load and can be widely applied to dynamic load scheduling and energy optimization of port shore power systems, providing reliable data support and technical assurance for the intelligent operation of green ports.

[0124] Figure 4This is a graph comparing the load forecast and actual load obtained from the prediction model using example data. The graph shows the model's predicted load value for the shore power system over a 24-hour period, comparing it with the trend of actual load value changes. In the graph, the solid line with circles represents the actual load value, and the dashed line with squares represents the model's predicted load value. The graph shows that the prediction curve accurately captures the trend of actual load changes, especially during peak load periods (e.g., 5 to 10 hours) and off-peak periods (e.g., 15 to 20 hours), where the prediction results are basically consistent with the actual load, verifying the model's high-precision prediction capability. This graph visually reflects the model's effective modeling capability for dynamic load fluctuations, providing important data support for the dynamic scheduling and optimized management of port shore power load.

[0125] This invention also discloses a shore power load forecasting system based on a self-attention mechanism, comprising:

[0126] The training sample construction unit is used to construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system.

[0127] Specifically, the training sample construction unit includes a data acquisition unit and a data preprocessing unit. The data acquisition unit is used to collect multimodal data of the port shore power system, including historical load data, environmental features, and ship information of the port shore power system. The data preprocessing unit is used to preprocess the multimodal data, remove missing and outlier values, sort the historical load data according to timestamps to construct time series samples, and fuse the time series samples with environmental features and ship information to form the training samples.

[0128] A shore power load forecasting model construction unit is used to construct a shore power load forecasting model based on a self-attention mechanism. The model includes an input layer, a feature embedding and location encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives the multimodal data and processes it to form a multimodal feature matrix. The feature embedding and location encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds location encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention outputs of the multiple heads. The feedforward neural network layer further performs nonlinear transformations on the self-attention outputs of the multiple heads. The residual connection and layer normalization layer adds the self-attention outputs of the multiple heads to the output features of the feedforward neural network layer to form residual outputs and performs layer normalization on the residual outputs. The output layer maps the output features of the residual connection and layer normalization layer to the target dimension.

[0129] The calculation process of each self-attention head in the multi-head self-attention mechanism layer includes calculating the self-attention output of the current head based on the feature representation matrix; and / or, the feedforward neural network layer consists of two fully connected network layers with an activation function introduced in between; and / or, the shore power load prediction model also includes a comparison unit for comparing the results of the output layer with the actual load value and measuring the prediction error through a loss function.

[0130] The model training unit is used to train the load prediction model constructed by the shore power load prediction model construction unit using the training samples constructed by the training sample construction unit.

[0131] Specifically, the model training unit includes a parameter definition unit, a prediction unit, a loss function calculation unit, a training round determination unit, a training round update unit, and a performance evaluation unit. The parameter definition unit defines the initial hyperparameters of the shore power load prediction model and sets the training epochs and maximum number of iterations. The prediction unit inputs training samples into the trained shore power load prediction model, processes them through each layer of the model, and generates preliminary prediction results. The loss function calculation unit calculates the loss function based on the preliminary prediction results, evaluates the difference between the predicted and actual values, and adjusts the model parameters and updates the learning state of the network. The training epoch judgment unit determines whether the current training epoch has reached the maximum number of iterations. If it has not reached the maximum number of iterations, it proceeds to the training epoch update unit; otherwise, it proceeds to the performance evaluation unit. The training epoch update unit increments the training epoch by 1 after each training epoch and returns to the prediction unit. The performance evaluation unit stops the training process when the training reaches the maximum number of iterations or the verification error converges to a stable range, saves the finally trained shore power load prediction model, and tests and evaluates its performance.

[0132] The load forecasting unit is used to input training samples into the trained load forecasting model, and output the load forecast value for the future time step after processing by the load forecasting model.

[0133] Furthermore, the structure and working principle of each unit in the system can be referred to the descriptions in steps S1 to S4 above, which will not be repeated here.

[0134] The advantages of this invention are as follows: 1. This invention applies the Transformer model to the field of shore power load forecasting, utilizing its powerful multi-head self-attention mechanism to comprehensively model the global dependencies between short-term and long-term load time series. Compared with traditional methods (such as LSTM, ARIMA, etc.), the Transformer model has significant advantages in handling complex nonlinear data, overcoming the limitations of traditional models in capturing long-term dependencies. Simultaneously, the parallel computing characteristics of the Transformer model significantly improve prediction efficiency, enabling it to adapt to the real-time forecasting needs of large-scale port load data, meeting the dynamic scheduling requirements of modern port shore power systems while significantly reducing computational complexity, thus adapting to the high-efficiency requirements of dynamic power management in port shore power systems. 2. This invention innovatively combines time-series load characteristics, environmental characteristics, and ship characteristics into a joint model, generating multimodal feature representations through feature embedding and location encoding. The introduction of multimodal features allows the model to fully capture the influencing factors of load changes, thereby improving the adaptability and accuracy of predictions, especially performing excellently in complex port environments where multiple factors interact, providing strong technical support for dynamic load forecasting. 3. This invention improves the structural design of the Transformer model by introducing modules such as multi-head self-attention, feedforward neural networks, and residual connections to enhance the model's predictive capabilities. The multi-head self-attention mechanism can dynamically extract global dependencies between time series and multimodal features, identifying the relative importance of features at different time steps. The feedforward neural network further mines deep feature interactions through nonlinear mapping, while residual connections and layer normalization effectively improve the model's training stability and convergence efficiency. These structural designs optimize the model's robustness and generalization ability in complex scenarios. 4. This invention trains the Transformer model using a round-by-round optimization training strategy, gradually improving the model's predictive performance. Combining the parallel processing capabilities of the multi-head self-attention mechanism and the feedforward neural network, the model can efficiently process large-scale port load data, ensuring real-time prediction and high accuracy. Compared to traditional methods, the model significantly improves prediction accuracy while significantly reducing computational complexity, making it suitable for dynamic scheduling and rapid response scenarios in actual shore power systems. 5. This invention designs a high-precision prediction model for the practical application needs of port shore power loads, suitable for complex load fluctuation scenarios. The forecast results can be used for dynamic load scheduling, optimizing power supply schemes based on future load changes to avoid power overload or undersupply. Simultaneously, by incorporating green energy applications, the model can support the optimal allocation of renewable energy, reducing port energy costs and carbon emissions.

[0135] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A shore power load forecasting method based on a self-attention mechanism, characterized in that, include: S1, Construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system; S2. Based on a self-attention mechanism, a shore power load prediction model is constructed. The model includes an input layer, a feature embedding and location encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives the multimodal data and processes it to form a multimodal feature matrix. The feature embedding and location encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds location encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention outputs of the multiple heads. The feedforward neural network layer further performs a nonlinear transformation on the self-attention outputs of the multiple heads. The residual connection and layer normalization layer adds the self-attention outputs of the multiple heads to the output features of the feedforward neural network layer to form a residual output and performs layer normalization on the residual output. The output layer maps the output features of the residual connection and layer normalization layer to the target dimension. S3, use the training samples in S1 to train the load prediction model in S2; S4. Input the training samples into the trained load prediction model, and output the load prediction value for the future time step after processing by the load prediction model.

2. The shore power load forecasting method based on a self-attention mechanism as described in claim 1, characterized in that, S1 includes: S11, Collect multimodal data of the port shore power system, including historical load data, environmental characteristics and ship information of the port shore power system; S12, preprocess the multimodal data to remove missing and outlier values, sort the historical load data according to timestamps to construct time series samples, and fuse the time series samples with environmental features and ship information to form the training samples.

3. The shore power load forecasting method based on a self-attention mechanism as described in claim 1, characterized in that, In S2, the calculation process for each self-attention head includes calculating the self-attention output of the current head based on the feature representation matrix; and / or, the feedforward neural network layer consists of two fully connected network layers with an activation function introduced in between; and / or, it also includes comparing the result of the output layer with the actual load value and measuring the prediction error through a loss function.

4. The shore power load forecasting method based on a self-attention mechanism as described in claim 3, characterized in that, The process of calculating the self-attention output of the current head based on the feature representation matrix includes: performing a linear mapping on the feature representation matrix to generate a query vector, a key vector, and a value vector; converting the dot product of the query vector and the key vector into an attention weight matrix through normalization and the Softmax function; and using the attention weight matrix to perform a weighted summation of the value vector to obtain the self-attention output of the current head.

5. The shore power load forecasting method based on a self-attention mechanism as described in claim 3, characterized in that, The calculation process of the feedforward neural network layer is as follows: the self-attention output of the multi-head is projected onto the high-dimensional space of the hidden layer, and complex features are extracted through the activation function to obtain the output features of the first layer; The output features of the first layer are then mapped back to the original embedding dimension to obtain the output features of the second layer.

6. The shore power load forecasting method based on a self-attention mechanism as described in claim 1, characterized in that, S3 includes: S31, defining the initial hyperparameters of the shore power load prediction model, and setting the training rounds and maximum number of iterations of the model; S32, input the training samples into the trained shore power load prediction model, and after passing through each layer of the model, generate preliminary prediction results; S33, Calculate the loss function based on the preliminary prediction results, evaluate the difference between the predicted value and the actual value, and adjust the model parameters and update the learning state of the network. S34. Determine whether the current training round has reached the maximum number of iterations. If it has not reached the maximum number of iterations, proceed to step S35; otherwise, proceed to step S36. S35, after each training round, increment the training round number by 1 and return to step S32; S36. When the training reaches the maximum number of iterations or the verification error converges to a stable range, stop the training process, save the finally trained shore power load prediction model, and test and evaluate its performance.

7. A shore power load forecasting system based on a self-attention mechanism, characterized in that, include: The training sample construction unit is used to construct training samples based on multimodal features, wherein the data in the training samples includes multimodal data of the port shore power system; A shore power load forecasting model construction unit is used to construct a shore power load forecasting model based on a self-attention mechanism. The model includes an input layer, a feature embedding and position encoding layer, a multi-head self-attention mechanism layer, a feedforward neural network layer, a residual connection and layer normalization layer, and an output layer. The input layer receives the multimodal data and processes it to form a multimodal feature matrix. The feature embedding and position encoding layer embeds the multimodal feature matrix to form an embedded feature matrix and adds position encoding to the embedded feature matrix to generate a feature representation matrix. The multi-head self-attention mechanism layer computes multiple self-attention heads in parallel based on the feature representation matrix and outputs the self-attention outputs of the multiple heads. The feedforward neural network layer further performs nonlinear transformations on the self-attention outputs of the multiple heads. The residual connection and layer normalization layer adds the self-attention outputs of the multiple heads to the output features of the feedforward neural network layer to form residual outputs and performs layer normalization on the residual outputs. The output layer is used to map the output features of the residual connection and the layer normalization layer to the target dimension; The model training unit is used to train the load prediction model constructed by the shore power load prediction model construction unit using the training samples constructed by the training sample construction unit. The load forecasting unit is used to input training samples into the trained load forecasting model, and output the load forecast value for the future time step after processing by the load forecasting model.

8. A shore power load forecasting system based on a self-attention mechanism as described in claim 7, characterized in that, The training sample construction unit includes a data acquisition unit and a data preprocessing unit. The data acquisition unit is used to collect multimodal data of the port shore power system, including historical load data, environmental features, and ship information of the port shore power system. The data preprocessing unit is used to preprocess the multimodal data, remove missing and outlier values, sort the historical load data according to timestamps to construct time series samples, and fuse the time series samples with environmental features and ship information to form the training samples.

9. A shore power load forecasting system based on a self-attention mechanism as described in claim 7, characterized in that, The calculation process of each self-attention head in the multi-head self-attention mechanism layer includes calculating the self-attention output of the current head based on the feature representation matrix; and / or, the feedforward neural network layer consists of two fully connected network layers with an activation function introduced in between; and / or, the shore power load prediction model also includes a comparison unit for comparing the results of the output layer with the actual load value and measuring the prediction error through a loss function.

10. A shore power load forecasting system based on a self-attention mechanism as described in claim 7, characterized in that, The model training unit includes a parameter definition unit, a prediction unit, a loss function calculation unit, a training epoch determination unit, a training epoch update unit, and a performance evaluation unit. The parameter definition unit defines the initial hyperparameters of the shore power load prediction model and sets the training epochs and maximum number of iterations. The prediction unit inputs training samples into the trained shore power load prediction model, processes them through each layer of the model, and generates preliminary prediction results. The loss function calculation unit calculates the loss function based on the preliminary prediction results, evaluates the difference between the predicted and actual values, and adjusts the model parameters and updates the learning state of the network. The training epoch judgment unit determines whether the current training epoch has reached the maximum number of iterations. If it has not reached the maximum number of iterations, it proceeds to the training epoch update unit; otherwise, it proceeds to the performance evaluation unit. The training epoch update unit increments the training epoch by 1 after each training epoch and returns to the prediction unit. The performance evaluation unit stops the training process when the training reaches the maximum number of iterations or the verification error converges to a stable range, saves the finally trained shore power load prediction model, and tests and evaluates its performance.

Images