Arid region load prediction method and device based on adaptive attention

Abstract

The invention provides an adaptive attention-based arid region load prediction method and device, and the method comprises the steps: constructing an integrated technology system for the load prediction of an arid region: designing a systematic feature processing flow, and collecting the historical power load data of the region and the core impact factor data of the arid region; the data integrity is ensured through abnormal value identification and replacement and missing value filling, feature structuring is realized through time feature discretization and meteorological feature preprocessing subsequently, and finally redundant factors are eliminated through correlation screening and dimensional differences are eliminated in a standardized manner, so that high-quality input is provided for prediction. Transform model architecture adaptive to load time sequence characteristics is adopted, time sequence information is reserved by combining an input embedding layer with position coding, an encoder layer captures associated characteristics of loads and influence factors through multi-head attention, feature expression is enhanced in cooperation with a feed-forward network, and finally load prediction is achieved through a classification output layer. And meanwhile, standardized features are directly linked with model training.

Description

Technical Field

[0001] This invention belongs to the field of power load forecasting technology, specifically relating to a method and apparatus for load forecasting in arid regions based on adaptive attention. Background Technology

[0002] Regional power load forecasting is a core foundation for power system dispatching, grid planning, and energy optimization, especially in arid regions where load is significantly influenced by meteorological conditions. Typical arid climate characteristics such as high temperatures, low humidity, and low rainfall lead to a surge in cooling loads, and load fluctuation patterns differ significantly from those in non-arid regions. However, load forecasting in arid regions faces two key challenges: first, the quality of raw data—historical load data and data on meteorological and temporal influencing factors often contain missing or outlier values, directly affecting the reliability of forecast inputs; second, the complexity of feature correlations—the nonlinear correlation between meteorological factors (such as extreme temperatures and low humidity) and load in arid regions, as well as the long-term dependence of time-series loads, are difficult to accurately capture using traditional methods. The current power system demands increasingly higher accuracy in load forecasting for arid regions. Large forecast deviations can easily lead to grid supply-demand imbalances, affecting power supply stability and energy utilization efficiency.

[0003] Currently, there are three main technical solutions in the field of regional load forecasting: First, traditional statistical models (such as ARIMA models) and basic machine learning models (such as random forests and support vector machines). These methods are poorly adapted to the nonlinear relationship between meteorology and load unique to arid regions and rely on manually designed features, making it impossible to automatically uncover deep patterns in time-series data. Second, deep learning methods based on recurrent neural networks (RNN, LSTM) can process time-series data, but they suffer from the gradient vanishing problem in long-sequence dependency scenarios, making it difficult to accurately capture the long-term fluctuation characteristics of load in arid regions. Third, some load forecasting methods introduce Transformer models. These methods often lack standardized feature preprocessing procedures—either they do not systematically handle missing / outliers in the data, or they do not perform targeted feature screening for meteorological factors in arid regions. Moreover, their attention mechanisms are mostly fixed structures, unable to adaptively focus on key features affecting load in arid regions (such as extreme high temperatures), resulting in insufficient prediction accuracy and robustness in arid region scenarios. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a load prediction method for arid regions based on adaptive attention.

[0006] The main objective of this invention is to construct a general-purpose visual foundation model based on self-supervised visual representations, using a unified latent diffusion model backbone as its core to support various visual perception and generation tasks. Specifically, this invention employs a pre-trained self-supervised feature encoder (such as the DINO series) to extract latent image representations, and introduces a lightweight residual encoder on top of this to enhance reconstruction capabilities and fine-grained information representation. The latent features are then input into the unified diffusion model backbone and reconstruction decoder for generation, or directly through the task decoder for perceptual prediction. To maintain the semantic structure of the self-supervised encoder and reduce training and deployment costs, this method fixes the encoder parameters and trains only the lightweight residual branch and the task-specific decoder, thereby achieving efficient task transfer and multi-task support while ensuring generation quality and inference efficiency.

[0007] The second objective of this invention is to propose a load prediction device for arid regions based on adaptive attention.

[0008] The third objective of this invention is to provide a computer device.

[0009] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above objectives, a first aspect of the present invention proposes a load prediction method for arid regions based on adaptive attention, comprising: S1, collect historical power load sequences in arid regions and data on influencing factors including temporal and meteorological characteristics; S2, outlier identification and missing value imputation are performed on the collected load sequence and influence factor data. Outliers are replaced by the mean of the adjacent normal data, and missing values ​​are imputed by linear interpolation. S3 converts time features into binary features and unique thermal coding features, and normalizes meteorological features. S4. Based on the correlation between the load sequence and each influencing factor, key features are screened, features with a correlation threshold greater than the preset value are retained, and the screened features are standardized. S5 trains a Transformer model with standardized features and an adaptive attention mechanism. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and a feedforward network to achieve time-series-meteorological coupled modeling, outputting load prediction results.

[0011] In one embodiment of the present invention, S2 includes: S21, the 3σ criterion is used to identify outliers, that is, for load characteristics or influencing factors, if a data point x satisfies... ,in The mean of the data. If the value is less than the standard deviation, it is considered an outlier. S22, replace the outlier with the mean of the three normal data points before and after the data point, and fill in the missing values ​​using linear interpolation. That is, if the data at time t... The missing part is filled using the following formula: .

[0012] In one embodiment of the present invention, S3 includes: S31 converts whether it is a holiday into a binary feature and the day of the week into a 7-dimensional one-hot encoded feature. S32 compresses the continuous meteorological characteristics of temperature and humidity into the [0,1] interval using the maximum-minimum method.

[0013] In one embodiment of the present invention, S4 further includes: S41, Calculate each influencing factor Pearson correlation coefficient with load series L The formula is ; S42, filter out The influencing factors were used as key features, and the Z-score normalization formula was adopted. ,in The characteristic mean, The characteristic standard deviation is denoted as .

[0014] In one embodiment of the present invention, S5 includes: S51, through linear transformation of the input embedding layer ( , (For embedding layer parameters) are mapped to a high-dimensional feature space; S52, Add position encoding matrix To preserve the sequential information of time-series data, the final input feature is .

[0015] In one embodiment of the present invention, the method further includes: S6 utilizes a dynamic weight adjustment mechanism for multi-head attention to automatically enhance the weight allocation of relevant features based on the meteorological characteristics of arid regions. The specific calculation formula is as follows: MultiHead , , in , , , Let be the parameters of the i-th attention head. , , Output projection parameters for multi-head attention.

[0016] To achieve the above objectives, a second aspect of the present invention provides a load prediction device for arid regions based on adaptive attention, comprising: The data acquisition module is used to collect historical power load sequences and data on influencing factors including temporal and meteorological characteristics in arid regions. The outlier and missing value processing module is used to identify outliers and impute missing values ​​in the collected load sequence and influence factor data. Outliers are replaced by the mean of the adjacent normal data, and missing values ​​are imputed by linear interpolation. The feature conversion and normalization module is used to convert time features into binary features and uniquely thermal coded features, and to normalize meteorological features. The key feature screening and standardization module is used to screen key features based on the correlation between the load sequence and each influencing factor, retain features whose correlation with the load threshold is greater than a preset value, and standardize the screened features. The feature input and model training module is used to train a Transformer model with an adaptive attention mechanism based on the standardized feature input. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and feedforward network to achieve time-series-meteorological coupled modeling, outputting load prediction results.

[0017] This invention discloses a method and apparatus for load forecasting in arid regions based on adaptive attention, which effectively improves the accuracy and robustness of power load forecasting in arid regions. By dynamically focusing on key meteorological features through an adaptive attention mechanism and combining a standardized feature preprocessing process, it significantly improves data quality and nonlinear correlation modeling capabilities.

[0018] To achieve the above objectives, a third aspect of this application provides a computer device comprising a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory, for implementing the adaptive attention-based load forecasting method for arid regions as described in the first aspect embodiment.

[0019] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the adaptive attention-based load forecasting method for arid regions as described in the first aspect embodiment.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a load forecasting method for arid regions based on adaptive attention, according to an embodiment of the present invention. Figure 2 This is a structural diagram of a drought-area load prediction device based on adaptive attention according to an embodiment of the present invention; Figure 3 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] The following description, with reference to the accompanying drawings, describes a method and apparatus for predicting load in arid regions based on adaptive attention, according to an embodiment of the present invention.

[0025] Example 1 Figure 1 This is a flowchart of a load forecasting method for arid regions based on adaptive attention according to an embodiment of the present invention, as shown below. Figure 1 As shown, it includes: S1 collects historical power load sequences in arid regions and data on influencing factors including temporal and meteorological characteristics.

[0026] In some implementations, load characteristic data is collected in time series format, denoted as... ,in Indicates the first The electrical load value at any given time, expressed in megawatts (MW). This represents the total number of data entries. This sequence typically originates from historical records at power dispatch centers or smart meters. The sampling frequency can be set hourly, daily, or weekly, depending on the time granularity requirements of the forecasting task. Influencing factor data includes time and meteorological characteristics. Time characteristics cover year, month, day, hour, day of the week, and whether it is a holiday, used to capture the periodic and regular changes in load; meteorological characteristics include temperature. ,humidity Rainfall These are used to model the impact of climate-specific factors in arid regions on loads.

[0027] During data acquisition, it is essential to ensure the spatiotemporal consistency and integrity of the data. For example, meteorological data should be strictly aligned with load data in the time dimension, with consistent sampling intervals and coverage of the same time window. Furthermore, data acquisition should adhere to standard protocols for power systems and meteorological observations, such as the ISO 8601 time format and the WMO standard for meteorological data, to guarantee data readability and processability.

[0028] S2, perform channel-level concatenation of the global and local features, and adjust the mean and variance of the local features through a distribution alignment mechanism to make them consistent with the distribution of the global features.

[0029] In some implementations, outlier detection uses... The criterion is to calculate the mean of each feature sequence. with standard deviation If a certain data point satisfy If the value is 0, it is considered an outlier. The identified outlier will be replaced by the mean of the three preceding and following normal data points. Specifically, let the outlier be located at the th ... At that time, its replacement value is This method effectively suppresses the interference of extreme values ​​on model training while preserving temporal continuity.

[0030] For handling missing values, linear interpolation is used. Let the first... Time data Missing data; the previous time step was... The data at the next moment is The formula for filling in the gaps is: This method is suitable for situations where adjacent data points in a time series change gradually, effectively restoring the temporal structure of the data and avoiding the problem of discontinuous model input caused by missing data.

[0031] Specifically, outlier identification relies on statistics. and Its calculation is based on a sliding window or global statistics of the current feature sequence. The window size can be set to a 24-hour or weekly cycle according to the data sampling frequency. The imputation accuracy of linear interpolation is closely related to the distribution density of missing data and is generally suitable for scenarios with a missing rate of less than 5%.

[0032] Furthermore, S2 includes: S21, the 3σ criterion is used to identify outliers, that is, for load characteristics L or influence factors f j If a data point x satisfies ,in The mean of the data. If the value is less than the standard deviation, it is considered an outlier.

[0033] This method, based on statistical principles, determines whether a data point exceeds the normal fluctuation range by calculating the degree of deviation between the data point and the mean. Specifically, for data points in a certain feature sequence... If it satisfies If it is, then it is determined to be an outlier. This is the mean of the characteristic sequence. It is its standard deviation. This criterion is based on the property of the normal distribution, that is, under an ideal normal distribution, about 99.7% of the data points are within the range of the mean ± 3 times the standard deviation, and points outside this range can be regarded as statistical outliers.

[0034] Specifically, this step first involves the load sequence. or impact factor sequence Perform statistical calculations to obtain its mean. with standard deviation Then, for each data point An outlier is detected if its deviation from the mean exceeds three times the standard deviation. To ensure the continuity and integrity of the data sequence, outliers are replaced by the mean of the three preceding and following normal data points. Located in the At that time, the replacement value is (The premise is) and (All are within the valid range).

[0035] S22, replace the outlier with the mean of the three normal data points before and after the data point, and fill in the missing values ​​using linear interpolation. That is, if the data at time t... The missing part is filled using the following formula: .

[0036] Specifically, in terms of outlier handling, the methods widely used in statistics are adopted. The criteria are used for identification and correction. Specifically, for any given time... data points If it is similar to the mean The deviation exceeds three times the standard deviation That is, satisfying If the value is not found in the range, it is considered an outlier. In some implementations, outliers are replaced by the arithmetic mean of the three normal data points before and after them. (Excluding outliers themselves), this method effectively suppresses the interference of extreme values ​​on model training while maintaining the continuity of the time series.

[0037] For handling missing values, this invention employs linear interpolation for imputation, suitable for scenarios where adjacent data points have strong temporal continuity. Specifically, if the first... Data at any time Missing, and its previous moment and the next moment If the data is valid, then the fill formula is: This formula is based on the assumption of a linear relationship and is suitable for scenarios with short-term missing data. It can effectively restore the integrity of the data sequence while avoiding the distortion of time-series features caused by simple mean imputation.

[0038] Specifically, In the guidelines and The calculation is based on a sliding window or global statistics of the current feature sequence, and the specific window size can be adjusted according to the data sampling frequency and fluctuation characteristics. In arid regions, this step is particularly crucial for improving data quality because the load is highly sensitive to meteorological factors.

[0039] S3 converts time features into binary features and uniquely thermally encoded features, and normalizes meteorological features.

[0040] In some implementations, time features include "whether it's a holiday" and "day of the week." "Whether it's a holiday" is converted to a binary feature, meaning it takes a value of 1 if a given time is a holiday and 0 otherwise. This feature can be directly used as model input without further encoding. "Day of the week" is represented using 7-dimensional one-hot encoding, mapping each day of the week to a 7-dimensional vector where only the weekday dimension has a value of 1, and the rest are 0. For example, if a given time is Wednesday, its encoding is [0, 0, 1, 0, 0, 0, 0], thus explicitly expressing the periodicity of time.

[0041] Furthermore, regarding meteorological characteristics (such as temperature) ,humidity Rainfall (etc.), this invention employs the Min-Max Normalization method to map the original continuous features to the [0, 1] interval, as shown in the formula:

[0042] in These are the original eigenvalues. and These are the minimum and maximum values ​​of the feature in the entire dataset, respectively. This method can effectively compress the data range, providing a good numerical basis for subsequent Z-score standardization, while avoiding feature distribution distortion caused by extreme values.

[0043] Furthermore, S3 includes: S31 converts whether it is a holiday or not into a binary characteristic, where 1 represents a holiday and 0 represents a non-holiday.

[0044] In some implementations, the construction process of this binary feature is as follows: First, the "whether it's a holiday" field is extracted from the original time series. This field is usually in Boolean or string form (e.g., "yes" or "no"). Then, it is converted into a binary variable using a mapping function, where 1 indicates that the current time is a holiday and 0 indicates a non-holiday state. This conversion operation can be performed using simple conditional checks or batch processing, such as using the `map` function or the `get_dummies` method of the `pandas` library in Python. The value range of this feature is strictly limited to... This ensures the numerical stability and computational efficiency of the model input.

[0045] Furthermore, this binary feature has significant interpretative value in the model. In arid regions, holidays are often accompanied by significant changes in residential electricity consumption behavior, such as decreased air conditioning usage and reduced industrial load. Therefore, explicitly encoding this feature in binary form helps the model identify and learn the nonlinear impact of holidays on load during training. In addition, this feature has good compatibility with other features (such as temperature and humidity) in the subsequent multi-head attention mechanism, facilitating feature fusion and dynamic focusing in the temporal dimension.

[0046] S32 converts the "day of the week" into a 7-dimensional unique thermal coding feature, and compresses continuous meteorological features such as temperature and humidity into the [0,1] interval using the maximum-minimum method.

[0047] In some implementations, a 7-dimensional one-hot encoding method is used for vectorization of the discrete time feature of "day of the week". Specifically, if a certain time is Wednesday, the corresponding 7-dimensional vector is [0, 0, 1, 0, 0, 0, 0], and so on. This encoding method can effectively avoid the pseudo-order relationship introduced by numerical encoding (such as 1~7), ensuring that the model does not misjudge the continuity or distance relationship between days of the week when processing time features. The number of dimensions of the one-hot encoding is the same as the number of week types, that is, 7 possible values, so the output dimension is... This is suitable for parallel processing and feature fusion in subsequent models.

[0048] For continuous meteorological characteristics such as temperature and humidity, the maximum-minimum method is used to compress them to the interval [0, 1], and the formula is as follows:

[0049] in, These are the original eigenvalues. and These are the minimum and maximum values ​​of the feature in the entire dataset, respectively. This method has the advantages of simple calculation and preservation of the original distribution trend, making it suitable for scenarios where meteorological data in arid regions fluctuate significantly. For example, if the temperature range of a certain region is [20, 45]℃, then... The normalized value of ℃ is This facilitates the model's ability to uniformly handle inputs of different dimensions.

[0050] S4. Based on the correlation between the load sequence and each influencing factor, key features are screened, features with a correlation threshold greater than the preset value are retained, and the screened features are standardized.

[0051] In some implementations, the influencing factors are calculated first. With load sequence Pearson correlation coefficient between Its mathematical expression is:

[0052] in, The mean of the load series. For the first The mean of the class impact factors, Indicates the first Class of impact factors in the first The value at time. This formula quantifies the degree of linear correlation between each influencing factor and the load, thereby screening out... The key features are selected as the main features, and low-correlation factors are eliminated to reduce the input dimension and computational complexity of the model.

[0053] Furthermore, the key features selected are standardized using the Z-score standardization method, the formula of which is:

[0054] in, These are the original eigenvalues. Its mean, Its standard deviation. After standardization, the mean of all features is 0 and the standard deviation is 1, thereby eliminating the dimensional differences between different features and improving the convergence efficiency and generalization ability of the model in the feature space.

[0055] Furthermore, S4 includes: S41, Calculate each influencing factor Pearson correlation coefficient with load series L The formula is .

[0056] This invention calculates various influencing factors. With load sequence Pearson correlation coefficient This allows for the selection of key features, thereby improving the feature quality and prediction accuracy of the model input. The Pearson correlation coefficient is a statistical indicator that measures the degree of linear correlation between two variables, with a value ranging from -1 to 1; a larger absolute value indicates a stronger correlation. In this invention, this step is used to evaluate each type of influencing factor. With load sequence The linear correlation between them is expressed by the formula:

[0057] in, Indicates the first Load value at any given time For load sequence The mean; Indicates the first Class of impact factors in the first The value at time, This represents the mean of the influencing factor. The formula quantifies the strength of the linear relationship between the influencing factor and the loading by using the ratio of covariance to standard deviation.

[0058] In some implementations, the present invention sets the correlation coefficient threshold to be... The key features were selected based on influencing factors that showed a significant linear relationship with the load. This threshold was set based on the typical correlation distribution between load and meteorological and temporal characteristics in arid regions, balancing feature representativeness with model simplicity. By eliminating low-correlation features, the dimensionality of the model input can be effectively reduced, the interference of redundant information on prediction results can be minimized, and the model training efficiency and generalization ability can be improved.

[0059] S42, filter out The influencing factors were used as key features, and the Z-score normalization formula was adopted. (in The characteristic mean, (Using the characteristic standard deviation) to eliminate dimensional differences.

[0060] In some implementations, the filtering threshold is set to This means that only the influencing factors that are strongly correlated with the load are retained as key features, thereby effectively reducing the dimensionality of the model input, improving computational efficiency and reducing noise interference.

[0061] The selected key features are then standardized using Z-score, with the following formula: ,in, For the original value of the feature, The mean of this feature. This represents the standard deviation of the feature. This standardization method can unify features with different dimensions and distribution ranges to a standard normal distribution with a mean of 0 and a standard deviation of 1, thereby eliminating dimensional differences and improving the model's sensitivity to features and the balance of weight allocation.

[0062] S5 trains a Transformer model with standardized features and an adaptive attention mechanism. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and a feedforward network to achieve time-series-meteorological coupled modeling, outputting load prediction results.

[0063] Specifically, the standardized feature matrix (in The length of the time series. (For model feature dimensions) First, a linear transformation is performed through the input embedding layer, as shown in the formula: ,in , The weights and bias parameters of the embedding layer, This represents the original dimension of the input features. Subsequently, positional encoding is added. To preserve the sequential information of the time series, the final input features are... .

[0064] In the encoder layer, the model adopts Multi-head attention consists of several parallel self-attention heads, each generating a query, key, and value vector through independent linear transformation matrices $W_i^Q, W_i^K, W_i^V$. The output of the multi-head attention is expressed by the formula... Perform splicing and projection, among which This is for outputting projection parameters. The calculation of each attention head follows the standard self-attention formula:

[0065] in This represents the key vector dimension for each head. Through a multi-head attention mechanism, the model can adaptively focus on key drivers of load changes in arid regions, such as extreme high temperatures or sudden drops in humidity.

[0066] Furthermore, after the multi-head attention output, the data is sequentially processed through residual connections and layer normalization to mitigate gradient vanishing and improve training stability. Subsequently, the data enters a feed-forward network (FFN), which typically consists of two fully connected layers with a non-linear activation function (such as ReLU) sandwiched in between to enhance the non-linear expressive power of the features.

[0067] Finally, the encoder outputs global features The data is fed into the classification output layer, where a linear transformation and a softmax function are applied to output the load prediction result, as shown in the formula:

[0068] in , For output layer parameters, This is for predicting the number of categories (such as load level or specific numerical range).

[0069] Furthermore, S5 includes: S51, through linear transformation of the input embedding layer ( , The embedding layer parameters are mapped to a high-dimensional feature space.

[0070] Specifically, in the Transformer model architecture of this invention, the Input Embedding Layer maps the original temporal features to a high-dimensional feature space through a linear transformation, thereby providing a structured and learnable input representation for the model's subsequent attention mechanism and feature fusion. Specifically, the mathematical expression of this step is as follows: ,in This represents the standardized input feature matrix. The length of the time series. For feature dimensions; This is the weight matrix of the embedding layer. For bias vectors, The embedding dimension set for the model is typically 64, 128, or 256, depending on the model complexity and computing resources.

[0071] In some implementations, this linear transformation is implemented using a fully connected layer, where each input feature vector... Through matrix multiplication and Multiply and add bias. The corresponding embedding vector is obtained. This process can be viewed as expanding the dimensions and semantically mapping the original features, enabling the model to better capture the potential relationship between load and multidimensional influencing factors in arid regions.

[0072] Furthermore, to preserve temporal information, features are embedded. It needs to be combined with positional encoding. ) Perform element-by-element addition to obtain the final input representation. Location encoding is typically constructed using sine and cosine functions, with dimensions consistent with the embedding layer output. This ensures that the model can perceive temporal location information when processing time-series data, thereby improving its ability to model load change trends.

[0073] S52, Add position encoding matrix To preserve the sequential information of time-series data, the final input feature is .

[0074] In some implementations, the position encoding matrix Typically, a fixed or learnable approach is used for construction. This invention preferably uses fixed-position encoding, which can be a combination of sine and cosine functions, specifically defined as: for position... and dimensions ,like If it is even, then ;like If it is an odd number, then This method can provide the model with smooth and periodic location information, which helps to handle the seasonal and periodic fluctuations that may exist in load data in arid regions.

[0075] In terms of implementation, the position encoding matrix Dimensions and Embedded Features Consistency, that is ,in Given the length of the input sequence, The feature embedding dimension is assigned to the model. During the model's forward propagation, With embedded features Adding each element together yields the final input features. This operation can be implemented using simple addition in deep learning frameworks such as PyTorch, without requiring additional nonlinear transformations.

[0076] The introduction of location coding significantly enhances the model's ability to model temporal dependencies. In load forecasting tasks in arid regions, since load changes have obvious temporal characteristics (such as daily cycles, weekly cycles, seasonality, etc.), the addition of location coding helps the model to more accurately locate key time points in the self-attention mechanism, thereby improving forecast accuracy.

[0077] S6 utilizes a dynamic weight adjustment mechanism for multi-head attention to automatically enhance the weight allocation of relevant features based on meteorological characteristics of arid regions (such as extreme high temperatures). The specific calculation formula is MultiHead. ,in , , , The parameters of the i-th attention head ( , ), Output projection parameters for multi-head attention.

[0078] In some implementations, this mechanism enables the model to automatically enhance the weight allocation of relevant features based on typical meteorological characteristics of arid regions (such as extreme high temperatures) through dynamic weight adjustment, thereby improving prediction accuracy and model robustness.

[0079] In the specific technical implementation, the formula for calculating multi-head attention is:

[0080] in, This indicates that the query, key, and value all originate from the input embedded feature $E_{in}$, which is obtained by processing the original loading and influence factor data through an embedding layer and positional encoding. The calculation process for each attention head $head^i$ is as follows: first, the input features are transformed using a learnable linear transformation matrix... , , Mapped to different subspaces, where , , This represents the number of attention heads. Then, the attention weights are calculated using a standard self-attention mechanism, i.e.:

[0081] In load forecasting scenarios in arid regions, this mechanism can dynamically adjust attention weights based on the distribution of key meteorological factors such as extreme high temperatures in the input features, making the model focus more on features highly correlated with load changes. For example, during periods of high temperatures, the model automatically increases the weights of features related to temperature, thereby more accurately predicting the surge in cooling load.

[0082] At the parameter level, This is the output projection parameter for multi-head attention, used to concatenate the outputs of multiple attention heads and map them back to the original feature dimension. This parameter is optimized through backpropagation during training to improve the model's ability to perceive load changes in arid regions.

[0083] An adaptive attention-based load forecasting method for arid regions, as described in this invention, can effectively improve the accuracy and robustness of power load forecasting in arid regions. By dynamically focusing on key meteorological features through an adaptive attention mechanism and combining it with a standardized feature preprocessing workflow, it significantly improves data quality and nonlinear correlation modeling capabilities.

[0084] Example 2 The following is a detailed description of a load prediction method for arid regions based on adaptive attention, according to an embodiment of the present invention.

[0085] Feature processing: S10: Feature Data Acquisition. Two types of raw data are collected as basic features: Load Feature Data: Historical electricity load sequence of the region, denoted as... ,in Indicates the first Load value at any given time (unit: MW). Total number of data entries; Influence factor characteristic data: including time characteristics (year, month, day, hour, day of the week, whether it is a holiday, etc.) and meteorological characteristics (temperature). ,humidity Rainfall (etc.), denoted as the set of impact factors. ,in For the first Class of impact factors This represents the total number of impact factors.

[0086] S20: Data cleaning, which involves processing missing and outlier values ​​in the collected raw data. Specifically, this includes: Outlier handling: using... Criteria for identifying outliers, i.e., for load characteristics or impact factor If a certain data point satisfy (in The mean of the data. If the value is less than the standard deviation, it is considered an outlier and replaced with the mean of the three normal data points before and after it. Missing value handling: For missing values ​​in the time series, linear interpolation is used to fill in the missing values; that is, if the first missing value is less than the standard deviation, the missing value is filled in. Time data Missing data; the previous time step was... The data for the next time step is The formula for filling in the gaps is: The above processing ensures the integrity and continuity of the data sequence.

[0087] S30: Feature Construction and Transformation. This involves converting raw features into structured features that the model can recognize, including: Time Feature Discretization: converting "whether it's a holiday" into binary features (1 for holidays, 0 for non-holidays), and converting "day of the week" into 7-dimensional one-hot encoded features; Meteorological Feature Normalization Preprocessing: for continuous meteorological features such as temperature and humidity, first compressing them using the maximum-minimum method to... This interval serves as preparation for subsequent standardization.

[0088] S40: Feature selection, selecting effective features based on the correlation between features and loads, specifically: calculating each influencing factor. With load sequence Pearson correlation coefficient The formula is:

[0089] in, The mean of the load series, For the first The mean of the class impact factors, For the first Class of impact factors in the first The value at any given time. Filter out. The influencing factors are used as key features, and low-correlation features are removed to reduce model complexity.

[0090] S50: Feature standardization. Key features after screening are standardized to eliminate dimensional differences using the Z-score standardization formula.

[0091] in, For the original value of the feature, The mean of this feature. This represents the standard deviation of the feature. After standardization, the mean of all features is 0 and the standard deviation is 1, ensuring that the model distributes the weights of each feature evenly.

[0092] Through the above steps, the original messy feature data can be transformed into high-quality, low-redundancy input features, providing reliable input for subsequent load forecasting models (such as neural networks, regression models, etc.) and significantly improving forecast accuracy.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0094] Description of the transformer model: Input embedding layer: The one-dimensional time series data obtained from S10 Mapped to a high-dimensional feature space, through linear transformation ( For embedding layer parameters, (For model feature dimension), to obtain embedded features Simultaneously, positional encoding is added to preserve the sequential information of the time-series data, and the final input feature is... ( (for the position encoding matrix) Encoder layer: the core of the model, employing... Multiple parallel self-attention heads (multi-head attention) capture the correlation features of the load factor. The multi-head attention calculation is as follows: MultiHead Concat (head) head head

[0095]

[0096] in, (The query, key, and value all come from the input features.) For the first Parameters of an attention head Output projection parameters for multi-head attention; Self-attention calculation follows the core formula (achieving dynamic focusing of fault features):

[0097] After the multi-head attention output, it sequentially passes through "residual connection + layer normalization" and "feedforward network". "Residual connectivity + layer normalization" enhances feature representation capabilities; Classification output layer: The global features output by the encoder layer. Mapped to the load forecast space, the load value probability is output through linear transformation and the Softmax function: .in, Classification layer parameters For the number of fault categories, such as "temperature". ,humidity Rainfall "correspond Each element in the array represents the probability that a sample belongs to the corresponding fault category.

[0098] Model training process: After feature normalization, the features are input into the model for training.

[0099] Example 3 To achieve the above embodiments, such as Figure 2 As shown, this embodiment also provides a drought load prediction device 10 based on adaptive attention. The device 10 includes a data acquisition module 100, an outlier and missing value processing module 200, a feature transformation and normalization module 300, a key feature screening and standardization module 400, and a feature input and model training module 500.

[0100] Data acquisition module 100 is used to collect historical power load sequences and influencing factor data including time characteristics and meteorological characteristics in arid regions; The outlier and missing value processing module 200 is used to identify outliers and fill in missing values ​​in the collected load sequence and influence factor data. Outliers are replaced by the mean of the adjacent normal data, and missing values ​​are filled by linear interpolation. The feature conversion and normalization module 300 is used to convert time features into binary features and unique thermal coded features, and to normalize meteorological features. The key feature screening and standardization module 400 is used to screen key features based on the correlation between the load sequence and each influencing factor, retain features whose correlation with the load threshold is greater than a preset value, and standardize the screened features. The feature input and model training module 500 is used to train a Transformer model containing an adaptive attention mechanism with standardized feature input. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and feedforward network to achieve time-series-meteorological coupled modeling, and outputs load prediction results.

[0101] Furthermore, the outlier and missing value processing module 200 is also used for: The 3σ criterion is used to identify outliers. That is, for load characteristics or influencing factors, if a data point x satisfies... (in The mean of the data. If the value is less than the standard deviation, it is considered an outlier. Replace outliers with the mean of the three normal data points before and after the data point, and fill in missing values ​​using linear interpolation. That is, if the data at time t... The missing part is filled using the following formula: .

[0102] Furthermore, the aforementioned feature transformation and normalization module 300 is also used for: Convert whether it is a holiday into a binary feature, and convert the day of the week into a 7-dimensional one-hot encoded feature; The continuous meteorological characteristics of temperature and humidity are compressed to the [0,1] interval using the maximum-minimum method.

[0103] Furthermore, the aforementioned key feature screening and standardization module 400 is also used for: Calculate each influencing factor Pearson correlation coefficient with load series L The formula is ; Filter out The influencing factors were used as key features, and the Z-score normalization formula was adopted. ,in The characteristic mean, The characteristic standard deviation is denoted as .

[0104] Furthermore, the aforementioned feature input and model training module 500 is also used for: Through linear transformation of the input embedding layer ( , (For embedding layer parameters) are mapped to a high-dimensional feature space; Add position encoding matrix To preserve the sequential information of time-series data, the final input feature is .

[0105] Furthermore, device 10 also includes: Through a dynamic weight adjustment mechanism for multi-head attention, the attention heads automatically enhance the weight allocation of relevant features based on the meteorological characteristics of arid regions. The specific calculation formula is as follows: MultiHead , , in , , , Let be the parameters of the i-th attention head. , , Output projection parameters for multi-head attention.

[0106] An adaptive attention-based load forecasting device for arid regions, according to an embodiment of the present invention, can effectively improve the accuracy and robustness of power load forecasting in arid regions. It dynamically focuses on key meteorological features through an adaptive attention mechanism and, combined with a standardized feature preprocessing process, significantly improves data quality and nonlinear correlation modeling capabilities.

[0107] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 3 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the method described above.

[0108] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0109] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0110] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A load forecasting method for arid regions based on adaptive attention, characterized in that, include: S1, collect historical power load sequences in arid regions and data on influencing factors including temporal and meteorological characteristics; S2, outlier identification and missing value imputation are performed on the collected load sequence and influence factor data. Outliers are replaced by the mean of the adjacent normal data, and missing values ​​are imputed by linear interpolation. S3 converts time features into binary features and unique thermal coding features, and normalizes meteorological features. S4. Based on the correlation between the load sequence and each influencing factor, key features are screened, features with a correlation threshold greater than the preset value are retained, and the screened features are standardized. S5 trains a Transformer model with standardized features and an adaptive attention mechanism. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and a feedforward network to achieve time-series-meteorological coupled modeling, outputting load prediction results.

2. The method as described in claim 1, characterized in that, The S2 includes: S21, the 3σ criterion is used to identify outliers, that is, for load characteristics or influencing factors, if a data point x satisfies... ,in The mean of the data. If the value is less than the standard deviation, it is considered an outlier. S22, replace the outlier with the mean of the three normal data points before and after the data point, and fill in the missing values ​​using linear interpolation. That is, if the data at time t... The missing part is filled using the following formula: .

3. The method as described in claim 1, characterized in that, The S3 includes: S31 converts whether it is a holiday into a binary feature and the day of the week into a 7-dimensional one-hot encoded feature. S32 compresses the continuous meteorological characteristics of temperature and humidity into the [0,1] interval using the maximum-minimum method.

4. The method as described in claim 1, characterized in that, The S4 further includes: S41, Calculate each influencing factor Pearson correlation coefficient with load series L The formula is ； S42, filter out The influencing factors were used as key features, and the Z-score normalization formula was adopted. ,in The characteristic mean, The characteristic standard deviation is denoted as .

5. The method as described in claim 1, characterized in that, The S5 includes: S51, through linear transformation of the input embedding layer , These are the parameters for the embedding layer, mapped to a high-dimensional feature space. S52, Add position encoding matrix To preserve the sequential information of time-series data, the final input feature is .

6. The method as described in claim 1, characterized in that, Also includes: S6 utilizes a dynamic weight adjustment mechanism for multi-head attention to automatically enhance the weight allocation of relevant features based on the meteorological characteristics of arid regions. The specific calculation formula is as follows: MultiHead , ， in , , , Let be the parameters of the i-th attention head. , , Output projection parameters for multi-head attention.

7. A load forecasting device for arid regions based on adaptive attention, characterized in that, include: The data acquisition module is used to collect historical power load sequences and data on influencing factors including temporal and meteorological characteristics in arid regions. The outlier and missing value processing module is used to identify outliers and impute missing values ​​in the collected load sequence and influence factor data. Outliers are replaced by the mean of the adjacent normal data, and missing values ​​are imputed by linear interpolation. The feature conversion and normalization module is used to convert time features into binary features and uniquely thermal coded features, and to normalize meteorological features. The key feature screening and standardization module is used to screen key features based on the correlation between the load sequence and each influencing factor, retain features whose correlation with the load threshold is greater than a preset value, and standardize the screened features. The feature input and model training module is used to train a Transformer model with an adaptive attention mechanism based on the standardized feature input. It dynamically adjusts the weight distribution of key features in arid regions through multi-head attention, and combines location encoding and feedforward network to achieve time-series-meteorological coupled modeling, outputting load prediction results.

8. The apparatus as claimed in claim 7, characterized in that, The outlier and missing value processing module is also used for: The 3σ criterion is used to identify outliers. That is, for load characteristics or influencing factors, if a data point x satisfies... If it is, then it is determined to be an outlier. The mean of the data. Standard deviation; Replace outliers with the mean of the three normal data points before and after the data point, and fill in missing values ​​using linear interpolation. That is, if the data at time t... The missing part is filled using the following formula: .

9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the adaptive attention-based load prediction method for arid regions as described in any one of claims 1-6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the adaptive attention-based load forecasting method for arid regions as described in any one of claims 1-6.

Images