Predicted power data determination method and device, storage medium and electronic device

Abstract

The invention discloses a prediction power data determination method and device, a storage medium and an electronic device, and the method comprises the steps: mapping original power data to a high-dimensional space, and obtaining first sequence data; performing weight processing on the first sequence data through a first encoder to obtain second sequence data; performing prediction processing on the second sequence data through a plurality of expert prediction sub-models to obtain a plurality of pieces of third sequence data; and aggregating the plurality of third sequence data by using a first decoder to obtain fourth sequence data, and determining predicted power data corresponding to the original power data based on the fourth sequence data. The problem that the data prediction process of large-scale power data is complex and poor in precision is solved.

Description

Technical Field

[0001] This application relates to the field of machine learning, and more specifically, to a method and apparatus for determining predictive power data, a storage medium, and an electronic device. Background Technology

[0002] In the field of power data forecasting, especially for forecasting power generation clusters, traditional methods largely rely on single models or linear combination models. These models face significant challenges and limitations when processing power data, particularly large-scale and complex time-series data. Specifically, existing technologies have the following main shortcomings:

[0003] 1. Uneven load: When processing large-scale power data, existing time series forecasting models are often based on a single neural network architecture, such as the classic Transformer model. Although the Transformer performs well in forecasting tasks, its computational complexity increases linearly with the sequence length. When processing long sequence data, the allocation of computing resources may be uneven, leading to some computing units being overloaded while others are underutilized, affecting the overall efficiency and prediction accuracy of the model.

[0004] 2. Limited Dynamic Adaptability: Existing models suffer from limited adaptability when dealing with complex and variable power data, such as mixed time-series data containing long-term trends and short-term abrupt changes. The model structure often struggles to dynamically adjust the contribution of its internal components according to the characteristics of the input data, resulting in insufficient ability to capture multi-scale time dependencies and thus affecting the model's ability to generalize to complex power patterns.

[0005] 3. Unstable Training: In Hybrid Expert (MoE) architectures, expert selection is typically achieved through a hard routing mechanism, where each input is assigned to only a fixed number of experts. This discrete routing approach can lead to vanishing or exploding gradients during training, making model training unstable and difficult to converge to the optimal solution.

[0006] 4. Insufficient ability to model sequence information: In existing time series forecasting models, attention mechanisms and location encoding methods may not be sufficient to fully capture long-distance dependencies and diverse contextual patterns in power data. This limits the model's accurate modeling of time patterns, especially when dealing with variable-length sequences.

[0007] 5. Limited prediction accuracy: The above-mentioned limitations work together to make it difficult for existing models to achieve a high level of prediction accuracy when dealing with large-scale power data, especially when dealing with variable and complex power time series data.

[0008] In summary, existing technologies suffer from several problems when processing large-scale power data forecasting, including uneven allocation of computing resources, poor dynamic adaptability, unstable training, insufficient modeling of sequence information, and limited prediction accuracy.

[0009] There is currently no effective solution to the problem of complex and inaccurate data prediction processes for large-scale power data in related technologies.

[0010] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0011] This application provides a method and apparatus for predicting power data, a storage medium, and an electronic device, which at least solves the problems of complex and poor accuracy in the data prediction process for large-scale power data.

[0012] According to one aspect of the embodiments of this application, a method for determining predicted power data is provided, applied to a power generation equipment cluster, comprising: mapping the original power data to a high-dimensional space to obtain a first sequence of data; performing weight processing on the first sequence of data through a first encoder to obtain a second sequence of data; performing prediction processing on the second sequence of data through multiple expert prediction sub-models to obtain multiple third sequence of data; aggregating the multiple third sequence of data using a first decoder to obtain a fourth sequence of data, and determining the predicted power data corresponding to the original power data based on the fourth sequence of data.

[0013] In an exemplary embodiment, mapping the original power data to a high-dimensional space to obtain a first sequence of data includes: parsing the original power data to obtain a set of values ​​and a set of timestamps corresponding to the original power data; mapping different scalar values ​​in the set of values ​​to a D-dimensional space using a preset linear projection method to obtain multiple spatial vectors, where D is a positive integer greater than or equal to 3; embedding time features into the multiple spatial vectors to obtain multiple sub-sequences; and combining the multiple sub-sequences based on the relative positional relationship of the different sub-vectors to generate the first sequence of data.

[0014] In an exemplary embodiment, before obtaining the second sequence data by weighting the first sequence data through the first encoder, the method further includes: determining the target number of subsequences contained in the first sequence data; generating a weight matrix based on a preset gating network and the target number, wherein the preset gating network is: W = Softmax(MLP(X), dim = -1) ∈ R T×E×CE represents the number of inputs to the expert prediction sub-model, C represents the maximum number of sub-sequences processed by each expert prediction sub-model in the first encoder in a single operation, the total number of sub-sequences shared by all expert prediction sub-models is m = E × C, and T represents the target number. The target weight corresponding to each expert prediction sub-model is determined through the weight matrix, and the calculation results of the associated expert prediction sub-models are weighted according to the target weight.

[0015] In an exemplary embodiment, before obtaining multiple third sequence data by predicting the second sequence data through multiple expert prediction sub-models, the method further includes: obtaining a causal mask list corresponding to the multiple expert prediction sub-models; and controlling the order in which the multiple expert prediction sub-models process the second sequence data according to the causal mask list.

[0016] In an exemplary embodiment, a fourth sequence of data is obtained by aggregating multiple third sequence data using a first decoder, including: obtaining a first formula for aggregation by the first decoder, wherein the first formula is: Z d =∑ e,c W d ·Expert d e (Y d '[:,e,c,:]), W d To decode the weight assignment matrix, Z d For the fourth sequence of data in the decoded output, Y d ' is the weighted copy generated during weighted distribution, e is the number of expert prediction sub-models input in the decoder, and c is the maximum number of sub-sequences processed by each expert prediction sub-model in the first decoder in a single operation; multiple third sequence data are processed by the first formula to obtain multiple fourth sequence sub-data; multiple fourth sequence sub-data are aggregated to obtain the fourth sequence data.

[0017] In an exemplary embodiment, before determining the predicted power data corresponding to the original power data based on the fourth sequence data, the method further includes: determining the data generation requirements corresponding to the original power data; performing hidden state mapping processing on the fourth sequence data based on the data generation requirements to obtain a target numerical sequence; and adjusting the target numerical sequence according to a preset hybrid loss function.

[0018] According to another aspect of the embodiments of this application, a device for determining predicted power data is also provided, applied to a power generation equipment cluster, comprising: a mapping module for mapping raw power data to a high-dimensional space to obtain a first sequence of data; a first processing module for performing weight processing on the first sequence of data through a first encoder to obtain a second sequence of data; a second processing module for performing prediction processing on the second sequence of data through multiple expert prediction sub-models to obtain multiple third sequence of data; and a determination module for aggregating the multiple third sequence of data using a first decoder to obtain a fourth sequence of data, and determining the predicted power data corresponding to the raw power data based on the fourth sequence of data.

[0019] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described method for determining predicted power data when it is run.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the predicted power data through the computer program.

[0021] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program and a method for determining the above-mentioned predicted power data when the computer program is executed by a processor.

[0022] This application, based on identifying the power generation equipment cluster to be predicted, involves: mapping the original power data corresponding to different power generation equipment to a high-dimensional space to obtain a first sequence of data; weighting the first sequence of data using a first encoder to obtain a second sequence of data; performing prediction processing on the second sequence of data using multiple expert prediction sub-models to obtain multiple third sequence of data; aggregating the multiple third sequence of data using a first decoder to obtain a fourth sequence of data; and determining the predicted power data corresponding to the original power data based on the fourth sequence of data. This solves the problem of complex and inaccurate data prediction processes for large-scale power data, improving both the efficiency and accuracy of power data prediction. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a hardware structure block diagram of a computer terminal for a method of determining predicted power data according to an embodiment of this application.

[0026] Figure 2 This is a flowchart of a method for determining predicted power data according to an embodiment of this application;

[0027] Figure 3 This is a flowchart illustrating a time series prediction model combining Soft-MoE and Transformer according to an embodiment of this application.

[0028] Figure 4 This is a schematic diagram of the input embedding layer processing flow according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the encoder structure according to an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of the decoder structure according to an embodiment of this application;

[0031] Figure 7 This is a structural block diagram of a device for determining predictive power data according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] The methods and embodiments provided in this application can be executed on a computer terminal, mobile terminal, or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure block diagram of a computer terminal for a method of determining predicted power data according to an embodiment of this application. Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a central processing unit (CPU) or a field-programmable gate array (FPGA)) and a memory 104 for storing data are also shown. The computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the network point adjustment method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0037] This embodiment provides a method for determining predicted power data. Figure 2 This is a flowchart of a method for determining predicted power data according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps S202-S208:

[0038] Step S202: Map the raw power data to a high-dimensional space to obtain the first sequence of data;

[0039] As an optional implementation, raw power data (such as current, voltage, and power) from the power generation cluster is transformed into a higher-dimensional representation to facilitate subsequent processing by deep learning models. This mapping is typically accomplished through the following operations:

[0040] Value embedding: The numerical values ​​in the original data (such as power readings) are transformed into a higher-dimensional vector through linear projection. This process increases the information capacity of the data and enables the data to be processed more effectively in the context of neural networks.

[0041] Temporal feature embedding: In addition to the numerical values ​​themselves, the temporal attributes of the data are also encoded. This can be achieved through periodic features (such as sine and cosine encoding) or by directly encoding timestamps (hours, days of the week, seasons, etc.), ensuring that the model can capture the periodicity and seasonality of the data over time.

[0042] Location Encoding: Since power data often forms time series, location encoding is used to identify the location information of each data point in the series, which is crucial for the model to understand and predict time series patterns. In this step, relative location encoding is used, which can automatically adapt to series of different lengths and can handle variable-length data without modifying the model architecture.

[0043] The data after the above operations is called "first sequence data", which contains the numerical information, time attributes and relative position information of the original power data.

[0044] Step S204: The first sequence data is weighted by the first encoder to obtain the second sequence data;

[0045] Optionally, the first encoder described above can be an encoder incorporating a Soft-MoE mechanism, used to process the first sequence data that has already been mapped to a high dimension. The Soft-MoE mechanism allows each data point to be weighted among all experts, rather than simply assigned to a single expert. This means that each data point can utilize the knowledge of multiple experts to varying degrees, which brings significant advantages in expert load balancing and model generalization ability. The data obtained after weighting is called the "second sequence data".

[0046] Step S206: The second sequence data is processed by multiple expert prediction sub-models to obtain multiple third sequence data;

[0047] Understandably, based on the second-series data, a set of specially designed expert prediction sub-models are used for in-depth analysis and prediction. These expert prediction sub-models are actually a series of independent, specially optimized small models, each focusing on different power data characteristics or patterns. Through parallel computing, each expert independently processes the second-series data and provides its own prediction results, generating multiple prediction sequences—the "third-series data." This multi-expert architecture can capture the diverse features in the data, thereby improving the accuracy and robustness of predictions.

[0048] Step S208: Use the first decoder to aggregate the multiple third sequence data to obtain fourth sequence data, and determine the predicted power data corresponding to the original power data based on the fourth sequence data.

[0049] The first decoder, incorporating a Soft-MoE mechanism, integrates the predictions from multiple experts (third-sequence data) into a unified prediction sequence—the "fourth-sequence data." Because the decoder also employs the Soft-MoE mechanism, it ensures that all expert predictions are appropriately weighted and fused. Based on the fourth-sequence data, the prediction results for the original power data can be directly derived, completing the entire prediction process.

[0050] The above steps, based on identifying the power generation equipment cluster to be predicted, involve: mapping the original power data corresponding to different power generation equipment to a high-dimensional space to obtain a first sequence of data; weighting the first sequence of data using a first encoder to obtain a second sequence of data; performing prediction processing on the second sequence of data using multiple expert prediction sub-models to obtain multiple third sequence of data; aggregating the multiple third sequence of data using a first decoder to obtain a fourth sequence of data; and determining the predicted power data corresponding to the original power data based on the fourth sequence of data. This approach solves the problem of complexity and poor accuracy in the data prediction process for large-scale power data, improving both the efficiency and accuracy of power data prediction.

[0051] In an exemplary embodiment, mapping the original power data to a high-dimensional space to obtain a first sequence of data includes: parsing the original power data to obtain a set of values ​​and a set of timestamps corresponding to the original power data; mapping different scalar values ​​in the set of values ​​to a D-dimensional space using a preset linear projection method to obtain multiple spatial vectors, where D is a positive integer greater than or equal to 3; embedding time features into the multiple spatial vectors to obtain multiple sub-sequences; and combining the multiple sub-sequences based on the relative positional relationship of the different sub-vectors to generate the first sequence of data.

[0052] Understandably, the raw power data collected from the power generation equipment cluster is parsed to separate the numerical components and time information, forming a value set and a timestamp set. The value set contains the specific numerical values ​​of the power data, such as power consumption, voltage, and current intensity. The timestamp set records the time information for each data point, used to determine the temporal sequence and periodicity of the data.

[0053] The scalar values ​​in the set of values ​​are mapped to a higher dimension (D-dimensional space) using a predefined linear projection method, where D is typically a positive integer greater than or equal to 3. This mapping process increases the representational power of the data, allowing numerical features to express their complexity in a higher dimension, and also facilitating processing by deep learning models (such as the Transformer model). For example, if the original data is electricity consumption (a scalar value), then through linear projection, this value will be transformed into a D-dimensional vector, allowing the model to capture the characteristics of electricity consumption from multiple dimensions. Furthermore, the vector mapped to the higher-dimensional space is then embedded with temporal features. Temporal feature embedding allows the model to understand the temporal attributes and potential periodic patterns of the data. Optionally, temporal features can be determined through periodic feature encoding or timestamp feature encoding. Periodic feature encoding (such as sine and cosine function encoding) is used to capture the periodic changes in the data, such as peak and off-peak electricity usage throughout the day. Timestamp feature encoding directly encodes timestamp information into a vector, which may include finer time granularity, such as hours, minutes, or days of the week. After being embedded with temporal features, each high-dimensional vector will carry richer temporal information, forming multiple subsequences.

[0054] Since electricity data is arranged in a time series, location encoding is required to enable the model to handle sequences of varying lengths. Location encoding helps the model understand the structure and positional relationships of the sequence by assigning a unique representation to each position within it.

[0055] In this embodiment, relative position encoding is used. Unlike absolute position encoding, relative position encoding can adapt to changes in sequence length. It does not require assigning a fixed code to each absolute position in the sequence. Instead, it calculates the code based on the relative positions between data points in the sequence, making the model more flexible and efficient when processing variable-length sequences.

[0056] Ultimately, all spatial vectors and subsequences that have undergone value embedding, temporal feature embedding, and location encoding will be integrated and spliced ​​together based on their relative positional relationships to form the "first sequence data." This sequence data contains the numerical features, temporal features, and location information of the power data, providing a rich and complete information foundation for subsequent forecasting processing.

[0057] In an exemplary embodiment, before obtaining the second sequence data by weighting the first sequence data through the first encoder, the method further includes: determining the target number of subsequences contained in the first sequence data; generating a weight matrix based on a preset gating network and the target number, wherein the preset gating network is: W = Softmax(MLP(X), dim = -1) ∈ R T×E×CE represents the number of inputs to the expert prediction sub-model, C represents the maximum number of sub-sequences processed by each expert prediction sub-model in the first encoder in a single operation, the total number of sub-sequences shared by all expert prediction sub-models is m = E × C, and T represents the target number. The target weight corresponding to each expert prediction sub-model is determined through the weight matrix, and the calculation results of the associated expert prediction sub-models are weighted according to the target weight.

[0058] Optionally, the weight matrix (W) is generated using a pre-defined gating network that processes the input sequence data using a multilayer perceptron (MLP). The MLP receives the first sequence data as input (X) and calculates the importance weight of each expert for each data point (or subsequence) using the Softmax function. The weight matrix has dimensions (T*E*C), where T is the target number, i.e., the total number of subsequences in the first sequence data. E is the number of expert prediction sub-models, i.e., how many expert models are available for data prediction analysis. C is the maximum number of subsequences that each expert model can process in a single pass, meaning that each expert model can process multiple data points, not just a single point. Each row of the weight matrix corresponds to a data point (subsequence), and each column corresponds to a slot in an expert model. The entire matrix represents the weight distribution of each data point across the various expert models. These weights determine how data points should be allocated among different expert models and the weighting ratio of each expert model's output.

[0059] After generating the weight matrix, the next step is to determine the target weight for each expert prediction sub-model. Here, "target weight" refers to the probability of a data point being assigned to a specific expert model, or in other words, the degree of contribution of that expert model should be considered when processing the data point. With the weight matrix, the weight of each data point in each expert model can be determined through a weight allocation mechanism. This means that each data point will be assigned a series of weight values, corresponding to different expert models. During weight processing, the features of the data point will be sent to all expert models simultaneously for processing according to the weight values ​​in the weight matrix. The output of each expert model will be weighted according to its respective data point weight. This weighting ensures that the data point does not depend solely on a single expert model, but is based on the common contribution of all expert models, thus obtaining a more comprehensive and balanced processing result—the second sequence data.

[0060] Furthermore, by interacting with all expert models through data points, some expert models may become overloaded due to processing too many data points, while other expert models are not left idle, thereby optimizing the use of computing resources and achieving load balancing. At the same time, the degree of collaboration between expert models is dynamically changed according to the characteristics of data points, enabling the models to more flexibly cope with different types of data and prediction scenarios, and achieving dynamic collaborative operation.

[0061] In summary, through the above implementation methods, by generating a weight matrix and performing weight processing, the first encoder with the Soft-MoE mechanism can more intelligently allocate and process data, thereby improving the accuracy of prediction while maintaining computational efficiency.

[0062] In an exemplary embodiment, before obtaining multiple third sequence data by predicting the second sequence data through multiple expert prediction sub-models, the method further includes: obtaining a causal mask list corresponding to the multiple expert prediction sub-models; and controlling the order in which the multiple expert prediction sub-models process the second sequence data according to the causal mask list.

[0063] In simple terms, in the Soft-MoE architecture, since data is distributed across multiple expert prediction sub-models for processing, a specific causal mask needs to be generated for each expert model to ensure they do not violate the inherent temporal order of the sequence when processing data. These masks are typically upper triangular matrices, where elements below the diagonal are 1s, indicating that the model can access data from previous time steps; elements above the diagonal are 0s, indicating that the model cannot access future data. After obtaining the causal masks for all expert prediction sub-models, the next step is to use these masks to control the order in which the expert models process the second sequence data (i.e., the sequence data after being weighted by the Soft-MoE encoder). Specifically, when a data point is assigned to an expert model, that expert model applies its corresponding causal mask to limit its attention scope, focusing only on data points preceding that data point and ignoring subsequent data points. This operation ensures that each expert prediction sub-model adheres to the autoregressive principle when making predictions, meaning they can only predict based on historical information and cannot "see" future data in advance. In this way, even if the expert prediction sub-models work in parallel, their prediction process remains orderly, following the prediction logic of the autoregressive model, thus ensuring the correctness and effectiveness of the prediction results.

[0064] In an exemplary embodiment, a fourth sequence of data is obtained by aggregating multiple third sequence data using a first decoder, including: obtaining a first formula for aggregation by the first decoder, wherein the first formula is: Z d =∑ e,c W d ·Expert d e (Y d '[:,e,c,:]), W d To decode the weight assignment matrix, Z d For the fourth sequence of data in the decoded output, Y d' is the weighted copy generated during weighted distribution, e is the number of expert prediction sub-models input in the decoder, and c is the maximum number of sub-sequences processed by each expert prediction sub-model in the first decoder in a single operation; multiple third sequence data are processed by the first formula to obtain multiple fourth sequence sub-data; multiple fourth sequence sub-data are aggregated to obtain the fourth sequence data.

[0065] Optional, Expert d e (Y d '[:,e,c,:]) represents the (e)th expert prediction sub-model in the (d)th layer decoder, Y d '[:,e,c,:] refers to the weighted copy of the expert prediction sub-model (e) and slot (c) generated during weighted distribution.

[0066] In the decoder, the output of each expert prediction sub-model (third-sequence data) is weighted and aggregated according to the formula above. This formula uses matrix multiplication and summation operations, based on W... d The value of W is used to adjust the contribution of each expert sub-model output. This means that the prediction result for each data point will be affected by the outputs of all expert sub-models simultaneously, but the degree of influence is determined by W. d The weight values ​​in the formula determine the fourth sequence data. The decoder summarizes all the fourth sequence sub-data processed by the formula to generate the complete fourth sequence data. This sequence data contains the comprehensive prediction results of all expert prediction sub-models. After aggregation and adjustment by the decoder, it can accurately reflect the predicted values ​​of the original power data.

[0067] In an exemplary embodiment, before determining the predicted power data corresponding to the original power data based on the fourth sequence data, the method further includes: determining the data generation requirements corresponding to the original power data; performing hidden state mapping processing on the fourth sequence data based on the data generation requirements to obtain a target numerical sequence; and adjusting the target numerical sequence according to a preset hybrid loss function.

[0068] Understandably, before converting fourth-series data into predicted power data, it's crucial to first define the specific requirements for power forecasting. These requirements might include the frequency of forecasting (e.g., minute-level, hourly, or daily), the range of forecasting (how many time steps into the future), and the required accuracy. Defining these requirements is a critical first step, guiding the adjustment and optimization of the model output and ensuring the forecast results are valuable in practical applications. After clarifying the data generation requirements, the next step is to further process the fourth-series data to generate predicted values ​​that meet the requirements. This typically involves mapping the high-dimensional vectors output by the decoder back to the numerical space of the original data—a process known as hidden-state mapping.

[0069] Specifically, hidden state mapping processing can include the following:

[0070] Linear mapping: This involves mapping each high-dimensional vector in the fourth sequence data back to the numerical range of the original power data through a linear projection. This may involve using one or more linear transformation layers, such as fully connected layers, to adjust the dimension and value of the vectors so that they can be interpreted as actual physical quantities such as power consumption and power.

[0071] Probabilistic output: In some cases, especially for prediction scenarios with high uncertainty, the model output can be mapped to a probability distribution, thereby providing confidence intervals or probability density functions for the prediction results, which helps in risk assessment and decision support.

[0072] After the hidden state mapping process, the model will generate a target numerical sequence, which directly reflects the prediction result of the original power data and usually includes the values ​​of the prediction time step.

[0073] To ensure the accuracy of the prediction results, the model needs to adjust the target numerical sequence based on a pre-defined hybrid loss function. The hybrid loss function typically combines the characteristics of multiple loss functions to comprehensively evaluate various aspects of the prediction results. For example, a hybrid loss function might include:

[0074] Mean Absolute Error (MAE): Used to assess the absolute difference between predicted and actual values, ensuring the accuracy of prediction results.

[0075] Mean Squared Error (MSE): Calculates the average of the squared differences between the predicted and actual values, and imposes a higher penalty for larger prediction errors.

[0076] Quantile Loss: Used to quantify the uncertainty of forecasts and ensure the predictive performance of the model at different data quantiles, such as the 10th, 50th (median), and 90th percentiles. This helps to understand the distribution characteristics of electricity demand, especially in the prediction of extreme events.

[0077] By minimizing the value of the hybrid loss function, the model can continuously adjust its parameters during training to optimize the prediction results, ensuring that the predicted power data is not only accurate, but also fully considers the distribution characteristics of the data, and adapts to different prediction needs and scenarios.

[0078] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the above method, the following description, in conjunction with embodiments, illustrates the process, but is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0079] An optional embodiment of this application provides a time series prediction method combining Soft-MoE and Transformer, replacing hard routing with continuous expert weight allocation, and achieving the following optimizations:

[0080] Load balancing: alleviates competition for expert resources through soft allocation mechanisms, such as superimposing expert weights on the input features at each time step, rather than forcibly selecting a single expert.

[0081] Dynamic collaboration: Allows different expert networks to dynamically adjust their contributions based on the complexity of the input features, for example, by using differentiable gating to achieve a continuous probability distribution of the expert portfolio.

[0082] Complete differentiability: Routing and aggregation in Soft-MoE are continuous operations, supporting end-to-end backpropagation, which solves the training instability problem caused by the discrete routing of traditional MoE.

[0083] No token loss: All tokens participate in the calculations of all experts, avoiding token loss caused by hard routing in traditional MoE.

[0084] Dynamic context modeling: Experts capture sequence information of different granularities through soft assignment, enhancing the encoder's ability to model long-distance dependencies and diverse contextual patterns.

[0085] Subsequently, by introducing Soft-MoE, the model maintains the computational efficiency advantage of sparse activation while improving its adaptability to complex time series patterns through a soft routing mechanism. That is, the Soft-MoE (Soft Mixture of Experts) mechanism is used to replace the feedforward network in the standard Transformer to build a more efficient time series prediction model.

[0086] As an optional implementation method, Figure 3 This is a schematic diagram of a time series prediction model combining Soft-MoE and Transformer according to an embodiment of this application. The time series prediction model adopts an Encoder-Decoder structure and mainly includes the following components: First component: Input embedding layer, which maps the original time series data to a high-dimensional space. Figure 4 This is a schematic diagram of the input embedding layer processing flow according to an embodiment of this application, which includes the following steps:

[0087] Step S402: Determine the original time series containing value data and timestamp data.

[0088] Step S404, Value Embedding: Linear projection maps scalar values ​​to d-dimensional space.

[0089] Step S406, Time Feature Embedding: ① Time features, i.e., periodic features (sin / cos encoding). ② Timestamp features (hours, days of the week, etc.).

[0090] Step S408: Determine the position code, that is, determine the relative position code to adapt to the variable length sequence.

[0091] Step S410: Determine the final embedded representation, such as a sequence format that can be value + time + position.

[0092] The second component is the Soft-MoE Transformer encoder, which consists of multiple layers of Soft-MoE Transformer blocks. It's a fusion design based on the Transformer architecture and Soft-MoE. Soft-MoE replaces the standard feedforward neural network (FFN) expert system architecture. Figure 5 This is a schematic diagram of the encoder structure according to an embodiment of this application. Each expert in the encoder part is an independent Transformer encoder feedforward network.

[0093] Optionally, the encoder's attention mechanism can be implemented using the standard Transformer multi-head self-attention mechanism, or an enhancement mechanism for the self-attention mechanism can be introduced. For example, time-aware position encoding using time-aware position encoding can be introduced; absolute position encoding injecting temporal position information using a sinusoidal function can be introduced; or relative position bias can be modeled by learningable parameters to determine the distance correlation between time steps.

[0094] Optionally, in the dynamic routing process of the aforementioned Soft-MoE mechanism: a slot is the basic unit for expert input processing, and each slot corresponds to a virtual processing position for an input token. The length of the input sequence is T (the number of tokens), and each token is assigned to all slots through a weight matrix for soft allocation.

[0095] 1) Given an input sequence X∈R n×d where n is the sequence length and d is the hidden dimension;

[0096] 2) Softmax weight assignment: For the input sequence X∈R n×d The gated network generates a weight matrix W∈R T×E×CW = Softmax(MLP(X), dim = -1), with normalized weights of 1 along the expert dimension. E is the number of experts, C is the maximum number of tokens processed by each expert in a single run (the number of slots per expert, which can also be understood as the expert's index), the total number of slots shared by all experts is m = E × C, and T is the total number of tokens in the input sequence.

[0097] 3) Weighted distribution: Each token generates E×C weighted copies. The outer product expands the dimension to T×E×C×d.

[0098] 4) Weights are assigned to experts based on the weight matrix, and expert calculations are performed in parallel. The aggregated result Y = ∑ e,c W·Expert e (X ′ [:,e,c,:]).

[0099] The third component is the Soft-MoE Transformer decoder. Generally, the decoder can be implemented using a standard Transformer decoder structure; however, a combined Soft-MoE and Transformer decoder implementation is also designed here. Each expert in the decoder part is an independent Transformer decoder feedforward network. Figure 6 This is a schematic diagram of the decoder structure according to an embodiment of this application. The input of the Soft-MoE Transformer decoder is the output of the encoder part, and its implementation is similar to that of the Soft-MoE Transformer encoder. The routing allocation adopts the same implementation method as the encoder. Here, Expert 1 to Expert E are a group of different experts who perform Transformer decoding calculations.

[0100] The difference lies in the Transformer encoder and decoder. The decoder employs a masked multi-head self-attention mechanism. Causal masking ensures autoregressive generation, allowing attention to historical information only at the current time step. Furthermore, strategies such as dynamic head dimension allocation can be used to adapt to time series prediction tasks of varying lengths, dynamically adjusting the number of attention heads based on the input sequence length (e.g., allocating more heads to longer sequences).

[0101] Output aggregation is the process by which the decoder integrates multiple expert predictions. The expert aggregation method based on Soft-MoE can be achieved using the following formula: Z d =∑ e,c W d ·Expert d e (Y d '[:,e,c,:]), where Wd Z is the weight assignment matrix for the decoder part. d Y is the output of the soft-MoE Transformer decoder. d 'This is the weighted copy generated during weighted distribution. The superscript and subscript d represent the encoder part, distinguishing it from the decoder part. [:,e,c,:] indicates taking all values ​​of e and c in the corresponding dimension. E is the number of experts, and c is the number of times the expert's index appears.

[0102] The fourth component is the output layer. Specifically, the output layer is used to adapt to the needs of sequence generation tasks, mapping the hidden states of the last layer of the decoder to predicted values. It can employ linear projection layers, probabilistic output layers, or task-oriented hybrid structures.

[0103] It should be noted that the training strategy for the above time series prediction model is as follows:

[0104] (1) The prediction loss function adopts MSE or MAE loss and expert load balancing loss.

[0105] (2) Based on the characteristics of the model structure, the training strategy is implemented in two stages:

[0106] Pre-training phase (EncoderFocus): A progressive learning rate (initial value 3e-4, cosine decay) can be used to prioritize optimizing the encoder's Soft-MoE dynamic routing and time-aware positional encoding. L2 regularization (λ = 0.01) is applied to timestamp feature embeddings (hours / weekdays, etc.) to enhance the generalization of periodic features. The encoder is pre-trained through a masked sequence reconstruction task (randomly masking 20% ​​of time segments), enabling the Soft-MoE expert system to prioritize learning local and global temporal patterns.

[0107] Fine-tuning phase (Decoder-Output Joint Tuning): With encoder parameters fixed, the causal attention and multi-expert aggregation modules of the decoder are fine-tuned using hierarchical learning rates (2e-4 for the decoder FFN layer and 5e-4 for the output layer). A hybrid loss function is introduced: MAE (Main Loss) + multiquantile loss (10% / 50% / 90th quantile, weight ratio 1:2:1) to balance deterministic prediction and uncertainty quantification.

[0108] In summary, the optional embodiments of this application utilize a Soft-MoE mechanism to avoid the load imbalance problem of classic MoE through soft weight allocation. The adaptive weight mechanism allows different experts to focus on local or global time-series features (such as abrupt events and long-term cycles), significantly improving prediction accuracy through multi-expert collaborative modeling.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0110] This embodiment also provides a device for determining predicted power data, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] Figure 7 This is a structural block diagram of a device for determining predicted power data according to an embodiment of this application. The device includes:

[0112] Mapping module 72 is used to map the raw power data to a high-dimensional space to obtain the first sequence of data;

[0113] The first processing module 74 is used to perform weight processing on the first sequence data through the first encoder to obtain the second sequence data;

[0114] The second processing module 76 is used to perform prediction processing on the second sequence data through multiple expert prediction sub-models to obtain multiple third sequence data;

[0115] The determination module 78 is used to aggregate the multiple third sequence data using the first decoder to obtain fourth sequence data, and to determine the predicted power data corresponding to the original power data based on the fourth sequence data.

[0116] The aforementioned device, based on determining the power generation equipment cluster to be predicted, proceeds as follows: It maps the original power data corresponding to different power generation equipment into a high-dimensional space to obtain a first sequence of data; it then performs weighted processing on the first sequence of data using a first encoder to obtain a second sequence of data; next, it performs prediction processing on the second sequence of data using multiple expert prediction sub-models to obtain multiple third sequence of data; finally, it uses a first decoder to aggregate the multiple third sequence of data to obtain a fourth sequence of data, and determines the predicted power data corresponding to the original power data based on the fourth sequence of data. This solves the problem of complexity and poor accuracy in the data prediction process for large-scale power data, improving both the efficiency and accuracy of power data prediction.

[0117] In an exemplary embodiment, the mapping module is further configured to parse the original power data to obtain a set of values ​​and a set of timestamps corresponding to the original power data; map different scalar values ​​in the set of values ​​to a D-dimensional space using a preset linear projection method to obtain multiple spatial vectors, where D is a positive integer greater than or equal to 3; embed time features into the multiple spatial vectors to obtain multiple sub-sequences; and combine the multiple sub-sequences based on the relative positional relationship of the different sub-vectors to generate a first sequence of data.

[0118] In an exemplary embodiment, the above apparatus further includes: a setting module, configured to determine the target number of subsequences contained in the first sequence data before obtaining the second sequence data by weighting the first sequence data through the first encoder; and to generate a weight matrix based on a preset gating network and the target number, wherein the preset gating network is: W = Softmax(MLP(X), dim = -1) ∈ R T×E×C E represents the number of inputs to the expert prediction sub-model, C represents the maximum number of sub-sequences processed by each expert prediction sub-model in the first encoder in a single operation, the total number of sub-sequences shared by all expert prediction sub-models is m = E × C, and T represents the target number. The target weight corresponding to each expert prediction sub-model is determined through the weight matrix, and the calculation results of the associated expert prediction sub-models are weighted according to the target weight.

[0119] In an exemplary embodiment, the above apparatus further includes: a third processing module, configured to obtain a causal mask list corresponding to the multiple expert prediction sub-models before performing prediction processing on the second sequence data through multiple expert prediction sub-models to obtain multiple third sequence data; and to control the order in which the multiple expert prediction sub-models process the second sequence data according to the causal mask list.

[0120] In an exemplary embodiment, the fourth determining module is further configured to obtain a first formula for aggregation by the first decoder, wherein the first formula is: Z d =∑ e,c W d ·Expertd e (Y d '[:,e,c,:]), W d To decode the weight assignment matrix, Z d For the fourth sequence of data in the decoded output, Y d ' is the weighted copy generated during weighted distribution, e is the number of expert prediction sub-models input in the decoder, and c is the maximum number of sub-sequences processed by each expert prediction sub-model in the first decoder in a single operation; multiple third sequence data are processed by the first formula to obtain multiple fourth sequence sub-data; multiple fourth sequence sub-data are aggregated to obtain the fourth sequence data.

[0121] In an exemplary embodiment, the above-mentioned apparatus further includes: an adjustment module, configured to determine the data generation requirements corresponding to the original power data before determining the predicted power data corresponding to the original power data based on the fourth sequence data; perform hidden state mapping processing on the fourth sequence data based on the data generation requirements to obtain a target numerical sequence; and adjust the target numerical sequence according to a preset hybrid loss function.

[0122] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0123] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0124] S1. Map the raw power data to a high-dimensional space to obtain the first sequence of data;

[0125] S2. The first sequence data is weighted by the first encoder to obtain the second sequence data;

[0126] S3. The second sequence data is predicted and processed by multiple expert prediction sub-models to obtain multiple third sequence data.

[0127] S4. Use the first decoder to aggregate the multiple third sequence data to obtain fourth sequence data, and determine the predicted power data corresponding to the original power data based on the fourth sequence data.

[0128] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0129] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0130] Embodiments of this application also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, performs the steps in any of the above method embodiments.

[0131] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0132] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0133] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0134] S1. Map the raw power data to a high-dimensional space to obtain the first sequence of data;

[0135] S2. The first sequence data is weighted by the first encoder to obtain the second sequence data;

[0136] S3. The second sequence data is predicted and processed by multiple expert prediction sub-models to obtain multiple third sequence data.

[0137] S4. Use the first decoder to aggregate the multiple third sequence data to obtain fourth sequence data, and determine the predicted power data corresponding to the original power data based on the fourth sequence data.

[0138] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0139] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0140] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0141] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining predictive power data, characterized in that, Applications include power generation equipment clusters, including: The raw power data is mapped to a high-dimensional space to obtain the first sequence of data; The first sequence data is weighted by the first encoder to obtain the second sequence data; the second sequence data is predicted by multiple expert prediction sub-models to obtain multiple third sequence data. The first decoder is used to aggregate the multiple third sequence data to obtain fourth sequence data, and the predicted power data corresponding to the original power data is determined based on the fourth sequence data.

2. The method according to claim 1, characterized in that, Mapping the raw power data to a high-dimensional space yields the first sequence of data, including: Parse the raw power data to obtain the value set and timestamp set corresponding to the raw power data; Using a preset linear projection method, different scalar values ​​in the value set are mapped to a D-dimensional space to obtain multiple space vectors, where D is a positive integer greater than or equal to 3; Temporal feature embedding is performed on the multiple spatial vectors to obtain multiple sub-sequences; The first sequence data is generated by combining the multiple sub-sequences based on the relative positional relationships of different sub-vectors.

3. The method according to claim 1, characterized in that, Before obtaining the second sequence data by weighting the first sequence data using the first encoder, the method further includes: Determine the target number of subsequences contained in the first sequence data; A weight matrix is ​​generated based on a preset gating network and the number of targets, wherein the preset gating network is: W = Softmax(MLP(X), dim = -1) ∈ R T×E×C E is the number of expert prediction sub-models input, C is the maximum number of sub-sequences processed by each expert prediction sub-model in the first encoder in a single operation, the total number of sub-sequences shared by all expert prediction sub-models is m = E × C, and T is the target number. The target weights for each expert prediction sub-model are determined using the weight matrix, and the calculation results of the associated expert prediction sub-models are weighted using the target weights.

4. The method according to claim 1, characterized in that, Before obtaining multiple third sequence data by performing prediction processing on the second sequence data through multiple expert prediction sub-models, the method further includes: obtaining a list of causal masks corresponding to the multiple expert prediction sub-models; The order in which the multiple expert prediction sub-models process the second sequence data is controlled according to the causal mask list.

5. The method according to claim 1, characterized in that, The first decoder is used to aggregate the multiple third sequence data to obtain fourth sequence data, including: Obtain the first formula used for aggregation by the first decoder, wherein the first formula is: Z d =∑ e,c W d ·Expert d e (Y d '[:,e,c,:]), W d To decode the weight assignment matrix, Z d For the fourth sequence of data in the decoded output, Y d ' is the weighted copy generated during weighted distribution, e is the number of expert prediction sub-models input in the decoder, and c is the maximum number of sub-sequences processed by each expert prediction sub-model in the first decoder in a single operation; The multiple third sequence data are processed using the first formula to obtain multiple fourth sequence sub-data. The fourth sequence data is obtained by summing up the multiple fourth sequence sub-data.

6. The method according to claim 1, characterized in that, Before determining the predicted power data corresponding to the original power data based on the fourth sequence data, the method further includes: Determine the data generation requirements corresponding to the original power data; Based on the data generation requirements, the fourth sequence data is subjected to hidden state mapping processing to obtain the target numerical sequence; The target numerical sequence is adjusted according to a preset hybrid loss function.

7. A device for determining predicted power data, characterized in that, Applications include power generation equipment clusters, including: The mapping module is used to map the raw power data to a high-dimensional space to obtain the first sequence of data. The first processing module is used to perform weight processing on the first sequence data through the first encoder to obtain the second sequence data; The second processing module is used to perform prediction processing on the second sequence data through multiple expert prediction sub-models to obtain multiple third sequence data. The determination module is used to aggregate the multiple third sequence data using a first decoder to obtain fourth sequence data, and to determine the predicted power data corresponding to the original power data based on the fourth sequence data.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images