Multi-task prediction method and system based on hybrid expert network, electronic device and medium

Abstract

This invention relates to the field of intelligent analysis technology for power systems, and particularly to a multi-task prediction method, system, electronic device, and medium based on a hybrid expert network. The method includes: acquiring multi-source heterogeneous operating data of the power system to generate an input tensor; generating a shared feature map through a power domain feature aggregator; generating an optimized feature sequence through a variable mixer; capturing cross-task common knowledge and outputting global expert features through a global hybrid expert network; outputting task expert features through a task-specific hybrid expert network; and fusing the global expert features and task expert features to generate prediction results for each specific task. This approach solves the technical problem of insufficient prediction accuracy in existing hybrid expert networks when facing multi-task intensive prediction scenarios in power systems, and improves the feature representation capability, task collaboration efficiency, and final prediction accuracy of multi-task learning.

Description

Technical Field

[0001] This invention relates to the field of intelligent analysis technology for power systems, and in particular to a multi-task prediction method, system, electronic device, and medium based on a hybrid expert network. Background Technology

[0002] Against the backdrop of the intelligent transformation of power systems, generative artificial intelligence technologies, especially large-scale modeling, are gradually being applied to key aspects such as power equipment monitoring, fault diagnosis, load forecasting, and grid dispatching, driving the power system towards higher efficiency and greater adaptability. Traditional power data analysis methods largely rely on physical models and empirical rules, making it difficult to effectively handle massive, high-dimensional, and nonlinear real-world operational data. In recent years, deep neural networks have played a crucial role in power system state perception and decision support due to their powerful nonlinear fitting and feature extraction capabilities. Among these, multi-task learning techniques, by sharing underlying feature representations, can simultaneously optimize multiple related tasks, such as simultaneously predicting indicators like transformer oil temperature, partial discharge, and insulation aging, significantly improving the model's generalization ability and computational efficiency. However, standard multi-task neural networks typically employ a hard-sharing mechanism, where all tasks are forced to share the same set of underlying parameters. This can easily lead to "negative transfer" problems when there are significant differences between tasks, causing performance degradation in some tasks. Therefore, researchers have begun exploring more flexible parameter sharing mechanisms to optimize the collaborative effects between tasks.

[0003] Mixture of Experts (MIB) As a dynamic model architecture, this technology, by introducing multiple expert sub-networks and gating mechanisms, can adaptively combine expert outputs based on input characteristics, improving the model's adaptability to complex tasks. This technology has achieved significant results in natural language processing and computer vision, and has shown application potential in scenarios such as power load forecasting and fault classification. Currently, research in the power sector is attempting to apply this technology... The framework is applied to multi-task intensive prediction. For example, existing technology (application publication number CN120182791A) discloses a multi-task learning method for intensive prediction that combines hybrid experts and Mamba models. This method constructs a hybrid parametric expert module, introduces task-specific parameter priors into the Mamba state-space model, and employs multi-directional Hilber scans to enhance sequence modeling capabilities, aiming to improve the accuracy of multi-task interactions and the global integration effect. However, the methods proposed in existing technologies are mainly aimed at image processing tasks (such as semantic segmentation and depth estimation), and their architecture design does not fully consider the spatiotemporal correlation, non-stationarity, and multi-source heterogeneity unique to power system data. Specifically, power intensive prediction tasks (such as minute-by-minute load forecasting and whole-network voltage fluctuation monitoring) require models to have fine-grained feature capture capabilities, dynamic inter-task knowledge transfer mechanisms, and efficient fine-tuning characteristics. However, the expert partitioning and routing strategies in existing technologies are still coarse-grained, lacking adaptability to local patterns and transient changes in power data, and their parameter priors are fixed, making it difficult to cope with the dynamic evolution of power task relationships. Furthermore, this method does not explicitly incorporate power grid topology or spatiotemporal attention mechanisms, resulting in insufficient feature representation in scenarios such as node state prediction and topology analysis, thus limiting its practical application in power systems. Therefore, existing hybrid expert networks (HANs) In multi-task intensive forecasting of power systems, it is difficult to achieve an effective balance between knowledge sharing among tasks and task-specific modeling, resulting in low efficiency of model parameters and insufficient prediction accuracy. Summary of the Invention

[0004] To address the aforementioned shortcomings or drawbacks, this invention provides a multi-task prediction method, system, electronic device, and medium based on a hybrid expert network, which can solve the technical problem of insufficient prediction accuracy in existing hybrid expert networks when facing multi-task intensive prediction scenarios in power systems.

[0005] This invention provides a multi-task prediction method based on a hybrid expert network, comprising: Acquire multi-source heterogeneous operation data of the power system, preprocess the multi-source heterogeneous operation data, and generate input tensors.

[0006] The input tensor is fed into a preset power domain feature aggregator, which performs multi-scale feature extraction and fusion on the input tensor to generate a shared feature map.

[0007] The shared feature map is input into a variable mixer, which performs local feature enhancement and global context dependency extraction to generate an optimized feature sequence.

[0008] The optimized feature sequences are input in parallel into a global hybrid expert network and a task-specific hybrid expert network. The global hybrid expert network captures common knowledge across tasks and outputs global expert features, while the task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features.

[0009] The global expert features and task expert features are fused to generate fused features corresponding to each specific task, which are then used to generate prediction results for each specific task.

[0010] According to a second aspect, the present invention provides a multi-task prediction system based on a hybrid expert network, comprising: The input tensor construction module is used to acquire multi-source heterogeneous operation data of the power system and preprocess the multi-source heterogeneous operation data to generate input tensors.

[0011] The shared feature map generation module is used to input the input tensor into a preset power domain feature aggregator, and then perform multi-scale feature extraction and fusion on the input tensor through the power domain feature aggregator to generate a shared feature map.

[0012] The optimized feature sequence generation module is used to input the shared feature map into the variable mixer, and perform local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence.

[0013] The hybrid expert feature generation module is used to input optimized feature sequences into a global hybrid expert network and a task-specific hybrid expert network in parallel. The global hybrid expert network captures common knowledge across tasks and outputs global expert features, while the task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features.

[0014] The multi-task prediction result generation module is used to fuse global expert features and task expert features to generate fused features corresponding to each specific task, which are then used to generate prediction results for each specific task.

[0015] According to a third aspect, the present invention provides an electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the multi-task prediction methods based on hybrid expert networks in the embodiments of the present invention.

[0016] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute any of the multi-task prediction methods based on hybrid expert networks in the embodiments of the present invention.

[0017] The present invention provides a multi-task prediction method based on a hybrid expert network. This method is achieved collaboratively through five core steps: input tensor generation, shared feature extraction, feature sequence optimization, hybrid expert feature generation, and multi-task prediction. Specifically, it acquires multi-source heterogeneous operating data of the power system and performs preprocessing operations to construct a unified and regular model input; input tensors are input to a pre-defined power domain feature aggregator to perform multi-scale feature extraction and fusion operations, extracting shared feature representations with domain commonalities from multi-source data; shared feature maps are input to a variable mixer to perform local feature enhancement and global context dependency extraction operations, generating optimized feature sequences that simultaneously contain local details and global semantics; the optimized feature sequences are input in parallel to a global hybrid expert network and a task-specific hybrid expert network. The former captures cross-task common knowledge and outputs global expert features, while the latter captures task-specific knowledge and outputs task expert features based on a task-aware dynamic routing mechanism, achieving decoupling and parallel learning of common knowledge and task-specific knowledge; feature fusion operations are performed on the global expert features and task expert features to generate prediction results for each specific task, outputting the final multi-task prediction.

[0018] In this technical solution, the present invention addresses the problem described in the background art of existing hybrid expert networks' difficulty in effectively balancing inter-task knowledge sharing and task-specific modeling when handling multi-task intensive forecasting in power systems. It achieves this by deploying a global hybrid expert network and a task-specific hybrid expert network in parallel, and designing a task-aware dynamic routing mechanism. This constructs a dual-path architecture that combines "commonality capture" and "specific extraction," realizing explicit separation and collaborative optimization of shared information and task-specific information. This solves the problem of negative migration or parameter redundancy caused by rigid architecture in existing technologies. Furthermore, it addresses the challenges of multi-source, heterogeneous, and strongly spatiotemporal power system data. To address the issue of insufficient feature extraction due to related characteristics, this invention introduces a power domain feature aggregator and a variable mixer to sequentially perform multi-scale feature fusion, local enhancement, and global dependency extraction. This forms a hierarchical feature learning path from fine-grained local patterns to macroscopic contextual semantics, overcoming the limitation of traditional single feature extraction methods in comprehensively capturing the inherent patterns of complex power data. Furthermore, to address the challenge of balancing model efficiency and accuracy in multi-task intensive prediction scenarios, this invention employs an end-to-end process involving feature aggregation, sequence optimization, and dual-path expert network collaboration. This enables efficient joint learning and accurate prediction of multiple related tasks within a unified framework. Therefore, the technical solution of this invention solves the technical problem of insufficient prediction accuracy in existing hybrid expert networks facing multi-task intensive prediction scenarios in power systems, improving the feature representation capability, task collaboration efficiency, and final prediction accuracy of multi-task learning. Attached Figure Description

[0019] Figure 1 This is a flowchart of a multi-task prediction method based on a hybrid expert network according to an embodiment of the present invention; Figure 2 This is a flowchart of a multi-task dense prediction method for the power field based on fine-grained hybrid expert networks, according to another embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a multi-task prediction system based on a hybrid expert network according to an embodiment of the present invention; Figure 4 This is a block diagram of an electronic device used to implement embodiments of the present invention. Detailed Implementation

[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] During the development of this invention, researchers, through extensive experiments and data analysis, revealed the intrinsic relationship between the knowledge sharing requirements among tasks and the task-specific modeling requirements in multi-task intensive prediction of power systems: traditional single shared encoders or static expert pool architectures struggle to achieve efficient transfer of common knowledge and accurate capture of fine-grained task features while maintaining parameter efficiency. Based on this relationship, this invention innovatively proposes this technical solution, utilizing a hybrid expert network (… Leveraging its modular nature, the system constructs a dual-path parallel architecture of "global expert network + task-specific expert network," combined with a task-aware dynamic routing mechanism and multi-level feature aggregation. This achieves an organic unity between cross-task knowledge sharing and task-specific feature extraction, embodying the core concept of "decoupling-cooperation-adaptation."

[0022] Specifically, through comparative experiments, the invention team discovered that traditional multi-task learning models and standard hybrid expert networks share common problems: rigid feature representation and imbalanced parameter allocation. Shared encoders lead to negative transfer between tasks, while static expert pools cannot adapt to the spatiotemporal dynamics of power data. These technical deficiencies significantly limit the model's prediction accuracy and generalization ability in complex power scenarios. However, the fine-grained expert partitioning strategy and dynamic routing mechanism proposed in this invention improve the model's adaptability and expression efficiency to multi-source heterogeneous power features, achieving a balance between prediction accuracy, parameter efficiency, and task scalability within a unified framework.

[0023] Therefore, according to the first aspect of the present invention, a multi-task prediction method based on a hybrid expert network is provided. This method can be applied to multi-task intensive prediction scenarios in power systems (hereinafter referred to as "the system"). The system can be deployed centrally in the cloud or distributed at the edge in a power monitoring center, substation edge computing nodes, or mobile inspection equipment to complete multi-task collaborative prediction such as load forecasting, voltage fluctuation monitoring, equipment status assessment, and fault diagnosis.

[0024] Specifically, this system can be deployed in various hardware environments, including but not limited to: high-performance server clusters in the cloud, host systems of provincial power grid dispatch centers, edge computing gateways in smart substations, and smart meters or inspection terminals equipped with computing units. This flexible deployment architecture allows the system to meet the high requirements of power systems for prediction accuracy and real-time performance, while also adapting to the resource-constrained and low-latency operation and maintenance needs of edge environments. In terms of its operational mechanism, the system achieves an end-to-end processing flow of multi-scale feature extraction, dynamic expert routing, and multi-task knowledge collaboration through modular design, thereby improving the model's prediction accuracy, parameter efficiency, and adaptability in complex power scenarios.

[0025] like Figure 1 As shown, the method may include: Step S110: Obtain multi-source heterogeneous operation data of the power system, and preprocess the multi-source heterogeneous operation data to generate input tensors.

[0026] Multi-source heterogeneous operational data refers to a set of operational parameters from different monitoring stages of the power system, with different sampling frequencies and data structures. This includes load curve data (recording the time-varying sequence of active / reactive power at each node), voltage fluctuation data (statistics of voltage deviation and fluctuation rate at monitoring points), equipment monitoring data (statistics such as transformer oil temperature and partial discharge), meteorological condition data (environmental parameters such as temperature, humidity, and wind speed), and time series data (timestamp information such as year, month, day, hour, and minute). Preprocessing operations include data cleaning (removing outliers and filling in missing values) and normalization (linearly scaling each parameter to a suitable value). (Interval) and time alignment (extracting synchronization segments according to a preset time window). The input tensor refers to a four-dimensional data array (dimension is batch size × time step × feature dimension × number of channels) that meets the input requirements of a neural network after standardization.

[0027] Specifically, the system can acquire multi-source data in real time from SCADA (Supervisory Control and Data Acquisition), PMU (Phasor Measurement Unit), and equipment sensor networks through the data acquisition interface module. The data preprocessing module performs missing value imputation, Z-score normalization (a data normalization method based on mean and standard deviation, used to convert raw data into a standard normal distribution with a mean of 0 and a standard deviation of 1), and time window sliding cutting operations. For example, the system collects 96 time-step data points from a regional power grid within 24 hours at 5-minute sampling intervals, including 8 types of features such as load, voltage, and equipment status. After preprocessing, a data point with dimensions of [missing information] is generated. The input tensor (batch size 32, time step 96, feature dimension 8, single channel).

[0028] Step S120: Input the input tensor into a preset power domain feature aggregator, and use the power domain feature aggregator to perform multi-scale feature extraction and fusion on the input tensor to generate a shared feature map.

[0029] In the power sector, a feature aggregator refers to a network composed of backbone networks (such as...). This architecture is a dedicated feature extraction architecture consisting of a hierarchical vision transformation architecture based on a sliding window mechanism and a feature fusion module. Multi-scale feature extraction refers to extracting feature maps with different spatiotemporal resolutions (such as 1 / 4, 1 / 8, 1 / 16, and 1 / 32 scales) through multi-stage downsampling operations of the backbone network. Shared feature maps refer to a unified feature representation generated after multi-scale feature upsampling, channel concatenation, and linear projection, which is used to support subsequent multi-task learning.

[0030] Specifically, the system can perform four-stage convolution processing on the input tensor through the backbone network to obtain feature maps at four scales; then, it uses bilinear upsampling to unify all feature maps to a 1 / 4 scale, and concatenates them along the channel dimension before processing. Convolution compresses the number of channels, generating shared feature maps. For example, the input tensor... After processing by the feature aggregator, the output shared feature map has a dimension of 1. (Time step compressed to 24, spatial dimension expanded to 24, number of channels increased to 256).

[0031] In other embodiments, the feature fusion module configured in this power sector feature aggregator integrates multi-source heterogeneous data features from the power system to generate a multi-task shared feature representation. This enables the model to effectively utilize information across time scales and spatial topology, thereby improving performance in multi-task learning such as load forecasting, state estimation, and fault early warning. The input image is represented as... (H, W, and 3 represent height, width, and number of channels, respectively), it can use a backbone network (such as...) The image is processed (e.g., ...) to generate four stages of image feature mapping. The mapping features that can be output from stages 1 to 4 are respectively... , , and The above features are upsampled to the same spatial resolution and then concatenated along the feature channel dimension to generate a high-dimensional feature map. (in (Total number of channels after concatenation). The feature channel dimension is reduced from [previous dimension] by a linear projection operation. Compression to C enables power information compression and noise suppression, providing a unified and highly expressive shared feature foundation for subsequent multi-task intensive prediction and efficient fine-tuning of fine-grained hybrid expert networks.

[0032] Step S130: Input the shared feature map into the variable mixer, and perform local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence.

[0033] The variable mixer refers to a feature optimization module composed of channel convolutional layers, variable convolutional layers, and a transformer layer. Local feature enhancement refers to learning spatial adaptive offsets through deformable convolutions to capture local distortion patterns in power data (such as voltage sag waveforms). Global context dependency refers to modeling long-term correlations across time steps (such as load periodicity) through a self-attention mechanism. Optimized feature sequence refers to a two-dimensional matrix (dimensions of sequence length × feature dimension) suitable for input to the expert network after serialization.

[0034] Specifically, the system can first use channel convolution ( Convolutional layers enable cross-channel information interaction, while variable convolutions learn the offset at each spatial location to enhance local feature perception. Finally, a transformer layer performs self-attention computation to capture global dependencies. For example, shared feature maps... After processing by a variable mixer, it is serialized into 576 feature vectors ( Each vector has a dimension of 256, and the output optimized feature sequence has a dimension of [missing value]. .

[0035] In other embodiments, the system can generate optimized feature sequences by realizing cross-channel information interaction, spatial adaptive convolution, and global context dependency extraction in a power field variable mixer using the following formula; Next, in this embodiment, the system can also achieve cross-channel information interaction through the following formula (1): (1) Formula (1) is the calculation expression for the channel convolution layer, which is used to perform point-by-point convolution and bias superposition on the input feature map to achieve cross-channel feature fusion. This represents the input feature map, with dimension 1. ,in and These represent the height and width of the feature map, respectively (in pixels). Number of feature channels; The weight parameters of the pointwise convolution kernel have a dimension of . (Convolution kernel height 1 pixel, width 1 pixel, number of input channels) Number of output channels ); The bias term has the following dimensions: Pointwise convolution refers to a convolution kernel with a size of 1. The convolution operation is used to adjust the number of feature channels and achieve cross-channel information interaction.

[0036] Furthermore, in this embodiment, the system can also perform nonlinear activation and normalization processing on the features after cross-channel interaction using the following formula (2): (2) Formula (2) represents the sequential application of the GELU (Gaussian Error Linear Unit) activation function to the feature X processed by formula (1). This is related to the Batch Normalization (BN) operation. The GELU activation function is used to introduce nonlinear transformations; the batch normalization operation is used to stabilize the training process and accelerate convergence.

[0037] Furthermore, in this embodiment, the system can also calculate the spatial adaptive convolution output of the variable convolutional layer using the following formula (3): (3) Formula (3) is the core calculation expression of the variable convolutional layer, which is used to adaptively sample and convolve the feature map according to the learned spatial offset. Indicates position The corresponding learnable spatial offset is obtained from the upper-layer features through a separate convolutional layer. The prediction was obtained from the middle; Variable convolution kernel weights; This represents the number of output channels. This operation allows the convolution kernel to dynamically adjust the sampling position based on the input content, thereby better capturing local deformation features in power data.

[0038] Furthermore, in this embodiment, the system can also achieve residual connections and feature enhancement after the variable convolutional layer through the following formula (4): (4) Formula (4) represents the variable convolution output. After GELU activation and batch normalization, the original input features are compared with those of the original input features. Element-by-element addition is performed to form residual connections. This is the enhanced feature map. Residual connections help alleviate the vanishing gradient problem during deep network training and preserve the original feature information.

[0039] Furthermore, in this embodiment, the system can also flatten the feature map and perform layer normalization processing using the following formula (5) to adapt it to the input requirements of the Transformer layer: (5) In formula (5), This indicates the three-dimensional feature map Flattened into a two-dimensional sequence, the sequence length is... ; Layer normalization (a normalization technique in deep learning models used to stabilize the training process of neural networks and accelerate convergence) is used to stabilize the distribution of sequential data. The normalized sequence features have a dimension of . .

[0040] Finally, in this embodiment, the system can also perform multi-head self-attention (MHSA, an attention mechanism used in deep learning) calculation in the transformer layer using the following formula (6) to extract global context dependencies: (6) Formula (6) is the formula for calculating scaled dot product attention. Q (query matrix), K (key matrix), and V (value matrix) are all derived from the input sequence. Obtained through linear projection; The dimension of the key vector is used to scale the dot product result and prevent gradient vanishing. The function is used to normalize the attention weights. Multi-head self-attention mechanisms can capture long-range dependencies between sequence elements from different subspaces by computing multiple attention heads in parallel.

[0041] Therefore, by combining the above formulas (1) to (6), the system can sequentially achieve cross-channel feature fusion, spatial adaptive convolution, residual enhancement, serialization processing and global context modeling, thereby generating optimized feature sequences rich in local details and global semantics from the shared feature map, providing high-quality feature representations for subsequent multi-task prediction.

[0042] Step S140: Input the optimized feature sequence into the global hybrid expert network and the task-specific hybrid expert network in parallel. The global hybrid expert network captures common knowledge across tasks and outputs global expert features. The task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features.

[0043] Global hybrid expert networks refer to shared networks composed of multiple expert units with identical structures, where all experts are permanently active, and common features are output through summation and aggregation. Task-specific hybrid expert networks are dynamic routing networks containing a subset of shared experts and a subset of experts specific to a given task. They utilize a task-aware gating mechanism (a task-aware dynamic routing mechanism used for...) Based on the characteristics of the input data and task requirements, the algorithm adaptively allocates and weights the outputs of different expert units (Experts). It calculates the affinity weight between the input and each expert, and selects Top-K experts for activation. The task-aware dynamic routing mechanism refers to an algorithm that dynamically calculates the activation weights of each expert based on the current input features. The formula is: (in As input features, For experts (characteristic centroid).

[0044] Specifically, the system can calculate the affinity score between the optimized feature sequence and each specific task expert through the routing decision-maker, and select the K experts with the highest scores (such as...). The expert network participates in the calculation and performs a weighted summation of the expert outputs; simultaneously, the global expert network processes and sums all inputs in parallel. For example, in a dual-task scenario of load forecasting and voltage warning, the task-specific expert network contains 10 experts (8 task-specific experts + 2 shared experts), with 4 experts activated for each task, outputting the task expert feature dimensions. The global expert network contains four experts and outputs global expert feature dimensions. .

[0045] In some embodiments, the system can calculate the final output y of the hybrid expert network module using the following formula (7): (7) Wherein, formula (7) is The basic architecture expression uses a differentiable gating mechanism to dynamically weight and aggregate the outputs of multiple expert units. y represents the final output vector of the module; N is a positive integer representing the total number of expert units in the network; x is the feature vector of the input sample. Indicates the first The output vector obtained by each expert unit after processing the input x; For the first The gate value corresponding to each expert unit is a scalar representing the probability weight that input x should be assigned to that expert.

[0046] Next, in this embodiment, the system can also calculate the above-mentioned gating weight value using formula (8). : (8) Formula (8) describes the specific computation process of the gating network. yes An activation function is used to apply an activation signal to a vector (here) Normalization is performed to ensure that the sum of the gating weights of all expert units is 1, i.e. . The weight parameter matrix of the gating function has dimensions (input feature dimension × total number of expert units N). Dot product operator. Represents the input feature vector x and the weight matrix The matrix multiplication operation between them results in a raw weight score vector of length N, which is then processed... The final gating weight distribution is obtained after function processing. .

[0047] Therefore, by combining the above formulas (7) and (8), the system can construct a hybrid expert network infrastructure containing a differentiable routing mechanism. This architecture can process input samples in parallel and dynamically and adaptively allocate the input to the most relevant expert units through a gating network based on the learned pattern features. It also performs a weighted summation of the outputs of all experts, thereby providing flexible and efficient feature computation and integration capabilities for subsequent complex tasks (such as the multi-task prediction of this invention).

[0048] Next, the system can calculate the input feature representation after layer normalization preprocessing using the following formula (9). : (9) Formula (9) is a layer normalization operation used to stabilize the distribution of input features in order to improve the training efficiency of the model. Represents the original input features; The layer normalization function, which standardizes the input features along the last dimension, is calculated using the following formula: ,in The mean, Let be the variance, and γ and β be learnable scaling and translation parameters. For a very small constant (e.g.) This is to prevent division by zero errors.

[0049] Next, in this embodiment, the system can also calculate the preliminary affinity score between specific task features and expert units using formula (10). : (10) Formula (10) is the affinity calculation expression, used to measure the affinity between the input feature and the first feature. The relevance of each expert unit. The input feature vector is the result of layer normalization. Indicates the first The feature centroid vector corresponding to a specific task expert unit is a learnable parameter; Indicates the transpose operation; It is a sigmoid activation function that maps the result of the dot product to a scalar value between 0 and 1. The larger this value, the higher the matching degree between the input features and the corresponding expert unit.

[0050] Furthermore, in this embodiment, the system can also calculate the gate value of a specific task expert unit through a dynamic gating mechanism composed of formulas (11) and (12). with normalized weights : (11) (12) Formula (11) is the Top-k selection function, used to dynamically select the most relevant expert units for the current task from all specific task expert units. One expert. Indicates the total number of expert units for a specific task; The preset number of activated experts is usually much smaller than... A fixed value, for example, can be set to ; Functions from sets Select the highest score One element is assigned a value of 0, and the rest of the elements are assigned a value of 0. Formula (12) is a weight normalization operation, which is used to ensure that the sum of the gating values ​​of all activated expert units is 1. The selected gating value; the denominator is the total number of activated expert units (i.e., ...). The sum of the gating values ​​of the experts.

[0051] Furthermore, in this embodiment, the system can also calculate the final output y of the hybrid expert network using formula (13): (13) Formula (13) is the integrated output expression of the hybrid expert network, which realizes the collaborative computation of shared experts and task-specific experts. Indicates the number of shared expert units; This indicates that the i-th expert unit (whether shared or task-specific) has a positive response to the input. The result of the processing is usually an eigenvector or tensor. The first term of formula (13) The first term represents the sum of the outputs of all shared expert units, used to capture common knowledge across tasks; the second term... Representative based on gating weight The weighted sum of the outputs of all task-specific expert units is used to capture task-specific fine-grained features. The final output y is the sum of these two parts.

[0052] Therefore, by combining the above formulas (9) to (13), the system can construct an efficient hybrid expert network that combines collaborative and shared experts with specific task experts. Through the dynamic routing mechanism, the system can achieve accurate matching and weighted aggregation of input features and the most relevant expert units, thereby effectively improving the feature expression ability, task adaptability and overall prediction accuracy of the model in multi-task intensive prediction scenarios.

[0053] Step S150: Perform feature fusion on the global expert features and the task expert features to generate fused features corresponding to each specific task, which are used to generate prediction results for each specific task.

[0054] Feature fusion refers to integrating globally shared features and task-specific features through element-wise addition. The fused feature is a unified feature representation that serves as input to the task head network. The task head network is a lightweight output layer (such as a fully connected layer) designed for each specific task to map the fused features to the prediction result.

[0055] Specifically, the system can add the global expert features and task expert features for the same task, then input the fused features into the fully connected layer corresponding to the task, and output the final predicted value. For example, for load forecasting tasks, the fused features... The load forecast is mapped to 96 steps through a fully connected layer, and the output dimension is... The voltage early warning task outputs the risk probability of 32 nodes, in terms of dimensions. .

[0056] In some embodiments, the system can calculate the output of the shared expert set using the following formula (14). : (14) Formula (14) is the output expression of the shared expert set, which distributes all input features to all shared experts through a deterministic allocation mechanism to capture common knowledge across tasks. This represents the final output feature of the shared expert set; This is a positive integer representing the total number of expert units in the shared expert set. In this embodiment, it can be set to... ; This indicates that the i-th shared expert unit has a good understanding of the input features. The output features obtained after processing; The feature vector input to the shared expert set is typically derived from the output of an upstream feature extraction module (e.g., a variable mixer). This formula ensures stable extraction of common features by simply summing and aggregating the outputs of all shared experts, without requiring dynamic routing.

[0057] Next, in this embodiment, the system can also calculate the final output of the global hybrid expert network in the power field using formula (15). : (15) Formula (15) is the output expression of the global hybrid expert network module. It adopts a residual connection structure, which adds the features processed by the global experts to the original input features in order to retain basic information and promote gradient flow. This represents the output features of a global hybrid expert network; This indicates that the global hybrid expert network module handles the input features. The processing result is that the module consists of multiple global expert units, and its internal calculation follows the summation form shown in formula (14); The raw feature maps input to this module are typically derived from the output of a feature aggregator in the power domain. This design enables the network to learn residual mappings of the input features, effectively mitigating the vanishing gradient problem in deep network training and enhancing feature reuse capabilities.

[0058] Furthermore, the Global (Global Hybrid Expert Network, also known as Global-MoEs) module consists of a group of permanently activated feedforward neural network experts, which does not include a dynamic routing mechanism. This module receives the feature sequence output by the variable mixer as input X, and extracts common features across tasks in parallel through feature interaction among experts while preserving the original input information. This design can effectively capture the shared knowledge and underlying physical laws among multiple tasks in the power system without interfering with the independence of each task, providing a robust global feature foundation for subsequent specific task predictions.

[0059] Therefore, by combining formulas (14) and (15) above, the system can construct a two-layer expert processing architecture that includes a shared expert set and a global hybrid expert network. This architecture first extracts common features across tasks stably through the shared expert set, and then further fuses and enhances the features through the global hybrid expert network within the residual connection framework. The two work together to provide a high-quality, information-rich common feature foundation for subsequent fusion with expert features specific to the task, thereby supporting the efficient learning of complex operating modes by the multi-task prediction model of the power system.

[0060] In some embodiments, the system can calculate the overall loss function for multi-task learning using the following formula (16). : (16) Formula (16) is the overall loss function expression for multi-task learning, which is used to weight and sum the loss functions of each specific task during joint training in order to balance the optimization objectives among different tasks. Represents the final total loss value, which is a scalar; symbol This indicates a summation operation; T is a positive integer representing the total number of specific tasks in the system; the subscript t is the task index. ; Indicates that the index is The loss function value corresponding to the specific task; Representation and Task The corresponding balancing factor (or weighting coefficient) is a configurable hyperparameter used to adjust the proportion of the task loss in the total loss.

[0061] Next, in this embodiment, the system can also define a specific task loss itemset using formula (17). Possible components: (17) Formula (17) defines the part in formula (16). The range of values ​​for is defined as a set containing six preset loss types. This represents the loss function for semantic segmentation tasks, used to measure the accuracy of pixel-level classification; The loss function represents the component segmentation task, used for finer-grained object component partitioning; The loss function represents the saliency detection task and is used to evaluate the model's ability to identify the most salient regions in an image; This represents the loss function for the depth estimation task, used to measure the error in predicting the depth information of the scene; This represents the loss function for the surface normal estimation task, used to evaluate the accuracy of predicting the orientation vector of the object's surface; This represents the loss function for the boundary detection task, used to measure the accuracy of object boundary prediction in an image. For multi-task prediction scenarios in power systems, these tasks can be mapped to load forecasting, voltage fluctuation monitoring, equipment condition assessment, etc.

[0062] Furthermore, in this embodiment, the system can also define a set of balance factors using formula (18). The possible components are given, corresponding to the task loss term in formula (17): (18) Formula (18) defines the part in formula (16). The range of values ​​for . These correspond to the task loss functions in formula (17). The balancing factors. Each balancing factor is a configurable positive scalar hyperparameter used to adjust the corresponding task loss term in the total loss. The relative weights in the model are used to meet the importance requirements of different tasks for model optimization. The balancing factor can be manually set to a fixed value (e.g., set to...). Alternatively, dynamic learning can be performed through gradient backpropagation or meta-learning algorithms.

[0063] Therefore, by combining the above formulas (16), (17), and (18), the system can define a complete and flexible multi-task learning training objective. This framework first clarifies the specific prediction tasks and their metrics using formula (17), then assigns or learns corresponding optimization weights for each task using formula (18), and finally aggregates the weighted losses of each task into a single optimizable objective function using formula (16). This mechanism enables the system to automatically balance multiple related but potentially conflicting learning objectives during end-to-end training, effectively coordinating and optimizing globally shared features and task-specific features, thereby improving the overall performance and generalization ability of the model in multi-task intensive prediction scenarios.

[0064] In another embodiment, the flowchart of a multi-task dense prediction method based on fine-grained hybrid expert networks in the power sector is as follows: Figure 2 As shown, this method automates the process from raw data to multi-task prediction results using an end-to-end approach (meaning it starts with the raw input data and ends with the desired output or decision, constructing a single, coherent, and complete processing flow or model). Specifically, the process begins with "power sector input corpus," which consists of multi-source heterogeneous operational data, including load curves, voltage fluctuation sequences, equipment status monitoring data, and meteorological information. This input data is fed into the "power sector feature aggregator," where it is processed through a backbone network (such as...). The system performs multi-stage downsampling and feature extraction, and then performs upsampling, channel stitching, and linear projection operations in the feature fusion module to generate a shared feature map containing multi-scale spatiotemporal patterns. Subsequently, the shared feature map is input into the "electric field variable mixer", which sequentially passes through channel convolutional layers to achieve cross-channel information interaction, through variable convolutional layers to perform spatial adaptive convolution to enhance the perception of local deformation features, and through the Transformer layer to extract global contextual dependencies, finally outputting an optimized feature sequence.

[0065] Subsequently, the process executes two branches in parallel: In the "Construction of Hybrid Expert Networks for Specific Tasks in the Power Sector" branch, the optimized feature sequence is optimized based on a task-aware dynamic routing mechanism, calculating the affinity weights with each task's expert network, dynamically activating the most relevant expert subsets, and weighted aggregating their outputs to obtain task expert features that characterize the specificity of the task; In the "Construction of Global Hybrid Expert Networks in the Power Sector" branch, the optimized feature sequence is processed in parallel and summed through multiple permanently activated global expert units, and then fused with the input features via residual connections to obtain global expert features that capture the commonalities across tasks. The outputs of the two branches—namely, the task expert features and the global expert features—are jointly input into the "Multi-Task Learning Objective Training and Prediction" module, where feature fusion is performed by element-wise addition, and the final "model prediction results" are generated through the lightweight task head networks corresponding to each task, such as the load curve for the next 24 hours, the probability of voltage exceeding limits at each node, and the fault warning level of key equipment. This process clearly demonstrates how the present invention, through cascaded aggregation, hybridization, expert routing, and fusion modules, collaboratively processes multi-source data from the power system and achieves efficient joint learning and accurate prediction for multiple tasks.

[0066] Therefore, according to the above implementation method, the system first acquires multi-source heterogeneous operation data of the power system and performs preprocessing operations to construct a unified and regular model input; the input tensor is input to a preset power domain feature aggregator to perform multi-scale feature extraction and fusion operations to extract shared feature representations with domain commonalities from multi-source data; the shared feature map is input to a variable mixer to perform local feature enhancement and global context dependency extraction operations to generate an optimized feature sequence that simultaneously contains local details and global semantics; the optimized feature sequence is input in parallel to a global hybrid expert network and a task-specific hybrid expert network, the former capturing cross-task common knowledge and outputting global expert features, and the latter capturing task-specific knowledge and outputting task expert features based on a task-aware dynamic routing mechanism, to achieve decoupling and parallel learning of common knowledge and task-specific knowledge; feature fusion operations are performed on the global expert features and task expert features to generate prediction results for each specific task, which are used to output the final multi-task prediction.

[0067] Specifically, in this implementation, the technical solution addresses the problem described in the background art: existing hybrid expert networks struggle to effectively balance knowledge sharing between tasks and task-specific modeling when handling multi-task intensive forecasting in power systems. This is achieved by deploying a global hybrid expert network and a task-specific hybrid expert network in parallel, and designing a task-aware dynamic routing mechanism. This constructs a dual-path architecture that combines "commonality capture" and "specific extraction," enabling explicit separation and collaborative optimization of shared information and task-specific information. This solves the negative migration or parameter redundancy problems caused by rigid architectures in existing technologies. Furthermore, it addresses the challenges of multi-source, heterogeneous, and highly complex power system data. To address the issue of insufficient feature extraction due to spatiotemporal correlation, this solution introduces a power domain feature aggregator and a variable mixer to sequentially perform multi-scale feature fusion, local enhancement, and global dependency extraction. This forms a hierarchical feature learning path from fine-grained local patterns to macroscopic contextual semantics, overcoming the limitation of traditional single feature extraction methods in comprehensively capturing the inherent patterns of complex power data. Furthermore, to address the challenge of balancing model efficiency and accuracy in multi-task intensive prediction scenarios, this solution employs an end-to-end process involving feature aggregation, sequence optimization, and dual-path expert network collaboration. This enables efficient joint learning and accurate prediction of multiple related tasks within a unified framework. Therefore, this implementation solves the technical problem of insufficient prediction accuracy in existing hybrid expert networks facing multi-task intensive prediction scenarios in power systems, improving the feature representation capability, task collaboration efficiency, and final prediction accuracy of multi-task learning.

[0068] In some embodiments, the multi-source heterogeneous operation data includes load curve data, voltage fluctuation data, equipment monitoring data, meteorological condition data, and time series data; acquiring the multi-source heterogeneous operation data of the power system and preprocessing the multi-source heterogeneous operation data to generate an input tensor includes: Data cleaning, normalization, and time series alignment are performed on load curve data, voltage fluctuation data, equipment monitoring data, meteorological condition data, and time series data to generate multi-channel time series data.

[0069] Multi-channel time series data refers to a multi-dimensional time series data structure formed by aligning time series data from multiple data sources, with each data source acting as an independent channel, in order to preserve the spatiotemporal characteristics of the original data.

[0070] Specifically, data cleaning operations include using box plots or... Criteria (using the mean of the data) and standard deviation To determine the expected fluctuation range of the vast majority of "normal" data points, and to identify and remove outliers (data points outside this range are considered potential outliers or boundary points requiring special attention), a linear interpolation method is used to fill in missing values. Normalization is performed using a minimum-maximum scaling method to linearly transform the range of each data parameter to... The time-series alignment operation extracts synchronized data segments at fixed intervals (e.g., 5 minutes) within a preset time window (e.g., 24 hours) to ensure strict alignment of all data sources on the time axis. For example, the system collects eight types of data, including load, voltage, and equipment status, from a regional power grid within 24 hours at 5-minute sampling intervals. After preprocessing, it generates multi-channel time-series data with a dimension of 96 time steps × 8 feature channels (i.e., a 96×8 matrix).

[0071] Tensor reshaping is performed on multi-channel time-series data to generate input tensors in three-dimensional tensor format.

[0072] Tensor reshaping refers to reorganizing two-dimensional time-series data into a three-dimensional tensor structure containing three dimensions: batch, time step, and feature channel, to adapt to the input requirements of deep learning models.

[0073] Specifically, the system transforms the dimensions of multi-channel time-series data into a three-dimensional form of batch size × time step × feature channels through a reshaping function. Here, batch size represents the number of samples, time step represents the number of time points within a preset time window, and feature channels represent the number of data types. For example, the above... Multi-channel time-series data are grouped into batches of 32, generating input tensors with dimension [missing value]. (That is, 32 samples, 96 time points for each sample, and 8 features for each time point).

[0074] Tensor reshaping operations include: The structure of multi-channel time series data is transformed into a three-dimensional tensor including batch size dimension, time step dimension, and feature channel dimension. The batch size dimension represents the number of samples, the time step dimension represents the number of time points within a preset time window, and the feature channel dimension represents the number of data types.

[0075] Among them, batch size refers to the number of samples processed in one iteration of deep learning training; time step refers to the number of consecutive time points contained in a single sample; and feature channel refers to the number of data feature types corresponding to each time point.

[0076] Specifically, the system transforms a two-dimensional time-series data matrix into a three-dimensional tensor through tensor reshaping operations, for example, using the Python programming language... The library's `reshape` function (used to redefine the dimensional layout of an array to match the specific format required by downstream computation tasks) implements dimensional transformation. For example, for multi-channel time-series data with a batch size of 32, a time step of 96, and 8 feature channels, the reshaped array generates a dimension of... The three-dimensional tensor can be directly input into a convolutional neural network or a recurrent neural network for processing.

[0077] Therefore, according to the above implementation method, the system can efficiently convert multi-source heterogeneous power data into a standardized tensor format, providing a unified and high-quality input basis for subsequent multi-task prediction by hybrid expert networks.

[0078] In some embodiments, the power sector feature aggregator is configured with a backbone network and a feature fusion module; the step of performing multi-scale feature extraction and fusion on the input tensor through the power sector feature aggregator to generate a shared feature map includes: The input tensor is processed in multiple stages by the backbone network to extract multiple primary feature maps of different scales, and then upsampling operations are performed on the primary feature maps of different scales.

[0079] The backbone network refers to the feature extraction network based on a hierarchical transformer architecture, such as... Multi-stage processing is used to downsample and transform input data through multiple consecutive stages. Multi-stage processing refers to passing the input tensor through multiple network stages (such as 4 stages) in sequence, with each stage outputting a feature map with a different spatial resolution. Primary feature mapping refers to the two-dimensional feature matrix output by each network stage, which represents the data pattern at different scales. Upsampling operation refers to enlarging the spatial size of low-resolution feature maps to high resolution through interpolation or transposed convolution, so that all feature maps have the same size.

[0080] Specifically, the system processes the input tensor sequentially through stages 1 to 4 of the backbone network. Stage 1 outputs 1 / 4 resolution features, stage 2 outputs 1 / 8 resolution features, stage 3 outputs 1 / 16 resolution features, and stage 4 outputs 1 / 32 resolution features. Then, bilinear interpolation is used to upsample all feature maps to the resolution of stage 1. For example, the input tensor dimension is... (Batch size 32, height 128 pixels, width 128 pixels, number of channels 3), after processing by the backbone network, the first stage output feature dimension is: The output feature dimension of the second stage is After upsampling, they were uniformly identified as .

[0081] The feature fusion module concatenates multiple primary feature maps of different scales that have undergone upsampling along the channel dimension to generate high-dimensional concatenated features.

[0082] Among them, the feature fusion module refers to the neural network component responsible for integrating multi-source features; the channel dimension refers to the dimension in the feature tensor that represents different data types; and the high-dimensional spliced ​​feature refers to the high-channel tensor generated by connecting multiple feature maps along the channel direction.

[0083] Specifically, the system concatenates multiple upsampled feature maps in channel order. For example, it concatenates four feature maps (with 96, 192, 384, and 768 channels) into a single feature map with 1440 channels. For instance, it concatenates feature maps of dimension [missing information - likely a specific dimension]. , , , The four feature maps are concatenated to generate a high-dimensional concatenated feature map with a dimension of 1. .

[0084] The feature fusion module performs a linear projection operation on the high-dimensional stitched features to compress feature channels and generate a shared feature map.

[0085] Linear projection operation refers to the operation performed through a fully connected layer or Convolution is a linear transformation that maps high-dimensional features to a low-dimensional space; feature channel compression refers to reducing the number of channels in a feature tensor to reduce computational complexity while retaining key information; shared feature maps refer to a unified feature representation generated after compression, which is used for subsequent multi-task learning.

[0086] Specifically, the system through Convolution compresses the number of channels in a high-dimensional concatenated feature to a target value, for example, from 1440 channels to 512 channels. For example, the high-dimensional concatenated feature has a dimension of... After linear projection, the output shared feature map has a dimension of 1. .

[0087] Therefore, according to the above implementation method, the system can efficiently integrate multi-scale features to generate a shared representation suitable for multi-task prediction in the power field.

[0088] In some embodiments, the variable mixer is configured with channel convolutional layers, variable convolutional layers, and transformer layers; the step of performing local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence includes: Cross-channel information exchange is performed on shared feature maps through channel convolutional layers.

[0089] Among them, channel convolutional layers refer to those that use pointwise convolution (…). The kernel size is The neural network layer is used to achieve information fusion between different feature channels; cross-channel information interaction refers to promoting the exchange and integration of information between feature channels through linear transformation and nonlinear activation.

[0090] Specifically, the system uses the weight matrix of the channel convolutional layer and bias terms Perform a convolution operation on the input shared feature map, and the calculation formula is as follows: Here, X is the input feature map; subsequently, the GELU (Gaussian Error Linear Unit) activation function and Batch Normalization (BN) are applied to enhance the non-linear representation and stability of the features. For example, if the shared feature map has a batch size of 32, a height of 32 pixels, a width of 32 pixels, and 256 channels, after processing by the channel convolutional layer, the output feature map dimensions remain unchanged. .

[0091] Spatially adaptive convolution operations are performed on the feature maps processed by channel convolution layers using variable convolution layers.

[0092] Among them, the variable convolutional layer refers to a dynamic convolutional network layer that can learn spatial offsets; the spatial adaptive convolution operation refers to dynamically adjusting the sampling position of the convolutional kernel according to the input feature content in order to capture local deformation features.

[0093] Specifically, the system first learns each spatial location from the input feature map. offset Then apply a variable convolution kernel. The formula for convolution calculation is: ; Subsequently, features are optimized through residual connections, GELU activation, and batch normalization. For example, the input feature map dimension is... The dimension of the output feature map of the variable convolutional layer remains the same. The offset learning module has the following parameter count: (Learn 2D offset at each position).

[0094] The feature maps processed by the variable convolutional layer are serialized and then input into the transformer layer.

[0095] Serialization refers to flattening the three-dimensional feature map into a one-dimensional sequence to adapt to the input requirements of the transformer layer.

[0096] Specifically, the system uses a flattening operation to merge the spatial dimensions (height and width) of the feature map into the sequence length, while preserving the channel dimension; for example, the feature map dimensions... Flattened into sequence dimension 1024 is the sequence length (32 by 32). For example, the flattened sequence can be directly input into the transformer layer without additional dimensional adjustment.

[0097] The self-attention computation is performed on the serialized features through the transformer layer to extract global contextual dependencies and generate an optimized feature sequence.

[0098] The transformer layer refers to a neural network module based on the self-attention mechanism (which is used to calculate the correlation weights between elements at different positions in a sequence, thereby dynamically generating a representation that incorporates global contextual information of the sequence for each position); self-attention calculation refers to calculating the dependency weights between sequence elements through the interaction of query, key, and value matrices.

[0099] Specifically, the system uses MHSA, first projecting the input sequence into a query, key, and value matrix, and then calculating the scaled dot product attention, as shown in the formula: ; in, The input sequence has the dimension of the key vector; the output is processed by Layer Normalization (LN) to generate an optimized feature sequence. For example, the input sequence has the dimension of... The dimension of the output sequence of the transformer layer remains unchanged, and is still 1. The number of attention heads is set to 8, and the key dimension is... .

[0100] Therefore, according to the above implementation method, the system can effectively fuse local details and global context through multi-level processing of the variable mixer to generate an optimized feature sequence suitable for multi-task prediction.

[0101] In some embodiments, the global hybrid expert network is configured with multiple global expert units; the step of capturing cross-task common knowledge and outputting global expert features through the global hybrid expert network includes: The optimized feature sequence is used as the input feature of the global hybrid expert network. Multiple global expert units perform parallel processing on the input feature to generate multiple expert output features.

[0102] Among them, global expert units refer to independent neural network modules with the same structure. Each expert unit is responsible for extracting cross-task common patterns from the input features; parallel processing means that all global expert units process the same input features at the same time to improve computational efficiency.

[0103] Specifically, the system distributes input features to each global expert unit, and each unit independently performs forward propagation computation to generate corresponding expert output features; the dimension of the expert output features is consistent with the dimension of the input features. For example, the optimized feature sequence dimension is batch size 32, time step size 576, and feature dimension 128 (i.e., ... The global expert unit has 4 units, and each unit outputs a feature dimension of 1. Four expert output features are generated.

[0104] Multiple expert output features are aggregated to generate aggregated output features.

[0105] Aggregation refers to merging multiple expert output features into a single feature representation through a summation operation, in order to integrate common knowledge across tasks.

[0106] Specifically, the system performs element-wise addition on all expert output features along the expert dimension to generate aggregated output features; the dimension of the aggregated output features is the same as the dimension of the individual expert output features. For example, 4 expert output features (each with a dimension of...) After summation and aggregation, the aggregated output feature dimension is . .

[0107] The aggregated output features are added to the input features to generate global expert features.

[0108] Among them, the addition operation refers to the residual connection mechanism (which introduces a "shortcut path" to pass the input directly to the subsequent layer and add it element-wise with the output after nonlinear transformation, making it easier for the network to learn identity mappings or small residuals). By adding the aggregated output features to the original input features element-wise, it achieves the fusion of common knowledge and original features and avoids the gradient vanishing problem.

[0109] Specifically, the system directly performs element-wise addition on the aggregated output features and input features to generate global expert features; the dimension of the global expert features is the same as the dimension of the input features. For example, the dimension of the aggregated output features is... The input feature (i.e., the optimized feature sequence) has a dimension of The sum of these dimensions generates the global expert feature dimension. The process conforms to the formula: ; in, Represents global expert characteristics. Indicates input features, This represents the computational results of the global hybrid expert network.

[0110] Therefore, according to the above implementation method, the system can efficiently capture common knowledge across tasks through parallel processing, aggregation and residual connection of the global expert network, providing a unified feature basis for multi-task prediction.

[0111] In some embodiments, the task-specific hybrid expert network is configured with a shared expert network set, a task-specific expert network set, and a routing decision-maker; the step of capturing task-specific knowledge and outputting task expert features through the task-aware dynamic routing mechanism of the task-specific hybrid expert network includes: By using the routing decision maker, the affinity weights of each expert network in the task-specific expert network set are calculated based on the optimized feature sequence.

[0112] Among them, the routing decision-maker refers to a neural network module based on a gating mechanism, which is used to evaluate the relevance between input features and various expert networks; the affinity weight refers to a scalar value calculated through similarity, which represents the degree of matching between input features and a specific expert network.

[0113] Specifically, the system projects the optimized feature sequence into a query vector through a routing decision unit, performs a dot product operation with the feature centroid vector of each expert network, and then applies... The function (S-type activation function) is normalized to generate affinity weights; the calculation formula is as follows: ,in To optimize the representation of feature sequences, For the first The feature centroid vector of an expert network. For example, optimize the feature sequence dimension to batch size 32, sequence length 576, and feature dimension 128 (i.e., The task-specific expert network set contains 10 expert networks, and the routing decision-maker outputs 10 affinity weights, each with a value between 0 and 1.

[0114] Based on affinity weights, multiple expert networks with the highest affinity weights are dynamically selected from the set of expert networks for a specific task to form an activated expert subset.

[0115] Dynamic selection refers to adaptively selecting experts based on the weight ranking results calculated in real time; activated expert subset refers to the expert network combination selected under the current input.

[0116] Specifically, the system sorts the affinity weights in descending order and selects the expert networks corresponding to the top K weights as the activated expert subset, where K is the preset number of activated experts. For example, after sorting the affinity weights, the first four weight values ​​are 0.95, 0.89, 0.85, and 0.80, corresponding to expert networks E1, E3, E5, and E7, which are selected to form the activated expert subset. .

[0117] By performing parallel processing on the optimized feature sequence using a shared set of expert networks, shared expert features are obtained. Furthermore, by activating each expert network in the expert subset, the optimized feature sequence is processed in parallel to obtain multiple task-specific expert features.

[0118] Parallel processing refers to multiple expert networks performing forward propagation computations simultaneously; shared expert features refer to the common knowledge representation output by a set of shared expert networks; and task-specific expert features refer to the task-specific features output by a subset of activated experts.

[0119] Specifically, the shared expert network set contains two shared expert networks, and the activated expert subset contains four task-specific expert networks. The system inputs the optimized feature sequences into these networks, and each network processes and outputs features independently. The shared expert feature dimension is [missing value]. Each task has its own unique expert feature dimension. For example, a shared expert network set outputs 2 features, and an activated expert subset outputs 4 features, generating a total of 6 expert features.

[0120] Based on affinity weights, the weighted sum of multiple task-specific expert features is obtained to obtain weighted task expert features.

[0121] Weighted summation refers to the linear combination of features based on weight coefficients.

[0122] Specifically, the system multiplies the expert features specific to each task by their corresponding affinity weight, and then sums all the weighted features; the calculation formula is: weighted features ;in For the first The affinity weight of each expert, This represents the output of the i-th expert. For example, if the weights of the four task-specific expert features are 0.95, 0.89, 0.85, and 0.80 respectively, the weighted sum will generate the weighted task expert feature dimension. .

[0123] The shared expert features and the weighted task expert features are summed to generate task expert features.

[0124] Among them, summation refers to the element-by-element addition operation, which is used to integrate common knowledge with task-specific knowledge.

[0125] Specifically, the system adds the shared expert features and the weighted task expert features element-wise along the same dimension to generate task expert features. For example, the shared expert feature dimension... Weighted task expert feature dimensions Add them together, and the output task expert feature dimension is: .

[0126] Therefore, according to the above implementation method, the system can accurately activate relevant expert networks through dynamic routing mechanism, efficiently integrate shared and task-specific features, and achieve collaborative optimization of multi-task prediction.

[0127] In some embodiments, feature fusion is performed on global expert features and task expert features to generate fused features corresponding to each specific task, which are used to generate prediction results for each specific task, including: For each specific task, the global expert features are added element-wise to the corresponding task expert features to generate the fused features for that specific task.

[0128] Among them, the element-wise addition operation refers to performing an addition operation on each corresponding element of two feature tensors in the same dimension, which is used to achieve feature fusion.

[0129] Specifically, the system performs element-wise addition on the global expert features and task expert features along the same dimensions to ensure consistency in feature dimensions and generate fused features. For example, the global expert feature dimensions are batch size 32, time step 576, and feature dimension 128 (i.e., ...). The task experts have the same feature dimension, and after adding them together, the fused feature dimension is... .

[0130] The fused features are input into the task head network corresponding to this specific task.

[0131] The task head network refers to a lightweight output layer designed for each specific task. It is usually composed of fully connected layers or convolutional layers and is used to map fused features into prediction results.

[0132] Specifically, the system inputs the fused features into a task-specific head network, which then performs linear or non-linear transformations based on task requirements. For example, for a load forecasting task, the task head network might be a fully connected layer, outputting load values ​​at 96 time steps, with a dimension of [missing information]. For voltage warning tasks, the task header network may output risk probabilities for 32 nodes, with the dimension being... .

[0133] The fused features are processed by the task head network to generate prediction results for the specific task.

[0134] The prediction results refer to the model's estimates of the power system's operating status, such as load values, voltage fluctuation probabilities, or equipment status indicators.

[0135] Specifically, the task head network uses activation functions (such as...) A function (or linear function) processes the fused features and outputs the final predicted value. For example, the Task Head Network uses... The activation function converts the fused features into probability values ​​between 0 and 1, representing the risk of voltage exceeding limits; or a linear activation function can be used to directly output the load power value in megawatts (MW).

[0136] Therefore, according to the above implementation method, the system can efficiently generate accurate prediction results for each specific task through feature fusion and task head network processing, and achieve multi-task collaborative optimization.

[0137] Figure 3 This is a structural block diagram of a multi-task prediction system based on a hybrid expert network according to an embodiment of the present invention.

[0138] like Figure 3 As shown, the multi-task prediction system based on a hybrid expert network includes: The input tensor construction module 210 is used to acquire multi-source heterogeneous operation data of the power system and preprocess the multi-source heterogeneous operation data to generate input tensors.

[0139] The shared feature map generation module 220 is used to input the input tensor to a preset power domain feature aggregator, and to perform multi-scale feature extraction and fusion on the input tensor through the power domain feature aggregator to generate a shared feature map.

[0140] The optimized feature sequence generation module 230 is used to input the shared feature map into the variable mixer, and perform local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence.

[0141] The hybrid expert feature generation module 240 is used to input the optimized feature sequence into the global hybrid expert network and the task-specific hybrid expert network in parallel. The global hybrid expert network captures common knowledge across tasks and outputs global expert features, while the task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features.

[0142] The multi-task prediction result generation module 250 is used to perform feature fusion on global expert features and task expert features to generate fused features corresponding to each specific task, which are then used to generate prediction results for each specific task.

[0143] The specific functions and examples of each module and submodule of the device in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0144] According to embodiments of the present invention, the above-described method of the present invention can be applied to an electronic device and a readable storage medium.

[0145] Figure 4 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0146] like Figure 4 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0147] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0148] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a multi-task prediction method based on a hybrid expert network. For example, in some embodiments, a multi-task prediction method based on a hybrid expert network can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of a multi-task prediction method based on a hybrid expert network described above can be performed. Alternatively, in other embodiments, computing unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform a multi-task prediction method based on a hybrid expert network.

[0149] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0154] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0155] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0156] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-task prediction method based on hybrid expert networks, characterized in that, include: Acquire multi-source heterogeneous operation data of the power system, and preprocess the multi-source heterogeneous operation data to generate input tensors; The input tensor is input to a preset power domain feature aggregator, which performs multi-scale feature extraction and fusion on the input tensor to generate a shared feature map. The shared feature map is input into a variable mixer, which performs local feature enhancement and global context dependency extraction to generate an optimized feature sequence. The optimized feature sequence is input in parallel into a global hybrid expert network and a task-specific hybrid expert network. The global hybrid expert network captures cross-task common knowledge and outputs global expert features, while the task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features. The global expert features and the task expert features are fused to generate fused features corresponding to each specific task, which are used to generate prediction results for each specific task.

2. The method according to claim 1, characterized in that, The multi-source heterogeneous operation data includes load curve data, voltage fluctuation data, equipment monitoring data, meteorological condition data, and time series data; The process of acquiring multi-source heterogeneous operation data of the power system and preprocessing the multi-source heterogeneous operation data to generate input tensors includes: Data cleaning, normalization, and time-series alignment are performed on the load curve data, voltage fluctuation data, equipment monitoring data, meteorological condition data, and time series data to generate multi-channel time series data. A tensor reshaping operation is performed based on the multi-channel time-series data to generate the input tensor in a three-dimensional tensor format; The tensor reshaping operation includes: The structure of the multi-channel time series data is converted into a three-dimensional tensor including batch size dimension, time step dimension and feature channel dimension. The batch size dimension represents the number of samples, the time step dimension represents the number of time points within the preset time window, and the feature channel dimension represents the number of data types.

3. The method according to claim 1, characterized in that, The power sector feature aggregator is configured with a backbone network and a feature fusion module; the step of performing multi-scale feature extraction and fusion on the input tensor through the power sector feature aggregator to generate a shared feature map includes: The input tensor is processed in multiple stages through the backbone network to extract multiple primary feature maps of different scales, and an upsampling operation is performed on the multiple primary feature maps of different scales. The feature fusion module concatenates the multiple primary feature maps of different scales that have undergone upsampling along the channel dimension to generate high-dimensional concatenated features. The feature fusion module performs a linear projection operation on the high-dimensional stitched features to compress feature channels and generate the shared feature map.

4. The method according to claim 1, characterized in that, The variable mixer is configured with channel convolutional layers, variable convolutional layers, and transformer layers; the step of performing local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence includes: Cross-channel information interaction is performed on the shared feature map through the channel convolutional layer; The variable convolutional layer performs a spatial adaptive convolution operation on the feature map processed by the channel convolutional layer. The feature map processed by the variable convolutional layer is serialized and then input into the transformer layer; The transformer layer performs self-attention computation on the serialized features to extract global contextual dependencies and generate the optimized feature sequence.

5. The method according to claim 1, characterized in that, The global hybrid expert network is configured with multiple global expert units; the step of capturing cross-task common knowledge and outputting global expert features through the global hybrid expert network includes: The optimized feature sequence is used as the input feature of the global hybrid expert network, and the input feature is processed in parallel by the multiple global expert units to generate multiple expert output features; The multiple expert output features are aggregated to generate aggregated output features; The aggregated output features are added to the input features to generate the global expert features.

6. The method according to claim 1, characterized in that, The task-specific hybrid expert network is configured with a shared expert network set, a task-specific expert network set, and a routing decision-maker; the step of capturing task-specific knowledge and outputting task expert features through the task-aware dynamic routing mechanism of the task-specific hybrid expert network includes: The affinity weights of each expert network in the specific task expert network set are calculated by the routing decision-maker based on the optimized feature sequence. Based on the affinity weights, multiple expert networks with the highest affinity weights are dynamically selected from the set of expert networks for the specific task to form an activated expert subset; The optimized feature sequence is processed in parallel by the shared expert network set to obtain shared expert features, and the optimized feature sequence is processed in parallel by each expert network in the activated expert subset to obtain multiple task-specific expert features. Based on the affinity weights, the multiple task-specific expert features are weighted and summed to obtain weighted task expert features; The shared expert features and the weighted task expert features are summed to generate the task expert features.

7. The method according to claim 1, characterized in that, The step of fusing the global expert features and the task expert features to generate fused features corresponding to each specific task, and using these features to generate prediction results for each specific task, includes: For each specific task, the global expert features are added element-wise to the corresponding task expert features to generate the fusion features for that specific task. The fusion features are input into the task head network corresponding to the specific task. The fused features are processed by the task head network to generate a prediction result for the specific task.

8. A multi-task prediction system based on a hybrid expert network, characterized in that, include: The input tensor construction module is used to acquire multi-source heterogeneous operation data of the power system and preprocess the multi-source heterogeneous operation data to generate input tensors; A shared feature map generation module is used to input the input tensor into a preset power domain feature aggregator, and to perform multi-scale feature extraction and fusion on the input tensor through the power domain feature aggregator to generate a shared feature map; An optimized feature sequence generation module is used to input the shared feature map into a variable mixer, and perform local feature enhancement and global context dependency extraction through the variable mixer to generate an optimized feature sequence; The hybrid expert feature generation module is used to input the optimized feature sequence into a global hybrid expert network and a task-specific hybrid expert network in parallel. The global hybrid expert network captures cross-task common knowledge and outputs global expert features, while the task-specific hybrid expert network captures task-specific knowledge based on a task-aware dynamic routing mechanism and outputs task expert features. The multi-task prediction result generation module is used to perform feature fusion on the global expert features and the task expert features to generate fused features corresponding to each specific task, so as to generate prediction results for each specific task.

9. An electronic device, characterized in that, include: At least one processor; and a memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, Computer instructions are used to cause a computer to perform the method according to any one of claims 1-7.

Images