Multi-modal feature and transfer learning non-intrusive load monitoring method and system

Abstract

The invention relates to the crossing field of a smart power grid and artificial intelligence, discloses a non-intrusive load monitoring method and system based on multi-modal features and transfer learning, and aims at the bottlenecks of insufficient long sequence modeling, single feature representation and weak cross-domain generalization in the prior art. The voltage-current track is coded into an HSV color image, and multi-modal input is constructed; an optimized TransDisNILM model is adopted to capture long sequence dependence and transient characteristics, and a sub-task gating network is integrated to realize power decomposition and state recognition dual-task cooperation; and quantitative source task selection, domain adaptive loss and hierarchical migration strategies are designed to improve the cross-domain adaptation efficiency. The method significantly improves the monitoring accuracy, generalization and practicability, reduces the data dependence, and is suitable for the scenes of intelligent electric meters, household energy management and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart grid and artificial intelligence, in particular to a multi-modal feature and transfer learning non-intrusive load monitoring method and system. BACKGROUND

[0002] The core of the non-intrusive load monitoring technology is to realize the accurate decomposition of individual electrical appliance energy consumption and running state by collecting user side total power consumption data, without the need to deploy monitoring devices for each electrical appliance, with the advantages of low cost and easy deployment, which is a key supporting technology for building smart grid supply and demand interaction system and realizing fine energy management of smart home.

[0003] However, the existing deep learning based NILM technical scheme still faces three major technical bottlenecks, which restricts its application effect in complex scenarios: 1. Insufficient long sequence dependence capture ability: washing machines, dishwashers and other household loads generally have the characteristics of multiple working state switching and long running period, and the network architecture of most CNN or RNN models is limited by the receptive field range or gradient disappearance problem, which makes it difficult to effectively model the long time span dynamic running mode of such loads, resulting in a significant decrease in the decomposition accuracy of multi-state loads.

[0004] 2. Single input feature representation dimension: most existing methods only use total power time series as model input, without fully mining the inherent impedance characteristics of loads contained in voltage-current (V-I) trajectory and other physical dimensions. When facing electrical appliances with similar power characteristics but different physical properties, the model's discrimination ability is severely insufficient.

[0005] 3. Weak cross-domain generalization ability and dependence on a large amount of labeled data: the training of NILM model usually relies on labeled data (source domain) of a specific family or region, when deployed to a new scene (target domain) with different load characteristics and user electricity usage habits, the model performance will drop sharply. Although some researches introduce transfer learning technology to improve cross-domain adaptability, most existing schemes use simple parameter fine-tuning or single alignment metric, lack of systematic design of source task screening, multi-target collaborative learning and hierarchical transfer strategy, resulting in unstable knowledge transfer effect and low transfer efficiency.

[0006] The closest prior art to the technical solution of the present application is the deep domain adaptation transfer learning network (TLN) proposed by Jun Lin et al. in the paper "Deep Domain Adaptation for Non-Intrusive Load Monitoring Based on a Knowledge Transfer Learning Network" published in the IEEE Transactions on Smart Grid journal in 2022. The core goal of this scheme is to improve the cross-domain generalization ability of the NILM model through transfer learning, and its specific technical solution is as follows: The design of the backbone feature extraction network: an improved time convolution network (TCN) based on the AlexNet architecture is used as the core of feature extraction, which contains 5 TCN residual blocks and 3 fully connected layers, which can extract dynamic time sequence features of load operation from the total load power sequence; Domain adaptation loss function construction: in the fc6-fc8 fully connected layer of the network, the maximum mean difference (MMD) loss and CORAL loss are fused to construct the domain adaptation loss function, which realizes the distribution alignment of source domain and target domain data in the feature space; Multi-loss joint optimization training: the mean square error (MSE) regression loss of load power decomposition and the above domain adaptation loss are jointly optimized, and an end-to-end transfer learning model that can adapt to both source and target domains is trained to improve the load decomposition performance of the model in unknown target domains.

[0007] However, this prior art scheme still has obvious defects and cannot solve the aforementioned industry common bottlenecks, which are as follows: Limitations of long sequence load modeling capability: although the time convolution network (TCN) used in this scheme has the advantages of causal convolution and scalable receptive field, it still has shortcomings in directly modeling the global dependence relationship of long-period multi-state loads, especially in the fine state decomposition scene of long-running period loads such as dishwashers and washing machines, the model performance still has a lot of room for improvement; The limitation of feature representation dimension has not been broken through: it only uses the total power sequence as the input feature, without introducing physical characteristic dimensions such as voltage-current trajectory, resulting in a single dimension of load feature information obtained by the model, which cannot effectively distinguish electrical appliances with similar power characteristics but different physical impedances, and the accuracy of load identification is limited; The migration learning strategy lacks systematic design: firstly, the source task selection lacks quantitative evaluation basis, and only relies on experimental experience or prior knowledge to select the migration source, without standardized evaluation index and theoretical guidance, so the efficiency of migration source selection is low and the effect is unstable; secondly, the task target is single, only focuses on the regression task of load power decomposition, does not combine the classification task information of equipment state identification, and limits the discriminability and generalization ability of model feature learning; thirdly, the cross-domain fine-tuning strategy is not designed specifically, and there is no clear differentiated parameter adjustment mechanism for different levels of network, so it is difficult to balance the preservation of general feature extraction ability and the adaptation of target domain specific characteristics, resulting in insufficient efficiency and stability of knowledge migration.

[0008] In summary, the defects of the existing NILM technology in long sequence modeling, multi-dimensional feature representation and systematic migration learning have become the core obstacles restricting its large-scale application, and new technical solutions are urgently needed to solve them. SUMMARY

[0009] In view of the deficiencies of the prior art, the present application provides a multi-modal feature and migration learning non-intrusive load monitoring method and system, which aims to solve the problems in the background art.

[0010] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a multi-modal feature and migration learning non-intrusive load monitoring method, comprising the following steps: Step S1, multi-modal data preprocessing: the total load power sequence is intercepted by a sliding window and a denoising recurrent graph is generated, the voltage and current trajectories are decomposed based on Fryze theory to obtain a color-coded V-I image, and the features of the three are integrated to form a multi-modal input; Step S2, mixed model construction and source domain pre-training: based on the multi-modal input, a TransDisNILM backbone network is built, after feature extraction through convolution enhancement and position coding, a 3-layer Transformer encoder is used for modeling; a gating network is integrated to combine power regression and state classification double tasks, and the source domain model is obtained by training and minimizing the error using source domain labeled data; Step S3, migration learning and cross-domain adaptation: relying on the source domain model, the source task is selected according to the average active channel number and feature importance, and a joint loss function containing task loss and MMD+CORAL domain adaptation loss is constructed; the strategy of freezing the backbone network and fine-tuning only the gating network is adopted to realize the adaptation of the source domain model to the target domain; Step S4, monitoring result output: finally, the target domain model after adaptation is used to output the load power values and running state probabilities synchronously, providing core data support for smart grid supply and demand interaction and home energy management.

[0011] Further, the specific process of step S1 is: Step S1.1: A sliding window interception method is used for the total load power sequence, the window length and the sliding step are set, and a plurality of power subsequences are obtained; based on each power subsequence, a two-dimensional recurrence graph is generated by a recurrence graph method: based on the Euclidean distance between elements in the power subsequence, a distance threshold is set, and the Heaviside step function is used to determine whether the distance between elements is less than the threshold, and then a recurrence matrix is constructed to form a recurrence graph; the binary recurrence graph generated is processed by median filtering to remove isolated noise points, and finally the recurrence graph features are obtained; Step S1.2: The voltage and current trajectories are preprocessed based on the Fryze power theory; the preprocessed voltage-current trajectory is normalized and mapped, and a three-dimensional encoding is performed in the HSV color space to generate a color-coded V-I trajectory image; Step S1.3: The original total load power sequence, the recurrence graph features, and the color-coded V-I trajectory image are integrated to form a multi-modal input.

[0012] Further, the specific process of step S2 is as follows: Step S2.1: A TransDisNILM hybrid deep perception is constructed as a backbone feature extraction network, and the multi-modal input of step 1.3 is connected to the coding layer of the backbone feature extraction network; a deep convolution enhancement module is introduced in the coding layer to strengthen feature extraction through one-dimensional convolution, ReLU activation function and maximum pooling operation; at the same time, a sinusoidal position coding is embedded to reserve the time sequence information through the association calculation of position and feature dimension; the encoder layer of the network is set to 3 layers of Transformer encoder, each layer containing a multi-head self-attention mechanism and a feedforward network: the self-attention mechanism realizes feature association modeling through the operation and Softmax normalization of the query, key and value matrix; the feedforward network adopts GELU activation function for nonlinear transformation, and each layer is supplemented with residual connection and layer normalization to ensure training stability; Step S2.2: A sub-task gating network is integrated at the output layer of the backbone feature extraction network to construct a double-task collaborative framework; Step S2.3: Based on the source domain labeled data and the true labels of each independent electrical equipment, the backbone feature extraction network is pre-trained, the mean square error is used as the loss function, the error between the model output of the total load power sequence and the true power of each load is minimized, the optimal model in the hypothesis space is determined, and the source domain model is obtained.

[0013] Further, step S2.2: The specific process of integrating the sub-task gating network SGN into the output layer of the backbone feature extraction network to construct the dual-task collaborative framework is as follows: The regression sub-network estimates the target load power value by applying two linear transformations and the Tanh activation function to the output features of the upstream perceptron, i.e., the backbone feature extraction network; the classification sub-network outputs the probability of the device operating state by applying a linear transformation and the Sigmoid activation function to the output features of the upstream perceptron; a gating fusion mechanism is introduced to multiply the regression result and the classification probability element-wise to achieve dual-task feature interaction and collaborative optimization, and obtain the final output.

[0014] Furthermore, the specific process of step S3 is as follows: Step S3.1: The source task is quantitatively selected by combining the average active channel count (AAC) with the random forest feature importance analysis. The average active channel count is obtained by comparing the activation values ​​of each load at different times and different channels with the preset threshold, and taking the mean value over the time dimension. Step S3.2: Construct a joint loss function to reduce the distribution difference between the source domain and the target domain. The total loss function is composed of a weighted sum of the task loss and the domain adaptive loss. Step S3.3: Adopt the hierarchical migration strategy of "freezing the shared layer and fine-tuning the specific layer": freeze the parameters of the TransDisNILM backbone network and fine-tune only the parameters of its sub-task gating network; based on the target domain data, complete the target domain model adaptation by minimizing the error between the model output and the actual power of each load in the target domain.

[0015] Furthermore, the task loss is the sum of the mean square error of each load power estimate and the binary cross-entropy of the equipment status identification, with the importance of the two adjusted by weighting coefficients; the domain adaptive loss is the weighted sum of the MMD loss and the CORAL loss.

[0016] Furthermore, the specific process of step S4 is as follows: using the target domain model obtained in step S3, the power estimate of each load and the probability of equipment operating status are output synchronously to provide accurate monitoring data support for the supply and demand interaction of smart grid and the refined management of household energy.

[0017] A non-invasive load monitoring system based on multimodal features and transfer learning, including: The multimodal data preprocessing module is used to extract the total load power sequence through a sliding window and generate a denoised recursive graph. It decomposes the voltage and current trajectories based on Fryze theory to obtain a color-coded VI image and integrates the features of the three to form a multimodal input. The hybrid model construction and source domain pre-training module is used to build the TransDisNILM backbone network based on multimodal input. After feature extraction through convolutional enhancement and positional encoding, it is modeled through a 3-layer Transformer encoder. The integrated gated network fuses the dual tasks of power regression and state classification. It is trained using source domain labeled data and the error is minimized to obtain the source domain model. The transfer learning and cross-domain adaptation module is used to select source tasks based on the source domain model, combined with the average number of active channels and feature importance, and to construct a joint loss function containing task loss and MMD+CORAL domain adaptation loss. It adopts a strategy of freezing the backbone network and only fine-tuning the gating network to adapt the source domain model to the target domain. The monitoring result output module is used to synchronously output the power values ​​and operating status probabilities of each load using the adapted target domain model, providing core data support for smart grid supply and demand interaction and home energy management.

[0018] An electronic device includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute a non-intrusive load monitoring method based on multimodal features and transfer learning.

[0019] A non-volatile computer storage medium storing computer-executable instructions that perform a non-intrusive load monitoring method based on multimodal features and transfer learning.

[0020] Compared with existing technologies, the present invention has the following advantages: Introducing an optimized Transformer architecture to enhance sequence modeling capabilities; To address the shortcomings of existing models in capturing long-term dependencies and rapidly changing local features, this invention employs a specially optimized Transformer model (TransDisNILM) as the backbone network. By introducing sinusoidal positional encoding to provide stable temporal information and adding deep convolutional pooling layers to enhance the ability to capture transient features during device start-up and shutdown, this model can more effectively coordinate the modeling of long-term operating patterns and short-term rapidly changing features of the load, fundamentally improving the model's expressive power and decomposition accuracy.

[0021] Constructing multimodal feature inputs to improve load discrimination capability; To address the issue of limited feature representation in existing methods, this invention converts the total load power sequence into a recursive graph (RP) and encodes the voltage-current trajectory as a color image, forming a multimodal input. The recursive graph captures the dynamic patterns of load operation by visualizing the recursive characteristics of the time-series states, while the color-coded VI trajectory image provides three-dimensional encoding of trajectory direction, power factor, and operational stability in HSV space. This provides the model with multi-dimensional complementary information from power time-series dynamics to electrical physical characteristics, significantly enhancing the ability to distinguish between loads with similar features.

[0022] Design a systematic transfer learning framework to improve cross-domain adaptation efficiency; To address the lack of systematic approach in existing transfer learning methods, this invention proposes a complete transfer learning framework: Systematic source task selection mechanism: A source task selection method based on average active channel number (AAC) and random forest feature importance analysis is proposed. By quantitatively evaluating the representativeness and migration value of the load, a strategy with sufficient theoretical basis is provided for the selection of migration sources.

[0023] Dual-task collaborative learning architecture: The model architecture integrates a sub-task gating network (SGN) to simultaneously perform dual-task learning of load power decomposition (regression task) and device status recognition (classification task). Through a gating fusion mechanism, deep interaction and collaborative optimization of the two tasks are achieved, making the features learned by the model more discriminative and generalizable.

[0024] Layered migration fine-tuning strategy: This strategy clarifies the use of "freezing the shared layer and fine-tuning specific layers" when applying across domains. By freezing the parameters of the TransDisNILM backbone network and only fine-tuning the parameters of the subtask gating networks, this strategy can effectively retain the general knowledge learned in the source domain, while using a small amount of target domain data to efficiently adapt to specific loads, thus maximizing the efficiency of knowledge transfer.

[0025] High monitoring accuracy: Through multimodal feature fusion and dual-task collaborative learning, the accuracy of load decomposition and state identification is significantly improved.

[0026] High generalization and adaptability: The systematic transfer learning strategy effectively alleviates the domain difference problem and maintains stable performance in cross-dataset testing.

[0027] Low data dependency and deployment cost: The layered migration mechanism only requires a small amount of target domain data to complete efficient adaptation, which lowers the threshold for practical application.

[0028] High practicality: It outputs power and status information synchronously, making it suitable for edge scenarios such as smart meters and home energy management.

[0029] This invention significantly improves the accuracy, adaptability, and deployability of load monitoring. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0031] like Figure 1 As shown, the present invention provides a technical solution: a non-invasive load monitoring method based on multimodal features and transfer learning, comprising the following steps: Step S1: Multimodal data preprocessing: The total load power sequence is extracted by a sliding window and a denoised recursive graph is generated. The voltage and current trajectories are decomposed based on Fryze theory to obtain a color-coded VI image. The features of the three are integrated to form a multimodal input.

[0032] The specific process of step S1 is as follows: Step S1.1: Apply a sliding window truncation method to the total load power sequence, setting the window length to [value missing]. With a sliding step size of 240, several power subsequences are obtained. Based on each power subsequence, a two-dimensional recursive graph is generated using the recursive graph method: taking the Euclidean distance between each element in the power subsequence as the basis, a distance threshold is set to 0.1 times the standard deviation of the sequence. The Heaviside step function is used to determine whether the distance between elements is less than the threshold, and then a recursive matrix is ​​constructed to form a recursive graph. The generated binary recursive graph is processed by 3×3 median filtering to remove isolated noise points, and finally the recursive graph features are obtained.

[0033] Among them, the recursion matrix of the two-dimensional recursion graph generated by the recursion graph method is calculated as follows: ; In the formula, Represents the nth element in the recursive matrix line, number Column elements; Represents the Heaviside step function; Indicates the distance threshold. Represents power subsequence Standard deviation; , They represent the power subsequences of the first and second power subsequences, respectively. The, the Data points.

[0034] Step S1.2: Based on Fryze power theory, the voltage... With current The trajectory (collected from the main power supply input of the monitoring scene, reflecting the "non-intrusive" nature of NILM) undergoes preprocessing: the current is decomposed into active and non-active components. The active component is calculated using active power and the effective voltage value, while the non-active component is the difference between the total current and the active component. Active power is calculated using the trapezoidal integration method, with a sampling frequency of 8Hz and a single integration period of 0.125 seconds. The preprocessed voltage-current (VI) trajectory is normalized to the [-1,1] interval and mapped to a 2N×2N grid, where N represents the grid size parameter. Three-dimensional encoding is then performed in the HSV color space to generate a color-coded VI trajectory image. The hue dimension represents the trajectory direction and is determined by the ratio of changes in adjacent voltages and currents; the saturation dimension represents the power factor and is calculated by the ratio of active power to apparent power; the brightness dimension represents the operational stability and is obtained from the mean value of 10 consecutive cycles of binary trajectory images.

[0035] Here, the current is decomposed into active and inactive components, which is expressed as: ; ; In the formula, express The active component of the current at any given moment; This represents active power (calculated using the trapezoidal integration method). This represents the integration period, corresponding to one cycle of 8Hz sampling. (representing differential operators); Indicates voltage The effective value; express The non-active component of the current at any given moment.

[0036] The preprocessed voltage-current (VI) trajectory is normalized to the [-1,1] interval, mapped to a 2N×2N grid, and then three-dimensionally encoded in the HSV color space to generate a color-coded VI trajectory image. Its encoding formula is: ; ; ; In the formula, Represents the HSV color space The hue value corresponding to each position is used to represent the direction of the VI trajectory at that position; Represents angle operations; , These represent the first digit of the preprocessed voltage-current (VI) trajectory. , Voltage sample value at time; Represents the maximum value of the voltage sequence; Represents the maximum value of the current sequence; , These represent the first digit of the preprocessed voltage-current (VI) trajectory. , Current sample value at time; This represents the saturation value in the HSV color space and is used to characterize the power factor. Indicates active power; Indicates apparent power; Represents the lightness value in the HSV color space; Indicates the number of consecutive periods; Indicates the first Binary image of the VI trajectory corresponding to each period.

[0037] Step S1.3: Integrate the original total load power sequence, the recursive graph features obtained in step S1.1, and the color-coded VI trajectory image obtained in step S1.2. This generates multimodal inputs, providing a foundation for subsequent model training.

[0038] Step S2, Hybrid Model Construction and Source Domain Pre-training: Based on multimodal input, a TransDisNILM backbone network is built. After convolutional enhancement and positional encoding to extract features, a 3-layer Transformer encoder is used to model the model. A gating network is integrated to fuse power regression and state classification tasks. The source domain labeled data is used for training and the error is minimized to obtain the source domain model.

[0039] The specific process of step S2 is as follows: Step S2.1: Construct a TransDisNILM hybrid deep perceptron as the backbone feature extraction network, and input the multimodal input from Step 1.3 into the encoding layer of the backbone feature extraction network. This encoding layer introduces a deep convolution enhancement module, which enhances feature extraction through one-dimensional convolution, ReLU activation function, and max pooling operation. At the same time, sinusoidal position encoding is embedded to preserve temporal information through the correlation calculation between position and feature dimension. The network's encoder layer consists of 3 Transformer encoder layers, each containing a multi-head self-attention mechanism and a feedforward network: the self-attention mechanism achieves feature correlation modeling through operations on the query, key, and value matrices and Softmax normalization; the feedforward network uses the GELU activation function for nonlinear transformation, and each layer is supplemented with residual connections and layer normalization to ensure training stability.

[0040] The encoding layer incorporates a depthwise convolution enhancement module, represented as follows: ; In the formula, This represents the output features of the deep convolutional enhancement module; This represents the max pooling operation; express Activation function; This represents a one-dimensional convolution operation; This represents the input data for the depthwise convolutional enhancement module.

[0041] The embedded sinusoidal position code, used to enhance timing information, is represented as follows: ; In the formula, Represents the pos-th position in the sequence, the... The positional encoding values ​​corresponding to each feature dimension are generated using a sine function; Represents the pos-th position in the sequence, the... The positional encoding values ​​corresponding to each feature dimension are generated using a cosine function; This represents the feature dimension of the model.

[0042] The self-attention mechanism is expressed as follows: ; In the formula, This represents the output of the self-attention mechanism; , , These represent the query matrix, key matrix, and value matrix, respectively, which are obtained from the input features through a linear transformation. express Activation function; express transpose; express The feature dimensions.

[0043] The formula for the feedforward network is expressed as follows: ; In the formula, This represents the output of the feedforward network. This represents the input characteristics of the feedforward network; Represents the activation function of the Gaussian error linear unit; , These represent the weight matrices for the first and second linear transformations, respectively. , These represent the bias terms corresponding to the first and second linear transformations, respectively.

[0044] Step S2.2: Integrate a subtask-gated network (SGN) into the output layer of the backbone feature extraction network to construct a dual-task collaborative framework: The regression subnetwork estimates the target load power value by applying two linear transformations to the output features of the upstream perceptron (backbone feature extraction network) and the Tanh activation function; the classification subnetwork outputs the device operating state (ON / OFF) probability by applying a linear transformation to the output features of the upstream perceptron (backbone feature extraction network) and the Sigmoid activation function; a gating fusion mechanism is introduced to multiply the regression result and the classification probability element-wise to achieve dual-task feature interaction and collaborative optimization, resulting in the final output.

[0045] The regression subnetwork estimates the target load power value by applying two linear transformations to the output features of the upstream perceptron (backbone feature extraction network) and the Tanh activation function, as follows: ; In the formula, Indicates the first Power estimates for each target load; Represents the hyperbolic tangent activation function; express The upstream features corresponding to the target load (i.e., the output of the backbone feature extraction network, and the features corresponding to the target load) are as follows: (High-dimensional abstract features of load matching); , denoted as the trainable weights of the first and second linear transformations of the regression subnetwork, respectively.

[0046] The classification subnetwork outputs the device operating state (ON / OFF) probability by linearly transforming the output features of the upstream perceptron (backbone feature extraction network) and applying the Sigmoid activation function, as follows: ; In the formula, Indicates the first The probability value of the operating status of each electrical device; express Activation function; This represents the trainable weights of the classification subnetwork.

[0047] This involves introducing a gating fusion mechanism, multiplying the regression result element-wise with the classification probability to achieve dual-task feature interaction and collaborative optimization, yielding the final output; the final output formula is expressed as: ; In the formula, Indicates the first The final power output result of the target load; This indicates an element-wise multiplication operation.

[0048] Step S2.3: Based on the source domain labeled data (a dataset that corresponds one-to-one with the real labels of each independent electrical device (including the actual power consumption value and operating status ON / OFF of the device) of the multimodal input data (total load power sequence, voltage / current trajectory, etc.) at the main power supply in the source domain scenario), complete the pre-training of the backbone feature extraction network. Using the mean squared error as the loss function, the optimal model in the hypothesis space is determined by minimizing the error between the model output of the total load power sequence and the actual power of each load, thus obtaining the source domain model.

[0049] Among them, the pre-training of the backbone feature extraction network is completed based on the source domain labeled data, and the learning objective of NILM is defined as: ; In the formula, The optimal backbone feature extraction network model obtained after pre-training is the model that minimizes the loss function in the model hypothesis space, i.e., the target model of the source domain pre-training. This indicates the number of time steps in the training data; This represents the mean squared error loss function, used to measure the error between the model output and the true value; This refers to the backbone feature extraction network, i.e., the model. exist Time-to-time multimodal input The output result; express Multimodal input at any time; express Time of the first The actual power label of each electrical device; This represents the set of all possible backbone feature extraction network models; This leads to the source domain model, whose training formula is: ; In the formula, This represents the source domain model obtained after source domain pre-training; This indicates the number of time steps corresponding to the source domain training data; This represents the loss function corresponding to the source domain training, used to measure the error between the source domain model output and the source domain true label; Indicating source domain scenario Multimodal input at any time; Indicating source domain scenario Time of the first The actual power label of each electrical device.

[0050] Step S3, Transfer Learning and Cross-Domain Adaptation: Based on the source domain model, source tasks are selected by combining the average number of active channels and feature importance, and a joint loss function containing task loss and MMD+CORAL domain adaptation loss is constructed; the strategy of freezing the backbone network and only fine-tuning the gating network is adopted to adapt the source domain model to the target domain.

[0051] The specific process of step S3 is as follows: Step S3.1: The source task is quantitatively selected by combining the average active channel count (AAC) with the feature importance analysis of random forest. The average active channel count is obtained by comparing the activation values ​​of each load at different times and different channels with a preset threshold (if the condition is met, it is counted), and the mean value over the time dimension is taken.

[0052] A combination of Average Active Channels (AAC) and Random Forest feature importance analysis was used to quantitatively select source tasks; the AAC calculation formula is as follows: ; In the formula, Indicates the first Average number of active channels per source task; This represents the total number of time steps involved in calculating AAC, i.e., the number of consecutive sampling moments of the data; The total number of channels representing the feature; Indicates the first Individual source task, first The time step, the first The feature activation values ​​corresponding to each channel; This indicates the preset activation threshold; This indicates an indicator function.

[0053] Step S3.2: Construct a joint loss function to reduce the distribution difference between the source domain and the target domain. The total loss function is composed of a weighted sum of task loss and domain adaptive loss: the task loss is the sum of the mean square error of each load power estimate and the binary cross-entropy of equipment status identification, and the importance of the two is adjusted by weight coefficients; the domain adaptive loss is the weighted sum of MMD loss and CORAL loss, where the CORAL loss is obtained by calculating the square of the F norm of the feature covariance matrix of the source domain and the target domain and dividing it by 4 times the square of the feature dimension, which is used to reduce the difference in feature distribution between the two domains.

[0054] The total loss function consists of the task loss. Domain Adaptive Loss Weighted composition, expressed as: ; In the formula, Indicates the first weighting coefficient; Mission loss Represented as: ; In the formula, Indicates mean square error; Indicates the first The actual power label of the target load; This represents the second weighting coefficient; Represents the binary cross-entropy; Indicates the first The actual status label of each device; This indicates the total number of independent electrical devices (target load) to be monitored; Domain Adaptive Loss The weighted sum of MMD loss and CORAL loss is expressed as: ; In the formula, Indicates the third weighting coefficient; This represents the loss due to the maximum mean difference. This indicates that the source domain data is in the model's... Feature distribution of layer output; This indicates that the target domain data is in the model's... Feature distribution of layer output; Indicates the relevant alignment loss; CORAL loss is expressed as: ; In the formula, The dimension representing the model features (i.e. the length of the feature vector) is a parameter used to normalize the loss value; This represents the covariance matrix corresponding to the features in the source domain; This represents the covariance matrix corresponding to the features of the target domain; This represents the Frobenius norm.

[0055] Step S3.3: Adopt the hierarchical migration strategy of "freezing the shared layer and fine-tuning specific layers": freeze the parameters of the TransDisNILM backbone network and fine-tune only the parameters of its sub-task gating network (SGN); based on the target domain data, complete the target domain model adaptation by minimizing the error between the model output and the actual power of each load in the target domain.

[0056] The fine-tuning formula is expressed as follows: ; In the formula, This represents the fitted model obtained after fine-tuning the target domain. This indicates the number of time steps corresponding to the target domain fine-tuning data; This represents the loss function corresponding to fine-tuning in the target domain; Indicating the target domain scenario Multimodal input data at any given time; Indicating the target domain scenario Time of the first The actual power label of each electrical device.

[0057] Step S4, Monitoring Result Output: Finally, using the adapted target domain model, the power values ​​and operating status probabilities of each load are output synchronously, providing core data support for smart grid supply and demand interaction and home energy management.

[0058] The specific process of step S4 is as follows: using the target domain model obtained in step S3, the accurate power estimate of each load and the probability of equipment operating status (ON / OFF) are output synchronously, providing accurate monitoring data support for the supply and demand interaction of smart grids and the refined management of household energy.

[0059] A non-invasive load monitoring system based on multimodal features and transfer learning, including: The multimodal data preprocessing module is used to extract the total load power sequence through a sliding window and generate a denoised recursive graph. It decomposes the voltage and current trajectories based on Fryze theory to obtain a color-coded VI image and integrates the features of the three to form a multimodal input. The hybrid model construction and source domain pre-training module is used to build the TransDisNILM backbone network based on multimodal input. After feature extraction through convolutional enhancement and positional encoding, it is modeled through a 3-layer Transformer encoder. The integrated gated network fuses the dual tasks of power regression and state classification. It is trained using source domain labeled data and the error is minimized to obtain the source domain model. The transfer learning and cross-domain adaptation module is used to select source tasks based on the source domain model, combined with the average number of active channels and feature importance, and to construct a joint loss function containing task loss and MMD+CORAL domain adaptation loss. It adopts a strategy of freezing the backbone network and only fine-tuning the gating network to adapt the source domain model to the target domain. The monitoring result output module is used to synchronously output the power values ​​and operating status probabilities of each load using the adapted target domain model, providing core data support for smart grid supply and demand interaction and home energy management.

[0060] An electronic device includes a processor, a memory, and a bus, wherein the processor and the memory are connected via the bus, the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute a non-intrusive load monitoring method based on multimodal features and transfer learning.

[0061] A non-volatile computer storage medium storing computer-executable instructions that perform a non-intrusive load monitoring method based on multimodal features and transfer learning.

[0062] Experimental Design and Results Analysis To verify the effectiveness of this invention, an experimental scheme was designed. The experiment mainly focuses on two key issues: first, evaluating the generalization ability of the method in cross-domain scenarios; second, verifying the transfer efficiency under the condition of scarce labeled data in the target domain. The experiment uses the REFIT dataset as the source domain and the UK-DALE dataset as the target domain, selecting four typical household appliances—refrigerators, washing machines, microwave ovens, and dishwashers—as monitoring objects. Seq2Point-CNN, BERT4NILM, and TransDisNILM are used as three representative baseline methods for comparative analysis.

[0063] Cross-dataset generalization performance test Cross-dataset generalization ability is an important metric for evaluating the practicality of non-intrusive load monitoring methods. This experiment simulates one of the most challenging scenarios in real-world deployments: the model is trained in the source domain (REFIT) and then directly applied to the target domain (UK-DALE) without any fine-tuning. This setup effectively tests whether the feature representations learned by the model possess sufficient domain invariance.

[0064] Table 1: Performance Comparison of Load Decomposition Across Datasets (MAE, Unit: Watts)

[0065] From the perspective of the accuracy of load decomposition, the experimental results shown in Table 1 exhibit a clear hierarchical distribution. The classic Seq2Point-CNN method shows the most limited performance, with an average MAE of 26.4W across the four types of home appliances. This reflects the inherent limitations of traditional deep learning methods in dealing with domain shifts. The Transformer-based BERT4NILM method shows some improvement, with the average MAE decreasing to 22.0W, indicating that the attention mechanism has certain advantages in capturing cross-domain invariant features. The recently proposed TransDisNILM method further improves the performance to 18.9W, demonstrating the potential of a specially optimized Transformer architecture for NILM tasks.

[0066] The method proposed in this invention achieved optimal performance across all tested appliance categories, with an average MAE of only 12.9W, a 31.7% reduction compared to TransDisNILM. This significant improvement can be attributed to the synergistic effect of multimodal feature fusion. Specifically, the recursive graph effectively preserves the dynamic patterns of load operation by visualizing the recursive characteristics of time-series states; while the color-coded VI trajectory provides complementary information from a physical characteristic perspective. The hue-coded trajectory direction and saturation reflect the power factor, while brightness characterizes operational stability. This multi-dimensional feature representation enables the model to understand load characteristics from different perspectives, thereby learning more discriminative feature representations.

[0067] Table 2: Performance Comparison of State Recognition Across Datasets (F1 Score)

[0068] In the device state recognition task, the performance of each method shows a similar trend to that in the load decomposition task, as shown in Table 2. The average F1 score of the proposed method reaches 0.88, significantly outperforming the baseline method. Particularly noteworthy is the more pronounced performance advantage of the proposed method on dishwashers and washing machines with complex state transitions, achieving F1 scores of 0.85 and 0.82 respectively, representing improvements of 11.8% and 12.3% compared to TransDisNILM. This advantage largely benefits from the dual-task collaborative learning mechanism designed in the subtask gating network. This mechanism achieves deep interaction between power estimation and state recognition through gating fusion, where the state probability serves as the power estimate for soft-switching dynamic modulation, enabling the two tasks to mutually promote each other during the shared feature learning process, enhancing the model's sensitivity to state changes.

[0069] Validation of migration efficiency under low data volume In real-world applications, acquiring large amounts of labeled target domain data is often costly or even impractical. Therefore, the ability of a method to quickly adapt to limited labeled data is crucial. This experiment simulates this real-world constraint. All methods are first pre-trained on the REFIT dataset, then fine-tuned using only 1% of the labeled data from the UK-DALE dataset, and finally evaluated on the full test set.

[0070] Table 3: Comparison of load decomposition performance after fine-tuning with low data volume (MAE, unit: watts)

[0071] Table 3 shows the load decomposition results under low data conditions. It can be observed that all methods show improved performance compared to direct testing across datasets, which is expected—even a small amount of labeled target domain data can provide valuable domain-specific information. However, the ability of different methods to utilize limited data varies significantly. Although Seq2Point-CNN and BERT4NILM show improvements, their average MAE remains at 20.2W and 16.8W respectively, indicating that these methods struggle to effectively adapt to the target domain distribution under data-scarce conditions.

[0072] The proposed method performs exceptionally well with low data volumes, further reducing the average MAE to 9.7W, a 32.6% improvement over TransDisNILM. This significant performance advantage stems from the synergistic effect of multiple design elements within the systematic transfer learning framework. First, the source task selection mechanism, based on the average number of active channels and the importance of random forest features, ensures the selection of the most representative and transferable source tasks through quantitative analysis, laying a solid foundation for subsequent transfer learning. Second, the hierarchical strategy of "freezing shared layers - fine-tuning specific layers" effectively prevents overfitting on limited target domain data while preserving general knowledge of the source domain. Most importantly, the joint domain adaptive loss function guides the model to learn domain-invariant features during the pre-training phase. The MMD loss minimizes distribution differences from the perspective of the regenerating kernel Hilbert space, while the CORAL loss further refines distribution matching by aligning second-order statistics. This dual constraint creates favorable conditions for rapid subsequent adaptation.

[0073] It is worth noting that the performance improvement of the method proposed in this invention varies across different types of home appliances. For loads with relatively simple state transitions, such as microwave ovens and electric kettles, the performance advantage mainly comes from the rich physical information provided by multimodal features; while for complex multi-state loads, such as dishwashers and washing machines, the role of the systematic transfer learning strategy is more critical. This indicates that the method proposed in this invention can adaptively leverage the advantages of each innovative component for loads with different characteristics.

[0074] Based on the above experimental results and analysis, the following conclusions can be drawn: This invention not only significantly outperforms existing mainstream methods in cross-dataset generalization ability, but also demonstrates higher transfer efficiency and adaptability under the real-world constraint of scarce labeled data in the target domain. These advantages stem from the organic combination of multimodal feature fusion, dual-task collaborative learning, and a systematic transfer strategy in the method, providing an effective solution for the practical application of non-intrusive load monitoring technology.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A non-invasive load monitoring method using multi-modal features and transfer learning, characterized in that, The method comprises the following steps: Step S1, multi-modal data preprocessing: the total load power sequence is intercepted by a sliding window and a denoising recurrence graph is generated, the voltage and current trajectories are decomposed based on Fryze theory to obtain a color-coded V-I image, and the three features are integrated to form a multi-modal input; Step S2, mixed model construction and source domain pre-training: based on the multi-modal input, a TransDisNILM backbone network is built, features are extracted through convolution enhancement and position coding, and then a 3-layer Transformer encoder is used for modeling; a gating network is integrated to fuse the power regression and state classification double tasks, and the source domain model is trained and error is minimized by using the source domain labeled data; Step S3, transfer learning and cross-domain adaptation: relying on the source domain model, the source task is screened in combination with the average active channel number and feature importance, a joint loss function containing the task loss and the MMD+CORAL domain adaptation loss is constructed; the strategy of freezing the backbone network and only fine-tuning the gating network is adopted to realize the adaptation of the source domain model to the target domain; Step S4, monitoring result output: finally, the target domain model after adaptation is used to synchronously output the load power values and the running state probability, thereby providing core data support for the supply and demand interaction of smart grids and home energy management.

2. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 1, wherein: The specific process of step S1 is as follows: Step S1.1: the total load power sequence is intercepted by a sliding window, the window length and the sliding step are set, and a plurality of power subsequences are obtained; based on each power subsequence, a two-dimensional recurrence graph is generated by a recurrence graph method: based on the Euclidean distance between elements in the power subsequence, a distance threshold is set, the Heaviside step function is used to determine whether the distance between elements is less than the threshold, and then a recurrence matrix is constructed to form a recurrence graph; the generated binary recurrence graph is subjected to median filtering to remove isolated noise points, and finally the recurrence graph feature is obtained; Step S1.2: the voltage and current trajectories are preprocessed based on the Fryze power theory; The preprocessed voltage-current trajectory is normalized and mapped, and a color-coded V-I trajectory image is generated in the HSV color space; Step S1.3: the original total load power sequence, the recurrence graph feature and the color-coded V-I trajectory image are integrated to form a multi-modal input.

3. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 2, wherein: The specific process of step S2 is as follows: Step S2.1: a TransDisNILM hybrid deep perceptron is constructed as a backbone feature extraction network, and the multi-modal input of step 1.3 is connected to the encoding layer of the backbone feature extraction network; a deep convolution enhancement module is introduced to the encoding layer to enhance feature extraction through one-dimensional convolution, ReLU activation function and maximum pooling operation; at the same time, sinusoidal position coding is embedded to reserve time sequence information through the association calculation of position and feature dimension; the encoder layer of the network is provided with 3-layer Transformer encoder, each layer containing multi-head self-attention mechanism and feedforward network: the self-attention mechanism realizes feature association modeling through the operation and Softmax normalization of query, key and value matrices; the feedforward network adopts GELU activation function for nonlinear transformation, and each layer is supplemented with residual connection and layer normalization to ensure training stability; Step S2.2: integrate a sub-task gating network at the output layer of the backbone feature extraction network to build a dual-task collaborative framework; Step S2.3: based on the source domain labeled data, complete the pre-training of the backbone feature extraction network with the real labels of each independent electrical equipment, take the mean square error as the loss function, minimize the error between the model output of the total load power sequence and the real power of each load, determine the optimal model in the hypothesis space, and obtain the source domain model.

4. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 3, wherein: Step S2.2: integrate a sub-task gating network SGN at the output layer of the backbone feature extraction network to build a dual-task collaborative framework. The specific process is as follows: the regression sub-network realizes the estimation of the target load power value by performing twice linear transformation and Tanh activation function on the output features of the upstream perceiver, i.e., the backbone feature extraction network; the classification sub-network outputs the equipment operating state probability by performing linear transformation and Sigmoid activation function on the output features of the upstream perceiver; the gating fusion mechanism is introduced to multiply the regression result and the classification probability element by element, realize the interaction and collaborative optimization of the dual tasks, and obtain the final output.

5. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 4, wherein: The specific process of step S3 is as follows: Step S3.1: adopt the combination of average active channel number AAC and random forest feature importance analysis to complete the source task quantitative selection; The average active channel number is obtained by comparing the activation values of each load at different time and different channels with a preset threshold and taking the time dimension mean value; Step S3.2: construct a joint loss function to reduce the distribution difference between the source domain and the target domain, and the total loss function is composed of task loss and domain adaptive loss; Step S3.3: adopt the hierarchical transfer strategy of "freezing shared layer and fine-tuning specific layer": freeze the parameters of the backbone network of TransDisNILM, and only fine-tune the parameters of the sub-task gating network thereof; based on the target domain data, complete the adaptation of the target domain model by minimizing the error between the model output and the real power of each load in the target domain.

6. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 5, wherein: The task loss is the sum of the mean square error of the load power estimation and the binary cross entropy of the equipment state recognition, and the importance of the two is adjusted by the weight coefficient; the domain adaptive loss is the weighted sum of the MMD loss and the CORAL loss.

7. The multi-modal feature and transfer learning based non-invasive load monitoring method of claim 6, wherein: The specific process of step S4 is as follows: use the target domain model obtained in step S3 to synchronously output the power estimation value and the equipment operating state probability of each load, and provide accurate monitoring data support for the supply and demand interaction of smart grids and the fine management of household energy.

8. A non-invasive load monitoring system using multi-modal features and transfer learning, characterized by, It includes: A multi-modal data preprocessing module for intercepting the total load power sequence by a sliding window and generating a denoising recurrent graph, decomposing the voltage and current trajectories based on Fryze theory to obtain a color-coded V-I image, and integrating the features of the three to form a multi-modal input; A hybrid model construction and source domain pre-training module for building a TransDisNILM backbone network based on the multi-modal input, extracting features through convolution enhancement and position coding, and modeling through a 3-layer Transformer encoder; integrating a gating network to fuse the power regression and state classification dual tasks, training and minimizing the error using source domain labeled data to obtain a source domain model; The migration learning and cross-domain adaptation module is used for relying on a source domain model, combining the average active channel number and the feature importance to screen a source task, and constructing a joint loss function containing a task loss and an MMD+CORAL domain adaptation loss; a strategy of freezing a main network and only fine-tuning a gating network is adopted to realize adaptation of the source domain model to a target domain; The monitoring result output module is used for synchronously outputting each load power value and an operation state probability by using the adapted target domain model, and providing core data support for smart grid supply and demand interaction and home energy management.

9. An electronic device, comprising: The computer executable instructions execute the non-intrusive load monitoring method with multi-modal features and migration learning according to any one of claims 1-7.

10. A non-transitory computer storage medium storing computer-executable instructions, the computer-executable instructions comprising instructions for: The computer executable instructions execute the non-intrusive load monitoring method with multi-modal features and migration learning according to any one of claims 1-7.

Images