A non-intrusive event detection method based on image features and retrieval

Abstract

The present application belongs to the technical field of electrical fingerprint identification, and specifically relates to a non-intrusive event detection method based on image features and retrieval. The present application determines the event or state of a certain electrical equipment within a period of time through a multi-layer full convolutional network, adds an attention score mechanism after network feature extraction, decomposes the image into several key points, and simultaneously gives the local features of the key points an attention score; through the network-extracted features and in combination with the attention score, the weight of the event occurring in the voltage and current change section of the circuit diagram is readjusted, and finally the matching event features in the feature library are retrieved to identify the event category. The present application improves the accuracy of switching event identification, and simultaneously has a light model order, a simple method, flexible use, simple and efficient training, high detection accuracy, and a wide range of application scenarios.

Description

Technical Field

[0001] This invention belongs to the field of electrical fingerprint recognition technology, specifically relating to a non-invasive event detection method based on image features and retrieval. Background Technology

[0002] Just as each person has a unique fingerprint, different electrical appliances, due to differences in their internal electrical components such as capacitance, inductance, and resistance, produce different electrical characteristics such as current and voltage when connected to a circuit. These unique electrical characteristics are what we call each appliance's "electrical fingerprint." By using non-intrusive monitoring methods to monitor the power consumption in circuits in real time, identify and record the operating status of electrical appliances, and even identify dangerous electrical use (illegal charging of electric bicycles, unauthorized use of electrical appliances in dormitories, etc.), this is of great significance for the rational allocation of electrical energy and electrical safety.

[0003] The key to electrical fingerprint recognition lies in the accuracy of identifying electrical appliance state transitions, that is, the accuracy of event detection during the electrical fingerprint recognition process. Currently, there are few satisfactory methods for event detection. Summary of the Invention

[0004] To address the need for high accuracy in electrical fingerprint recognition, this invention provides a non-intrusive event detection method based on image features and retrieval for event detection during the electrical fingerprint recognition process, thereby improving the accuracy of electrical fingerprint recognition.

[0005] The non-intrusive event detection method based on image features and retrieval provided by this invention calculates the active power of voltage and current data of a single electrical appliance over a certain period of time, and plots it as a line graph using the matlibplot library. The plotted line graph is then fed into a trained Local Feature Retrieval Network (DeLF) with attention scores for electrical event recognition, yielding the recognition result. Based on the recognition result, the occurrence time and current status of the current electrical event are obtained. The specific steps are as follows:

[0006] Step 1: Collect data for training the DeLF network model. This data consists of electrical data (voltage, current) of multiple known electrical devices (labels) in normal operation, gear switching, and other states. The data is grouped according to a pre-set sampling time and frequency (e.g., a single device at a sampling frequency of 1kHz within 60 seconds). m groups are collected for each event state (n states), resulting in a time series consisting of m×60000 pairs of instantaneous current and voltage values ​​under n states. The active power is then calculated.

[0007] Step 2: Process the active power collected in Step 1 and plot it as a line graph using the matlibplot library, with the horizontal axis representing time and the vertical axis representing active power values, generating two-dimensional image data; then, label the active power values ​​on the image during this time period based on the known device type (or a single special device) and the known state of the device at that time, according to Step 1; the labeled image dataset is used as the training set for the DeLF model (the test set can be allocated proportionally according to the training set);

[0008] Step 3: Construct the DeLF model and train it using the training set formed in the above steps; the DeLF model is constructed based on the fully convolutional network FCN and CNN.

[0009] Step 4: Real-time acquisition of electrical data in the circuit to form image data, inputting it into the trained DeLF network model, DeLF performs feature extraction to form a feature map, then performs feature retrieval, and finally obtains the switching event that occurred in a certain device during that period.

[0010] Step 5: Based on the prediction results of the DeLF model, determine which feature category the current feature belongs to, thereby determining the switching of appliance states in the current data segment and realizing the judgment of appliance switching events.

[0011] This invention includes the construction of a training set, the construction and initialization of a DeLF network model, the training of the model, and the acquisition and classification of real-time data.

[0012] The following is a further detailed explanation of each step:

[0013] Step 1 involves collecting electrical data during specific electrical state transitions used to train the DeLF model; specifically:

[0014] This step requires collecting electrical data of the specific target appliance to be detected during state switching. The collected electrical data uses five known appliances with high power consumption, numbered as follows: dishwasher (DW), refrigerator (FR), kettle (KE), microwave oven (MW), and washing machine (WM). Subsequently, electrical data of the specific target appliance used to train the DeLF model during normal operation is collected. When collecting electrical data, any collectable appliance should be selected, and the data acquisition should start from time 0, with a collection frequency of 1kHz and a total collection time of 60s. The collected data will then be 60,000 pairs of instantaneous voltage and current arranged in time series. Values ​​within a specified threshold range are selected from the collected data as a window, such as a 2s window containing 2000 sets of data. The window is then slid (to facilitate the subsequent data to form an image for the model).

[0015] Acquire target device state switching data for training the DeLF model; the opening and closing of the target device are state changes that need to be detected, i.e., events that need to be detected. For example, data collection starts at 0ms, the target device is "opened" at the first time period (e.g., 5s), the target device is "closed" at the second time period (e.g., 55s), and data collection stops at 60s; take 2000 window vectors within a certain time period (e.g., adjust the state at 15s, take 14s to 16s) as the training data for the target device gear 1.

[0016] Taking DW as the target device as an example, both the opening and closing of DW are state changes that need to be detected. Data acquisition starts at 0ms, DW is "opened" at 5s, DW is closed at 55s, and acquisition stops at 60s. At 15s, the state is adjusted and 2000 sets of window vectors from 14s to 16s are taken as training data for DW gear 1.

[0017] Step 2 involves processing the electrical data collected in Step 1. First, the data is calculated into an active power sequence arranged in time series. Then, the sequence is plotted as a line graph using the matlibplot library, with the time axis as the horizontal axis and the active power value as the vertical axis, generating two-dimensional image data. Based on the records in Step 1, the opening and closing of the DW device and the switching events occurring between 14s and 16s are labeled on the two-dimensional image data generated in Step 2. The remaining devices are operated in the same manner as in Step 2, ultimately forming an image dataset as the training set for the DeLF model.

[0018] In the basic training data of the DeLF network model, each window vector contains 2000 sets of one-to-one instantaneous values ​​of current and voltage, which are processed as follows:

[0019] (1) The active power is obtained by calculation based on the corresponding current and voltage;

[0020] (2) Plot the 2000 consecutive active power values ​​obtained in the previous step to obtain a (X) graph. t1 X t2 , ..., X t2000 The time series (X is the active power value at this instant, and t is a certain time window) is plotted as a line graph with time as the horizontal axis and active power as the vertical axis using the matlibplot library;

[0021] (3) Take the image data obtained above as the target image to be identified, and use it and the label of the original feature as the corresponding label to obtain a complete feature;

[0022] (4) After processing all the original features as described above, the training set of the DeLF model is obtained.

[0023] Step 3 involves constructing the DeLF model using the training set generated in the previous step. The specific process is as follows:

[0024] (1) Based on the sampling data of the device in actual application, determine the sampling window with different threshold sizes;

[0025] (2) Construct the DeLF network using a feedforward neural network and add attention fractional units;

[0026] (3) DeLF is mainly constructed based on the fully convolutional network FCN and CNN; firstly, the fully connected layer in ResNet

[50] is used to extract dense local features, with Conv4× as the output, and outputting features of size 14×14; after the output, the attention mechanism module is connected to obtain the relevant score of local feature expression; then an image pyramid with seven layers of FCN is constructed to handle the scale change of each layer of context image; firstly, the input image is cropped in the center to obtain a square image, then the square image is cropped in the center again and the size is adjusted to 250×250, and finally a random cropping of size 224×224 is used to extract 14×14 local features for training; then the features are summed to 1×40 dimensions.

[0027] (4) Since the number of extracted local features is too large, it affects the training time and testing performance. The importance of features in the image is also different. Therefore, an attention mechanism is used to weight the local features. First, weighted summation pooling is used to reduce the dimensionality of the features. The weight is the predicted value of the attention mechanism module. In subsequent practical applications, a feature map retrieval process is added. In this process, the PQ algorithm and asymmetric computation are used to speed up the comparison retrieval process.

[0028] The attention mechanism layer is treated as a normal CNN training process. It performs 1×1 convolutions on local features from N channels, followed by two more CNN convolution layers for training, to obtain the attention score. The attention score of the local features is represented as:

[0029] α n =α(f n ;θ),f n ∈R d n=1,…,N

[0030] Then, multiply the predicted feature value by the previously obtained local features, as shown below:

[0031]

[0032] Attention scores use a cross-entropy function as the loss function. The gradient between each layer is calculated by backpropagation using the loss between the true and predicted values. The weights in the network are updated through the loss function and backpropagation, and then the local feature function is obtained. When the loss decreases to a certain value and tends to stabilize, the network is basically trained. Then each local feature can also be used by the CNN network to obtain local features with attention scores.

[0033] (5) Establish feature indexes for the training data. In subsequent actual use, based on the electrical data characteristics of the input device, use the indexes already established in the database and the PQ algorithm and asymmetric computation to achieve nearest neighbor search, so as to quickly retrieve the device's status. The specific process is as follows: During the training process, each image... Figure 1 Thousands of local features are used to divide the 40-dimensional feature into 4×10 parts. K-means clustering is then performed on each part, with N centers selected. After clustering, there are N classes, and the cluster centers can be used to represent the cluster objects associated with them. A binary ID is then introduced to represent the index, and a compressed vector is constructed based on the number of index bits. A codebook is built using the index values ​​of the N clusters, and the codewords correspond to the specific feature values. After indexing, asymmetric computation is performed on the image to be searched. X and Y represent the local features of the query image and the image in the database, respectively, and q represents the codeword corresponding to the index value after performing the PQ operation on X and Y. Symmetric computation uses the distance between the codewords of the two sets of features to represent the distance between the features. Asymmetric computation does not calculate the codeword for X; instead, the distance from X to q(Y) is used to represent the distance from X to Y. In subsequent practical applications, the input image feature X can be directly used for distance calculation. Since the clustered dataset is much smaller than the original dataset, the number of calculations for q(y) will also be much less.

[0034] In step 4, the electrical data from the real-time acquisition circuit is converted into image data, which is then input into the trained DeLF network model. DeLF performs feature extraction to form a feature map, followed by feature retrieval, ultimately determining the switching events that occurred in a specific device during that time period; specifically:

[0035] (1) The current and voltage of the circuit under test are sampled at a frequency of 1kHz. When the circuit under test is turned on at 0 seconds, the previous 60,000 sets of data are taken out as a block and the collector continues to collect until the next 60,000 sets are collected and then taken out. The process is continuously cyclical until the detection stops.

[0036] (2) After each block is extracted, the data inside is the original current and voltage data. The active power is calculated and a two-dimensional image is generated. Then the image is sent to the trained DeLF model for recognition. During the recognition process, the corresponding image is matched through step 3 (5). Finally, the data for a certain period of time and the switching event label of a certain device are obtained according to the recognition result.

[0037] Step 5 describes determining which switching events occurred in a certain electrical device within a certain period of time based on the results obtained from the DeLF model, thus ultimately achieving the determination of electrical switching events. Specifically:

[0038] Based on the identification results obtained in the previous step, we can obtain the changes in the electrical appliance status in the current data segment, such as DW being on, DW being off, or DW being in standby mode without any other status changes, etc. If the switch of the electrical appliance status of DW being on is detected, then the DW device will be in the working state until the DW being off is detected. The same applies to other electrical devices.

[0039] In non-intrusive load monitoring scenarios, the method of this invention can identify the occurrence and switching of electrical devices under test in the monitored circuit within a certain time. This allows for the identification of the currently operating electrical appliances in the circuit, achieving the goal of "electrical fingerprint recognition" to ensure rational electricity use and electrical safety.

[0040] This invention uses a non-intrusive event detection method based on image features and retrieval methods to accurately input the "electrical fingerprint" information of a single device in different states to the fully connected layer, thereby amplifying the "attention" to certain hidden events (such as low-power gear shifting) and filtering out certain data convexities, thereby reducing the difficulty of distinguishing feature categories and improving the accuracy of the discrimination model in recognition.

[0041] This invention uses a multi-layer fully convolutional network to determine the events or states of an electrical device over a period of time. After feature extraction by the network, an attention score mechanism is added to decompose the image into several key points, and attention scores are assigned to the local features of the key points. Based on the features extracted by the network and combined with the attention scores, the weights of events occurring during voltage and current changes in the circuit diagram are readjusted. Finally, by retrieving matching event features from the feature library, the event category is identified. This invention improves the accuracy of switching event recognition. Due to its lightweight model, simple method, flexible use, easy and efficient training, and high detection accuracy, this invention achieves this goal.

[0042] Compared with existing technologies, this invention can identify changes in the working state of electrical equipment in a circuit with high accuracy, thereby achieving the purpose of monitoring the circuit and warning of dangerous electricity use. It has a wide range of applications and provides a real-time, efficient and highly accurate technical solution for the rational allocation of electrical energy and electrical safety. Attached Figure Description

[0043] Figure 1 This is a flowchart of the present invention.

[0044] Figure 2 This is a schematic diagram of the PQ algorithm and the retrieval process. Detailed Implementation

[0045] This invention includes the construction of a training set, the setup and initialization of a DeLF model, model training, and how to collect real-time data and identify switching events. This method requires the prior collection of a certain amount of electrical data to form the training set for the DeLF model. A two-dimensional image formed from the time series of active power is used as a unique feature to identify different states of a single device.

[0046] The technical solution of the present invention will be further described in detail below with specific implementation examples.

[0047] Example 1

[0048] See Figure 1 This invention uses current and voltage data obtained through non-intrusive load monitoring as input, processes them graphically, and then feeds them into a trained DeLF model for identification, thereby obtaining the electrical switching events of the device. The method can be divided into the following steps:

[0049] Step 1: Collect electrical data during specific electrical state transitions to train the DeLF model.

[0050] This step requires collecting electrical data on the specific target appliance to be detected during state transitions. The collected electrical data uses five known high-power appliances, numbered as follows: dishwasher (DW), refrigerator (FR), kettle (KE), microwave oven (MW), and washing machine (WM). The specific data collection includes:

[0051] When collecting electrical data, any collectable electrical appliance is selected, and the data acquisition begins at time 0. The acquisition frequency is 1kHz, and n states are collected, with 4 sets for each state. The total acquisition time is set to 60s, resulting in 4×60000n sets of instantaneous current and voltage values. When the device starts working, this invention continuously acquires the current and voltage data of the current circuit at a rate of 1000 sets per second. Values ​​within a specified threshold range are selected from the collected data as a window, such as 2000 sets of data within a 2s window, and the window is scrolled.

[0052] When a dishwasher is used as a target device, all state changes of the device from the moment it is turned on until the moment it is turned off are the objects to be identified and detected. Taking the dishwasher turning on as an example: Assuming that the dishwasher starts working at 2 seconds, the present invention starts from the 1st second (moment) to the 3rd second (moment), with a window size of 2000, and takes data blocks to obtain 2000 sets of event sequences containing current and voltage. These window vectors are the original feature data of the corresponding label "dishwasher turned on".

[0053] The target device in this invention uses five data-collecting electrical appliances, and their corresponding states are as follows:

[0054] (1) Dishwasher (DW): On + Off;

[0055] (2) Refrigerator (FR): Open + Close;

[0056] (3) Kettle (KE): Turn on (heat) + from heating to keep warm + from keep warm to heating + turn off;

[0057] (4) Microwave oven (MW): Turn on (level 1) + switch from level 1 to level 2 + turn off;

[0058] (5) Washing machine (WM): Turn on (speed 1) + Turn on (speed 2) + Turn off.

[0059] The final feature library contains (8+8+16+12+12)×6000=336000 features.

[0060] Step 2: Process the electrical data collected in Step 1 to form image data, which will serve as the training set for the DeLF model.

[0061] In the basic training data of the DeLF model, each window vector contains 2000 sets of one-to-one instantaneous values ​​of current and voltage, which are processed as follows:

[0062] (1) Calculate the active power from the corresponding current and voltage to obtain a (X) t1 X t2 , ..., X t2000The time series of X (where X is the active power value at this instant, and t is a certain time);

[0063] (2) Plot a line graph of the 2000 active power values ​​obtained in the previous step, arranged in a continuous time series.

[0064] (3) Take the two-dimensional image dataset obtained above as the target image to be identified, and use it and the label of the original feature as the corresponding label to obtain a complete feature;

[0065] (4) After processing all the original features as described above, the training set of the DeLF model is obtained.

[0066] Step 3: Build the DeLF model and train it using the training set formed in the previous step.

[0067] Construct a DeLF model and train it using the training set obtained in the previous step:

[0068] (1) Based on the sampling data of the device in actual application, determine the sampling window with different threshold sizes;

[0069] (2) Construct the DeLF network using a feedforward neural network and add attention fractional units;

[0070] (3) DeLF is mainly constructed based on the fully convolutional network FCN and CNN; firstly, the fully connected layer in ResNet

[50] is used to extract dense local features, with Conv4× as the output, and outputting features of size 14×14; after the output, the attention mechanism module is connected to obtain the relevant score of local feature expression; then an image pyramid with seven layers of FCN is constructed to handle the scale change of each layer of context image; firstly, the input image is cropped in the center to obtain a square image, then the square image is cropped in the center again and the size is adjusted to 250×250, and finally a random cropping of size 224×224 is used to extract 14×14 local features for training; then the features are summed to 1×40 dimensions.

[0071] (4) Since the number of extracted local features is too large, it affects the training time and testing performance. The importance of features in the image is also different. Therefore, an attention mechanism is used to weight the local features. First, weighted summation pooling is used to reduce the dimensionality of the features. The weight is the predicted value of the attention mechanism module. In subsequent practical applications, a feature map retrieval process is added. In this process, the PQ algorithm and asymmetric computation are used to speed up the comparison retrieval process.

[0072] The attention mechanism layer is treated as a normal CNN training process. It performs 1×1 convolutions on local features from N channels, followed by two more CNN convolution layers for training, to obtain the attention score. The attention score of the local features is represented as:

[0073] α n =α(f n ;θ),f n ∈R d n=1,…,N

[0074] Then, multiply the predicted feature value by the previously obtained local features, as shown below:

[0075]

[0076] Attention scores use a cross-entropy function as the loss function. The gradient between each layer is calculated by backpropagation using the loss between the true and predicted values. The weights in the network are updated through the loss function and backpropagation, and then the local feature function is obtained. When the loss decreases to a certain value and tends to stabilize, the network is basically trained. Then each local feature can also be used by the CNN network to obtain local features with attention scores.

[0077] (5) Establish feature indexes for the training data. In subsequent actual use, based on the electrical data characteristics of the input device, use the indexes already established in the database and the PQ algorithm and asymmetric computation to achieve nearest neighbor search, so as to quickly retrieve the device's status. The specific process is as follows: During the training process, each image... Figure 1 Thousands of local features are used to divide the 40-dimensional feature into 4×10 parts. K-means clustering is then performed on each part, with N centers selected. After clustering, there are N classes, and the cluster centers can be used to represent the cluster objects associated with them. A binary ID is then introduced to represent the index, and a compressed vector is constructed based on the number of index bits. A codebook is built using the index values ​​of the N clusters, and the codewords correspond to the specific feature values. After indexing, asymmetric computation is performed on the image to be searched. X and Y represent the local features of the query image and the image in the database, respectively, and q represents the codeword corresponding to the index value after performing the PQ operation on X and Y. Symmetric computation uses the distance between the codewords of the two sets of features to represent the distance between the features. Asymmetric computation does not calculate the codeword for X; instead, the distance from X to q(Y) is used to represent the distance from X to Y. In subsequent practical applications, the input image feature X can be directly used for distance calculation. Since the clustered dataset is much smaller than the original dataset, the number of calculations for q(y) will also be much less.

[0078] Step 4: Real-time acquisition of electrical data in the circuit to form image data, inputting it into the trained DeLF network model, DeLF performs feature extraction to form a feature map, and then performs feature retrieval to finally obtain the switching event that occurred in a certain device during that period.

[0079] (1) The current and voltage of the circuit under test are sampled at a frequency of 1kHz. When the circuit under test is turned on at 0 seconds, the previous 60,000 sets of data are taken out as a block after each data collection reaches 60,000 sets. The collector continues to collect data until the next 60,000 sets are collected and then taken out. The process is continuously cyclical until the detection stops.

[0080] (2) After each block is extracted, the data inside is the original current and voltage data. The active power is calculated and a two-dimensional image is generated. Then the image is sent to the trained DeLF model for recognition. During the recognition process, the corresponding image is matched through step 3 (5). Finally, the data for a certain period of time and the switching event label of a certain device are obtained according to the recognition result.

[0081] Step 5: Based on the results obtained from the DeLF model, determine which switching events occurred in a certain electrical device within a certain period of time, and finally realize the judgment of electrical switching events.

[0082] Based on the identification results obtained in the previous step, we obtain the changes in the electrical appliance status in the current data segment, such as whether the DW is on, off, or in standby mode without any other status changes. If the switch to the DW being on is detected, the DW device will remain in an on / off state until the off state is detected.

[0083] The current and voltage of the circuit under test are sampled at a frequency of 1kHz. Assuming that the circuit under test is turned on and starts working at 0 milliseconds, every time 60,000 sets of data are collected, the previous 60,000 sets are taken out as a block, and the data collector continues to collect until the next 60,000 sets are collected. Then the data is taken out and the process is continuously looped until all data is collected.

[0084] Once a data block is retrieved, it contains raw current and voltage data. Active power is calculated, and the results are arranged according to the device's operating time sequence to form a two-dimensional image. This image is then fed into a trained DeLF (Deep Filtering Algorithm) for recognition. The result represents the status of appliance switching events contained in the current data block, such as: refrigerator turned on, washing machine turned off, washing machine switched to the second speed setting, etc. During real-time detection, if "refrigerator turned on" is detected, it indicates that the refrigerator has been turned on and is operating. For a period of time afterward, until "refrigerator turned off" is detected, the refrigerator remains operational.

[0085] In this embodiment, multiple simulation experiments were conducted on all states of all devices through real-time detection. The state switching was carried out approximately 2400 times, with each state feature appearing an average of about 200 times. The experimental results are detailed in Table 1 below.

[0086] Table 1. Detection accuracy rate (%) for each device

[0087]

[0088]

[0089] The above experiments demonstrate that the average accuracy of the present invention in detecting device state switching features is 97.09%.

[0090] The above is merely one embodiment of the present invention and is not intended to limit the present invention. Any modifications and improvements made without departing from the concept of the present invention shall fall within the scope of protection of the claims of the present invention.

Claims

1. A non-intrusive event detection method based on image feature and retrieval, characterized in that, The active power is calculated including the voltage and current data of the extracted single electrical appliance in a period of time, and a broken line graph is drawn by the matlibplot library; the drawn broken line graph is input into the trained local feature retrieval network DeLF with attention score, electrical event identification is performed, the identification result is obtained, and the current electrical event occurrence time and current state are obtained according to the identification result; the specific steps are as follows: Step 1: collect data for training the DeLF network model, the data is the electrical data of a plurality of known electrical appliances in normal operation and switching gear state, including voltage and current; according to the pre-set sampling time and frequency, a group is formed, m groups are collected under each event state, a time series composed of m*60000 pairs of instantaneous current and voltage under n states is obtained, and then the active power is calculated; here the frequency is 1kHz sampling frequency of a single device in 60 seconds; Step 2: process the active power collected in step 1, draw a broken line graph by the matlibplot library, the horizontal axis is the time axis, and the vertical axis is the active power value, generate two-dimensional image data; then according to the known identification type in step 1 and the known state of the device at that time, the active power value on the image in this period of time is labeled; the image data set after labeling is used as the training set of the DeLF model; Step 3: build the DeLF model and train it using the training set formed in the above steps; the DeLF model is constructed according to the fully convolutional network FCN and CNN; Step 4: real-time collection of electrical data in the circuit to form image data, input into the trained DeLF network model, DeLF feature extraction, form feature map, then feature retrieval, finally get the switching event of the device in this period; Step 5: according to the prediction result of the DeLF model, determine which feature category the current feature belongs to, so as to determine the switching of the electrical state of the current data segment, realize the judgment of the electrical switching event.

2. The image feature and retrieval based non-intrusive event detection method of claim 1, wherein, The electrical appliances for training the DeLF network model in step 1 are in normal operation and switching gear state, wherein the electrical appliances are 5 known electrical appliances with high power consumption, namely: dishwasher (DW), refrigerator (FR), kettle (KE), microwave oven (MW) and washing machine (WM); then collect the electrical data of the specific target electrical appliance in normal operation; when collecting electrical data, any one of the collectable electrical appliances can be selected, the collection starts from 0 time, the collection frequency is 1kHz, and the total collection time is set to 60s, so the collected data is 60000 pairs of instantaneous voltage and current arranged in time sequence; select the values in the specified threshold range as a window, and slide the window. Obtaining the target device state switching data for training the DeLF model; the opening and closing of the target device are both state changes to be detected, i.e. events to be detected; specifically, data acquisition starts at 0 ms, the target device is "opened" at the first time period 5 s, the target device is closed at the second time period 55 s, and the acquisition is stopped at 60 s; wherein 2000 groups of window vectors in a certain time period are taken as the training data of the target device gear 1.

3. The image feature and retrieval based non-intrusive event detection method of claim 2, wherein, The electrical data collected in step 1 is processed in step 2, which is first calculated as an active power sequence arranged in time series, and then the sequence is plotted into a line chart by using the matlibplot library, with the time axis as the horizontal axis and the active power value as the vertical axis, to generate two-dimensional image data; and according to the records in step 1, the opening and closing of the electrical equipment and the switching event occurring at 14 s to 16 s are labeled on the generated two-dimensional image data; finally, the image data set is formed as the training set of the DeLF model; In the basic training data of the DeLF network model, each window vector contains 2000 groups of corresponding instantaneous values of current and voltage, which are processed as follows: (1) Calculate the active power from the corresponding current and voltage; (2) image plot of 2000 continuous active power obtained in the previous step, to obtain a time sequence of (X t1 , X t2 , …, X t2000 ) is drawn into a line chart with time as the horizontal coordinate and active power as the vertical coordinate through the matlibplot library; (3) The above obtained image data is taken as the target image to be recognized, and the label of the original feature is taken as the corresponding label to obtain a complete feature; (4) After processing all original features, the training set of the DeLF model is obtained.

4. The image feature and retrieval based non-intrusive event detection method of claim 3, wherein, In step 3, the DeLF model is constructed and trained with the training set, and the specific process is as follows: (1) According to the sampling data of the actual application electrical equipment, determine the sampling window size of different threshold values; (2) Construct the DeLF network with a feedforward neural network and add an attention score unit; (3) Construct the DeLF model according to the fully convolutional network FCN and CNN; first use the fully connected layer in ResNet[50] to extract dense local features, with Conv4x as the output, outputting a feature of size 14x14; after output, connect the attention mechanism module to obtain the related score of local feature expression; then construct an image pyramid with seven layers of FCN to process the context image scale change at each layer; first, center crop the input image to obtain a square image, then continue to center crop the square image to adjust the size to 250x250, and finally use random cropping of size 224x224 to extract 14x14 local features for training; then sum to 1x40-dimensional features; (4) Use the attention mechanism to weight the local features; first use weighted sum pooling to reduce the dimension of the features, and the weight is the predicted value of the attention mechanism module; and add a feature map retrieval process in the subsequent actual application, in which the PQ algorithm and asymmetric calculation are used to speed up the comparison retrieval process.

5. The image feature and retrieval based non-intrusive event detection method of claim 4, wherein, The predicted value of the attention mechanism module in step 3 (4) is calculated as follows: The attention mechanism layer is regarded as a common CNN training process, and the local features of N channels are convolved by 1*1, and then two layers of CNN convolution training are performed to obtain the attention score. The attention score of the local feature is represented as: α n = α(f n ; θ), f n ∈ R d ,n = 1,..., N; And the previous local feature is multiplied to obtain the feature prediction value, which is represented as: The attention score uses a cross-entropy function as a loss function, and the loss between the true value and the prediction value is used to calculate the gradient between each layer through back propagation. The weights in the network are updated through the loss function and back propagation, and then the local feature function is obtained. When the loss decreases to a certain value and tends to be stable, the network is basically trained. Then each local feature is obtained through the CNN network to obtain the local feature with the attention score.

6. The image feature and retrieval based non-intrusive event detection method of claim 5, wherein, During training, each image has one thousand local features, and the 40-dimensional features are divided into 4*10 parts. Then K-means clustering is performed on each part, N center points are selected, and after clustering, there are N classes. The cluster center is used to replace the associated cluster objects in this class. Then a binary Id is introduced to represent the index, and a compression vector is constructed according to the index bit number. A codebook is established by the N index values of the clustering, and the code word is the specific feature value. After establishing the index, the asymmetric calculation is performed on the image to be searched. X and Y represent the local features of the query image and the image in the database, respectively. q represents the index value corresponding to the code word after PQ operation on XY. Symmetric calculation is to replace the distance between features with the distance between two groups of feature code words. Asymmetric calculation does not calculate the code word of X, but replaces the distance between X and Y with the distance between X and q(y).

7. The image feature and retrieval based non-intrusive event detection method of claim 6, wherein, In step 4, the electrical data in the real-time acquisition circuit forms image data, which is input into the trained DeLF network model. DeLF performs feature extraction to form a feature map, and then performs feature retrieval to finally obtain the switching event of the device in the time period. The specific operation is as follows: (1) The current and voltage of the circuit to be detected are sampled at a frequency of 1 kHz. When the circuit to be detected is turned on at 0 seconds, 60000 groups of data are taken out as a block every time the data reaches 60000 groups. The collector continues to collect until the next 60000 groups are taken out. The process is continuously processed until the detection is stopped. (2) The data in each block is the original current and voltage data. The active power is calculated, and a two-dimensional image is generated. Then the image is input into the trained DeLF model for recognition. In the recognition process, step 3 is used to match the corresponding image. Finally, the switching event label of the device corresponding to the data in a certain period of time is obtained according to the recognition result.

8. The image feature and retrieval based non-intrusive event detection method of claim 7, wherein, The prediction result of the DeLF model in step 5 is used to determine to which feature category the current feature belongs, so as to determine the switching of the existing electrical appliance state of the current data segment, and realize the judgment of the electrical appliance switching event. Specifically: For a certain electrical device, the electrical device is opened, or the electrical device is closed, or the electrical device is always in a standby state without other state changes; if the switching of the electrical device opening is monitored, the electrical device is in an open working state until the electrical device closing state is detected.

Images