CART decision tree-based distribution network terminal user state perception method and system

Abstract

The application is based on the distribution network terminal user state perception system of CART decision tree and intelligent terminal multi-element heterogeneous big data mining, fully mines the intelligent terminal big data information, and realizes more comprehensive perception on the distribution network state. First, the multi-element heterogeneous data is structured and expressed, and the data defect problem caused by the great difference of user side terminal equipment is solved. Then, the distribution network terminal user state perception system of CART decision tree is designed, the terminal user state is divided into "normal & someone", "power failure & someone", "power failure & no one", and "other" categories, and then the algorithm for pruning processing of the CART decision tree is designed, so that the speed and accuracy are further improved.

Description

Technical Field

[0001] This invention relates to the fields of distribution network big data mining technology and artificial intelligence technology, specifically a distribution network end-user status perception method and system based on CART decision tree. Background Technology

[0002] With the popularization of smart homes and the rapid development of the power industry, a wealth of diverse and heterogeneous power data resources have accumulated on the home user side, including data from smart routers, smart air conditioners, and smart refrigerators. Among smart home terminal devices, those with high penetration rates and network communication capabilities possess rich user information in their terminal big data. Fully leveraging the potential value of this massive and multi-source user-side data (home smart monitoring, home broadband, smart home systems, etc.) is crucial for achieving scientific management and control of power distribution networks and improving the development of the receiving-end energy internet.

[0003] Taking power outage sensing research at energy internet terminals as an example, early power distribution networks had low levels of intelligence, and power outage sensing and repair relied mainly on manual dispatching and decision-making. Recently, the main technical path for smart grid status sensing is the installation of end-point power sensing modules, connecting sensing lines to the outgoing line of the circuit breaker switch after the meter to achieve real-time and accurate sensing of power outage information after the user's meter. The current penetration rate of smart meter power outage sensing is low, mainly due to high retrofit costs, making large-scale commercial deployment difficult. Currently, smart IoT terminal devices are widely used, with high penetration rates for home broadband and smart appliances on the user side. Research on mining distribution network status through smart IoT terminal device status is gradually underway. Currently, there are methods to determine whether a household has experienced a power outage by checking if the optical modem is online, and this is used to mine the power outage status of residential areas / communities. For example, the method, system, processing equipment, and medium disclosed in application number 2021111076201 for profiling user electricity usage habits and formulating maintenance plans utilize data from home optical modems to determine whether there is a power outage and characterize user electricity usage habits, facilitating on-site maintenance at appropriate times. This method uses historical data to create a profile of users' electricity usage habits, but it cannot keep track of whether users are at home in real time, making it impossible to schedule maintenance plans reasonably. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to use a smart terminal to monitor users in real time whether there is a power outage and whether anyone is at home.

[0005] The present invention solves the above-mentioned technical problems through the following technical means:

[0006] The CART decision tree-based method for distribution network end-user status awareness includes the following methods:

[0007] Step 1: Classify home smart terminals, obtain historical data for each type of home smart terminal, and perform information structured expression of smart terminal devices to obtain multi-dimensional feature data;

[0008] Step 2: Use the multiple sets of multidimensional feature data from Step 1 as training samples to train the CART decision tree and obtain the initial CART model;

[0009] Step 3: Prune the initial CART model to obtain the target CART model. The specific pruning process is as follows:

[0010] The initial CART model is defined as an acyclic graph consisting of vertices and edges, where a vertex is an arbitrary node and vertices are connected by edges; the set of weights for all edges in the acyclic graph is specified as W.

[0011] The initial model parameters for CART are Φ org ;

[0012] The CART target model parameter is Φ prun =f(W,Φ org );

[0013] The initial output of the CART model is Opt. org =CATR org (x);

[0014] The output parameters of the pruned CATR model are Opt. prun =CATR prun (x);

[0015] Constructing the loss function

[0016] Based on this loss function, the initial CART model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated.

[0017] Step 4: Apply the target model obtained in Step 3 to analyze in real time whether the power is interrupted or restored to the household and whether anyone is at home.

[0018] This invention presents a method for perceiving the status of distribution network terminal users based on CART decision trees and multi-dimensional heterogeneous big data mining from smart terminals. It fully leverages the big data information from smart terminals to achieve a more comprehensive perception of the distribution network status. First, the multi-dimensional heterogeneous data is structured to address the data incompleteness problem caused by significant differences in user-side terminal devices. Further pruning of the CART decision tree is performed, and parameters are optimized using acyclic graph construction and gradient descent, effectively solving the overfitting problem that easily occurs in the initial CATR model.

[0019] Furthermore, in step 2, the class imbalance problem in the training samples is addressed using the following method:

[0020] Assuming there are N training samples, let the dataset be D = {(x1, y1), ..., (x...}. N ,y N )};

[0021] Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is

[0022] At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is R n This refers to the total amount of data used by the current node;

[0023] The training samples are weighted, with weights w respectively. B =N B / N, The total of the modified data is:

[0024] Furthermore, in step 1, home smart terminals are divided into smart gateway devices, long-duration smart home appliances, and short-duration smart home appliances. The structured representation format of a smart gateway device is as follows:

[0025] feature_net = {existence signal; flow signal; power-on signal; timestamp}

[0026] The presence signal has a value range of {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device; the power-on signal is obtained by performing an OR operation on the power-on signals of multiple gateway devices, and the flow signal is obtained by adding the flow signals of multiple gateway devices;

[0027] The format of long-term intelligent terminal devices is as follows:

[0028] feature_long_electual = {existence signal; power-on signal; timestamp}

[0029] The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signal of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0.

[0030] The format of short-term intelligent terminal devices is as follows:

[0031] feature_short_electuical = {existence signal; power-on signal; timestamp}

[0032] The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device. The power-on signal is generated within 10 minutes; for smart TVs, smart air conditioners, and smart speakers, the power-on signal is ORed. The default value for the power-on signal is 0.

[0033] Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature:

[0034] Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT;

[0035] Furthermore, the three types of smart terminals in step 1 are:

[0036] Smart gateway devices = {optical modem, router}

[0037] Long-lasting smart home appliances = {smart refrigerator, smart doorbell}

[0038] Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

[0039] Furthermore, the CART decision tree construction process in step 3 is as follows:

[0040] Input the training set D, the Gini coefficient threshold, and the sample size threshold into the CART decision tree algorithm; output the decision tree T; where the Gini coefficient threshold is 0.2 and the sample size threshold is 2000.

[0041] The algorithm starts from the root node and recursively builds the CART classification tree using the training set.

[0042] (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node;

[0043] (2) Calculate the Gini coefficient of the sample set D using the above formula. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node.

[0044] (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D;

[0045] (4) In the training set, except for the flow feature which is a continuous value, the other features are all binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected. Based on this optimal feature and optimal feature value, the dataset is divided into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2.

[0046] (5) Recursively call steps (1)-(4) on the left and right child nodes to generate decision tree T.

[0047] The formula for calculating the Gini coefficient is as follows:

[0048] Suppose there are K categories of electricity usage status, and the probability of the k-th category is pk. The expression for the Gini coefficient of the probability distribution is:

[0049]

[0050] Corresponding to the above method, the present invention also provides a distribution network terminal user status awareness system based on CART decision tree, including the following methods:

[0051] The sample data acquisition module is used to classify home smart terminals, acquire historical data for each type of home smart terminal, and perform information structured expression of smart terminal devices to obtain multi-dimensional feature data.

[0052] The model building module is used to train the CART decision tree using multiple sets of multi-dimensional feature data as training samples to obtain the initial CART model.

[0053] The pruning module is used to prune the initial CART model to obtain the target CART model. The specific pruning process is as follows:

[0054] The initial CART model is defined as an acyclic graph consisting of vertices and edges, where a vertex is an arbitrary node and vertices are connected by edges; the set of weights for all edges in the acyclic graph is specified as W.

[0055] The initial model parameters for CART are Φ org ;

[0056] The CART target model parameter is Φ prun =f(W,Φ org );

[0057] The initial output of the CART model is Opt. org =CATRorg (x);

[0058] The output parameters of the pruned CATR model are OPt prun =CATR prun (x);

[0059] Constructing the loss function

[0060] Based on this loss function, the initial CART model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated.

[0061] The real-time analysis module is used to apply the obtained CART target model to analyze in real time whether the power is interrupted or restored to the household and whether anyone is at home.

[0062] Furthermore, the model building module addresses the class imbalance problem in the training samples using the following method:

[0063] Assuming there are N training samples, let the dataset be D = {(x1, y1), ..., (x...}. N ,y N )};

[0064] Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is

[0065] At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is R n This refers to the total amount of data used by the current node;

[0066] The training samples are weighted, with weights w respectively. B =N B / N, The total of the modified data is:

[0067] Furthermore, the sample data acquisition module categorizes home smart terminals into smart gateway devices, long-duration smart home appliances, and short-duration smart home appliances. The structured representation format of a smart gateway device is as follows:

[0068] feature_net = {existence signal; flow signal; power-on signal; timestamp}

[0069] The presence signal has a value range of {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device; the power-on signal is obtained by performing an OR operation on the power-on signals of multiple gateway devices, and the flow signal is obtained by adding the flow signals of multiple gateway devices;

[0070] The format of long-term intelligent terminal devices is as follows:

[0071] feature_long_electual = {existence signal; power-on signal; timestamp}

[0072] The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signal of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0.

[0073] The format of short-term intelligent terminal devices is as follows:

[0074] feature_short_electuical = {existence signal; power-on signal; timestamp}

[0075] The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device. The power-on signal is generated within 10 minutes; for smart TVs, smart air conditioners, and smart speakers, the power-on signal is ORed. The default value for the power-on signal is 0.

[0076] Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature:

[0077] Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT;

[0078] Furthermore, the three types of smart terminals in the sample data acquisition module are:

[0079] Smart gateway devices = {optical modem, router}

[0080] Long-lasting smart home appliances = {smart refrigerator, smart doorbell}

[0081] Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

[0082] Furthermore, the CART decision tree construction process in the pruning module is as follows:

[0083] Input the training set D, the Gini coefficient threshold, and the sample size threshold into the CART decision tree algorithm; output the decision tree T; where the Gini coefficient threshold is 0.2 and the sample size threshold is 2000.

[0084] The algorithm starts from the root node and recursively builds the CART classification tree using the training set.

[0085] (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node;

[0086] (2) Calculate the Gini coefficient of the sample set D using the above formula. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node.

[0087] (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D;

[0088] (4) In the training set, except for the flow feature which is a continuous value, the other features are all binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected. Based on this optimal feature and optimal feature value, the dataset is divided into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2.

[0089] (5) Recursively call steps (1)-(4) on the left and right child nodes to generate decision tree T.

[0090] The formula for calculating the Gini coefficient is as follows:

[0091] Suppose there are K categories of electricity usage status, then the probability of the k-th category is pk, and the Gini coefficient of the probability distribution is expressed as:

[0092]

[0093] The advantages of this invention are:

[0094] This invention presents a method for perceiving the status of distribution network terminal users based on CART decision trees and multi-dimensional heterogeneous big data mining from smart terminals. It fully leverages the big data information from smart terminals to achieve a more comprehensive perception of the distribution network status. First, the multi-dimensional heterogeneous data is structured to address the data incompleteness problem caused by significant differences in user-side terminal devices. Further pruning of the CART decision tree is performed, and parameters are optimized using acyclic graph construction and gradient descent, effectively solving the overfitting problem that easily occurs in the initial CATR model.

[0095] This invention addresses the problem of imbalanced training data by proposing a novel method for balancing data categories. According to the proposed method, the distribution balance between different data categories is well guaranteed, resulting in better generalization performance of the model.

[0096] Specifically, smart terminals are categorized into long-term online terminals and short-term online terminals, with the classification criterion being whether the terminal device's power-on is related to whether the resident is not at home. Long-term online terminal devices, such as optical modems, refrigerators, and surveillance cameras, are unrelated to whether the resident is at home and need to be turned on for extended periods. Short-term online terminal devices, such as air conditioners and televisions, are related to whether the resident is at home and are only turned on when the resident is home. The big data information of long-term online terminal devices contains power outage information about the distribution network status; if there is a power outage on the user's side, the long-term online terminal device signal will be lost. The big data information of short-term online terminal devices contains information about whether the user's household is occupied; if the short-term online terminal device signal is present, then the user's household is likely occupied. Then, a CART decision tree-based distribution network terminal user status perception method is designed, classifying terminal user status into four categories: "Normal & Occupied," "Power Outage & Occupied," "Power Outage & Occupied," and "Other." The terminal user data output from the terminal user data structuring module is input into the CART decision tree, which outputs the confidence scores of the four statuses, and the category corresponding to the highest confidence score is taken as the terminal user status. The above method can effectively detect power outages and whether anyone is present on the distribution network terminal user side. Attached Figure Description

[0097] Figure 1 This is a flowchart of the terminal data structuring process in Embodiment 1 of the present invention;

[0098] Figure 2 This is a schematic diagram of the CART decision tree output results in Embodiment 1 of the present invention;

[0099] Figure 3 This is a schematic diagram of the pruning process of the initial CART model in Embodiment 1 of the present invention. Detailed Implementation

[0100] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0101] Example 1

[0102] To address the issue of end-user status awareness in power distribution networks, this embodiment proposes a method based on CART decision trees and multi-dimensional heterogeneous big data mining from smart terminals for end-user status awareness, identifying whether there is a power outage and whether anyone is present at the end-user's location. The specific method consists of three parts: structured representation of end-user data, CART decision tree training for predicting end-user status, and CART decision tree inference deployment. The algorithm principles and actual algorithm process for each of these three parts are detailed below.

[0103] In this embodiment, a distribution network terminal user is first defined as a single household. The terminal user data consists of the data of all network-connected devices within that household.

[0104] First, the user data from smart terminals is structured. Specifically, terminal user data structuring involves transforming the highly differentiated data from connected device users into standardized structured feature information, which is then input into subsequent algorithm modules for data mining. The terminal data structuring processing flow is as follows: Figure 1 As shown.

[0105] The distribution network terminal user status awareness method based on CART decision tree provided in this embodiment specifically includes the following methods:

[0106] Step 1: Classify home smart terminals into smart gateway devices, long-duration smart home appliances, and short-duration smart home appliances; specifically:

[0107] 1) Original signal from the smart terminal:

[0108] The smart terminal devices used by end users of power distribution networks vary widely. To address this diversity, we have identified the seven most prevalent smart terminal devices: optical modems, routers, smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers. Signal analysis for these seven smart terminal devices is as follows:

[0109] Optical modems and routers are gateway devices with high online rates. Under normal power conditions, these devices are basically online 24 / 7, and they are widely used. The following signal representation is designed for such devices:

[0110] feature_net = {Device category; Traffic signal; Device power-on signal; Timestamp}

[0111] The device category value range is {optical modem, router}, which can be queried by the device's MAC address. The traffic signal is the current traffic value of the gateway, with a default value of 0. The device online signal value range is {0, 1}, where 0 represents that the device is not powered on and 1 represents that the device is powered on, with a default value of 0.

[0112] Smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers are home appliances. Online rates vary across different product categories, with moderate coverage and a rapid growth phase. The following signal representation is designed for these devices:

[0113] feature_electual = {Device type; Device power-on signal; Timestamp}

[0114] The device category value range is {Smart TV, Smart Refrigerator, Smart Air Conditioner, Smart Doorbell, Smart Speaker}, which can be queried by the device's MAC address. The device online signal value range is {0, 1}, where 0 represents that the device is not powered on and 1 represents that the device is powered on. The default value is 0, which means that the device is not powered on.

[0115] 2) Terminal Classification:

[0116] This embodiment analyzes the seven most widely used smart terminals: optical modems, routers, smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers. These seven smart terminals are categorized into three types: smart gateway devices, long-term smart home appliances, and short-term smart home appliances. Optical modems and routers are classified as smart gateway devices. Home appliances are further divided into long-term and short-term smart home appliances based on their power-on status and whether the user is present. Smart TVs, smart speakers, and smart air conditioners are likely to be turned on only when someone is present, therefore their status is related to whether the user is present, and they are classified as short-term smart home appliances. Smart refrigerators and smart doorbells remain on for extended periods regardless of whether the user is present, and therefore are classified as long-term smart home appliances. In summary, the terminal classification mapping relationship is as follows:

[0117] Smart gateway devices = {optical modem, router}

[0118] Long-lasting smart home appliances = {smart refrigerator, smart doorbell}

[0119] Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

[0120] Obtain historical data from home smart terminals and represent the information of smart terminal devices in a structured manner:

[0121] The structured representation format of a smart gateway device is as follows:

[0122] feature_net = {existence signal; flow signal; power-on signal; timestamp}

[0123] The presence signal has a value range of {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device; the power-on signal is obtained by performing an OR operation on the power-on signals of multiple gateway devices, and the flow signal is obtained by adding the flow signals of multiple gateway devices;

[0124] The format of long-term intelligent terminal devices is as follows:

[0125] feature_long_electual = {existence signal; power-on signal; timestamp}

[0126] The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signal of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0.

[0127] The format of short-term intelligent terminal devices is as follows:

[0128] feature_short_electuical = {existence signal; power-on signal; timestamp}

[0129] The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device. The power-on signal is generated within 10 minutes; for smart TVs, smart air conditioners, and smart speakers, the power-on signal is ORed. The default value for the power-on signal is 0.

[0130] Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature:

[0131] Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT; GT is obtained from a questionnaire survey of a specific user's power distribution network.

[0132] Step 2: Use the above-mentioned multiple sets of 7-dimensional feature data as training samples to train the CART decision tree and obtain the target model;

[0133] The CART decision tree construction process is as follows:

[0134] CART Decision Tree Design and Training: Since the output features of the data structuring module are 7-dimensional features, each with a different physical meaning, a decision tree algorithm is used to predict and classify these features. A decision tree is a tree structure similar to a flowchart: each internal node represents a test on an attribute, each branch represents an attribute output, and each leaf node represents a class or class distribution. The top level of the tree is the root node. Decision trees require supervised learning. Supervised learning involves providing a set of samples, each with a set of attributes and a classification result (i.e., the classification result is known). By learning from these samples, a decision tree is obtained that can correctly classify new data.

[0135] 1) Training set acquisition

[0136] 10,000 training sample pairs (feature, gt) were obtained based on real-world scene sampling, as detailed below:

[0137]

[0138] In practice, the construction process of the training set cannot guarantee the optimal proportion of data for each category, leading to unsatisfactory training results. This embodiment uses the following method to solve the problem of class imbalance in the training set.

[0139] Assuming there are N data points, let the dataset be D = {(x1, y1), ..., (x... N ,y N )};

[0140] Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is

[0141] At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is To address the issue of imbalanced data, we weight the sampled dataset, with weights w. B =N B / N, The total of the modified data is:

[0142] 2) CART decision tree construction

[0143] First, construct the initial CART model: The algorithm inputs are the training set D, the Gini coefficient threshold, and the sample size threshold; the output is the decision tree T; where the Gini coefficient threshold is 0.2, and the sample size threshold is 2000; the Gini coefficient calculation formula is:

[0144] Suppose there are K categories, in this example K is 2, representing the two states of power outage and no power outage. The probability of the k-th category is pk. The Gini coefficient expression for the probability distribution is:

[0145]

[0146] The algorithm starts from the root node and recursively builds the CART classification tree using the training set.

[0147] (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node;

[0148] (2) Calculate the Gini coefficient of the sample set D using the above formula. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node.

[0149] (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D;

[0150] (4) In the training set, except for the flow feature which is a continuous value, the other features are all binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected. Based on this optimal feature and optimal feature value, the dataset is divided into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2.

[0151] (5) Recursively call steps (1)-(4) on the left and right child nodes to generate the decision tree T, as follows: Figure 2 As shown.

[0152] The decision tree above fits the actual situation well.

[0153] Step 3: Then, prune the initial CART model to prevent overfitting.

[0154] The pruning scheme designed in this embodiment is a post-processing pruning scheme, that is, the final base model is generated first, and then pruning is performed. Figure 3 As shown, the specific steps are as follows:

[0155] (1) Consider the decision tree as an acyclic graph consisting of vertices and edges. A vertex is an arbitrary node, and vertices are connected by edges. The pruning process involves removing some edges according to certain rules. If the weight of an edge is 0, it means that there is an edge connection between two vertices; if it is 1, it means that the vertex is not connected. During the pruning process, the structure of the decision tree is maintained by assigning 0 weights to the edges connecting to the pruned subtree S. Figure 3 As shown.

[0156] (2) When pruning the CATR decision tree, we need to calculate the weight of each node of the model on the validation set. By minimizing the loss function on the validation set, we can iteratively optimize the weights. The specific algorithm is as follows:

[0157] Specify as Figure 3 The set of weights for such edges is W;

[0158] The original generated CATR model parameters are Φ org ;

[0159] The parameters of the pruned CATR model are Φ prun =f(W,Φ org );

[0160] The output of the original CATR model is Opt org =CATR org (x);

[0161] The parameters of the pruned CATR model are Opt. prun =CATR prun (x);

[0162] Constructing the loss function

[0163] Based on this loss function, the model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated.

[0164] Ultimately, this patent applies the pruned CATR decision tree described above to the power outage perception and analysis system of the distribution network. The actual results are good, and it can effectively predict the power outage status and whether anyone is present on the end user side. After pruning, the model overfitting problem is further reduced.

[0165] Step 4: Apply the target model obtained in Step 3 to analyze in real time whether the power is interrupted or restored to the household and whether anyone is at home.

[0166] This embodiment addresses the diverse and heterogeneous data from smart distribution network terminals by designing a structured data representation for individual users. This representation includes smart gateway signals, long-term smart terminal signals, and short-term smart terminal signals, embedding power outage information and occupancy status information from the user side. Furthermore, it resolves the data discrepancies arising from the diverse types of terminals on the user side. Further pruning of the CART decision tree was performed, and parameter optimization was achieved through constructing an acyclic graph and using gradient descent, effectively resolving the overfitting problem inherent in the initial CATR model.

[0167] To address the problem of imbalanced training data classes, we propose a novel method for balancing these classes. Our proposed method effectively ensures the balance of distributions across different classes, resulting in better generalization performance of the model.

[0168] This embodiment designs a multivariate heterogeneous data mining algorithm based on CART decision trees. First, the end-user status is categorized into normal, power outage & other, power outage & occupied, power outage & unoccupied, and other. A CART decision tree is then designed and trained to predict the end-user status of the distribution network.

[0169] Example 2

[0170] Corresponding to Embodiment 1, this embodiment provides a distribution network terminal user status awareness system based on CART decision tree, including:

[0171] The sample data acquisition module categorizes home smart terminals into smart gateway devices, long-duration smart home appliances, and short-duration smart home appliances; specifically:

[0172] 1) Original signal from the smart terminal:

[0173] The smart terminal devices used by end users of power distribution networks vary widely. To address this diversity, we have identified the seven most prevalent smart terminal devices: optical modems, routers, smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers. Signal analysis for these seven smart terminal devices is as follows:

[0174] Optical modems and routers are gateway devices with high online rates. Under normal power conditions, these devices are basically online 24 / 7, and they are widely used. The following signal representation is designed for such devices:

[0175] feature_net = {Device category; Traffic signal; Device power-on signal; Timestamp}

[0176] The device category value range is {optical modem, router}, which can be queried by the device's MAC address. The traffic signal is the current traffic value of the gateway, with a default value of 0. The device online signal value range is {0, 1}, where 0 represents that the device is not powered on and 1 represents that the device is powered on, with a default value of 0.

[0177] Smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers are home appliances. Online rates vary across different product categories, with moderate coverage and a rapid growth phase. The following signal representation is designed for these devices:

[0178] feature_electual = {Device type; Device power-on signal; Timestamp}

[0179] The device category value range is {Smart TV, Smart Refrigerator, Smart Air Conditioner, Smart Doorbell, Smart Speaker}, which can be queried by the device's MAC address. The device online signal value range is {0, 1}, where 0 represents that the device is not powered on and 1 represents that the device is powered on. The default value is 0, which means that the device is not powered on.

[0180] 2) Terminal Classification:

[0181] This embodiment analyzes the seven most widely used smart terminals: optical modems, routers, smart TVs, smart refrigerators, smart air conditioners, smart doorbells, and smart speakers. These seven smart terminals are categorized into three types: smart gateway devices, long-term smart home appliances, and short-term smart home appliances. Optical modems and routers are classified as smart gateway devices. Home appliances are further divided into long-term and short-term smart home appliances based on their power-on status and whether the user is present. Smart TVs, smart speakers, and smart air conditioners are likely to be turned on only when someone is present, therefore their status is related to whether the user is present, and they are classified as short-term smart home appliances. Smart refrigerators and smart doorbells remain on for extended periods regardless of whether the user is present, and therefore are classified as long-term smart home appliances. In summary, the terminal classification mapping relationship is as follows:

[0182] Smart gateway devices = {optical modem, router}

[0183] Long-lasting smart home appliances = {smart refrigerator, smart doorbell}

[0184] Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

[0185] Obtain historical data from home smart terminals and represent the information of smart terminal devices in a structured manner:

[0186] The structured representation format of a smart gateway device is as follows:

[0187] feature_net = {existence signal; flow signal; power-on signal; timestamp}

[0188] The presence signal has a value range of {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device; the power-on signal is obtained by performing an OR operation on the power-on signals of multiple gateway devices, and the flow signal is obtained by adding the flow signals of multiple gateway devices;

[0189] The format of long-term intelligent terminal devices is as follows:

[0190] feature_long_electual = {existence signal; power-on signal; timestamp}

[0191] The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signal of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0.

[0192] The format of short-term intelligent terminal devices is as follows:

[0193] feature_short_electuical = {existence signal; power-on signal; timestamp}

[0194] The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device. The power-on signal is generated within 10 minutes; for smart TVs, smart air conditioners, and smart speakers, the power-on signal is ORed. The default value for the power-on signal is 0.

[0195] Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature:

[0196] Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT; GT is obtained from a questionnaire survey of a specific user's power distribution network.

[0197] The model building module uses the above-mentioned multiple sets of 7-dimensional feature data as training samples to train the CART decision tree and obtain the target model;

[0198] The CART decision tree construction process is as follows:

[0199] CART Decision Tree Design and Training: Since the output features of the data structuring module are 7-dimensional features, each with a different physical meaning, a decision tree algorithm is used to predict and classify these features. A decision tree is a tree structure similar to a flowchart: each internal node represents a test on an attribute, each branch represents an attribute output, and each leaf node represents a class or class distribution. The top level of the tree is the root node. Decision trees require supervised learning. Supervised learning involves providing a set of samples, each with a set of attributes and a classification result (i.e., the classification result is known). By learning from these samples, a decision tree is obtained that can correctly classify new data.

[0200] 1) Training set acquisition

[0201] 10,000 training sample pairs (feature, gt) were obtained based on real-world scene sampling, as detailed below:

[0202]

[0203]

[0204] In practice, the construction process of the training set cannot guarantee the optimal proportion of data for each category, leading to unsatisfactory training results. This embodiment uses the following method to solve the problem of class imbalance in the training set.

[0205] Assuming there are N data points, let the dataset be D = {(x1, y1), ..., (x... N ,y N )};

[0206] Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is

[0207] At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is To address the issue of imbalanced data, we weight the sampled dataset, with weights w. B =N B / N, The total of the modified data is:

[0208] 2) CART decision tree construction

[0209] First, construct the initial CART model: The algorithm inputs are the training set D, the Gini coefficient threshold, and the sample size threshold; the output is the decision tree T; where the Gini coefficient threshold is 0.2, and the sample size threshold is 2000; the Gini coefficient calculation formula is:

[0210] Assuming there are K categories, in this embodiment K is 2, representing two states: power outage and no power outage. The probability of the k-th category is pk, and the Gini coefficient expression for the probability distribution is:

[0211]

[0212] The algorithm starts from the root node and recursively builds the CART classification tree using the training set.

[0213] (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node;

[0214] (2) Calculate the Gini coefficient of the sample set D using the above formula. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node.

[0215] (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D;

[0216] (4) In the training set, except for the flow feature which is a continuous value, the other features are all binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected. Based on this optimal feature and optimal feature value, the dataset is divided into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2.

[0217] (5) Recursively call steps (1)-(4) on the left and right child nodes to generate the decision tree T, as follows: Figure 2 As shown.

[0218] The decision tree above fits the actual situation well.

[0219] The pruning module is used to prune the initial CART model to prevent overfitting.

[0220] The pruning scheme designed in this embodiment is a post-processing pruning scheme, that is, the final base model is generated first, and then pruning is performed. The specific steps are as follows:

[0221] (1) Consider the decision tree as an acyclic graph consisting of vertices and edges. A vertex is an arbitrary node, and vertices are connected by edges. The pruning process involves removing some edges according to certain rules. If the weight of an edge is 0, it means that there is an edge connection between two vertices; if it is 1, it means that the vertex is not connected. During the pruning process, the structure of the decision tree is maintained by assigning 0 weights to the edges connecting to the pruned subtree S. Figure 3 As shown.

[0222] (2) When pruning the CATR decision tree, we need to calculate the weight of each node of the model on the validation set. By minimizing the loss function on the validation set, we can iteratively optimize the weights. The specific algorithm is as follows:

[0223] Specify as Figure 3 The set of weights for such edges is W;

[0224] The original generated CATR model parameters are Φ org ;

[0225] The parameters of the pruned CATR model are Φ prun =f(W,Φ org );

[0226] The output of the original CATR model is Opt org =CATRorg (x);

[0227] The parameters of the pruned CATR model are Opt. prun =CATR prun (x);

[0228] Constructing the loss function

[0229] Based on this loss function, the model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated.

[0230] Ultimately, this patent applies the pruned CATR decision tree described above to the power outage perception and analysis system of the distribution network. The actual results are good, and it can effectively predict the power outage status and whether anyone is present on the end user side. After pruning, the model overfitting problem is further reduced.

[0231] The real-time analysis module uses the obtained target model to analyze in real time whether the power is interrupted or restored to the residents and whether anyone is at home.

[0232] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for sensing the status of distribution network terminal users based on CART decision trees, characterized in that, Including the following methods: Step 1: Classify home smart terminals, obtain historical data for each type of home smart terminal, and perform information structured expression of smart terminal devices to obtain multi-dimensional feature data; In step 1, home smart terminals are divided into smart gateway devices, long-term smart home appliances, and short-term smart home appliances. The structured representation format of a smart gateway device is as follows: feature_net = {existence signal; flow signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device. The power-on signal is obtained by performing an "OR" operation on the power-on signals of multiple gateway devices, and the traffic signal is obtained by adding the traffic signals of multiple gateway devices. The format of long-term intelligent terminal devices is as follows: feature_long_electual = {existence signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signals of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0. The format of short-term intelligent terminal devices is as follows: feature_short_electuical = {existence signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device; the power-on signal is the power-on signal of smart TVs, smart air conditioners, and smart speakers within 10 minutes, which is obtained by "OR" operation; the default value of the power-on signal is 0. Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature: Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT; Step 2: Use the multiple sets of multidimensional feature data from Step 1 as training samples to train the CART decision tree and obtain the initial CART model; at each decision node, weight the data samples to solve the class imbalance problem. Step 3: Prune the initial CART model to obtain the target CART model. The specific pruning process is as follows: The initial CART model is defined as an acyclic graph consisting of vertices and edges, where a vertex is an arbitrary node and vertices are connected by edges; the set of weights for all edges in the acyclic graph is specified as W. The initial model parameters for CART are Φ org ; The CART target model parameter is Φ prun =f(W,Φ org ); The initial output of the CART model is Opt. otg =CATR org (x); The output parameters of the pruned CATR model are Opt. prun =CATR prun (x); Constructing the loss function Based on this loss function, the initial CART model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated. Step 4: Apply the target model obtained in Step 3 to analyze in real time whether the power is interrupted or restored to the household and whether anyone is at home.

2. The distribution network terminal user status awareness method based on CART decision tree according to claim 1, characterized in that, The following method is used to address the class imbalance problem in the training samples in step 2: If there are N training samples, then the dataset is denoted as D = {(x1, y1), ..., (x... N ,y N )}; Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is R n This refers to the total amount of data used by the current node; The training samples are weighted, with weights w respectively. B =N n / N, The total of the modified data is:

3. The distribution network terminal user status awareness method based on CART decision tree according to claim 1, characterized in that, In step 1, the smart terminals are divided into three categories: Smart gateway devices = {optical modem, router} Long-lasting smart home appliances = {smart refrigerator, smart doorbell} Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

4. The distribution network terminal user status awareness method based on CART decision tree according to claim 1, characterized in that, The CART decision tree construction process in step 3 is as follows: Input the training set D, the Gini coefficient threshold, and the sample size threshold into the CART decision tree algorithm; output the decision tree T; where the Gini coefficient threshold is 0.2 and the sample size threshold is 2000. The algorithm starts from the root node and recursively builds the CART classification tree using the training set. (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node; (2) Calculate the Gini coefficient of the sample set D. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node. (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D; (4) In the training set, the flow feature is a continuous value, and the other features are binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.

5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value of each feature with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected to divide the dataset into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2. (5) Recursively call steps (1)-(4) on the left and right child nodes to generate decision tree T; The formula for calculating the Gini coefficient is as follows: There are K categories of electricity usage status, and the probability of the k-th category is pk. The expression for the Gini coefficient of the probability distribution is:

5. A distribution network terminal user status awareness system based on CART decision tree, characterized in that, Including the following methods: The sample data acquisition module is used to classify home smart terminals, acquire historical data for each type of home smart terminal, and perform structured information representation of smart terminal devices to obtain multi-dimensional feature data. The sample data acquisition module categorizes home smart terminals into smart gateway devices, long-duration smart home appliances, and short-duration smart home appliances. The structured representation format for smart gateway devices is as follows: feature_net = {existence signal; flow signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no gateway device and 1 represents the presence of a gateway device. The power-on signal is obtained by performing an "OR" operation on the power-on signals of multiple gateway devices, and the traffic signal is obtained by adding the traffic signals of multiple gateway devices. The format of long-term intelligent terminal devices is as follows: feature_long_electual = {existence signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no long-term smart terminal device and 1 represents the presence of a long-term smart terminal device; the power-on signal is the "OR" operation between the power-on signals of the smart refrigerator and the smart doorbell within the set time (minutes); the default value of the power-on signal is 0. The format of short-term intelligent terminal devices is as follows: feature_short_electuical = {existence signal; power-on signal; timestamp} The signal value range is {0, 1}, where 0 represents no short-term smart terminal device and 1 represents the presence of a short-term smart terminal device; the power-on signal is the power-on signal of smart TVs, smart air conditioners, and smart speakers within 10 minutes, which is obtained by "OR" operation; the default value of the power-on signal is 0. Finally, the structured information of the smart terminal user is represented as a 7-dimensional feature: Feature = {(existence signal; flow signal; power-on signal), (existence signal; power-on signal;), (existence signal; power-on signal;)}, and the tag data corresponding to the timestamp is denoted as GT; The model building module is used to train the CART decision tree using multiple sets of multi-dimensional feature data as training samples to obtain the initial CART model; at each decision node, the data samples are weighted to solve the problem of class imbalance. The pruning module is used to prune the initial CART model to obtain the target CART model. The specific pruning process is as follows: The initial CART model is defined as an acyclic graph consisting of vertices and edges, where a vertex is an arbitrary node and vertices are connected by edges; the set of weights for all edges in the acyclic graph is specified as W. The initial model parameters for CART are Φ org ; The CART target model parameter is Φ prun =f(W,Φ org ); The initial output of the CART model is Opt. org =CATR org (x); The output parameters of the pruned CATR model are Opt. orun =CATR prun (x); Constructing the loss function Based on this loss function, the initial CART model is pruned and optimized, and the weights W of the edges in the acyclic graph are updated. The real-time analysis module is used to apply the obtained CART target model to analyze in real time whether the power is interrupted or restored to the household and whether anyone is at home.

6. The distribution network terminal user status awareness system based on CART decision tree according to claim 5, characterized in that, The model building module addresses the class imbalance problem in training samples using the following method: If there are N training samples, then the dataset is denoted as D = {(x1, y1), ..., (x... N ,y N )}; Let N be the set of data belonging to label B. B =|{j|y j =B}|, the data set without label B is At each decision node, the set of data belonging to label B is N. nB =|{j|x j ∈R n ∩y i =B}|, the data set without label B is R n This refers to the total amount of data used by the current node; The training samples are weighted, with weights w respectively. b =N B / N, The total of the modified data is:

7. The distribution network terminal user status awareness system based on CART decision tree according to claim 6, characterized in that, The smart terminals in the sample data acquisition module are divided into three categories: Smart gateway devices = {optical modem, router} Long-lasting smart home appliances = {smart refrigerator, smart doorbell} Short-term smart home appliances = {smart TV, smart air conditioner, smart speaker}.

8. The distribution network terminal user status awareness system based on CART decision tree according to claim 5, characterized in that, The CART decision tree construction process in the pruning module is as follows: Input the training set D, the Gini coefficient threshold, and the sample size threshold into the CART decision tree algorithm; output the decision tree T; where the Gini coefficient threshold is 0.2 and the sample size threshold is 2000. The algorithm starts from the root node and recursively builds the CART classification tree using the training set. (1) For the dataset D of the current node, if the number of samples is less than the threshold or there are no features, return to the decision subtree and stop the recursion of the current node; (2) Calculate the Gini coefficient of the sample set D. If the Gini coefficient is less than the threshold, return to the decision tree subtree and stop recursion at the current node. (3) Calculate the Gini coefficient of each feature value of the current node with respect to the dataset D; (4) In the training set, the flow feature is a continuous value, and the other features are binary discrete data. For binary discrete features, the dataset is divided with a threshold of 0.

5. For continuous values, the continuous features are discretized and the optimal threshold is found by stepwise binary division. Among the Gini coefficients of each feature value of each feature with respect to the dataset D, the feature A with the smallest Gini coefficient and the corresponding feature value a are selected to divide the dataset into two parts D1 and D2. At the same time, the left and right nodes of the current node are established. The dataset D of the left node is D1 and the dataset D of the right node is D2. (5) Recursively call steps (1)-(4) on the left and right child nodes to generate decision tree T; The formula for calculating the Gini coefficient is as follows: There are K categories of electricity usage status, and the probability of the k-th category is pk. The expression for the Gini coefficient of the probability distribution is:

Images