Electric power marketing common data set preprocessing method based on weighted principal component analysis method

Abstract

The invention relates to the field of power marketing business search, in particular to a power marketing common data set preprocessing method based on a weighted principal component analysis method. The data set preprocessing method comprises the following steps: standardizing original data; solving a correlation coefficient matrix V; assigning a weight to each sample; solving a feature root and a feature vector of the covariance matrix; calculating the contribution rate and the cumulative contribution rate of the principal component; and projecting the sample points to the selected feature vectors to obtain a new data set. According to the method, a multivariable problem in a high-dimensional space is converted into a low-latitude and few-variable problem, and corresponding weights are set due to diversified characteristics of power data, different characteristics in the same data table and different effects of samples in the recognition process, so that the recognition rate of sample data at different degrees is enhanced, and the recognition efficiency is improved. Therefore, effective feature extraction is performed on the sample, and the influence of outliers is eliminated, so that the recognition rate of sample data is improved.

Description

Technical Field

[0001] The present invention relates to the field of commercial search for power marketing, and in particular to a method for preprocessing a common data set for power marketing based on a weighted principal component analysis method. Background Art

[0002] In the professional data collection of power marketing, many data contain a large amount of information overlap. In order to improve the professional data analysis capabilities of power marketing, fully explore the deep value of data, deepen the carrying capacity of sparse data, release the ability to maximize data features, reduce the burden of resource storage and analysis, and promote the development of data business. Summary of the Invention

[0003] This paper proposes a method to perform redundancy analysis on power marketing datasets based on weighted principal component analysis to improve the recognition rate of sample data. The specific technical solution is as follows:

[0004] A method for preprocessing common data sets of power marketing based on weighted principal component analysis includes the following steps:

[0005] S100: standardize the raw data;

[0006] S200: Calculate the covariance matrix V based on the standardized data matrix; the formula is as follows:

[0007]

[0008] Where n is the number of data samples, is the normalized data matrix, is the transpose of the normalized data matrix;

[0009] S300: assign a weight to each sample;

[0010] Consider the obtained original data as matrix A n×m The arithmetic mean of its normalized n-row vectors is approximated as the weight vector, that is:

[0011]

[0012] Where a i ' j is the matrix a ij The elements of are normalized values;

[0013] S400: Find the eigenvalues ​​and eigenvectors of the covariance matrix;

[0014] S500: Sort the eigenvalues, select the largest k among them, and then use the corresponding k eigenvectors as column vectors to form an eigenvector matrix, and calculate the contribution rate and cumulative contribution rate of the principal component;

[0015] Contribution rate α i : The proportion of the eigenvalue corresponding to the i-th principal component in the sum of all eigenvalues ​​of the covariance matrix. The calculation formula is as follows:

[0016]

[0017] Where λ i is the eigenvalue corresponding to the i-th principal component. The larger the ratio, the stronger the ability of the i-th principal component to integrate the original index information;

[0018] Cumulative contribution rate M k : The proportion of the sum of the eigenvalues ​​of the first k principal components in the total sum of all eigenvalues. The calculation formula is as follows:

[0019]

[0020] S600: Project the sample points onto the selected feature vector to obtain a new data set.

[0021] Preferably, in S100, the original data is normalized to between 0 and 1.

[0022] Preferably, the normalization formula is as follows:

[0023]

[0024] Where a i v j is the matrix a ij The normalized values ​​of the elements.

[0025] Preferably, the fields of power generation industry (gc_ind_cls), consumption mode (e_consp_mode), contract capacity (ctrt_cap), electricity user behavior classification (ind_cls), electricity user classification (cust_cls), electricity user voltage level (voltage), electricity user electricity consumption category (ec_categ), electricity user market attributes (dereg_attr_cls), power generation (gpq), grid-connected power (net_qty), and self-generated and self-consumed power (sce_qty) are selected.

[0026] The advantages of the present invention over the prior art are:

[0027] The present invention transforms the multivariate problem in high-dimensional space into a low-dimensional problem with few variables. At the same time, due to the diverse characteristics of power data and the different roles played by different features and samples in the same data table in the recognition process, corresponding weights are set to enhance the recognition rate of sample data of different degrees, thereby effectively extracting features of the samples and eliminating the influence of outliers, thereby improving the recognition rate of sample data.

[0028] In the embodiment, taking the marketing data set as an example, the weighted principal component analysis method is used for feature screening, which reduces the data processing volume of the data set by 10%-30%, while increasing the deep value of the data by about 20% and the calculation speed by 20%-50%. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of the method for preprocessing common data sets for power marketing according to the present invention.

[0030] Figure 2 This is the normalization result of the sample data of the present invention.

[0031] Figure 3 It is the covariance matrix result of the sample data after standardization of the present invention.

[0032] Figure 4 It is the characteristic root result of the covariance matrix of the sample data of the present invention.

[0033] Figure 5 It is the eigenvector result of the covariance matrix of the sample data of the present invention.

[0034] Figure 6 It is the cumulative contribution rate data result of the sample data of the present invention.

[0035] Figure 7 This is the new dataset result after the sample data passes through S600. DETAILED DESCRIPTION

[0036] A method for preprocessing common data sets of power marketing based on weighted principal component analysis includes the following steps:

[0037] S100: Normalize the original data to eliminate the influence of variables in terms of quantity or dimension;

[0038] To ensure that each feature is treated fairly during the analysis process, and to avoid over-emphasis on certain features due to their large values, which may lead to the neglect of other features; at the same time, standardized data can make calculations more accurate and efficient.

[0039] Taking the distributed power user electricity information table in the marketing professional data set as an example, first eliminate the category feature data fields (standard time (stat_date), province code (pro_org_no), organization code (mgt_org_code), power code (gc_no), power name (gc_name), user code (cust_id), user name

[0040] (cust_name)), select the power generation industry (gc_ind_cls), consumption method

[0041] (e_consp_mode), contract capacity (ctrt_cap), user behavior classification (ind_cls), user classification (cust_cls), user voltage level

[0042] (voltage), electricity consumption category of electricity users (ec_categ), market attributes of electricity users (dereg_attr_cls), power generation (gpq), grid-connected power (net_qty), self-generated and self-consumed power (sce_qty);

[0043] The original data can be scaled to any range [0, 1]. The normalized results of the sample data are shown in Figure 2 , the normalization formula is as follows:

[0044]

[0045] Where a i ' j is the matrix a ij The normalized value of the elements of

[0046] S200: Calculate the covariance matrix based on the standardized data matrix. The covariance matrix is ​​a symmetric matrix whose elements represent the covariances between different variables. The formula for calculating the covariance matrix V of the standardized data is as follows:

[0047]

[0048] Where n is the number of data samples, is the normalized data matrix, is the transpose of the normalized data matrix;

[0049] Figure 2 The data is obtained after passing through S200 Figure 3 The covariance matrix shown;

[0050] S300: Assign a weight to each sample; used to correct the deviation between the sample structure and the overall structure, and regard the obtained raw data as matrix An×m The arithmetic mean of its normalized n-row vectors is approximated as the weight vector, that is,

[0051]

[0052] Where a i ' j is the matrix a ij The elements of A are normalized values. n×m The elements of are normalized by row, then the normalized rows are added together, and finally the added vector is divided by n to obtain the weight vector;

[0053] S400: Find the eigenvalues ​​and eigenvectors of the covariance matrix; refer to the eigenvalue results of the sample data. Figure 4 , eigenvector result reference Figure 5 ;

[0054] S500: Sort the eigenvalues, select the largest k of them, and then use the corresponding k eigenvectors as column vectors to form an eigenvector matrix. Calculate the contribution rate and cumulative contribution rate of the principal component to measure how much information the transformed data extracts from the original data.

[0055] Contribution rate α i : The proportion of the eigenvalue corresponding to the i-th principal component in the sum of all eigenvalues ​​of the covariance matrix. The calculation formula is as follows:

[0056]

[0057] Where λ i is the eigenvalue corresponding to the i-th principal component. The larger the ratio, the stronger the ability of the i-th principal component to integrate the original index information;

[0058] Cumulative contribution rate M k : The proportion of the sum of the eigenvalues ​​of the first k principal components in the total sum of all eigenvalues. The calculation formula is as follows:

[0059]

[0060] The cumulative contribution rate results of the sample data are shown in Figure 6 The larger the ratio is, the more comprehensive the first k principal components are in representing the information of the original data.

[0061] S600: Project the sample points onto the selected feature vector to obtain a new data set. Figure 7 .

Claims

1. A method for preprocessing common data sets of power marketing based on weighted principal component analysis, characterized in that: The process includes the following: S100: Standardize the raw data; S200: Calculate the covariance matrix V based on the standardized data matrix; the formula is as follows: Where n is the number of data samples, is the standardized data matrix, is the transpose of the normalized data matrix; S300: assign a weight to each sample; The obtained original data is regarded as matrix A n×m The arithmetic mean of its normalized n-row vectors is approximated as the weight vector, that is: In the formula, a i ' j is the matrix a ij The elements of are normalized values; S400: Find the characteristic roots and eigenvectors of the covariance matrix; S500: sorting the eigenvalues, selecting the largest k eigenvalues, and then using the corresponding k eigenvectors as column vectors to form an eigenvector matrix, and calculating the contribution rate and cumulative contribution rate of the principal component; Contribution rate α i : The proportion of the eigenvalue corresponding to the i-th principal component in the sum of all eigenvalues ​​of the covariance matrix. The calculation formula is as follows: In the formula, λ i is the eigenvalue corresponding to the i-th principal component. The larger the ratio, the stronger the ability of the i-th principal component to integrate the original index information; Cumulative contribution rate M k : The proportion of the sum of the eigenvalues ​​of the first k principal components in the total sum of all eigenvalues. The calculation formula is as follows: S600: Project the sample points onto the selected feature vector to obtain a new data set.

2. According to claim 1, a method for preprocessing common data sets of power marketing based on weighted principal component analysis is characterized in that: In S100, the original data is standardized to between 0 and 1.

3. According to claim 2, a method for preprocessing common data sets of power marketing based on weighted principal component analysis is characterized in that: The normalization formula is as follows: In the formula, a i ' j is the matrix a ij The normalized values ​​of the elements.

4. A method for preprocessing common data sets of power marketing based on weighted principal component analysis according to any one of claims 1 to 3, characterized in that: Select the fields of power generation industry (gc_ind_cls), consumption mode (e_consp_mode), contract capacity (ctrt_cap), electricity user behavior classification (ind_cls), electricity user classification (cust_cls), electricity user voltage level (voltage), electricity user electricity consumption category (ec_categ), electricity user market attributes (dereg_attr_cls), power generation (gpq), grid-connected electricity (net_qty), and self-generated and self-consumed electricity (sce_qty).

Images