Electric quantity transaction data quality verification method, system and device and medium

Abstract

The invention belongs to the technical field of electric quantity transaction data quality verification, and discloses an electric quantity transaction data quality verification method, system, device and medium, and the method comprises the steps: extracting all feature data contained in an electric quantity transaction data curve, carrying out the feature selection, and obtaining the feature data after redundant information and irrelevant features are removed; reconstructing the feature data after feature selection through an integrated depth auto-encoder to obtain a low-dimensional vector and a reconstruction error feature; and inputting the low-dimensional vector and the reconstruction error feature into an estimation network, performing density estimation and energy calculation of an abnormal sample, judging whether a threshold value is exceeded or not, and completing quality verification. According to the method, the feature vectors are screened, redundant information and irrelevant features are eliminated, only features which contribute to anomaly detection remarkably are reserved, sample noise can be suppressed, and the robustness of the model is improved. Data abnormity can be effectively identified and processed, and the efficiency and reliability of power market settlement are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electricity transaction data quality verification, and in particular relates to an electricity transaction data quality verification method, system, device and medium. Background Art

[0002] The efficient operation of the power trading market is crucial to ensuring the stability of power supply, promoting the effective use of clean energy, and promoting the sustainable development of the energy industry. In this market, the trading activities of power buyers and sellers generate a large amount of data, including trading volume, supply and demand status, settlement information, etc., which are characterized by large data volume, complex structure and wide distribution. The quality of power market transaction data directly affects the judgment of market operation status and the accuracy of settlement business. Therefore, ensuring the accuracy of data is the basis for improving power market efficiency and system stability.

[0003] In the actual electricity trading and settlement business, the abnormal power data of the unit may trigger frequent settlement needs, which may be caused by various factors such as equipment failure and complex trading and settlement processes. For example, hardware problems of the energy meter (such as loose wiring or damaged meter needles) may lead to metering errors, and improper management of energy meter data (such as loss of monthly base code, data jump or untimely maintenance of metering point information) may also cause data anomalies. In addition, the complexity of the electricity settlement business itself, including the untimely synchronization of marketing and transaction data, the high correlation between data, and the possible unreasonable power fluctuations (even negative values), further exacerbates the possibility of data anomalies.

[0004] Based on the above problems, the existing technology generates low-dimensional vectors and corresponding reconstruction errors through a deep autoencoder, and inputs this information into the Gaussian mixture model. The end-to-end training method is used to jointly optimize the autoencoder and mixture model parameters, and an additional estimation network is introduced to assist in the learning of mixture model parameters, thereby balancing the autoencoder reconstruction, low-dimensional vector density estimation and regularization.

[0005] A method for detecting electricity theft by users based on a deep autoencoder Gaussian mixture model has also been proposed. First, the data is subjected to an augmented Dickey-Fuller test to obtain a dimension of electricity consumption data with stability. Then, the potential features of the data are extracted through a compression network, and the sample energy reflecting the degree of abnormality is obtained using an estimation network and a Gaussian mixture model. Finally, the network parameters are jointly optimized based on an end-to-end learning method to avoid model decoupling, and users whose sample energy exceeds the abnormal threshold are identified as electricity theft, thereby realizing the detection of user electricity theft.

[0006] The electricity curve contains features that can identify anomalies. However, in the prior art, object data is directly used for anomaly detection, and no features suitable for anomaly detection of electricity trading data are proposed. If the original data is directly used for anomaly detection without extracting features specifically for electricity trading data, a large amount of redundant information contained in the original data will increase the computational burden and affect the efficiency and accuracy of the model. In addition, electricity trading data has its own particularities, such as changes in curve shape and peak and valley time points, which are crucial for anomaly detection. If special attention is not paid, the model may not be able to accurately capture these important features. In the power market trading data, the data curves uploaded by marketing and dispatching cover information of multiple types, multiple units, and multiple market members, resulting in inconsistent data curve shapes and peak and valley time points. The inconsistency of data curve shapes and peak and valley time points means that the changes in the data set are very large. This change may make it difficult for the model to learn general rules applicable to all types of data, thereby affecting the generalization ability of the model. And due to the diversity of data, the definition of what is normal and what is abnormal becomes unclear. This will make it difficult for the model to accurately identify anomalies, and it is easy to have false positives or false negatives. Summary of the invention

[0007] The purpose of the present invention is to provide a method, system, device and medium for verifying the quality of electricity transaction data, so as to solve the problem that the model is difficult to accurately identify anomalies and is prone to false positives or false negatives.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a method for verifying the quality of electricity transaction data, comprising:

[0010] Extract all feature data contained in the electricity transaction data curve, perform feature selection, and obtain feature data after removing redundant information and irrelevant features;

[0011] The feature data after feature selection is reconstructed through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features;

[0012] The low-dimensional vector and reconstruction error features are input into the estimation network to perform density estimation and energy calculation of abnormal samples, determine whether they exceed the threshold, and complete quality verification.

[0013] Furthermore, all characteristic data contained in the electricity transaction data curve are extracted, including:

[0014] Range, kurtosis, second-order center distance, normalized amplitude absolute value standard deviation, centroid, correlation coefficient, time domain waveform skewness, standard deviation, dispersion coefficient, waveform entropy and waveform fluctuation.

[0015] Furthermore, after feature selection, feature data is obtained after removing redundant information and irrelevant features, including:

[0016] Relevance vector machine (RVM) is used for feature screening. By constraining the sparseness of weight parameters, the weights of features are identified and assigned, and features with weights close to zero are eliminated.

[0017] Furthermore, the feature data after feature selection is reconstructed by integrating the deep autoencoder to obtain low-dimensional vectors and reconstruction error features, including:

[0018] The integrated deep autoencoder includes an encoder and a decoder. The encoder is composed of column-stacked neural network layers, which receives input data and converts it into a low-dimensional representation. The decoder has the same structure as the encoder, receives the low-dimensional vector generated by the encoder, and restores it back to a reconstructed sample that is approximate to the original input data, obtaining the sample low-dimensional vector and the reconstruction error characteristics.

[0019] Furthermore, the low-dimensional vector and the reconstruction error feature are input into the estimation network to perform density estimation and energy calculation of abnormal samples, including:

[0020] The estimation network is a multi-layer neural network, which is used to predict the probability of each sample in each mixture component of the Gaussian mixture model GMM, and estimate the parameters of the Gaussian mixture model GMM based on these probabilities: mixture component probability, mean and covariance matrix, and calculate the sample energy reflecting the degree of sample abnormality.

[0021] Furthermore, it is determined whether the threshold is exceeded and quality verification is completed, including:

[0022] The lower the energy, the more consistent the sample is with the model distribution. Conversely, it indicates anomalies. The sample energy and reconstruction error characteristics are calculated to obtain the objective function and calculate the sample loss. The objective function and AdamW optimizer are used to simultaneously update the parameters of the deep autoencoder and the estimation network; the test set with abnormal samples is used for model testing, and a threshold is set. The sample percentile is equal to the proportion of normal data in the test set. The sample energy of the sample at this percentile is set as the threshold. The subsequent input sample is calculated through the model to determine whether it exceeds the threshold to achieve anomaly detection.

[0023] In a second aspect, the present invention provides a system for verifying the quality of electricity transaction data, comprising:

[0024] The data acquisition module is used to extract all characteristic data contained in the electricity transaction data curve, and after feature selection, obtain the characteristic data after eliminating redundant information and irrelevant features;

[0025] The feature reconstruction module is used to reconstruct the feature data after feature selection through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features;

[0026] The output judgment module is used to input the low-dimensional vector and reconstruction error features into the estimation network, perform density estimation and energy calculation of abnormal samples, determine whether it exceeds the threshold, and complete quality verification.

[0027] Furthermore, it is determined whether the threshold is exceeded and quality verification is completed, including:

[0028] The lower the energy, the more consistent the sample is with the model distribution. Conversely, it indicates anomalies. The sample energy and reconstruction error characteristics are calculated to obtain the objective function and calculate the sample loss. The objective function and AdamW optimizer are used to simultaneously update the parameters of the deep autoencoder and the estimation network; the test set with abnormal samples is used for model testing, and a threshold is set. The sample percentile is equal to the proportion of normal data in the test set. The sample energy of the sample at this percentile is set as the threshold. The subsequent input sample is calculated through the model to determine whether it exceeds the threshold to achieve anomaly detection.

[0029] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a method for verifying the quality of electricity transaction data when executing the computer program.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for verifying the quality of electricity transaction data are implemented.

[0031] Compared with the prior art, the present invention has the following technical effects:

[0032] Targeted feature extraction and selection: In order to address the problem of underutilized anomaly identification features in electricity curves, this method first clearly proposes a feature extraction method for volatility, trend, and variability of power trading data. This includes using indicators such as range, kurtosis, second-order center distance, and normalized amplitude absolute value standard deviation to characterize the variability of data, as well as capturing the trend and volatility of data through features such as centroid, correlation coefficient, and time domain waveform skewness. Furthermore, feature selection is performed through relevance vector machine (RVM), which automatically identifies and eliminates irrelevant features, ensuring the pertinence and effectiveness of the extracted features and improving the accuracy of anomaly detection.

[0033] Enhanced adaptability and generalization: In the face of the complexity of power market transaction data, including the coverage of multi-type, multi-unit, and multi-market member information, as well as inconsistent data curve shapes and peak and valley time points, this method integrates deep autoencoders (DAE) and Gaussian mixture models (GMM), which not only achieves effective data compression and low-dimensional vectors, but also enhances the model's learning ability for various complex data patterns through end-to-end training. In particular, through the optimized parameter estimation network, the model can better adapt to different types of abnormal patterns, improve generalization ability, and accurately identify anomalies even in the face of inconsistent data curves.

[0034] Improve model accuracy and detection efficiency: In order to improve the generalization ability and model accuracy of the deep autoencoder Gaussian mixture model, the model performance was improved by improving the parameter optimizer and using Manhattan distance and cosine similarity as the reconstruction error function. By using the reconstruction error characteristics of the deep autoencoder and the sample energy of the Gaussian mixture model, as well as the regularization penalty term of the covariance matrix as the objective function, the end-to-end joint optimization of the model was achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the present invention.

[0036] Figure 2 This is a system structure diagram of the present invention.

[0037] Figure 3 It is a schematic diagram of the components of the present invention.

[0038] Figure 4 This is a logic block diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0041] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0042] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0043] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0044] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0045] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0046] Explanation of relevant terms

[0047] Relevance Vector Machine (RVM)

[0048] Relevance vector machine is a machine learning method and a variant of support vector machine (SVM), proposed by Michael Tipping in 2001. Similar to SVM, RVM is also used for classification and regression tasks, but its core idea lies in the sparsity of the model, that is, by finding the "relevance vector" in the data (a small number of sample points that make an important contribution to the prediction results) to approximate the behavior of the entire data set. This not only reduces the complexity of the model, but also improves computational efficiency and interpretability. RVM is implemented through a Bayesian framework and can automatically determine the complexity of the model, including regularization parameters and the number of relevance vectors.

[0049] Deep Autoencoding Gaussian Mixture Model (DAGMM)

[0050] The Deep Autoencoder Gaussian Mixture Model is an anomaly detection method that combines deep learning with probabilistic models. The model usually consists of two main parts: a Deep Autoencoder and a Gaussian Mixture Model (GMM). The Deep Autoencoder is responsible for learning the low-dimensional vectors of the data, aiming to capture the effective features of the data and reconstruct the input data; while the Gaussian Mixture Model is used to model the distribution of these low-dimensional features. By jointly training these two components, DAGMM can learn the normal pattern of the data and use it as a benchmark to identify abnormal samples. This method uses the powerful expressive power of deep learning and the flexibility of GMM to effectively improve the performance of anomaly detection.

[0051] Example 1, please refer to Figure 1 The present invention provides a method for verifying the quality of electricity transaction data, comprising:

[0052] Extract all feature data contained in the electricity transaction data curve, perform feature selection, and obtain feature data after removing redundant information and irrelevant features;

[0053] The feature data after feature selection is reconstructed through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features;

[0054] The low-dimensional vector and reconstruction error features are input into the estimation network to perform density estimation and energy calculation of abnormal samples, determine whether they exceed the threshold, and complete quality verification.

[0055] The present invention screens feature vectors, removes redundant information and irrelevant features, and retains only features that contribute significantly to anomaly detection. This method can suppress sample noise and improve the robustness of the model. It can effectively identify and process data anomalies and improve the efficiency and reliability of power market settlement.

[0056] Embodiment 2, the present invention provides a method for verifying the quality of electricity transaction data, specifically comprising:

[0057] Extract features reflecting volatility, trend and change from the original data, such as statistical indicators such as range, kurtosis, second-order center distance, normalized amplitude absolute value standard deviation, as well as correlation coefficient characteristics, time domain waveform skewness characteristics, etc. These features together constitute the feature vector that describes the characteristics of the data.

[0058] The relevance vector machine (RVM) algorithm is used to screen feature vectors, remove redundant information and irrelevant features, and retain only features that contribute significantly to anomaly detection. This method can suppress sample noise and improve the robustness of the model.

[0059] The compression network uses a deep autoencoder to encode and decode the input features in order to reconstruct the input data and learn the low-dimensional vector of the data in the process. The output of this process is a low-dimensional feature representation and a reconstruction error feature. The low-dimensional vector can better reveal the essential structure of the data, while the reconstruction error feature reflects the difference between the input data and the reconstructed data, which is part of the subsequent calculation loss.

[0060] The output of the compression network, i.e., the low-dimensional feature representation and the reconstruction error feature, is received by the estimation network, which further analyzes the data in depth through the neural network model, including calculating the probability of the sample under GMM and the sample energy to determine anomalies.

[0061] See also Figure 3 ,in:

[0062] Feature extraction

[0063] When there are larger or smaller values ​​in the electricity data that deviate from normal data, the variability feature can reflect the degree of dispersion within the electricity data set and extract the distribution state of the data values ​​around the central value. Peak and trough periods are common in daily electricity data. Extracting trend features helps identify abnormal data that deviates from the overall growth or decline trend over time. When the electricity data involves large increases and decreases in a short period of time (such as hourly and minute granularity) and there are large differences between the previous and subsequent time periods, the volatility feature can reflect the rapid and irregular changes in the electricity data in a short period of time.

[0064] Feature Selection Network

[0065] When there are larger or smaller values ​​in the electricity data that deviate from normal data, the variability feature can reflect the degree of dispersion within the electricity data set and extract the distribution state of the data values ​​around the central value. Peak and trough periods are common in daily electricity data. Extracting trend features helps identify abnormal data that deviates from the overall growth or decline trend over time. When the electricity data involves large increases and decreases in a short period of time (such as hourly and minute granularity) and there are large differences between the previous and subsequent time periods, the volatility feature can reflect the rapid and irregular changes in the electricity data in a short period of time.

[0066] Compressed Network

[0067] The compression network is essentially a deep autoencoder (DAE), which consists of an encoder and a decoder. The encoder is composed of a series of stacked neural network layers, which receives input data and gradually converts it into lower and lower dimensional representations. The decoder has the same structure as the encoder, which receives the low-dimensional vector generated by the encoder and restores it back to a reconstructed sample that is similar to the original input data. The compression network uses the autoencoder to reconstruct feature data, which makes data features easier to identify and reduces the time required for detection. It also obtains the sample low-dimensional vector and the reconstruction error feature.

[0068] The output of the compression network consists of two parts: one is the low-dimensional vector obtained by nonlinear dimensionality reduction of the input data, and the other is the reconstruction error feature based on the difference measurement between the original data and its reconstructed sample. Both are further sent to the estimation network to perform density estimation and energy calculation of abnormal samples under the framework of Gaussian mixture model.

[0069] Estimation Network

[0070] The estimation network is a multi-layer neural network that is used to predict the probability of each sample in each mixture component of the GMM and estimate the parameters of the GMM based on these probabilities: the mixture component probability, mean, and covariance matrix, which are used to calculate the sample energy that reflects the degree of sample abnormality.

[0071] In actual situations, power trading data conforms to the assumption that most data are normal values, and abnormal samples are usually far away from normal samples. Therefore, the probability that each dimensional component of the abnormal value belongs to a Gaussian mixture component is low, which is manifested in the high calculated sample energy in DAGMM. The abnormal sample energy can be obtained as the threshold for abnormal discrimination based on the abnormal proportion of the test sample during the model training and testing phase.

[0072] Example 3, please refer to Figure 4 , a method for verifying the quality of electricity transaction data, specifically comprising:

[0073] 1. Collect normal power data over a period of time for model training. Power data is usually recorded every 15 minutes, with 96 points of data per day. During the data preprocessing stage, normalize the data for each day.

[0074] 2. Extract the statistical features of 11 dimensions of variability, volatility, and trend of daily electricity data to form a feature vector. RVM automatically identifies and assigns weights of important features through sparse constraints on weight parameters, and removes features with weights close to zero.

[0075] 3. The reduced-dimensional feature vector set is input into the model training, and the original data is decoded and re-encoded by the compression network to obtain the reconstructed data. The compression network feeds the low-dimensional vector and the reconstructed error features to the estimation network.

[0076] 4. The input vector passes through a multi-layer neural network, and a Gaussian mixture model is connected after the multi-layer neural network to estimate the feature density and calculate the sample energy of each sample. The lower the energy, the more consistent the sample is with the model distribution. Conversely, it may indicate anomalies.

[0077] 5. Calculate the sample energy and reconstruction error to obtain the objective function and calculate the sample loss. Use the objective function and AdamW optimizer to simultaneously update the parameters of the deep autoencoder and the estimation network.

[0078] 6. Use the test set with abnormal samples for model testing and set a reasonable threshold. The sample percentile is equal to the proportion of normal data in the test set, and the sample energy of the sample at this percentile is set as the threshold. The subsequent input samples are calculated through the model to determine whether they exceed the threshold and achieve anomaly detection.

[0079] See also Figure 2 In one embodiment of the present invention, a system for verifying the quality of electricity transaction data is provided, which can be used to implement the above-mentioned method for verifying the quality of electricity transaction data. Specifically, the system includes:

[0080] The data acquisition module is used to extract all characteristic data contained in the electricity transaction data curve, and after feature selection, obtain the characteristic data after eliminating redundant information and irrelevant features;

[0081] The feature reconstruction module is used to reconstruct the feature data after feature selection through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features;

[0082] The output judgment module is used to input the low-dimensional vector and reconstruction error features into the estimation network, perform density estimation and energy calculation of abnormal samples, determine whether it exceeds the threshold, and complete quality verification.

[0083] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0084] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a method for verifying the quality of electricity trading data.

[0085] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of a method for verifying the quality of electricity transaction data in the above embodiment.

[0086] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for verifying the quality of electricity transaction data, characterized in that: include: Extract all feature data contained in the electricity transaction data curve, perform feature selection, and obtain feature data after removing redundant information and irrelevant features; The feature data after feature selection is reconstructed through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features; The low-dimensional vector and reconstruction error features are input into the estimation network to perform density estimation and energy calculation of abnormal samples, determine whether they exceed the threshold, and complete quality verification.

2. A method for verifying the quality of electricity transaction data according to claim 1, characterized in that: Extract all characteristic data contained in the electricity transaction data curve, including: Range, kurtosis, second-order center distance, normalized amplitude absolute value standard deviation, centroid, correlation coefficient, time domain waveform skewness, standard deviation, dispersion coefficient, waveform entropy and waveform fluctuation.

3. A method for verifying the quality of electricity transaction data according to claim 1, characterized in that: After feature selection, feature data is obtained after removing redundant information and irrelevant features, including: Relevance vector machine (RVM) is used for feature screening. By constraining the sparseness of weight parameters, the weights of features are identified and assigned, and features with weights close to zero are eliminated.

4. A method for verifying the quality of electricity transaction data according to claim 1, characterized in that: The feature data after feature selection is reconstructed through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features, including: The integrated deep autoencoder includes an encoder and a decoder. The encoder is composed of column-stacked neural network layers, which receives input data and converts it into a low-dimensional representation. The decoder has the same structure as the encoder, receives the low-dimensional vector generated by the encoder, and restores it back to a reconstructed sample that is approximate to the original input data, obtaining the sample low-dimensional vector and the reconstruction error characteristics.

5. The method for verifying the quality of electricity transaction data according to claim 1, characterized in that: The low-dimensional vector and reconstruction error features are input into the estimation network to perform density estimation and energy calculation of abnormal samples, including: The estimation network is a multi-layer neural network, which is used to predict the probability of each sample in each mixture component of the Gaussian mixture model GMM, and estimate the parameters of the Gaussian mixture model GMM based on these probabilities: mixture component probability, mean and covariance matrix, and calculate the sample energy reflecting the degree of sample abnormality.

6. A method for verifying the quality of electricity transaction data according to claim 5, characterized in that: Determine whether the threshold is exceeded and complete quality verification, including: The lower the energy, the more the sample conforms to the model distribution, and vice versa, it means there is an anomaly. The sample energy and reconstruction error characteristics are calculated to obtain the objective function, and the sample loss is calculated. The objective function and AdamW optimizer are used to simultaneously update the parameters of the deep autoencoder and the estimation network. The test set with abnormal samples is used for model testing, and a threshold is set. The sample percentile is equal to the proportion of normal data in the test set. The sample energy of the sample at this percentile is set as the threshold. The subsequent input sample calculates its sample energy through the model to determine whether it exceeds the threshold to achieve anomaly detection.

7. A power transaction data quality verification system, characterized in that: include: The data acquisition module is used to extract all characteristic data contained in the electricity transaction data curve, and after feature selection, obtain the characteristic data after eliminating redundant information and irrelevant features; The feature reconstruction module is used to reconstruct the feature data after feature selection through an integrated deep autoencoder to obtain a low-dimensional vector and reconstruction error features; The output judgment module is used to input the low-dimensional vector and reconstruction error features into the estimation network, perform density estimation and energy calculation of abnormal samples, determine whether it exceeds the threshold, and complete quality verification.

8. The electricity transaction data quality verification system according to claim 7 is characterized in that: Determine whether the threshold is exceeded and complete quality verification, including: The lower the energy, the more the sample conforms to the model distribution, and vice versa, it means there is an anomaly. The sample energy and reconstruction error characteristics are calculated to obtain the objective function, and the sample loss is calculated. The objective function and AdamW optimizer are used to simultaneously update the parameters of the deep autoencoder and the estimation network. The test set with abnormal samples is used for model testing, and a threshold is set. The sample percentile is equal to the proportion of normal data in the test set. The sample energy of the sample at this percentile is set as the threshold. The subsequent input sample calculates its sample energy through the model to determine whether it exceeds the threshold to achieve anomaly detection.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for verifying the quality of electricity transaction data as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a method for verifying the quality of electricity transaction data as described in any one of claims 1 to 6 are implemented.

Images