Electric power data quality detection method

Abstract

The invention discloses an electric power data quality detection method, and relates to the technical field of data detection, and the method comprises the steps: inputting electric power data through user identity authentication, evaluating the electric power data, and setting the electric power data quality detection method; power data quality detection is carried out based on a set power data quality detection method, and a result of power data quality detection is generated into a report to be output; and constructing a visual interface to display the result of the electric power data quality detection in real time, performing security encryption on the result of the electric power data quality detection, and storing, collecting and analyzing the generated electric power data. According to the method, the electric power data is input through user identity authentication, the electric power data quality detection method is set for electric power data quality detection, and the result of electric power data quality detection is generated into a report to be output, so that the accuracy and efficiency of detection are improved, and the accuracy, real-time performance and safety of data detection are improved; and a solid guarantee is provided for stable operation of a power system.

Description

Technical Field

[0001] The present invention relates to the technical field of data detection, and in particular to a method for detecting power data quality. Background Art

[0002] With the continuous development of smart grids and the widespread application of power data, the data quality problem of the power system has gradually become an important factor affecting the reliability and stability of the power system. In the existing technology, the methods of power data quality detection mainly include manual detection and rule-based automatic detection. Manual detection usually relies on the experience of experts and manual operations, which is inefficient and prone to errors. The rule-based automatic detection method verifies the data through a series of preset rules. Although it improves the detection efficiency to a certain extent, it is insufficient to cope with the complexity and variability of the data, and it is difficult to identify potential abnormal data and data quality problems.

[0003] The existing power data quality detection methods still have some shortcomings. The data integrity and consistency detection methods are relatively single and cannot fully cover all aspects of data quality. Although the existing machine learning models can perform intelligent analysis and anomaly detection on data, they often require a large amount of labeled data during the model training process. The cost of obtaining labeled data is high and the time is long. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing power data quality detection method, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the data integrity and consistency detection methods are relatively single and cannot comprehensively cover all aspects of data quality. Although the existing machine learning models can perform intelligent analysis and anomaly detection on data, a large amount of labeled data is often required during the model training process, and the cost and time of obtaining labeled data are high.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for detecting power data quality, which includes inputting power data through user identity authentication, evaluating the power data and setting the power data quality detection method; performing power data quality detection based on the set power data quality detection method, generating a report based on the results of the power data quality detection and outputting it; constructing a visual interface to display the results of the power data quality detection in real time, implementing secure encryption on the results of the power data quality detection, and storing, collecting and analyzing the generated power data.

[0007] As a preferred solution of the power data quality detection method of the present invention, wherein: the input of power data through user identity authentication refers to the user obtaining a digital certificate through real-name verification, including the user's public key and identity information;

[0008] The user installs the digital certificate on a USB key and connects it to a computer by inserting the USB key. When the user uploads power detection data, the system verifies the digital certificate based on the digital certificate of the requested electronic key and the pre-stored CA public key. If the verification is successful, the user is reminded to upload the power data. If the verification fails, the user is reminded to obtain a digital certificate through real-name verification for upload.

[0009] As a preferred solution of the power data quality detection method of the present invention, wherein: the power data quality detection method for evaluating the power data refers to using a statistical analysis method to extract the average value g, standard deviation b, and maximum value g of the power data. max and the minimum value g min , and merged into a comprehensive feature vector, the formula is:

[0010] F=[g,b,g max ,g min ],

[0011] Where F is the comprehensive feature vector;

[0012] Collect historical power data and use statistical analysis to extract features to obtain the historical comprehensive vector F ’ , generate training set;

[0013] Construct a convolutional neural network (CNN) model, including input layer, convolution layer, pooling layer, and fully connected output layer;

[0014] Define the input layer as the comprehensive feature vector F;

[0015] The historical comprehensive feature vector F′ output by the convolutional layer is processed as follows:

[0016] C l =ReLU(W l *F′+b l ),

[0017] Among them C l is the output of the convolutional layer at layer l, W l is the weight of the lth layer, b l is the bias of the lth layer, and ReLU is the activation function;

[0018] Using the maximum pooling operation, the formula is:

[0019] P l =max(C l ),

[0020] Where P l is the pooling output of the lth layer, and max is the pooling operation of taking the maximum value of the region;

[0021] The node of the pooling layer is input into the fully connected layer, and the formula is:

[0022] Y=W f *P+b f ,

[0023] Where Y is the evaluation output, W f is the weight of the fully connected layer, b f is the bias term of the fully connected layer;

[0024] Use the training set to train the convolutional neural network (CNN) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters;

[0025] The comprehensive feature vector F is input into the trained convolutional neural network CNN model to obtain the power data evaluation result Y;

[0026] Based on setting the inspection threshold Q according to expert opinions, if the power data evaluation result Y>inspection threshold Q, it is set as advanced power data quality detection, and if the power data evaluation result Y≤inspection threshold Q, it is set as basic power data quality detection.

[0027] As a preferred solution of the power data quality detection method of the present invention, wherein: the power data quality detection based on the set power data quality detection method includes:

[0028] Basic power data quality testing, including the use of data quality testing tools to test the integrity, consistency, numerical repeatability and timeliness of power data;

[0029] Define business rules based on voltage standards and use validation tools to check compliance of power data with business rules;

[0030] Advanced power data quality detection, including collecting historical advanced power data and generating action sets;

[0031] Constructing an isolation forest model, including estimating the proportion of outliers using statistical analysis on historical advanced power data, setting a value of the pollution rate according to the proportion of outliers, and randomly selecting an integer as a random state value;

[0032] The Isolation Forest model is trained using the set of operations. Each tree randomly draws a subsample from the dataset and is built independently on the subsample. At the node of each tree, a high-level power data and a split value are randomly selected to split the data until the data is isolated.

[0033] The advanced power data is input into the trained isolation forest model to obtain the outliers of the advanced power data;

[0034] Based on the normal distribution theory, an abnormal threshold is set, the mean and standard deviation of the advanced power data are calculated, and the standard score is calculated using the Z-score method. The advanced power data whose absolute value of the standard score exceeds the abnormal threshold is marked as an abnormal value;

[0035] Based on the theoretical distribution, the expected probability distribution G is set, and the occurrence probability of the advanced power data is calculated to set the actual probability distribution U;

[0036] Use the formula to calculate the KL divergence D between two distributions KL (U‖G), the formula is:

[0037]

[0038] Where U(d) is the probability of the advanced power data d in the actual distribution, and G(d) is the probability of the advanced power data d in the expected distribution.

[0039] As a preferred solution of the power data quality detection method of the present invention, wherein: generating a report from the power data quality detection result for output refers to using SQL query to aggregate all power data quality detection results, including the integrity, effectiveness, precision, repeatability, timeliness, accuracy and outlier detection results of basic detection and advanced detection;

[0040] Use ReportLab's function to generate the final PDF file and set the automatic save path.

[0041] As a preferred solution of the power data quality detection method described in the present invention, wherein: the construction of a visualization interface to display the results of the power data quality detection in real time refers to using the visualization tool Matplotlib to construct a visualization interface, laying out a chart area in the middle of the page, displaying the results of the power data quality detection in real time, and adding user controls to the sidebar of the page, including a date selector and a drop-down menu.

[0042] As a preferred solution of the power data quality detection method of the present invention, wherein: the secure encryption of the power data quality detection result refers to using a strong random number generator to create a 256-bit encryption key, and storing the generated key in the key management service AWS;

[0043] Use AES to encrypt the power data input by the user;

[0044] When evaluating the power data, the decryption key is obtained from the key management service AWS, and the encrypted data is decrypted using the AES decryption algorithm and key to restore the original data.

[0045] As a preferred solution of the power data quality detection method described in the present invention, wherein: the storage of the power data collected and analyzed refers to storing the collected power data and the power data quality detection results generated by the analysis in a cloud database, the cloud database is sorted in chronological order and marked with corresponding tags, and the collected power data and the power data quality detection results generated by the analysis are synchronously backed up in the cloud, and the collected power data, the power data quality detection results generated by the analysis and the backup data are regularly checked for integrity.

[0046] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned power data quality detection method when executing the computer program.

[0047] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned power data quality detection method when executed by a processor.

[0048] The beneficial effects of the present invention are as follows: the present invention inputs power data through user identity authentication, evaluates the power data and sets a power data quality detection method, performs power data quality detection based on the set power data quality detection method, generates a report based on the result of the power data quality detection and outputs it, thereby improving the accuracy and efficiency of detection, and enhancing the accuracy, real-time and security of data detection, providing a solid guarantee for the stable operation of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0050] Figure 1 The figure is a flow chart of the power data quality detection method.

[0051] Figure 2 It is a structural diagram of the power data quality detection method. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0053] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0055] Example 1, reference Figure 1 and Figure 2 , is the first embodiment of the present invention, which provides a method for detecting power data quality, the method for detecting power data quality comprising:

[0056] S1. Input power data through user identity authentication, evaluate the power data and set the power data quality detection method;

[0057] Specifically, inputting power data through user identity authentication means that the user obtains a digital certificate through real-name verification, including the user's public key and identity information;

[0058] The user installs the digital certificate on a USB key and connects it to a computer by inserting the USB key. When the user uploads power detection data, the system verifies the digital certificate based on the digital certificate of the requested electronic key and the pre-stored CA public key. If the verification is successful, the user is reminded to upload the power data. If the verification fails, the user is reminded to obtain a digital certificate through real-name verification for upload.

[0059] Through the dual verification of digital certificates and CA public keys, the security of the power data upload process is ensured, preventing intrusion and data tampering by illegal users. This PKI-based security mechanism has been widely used in finance, medical and other fields, proving its efficiency and security. Since each user who uploads data has undergone real-name verification and their digital certificates are issued by authoritative organizations, this greatly enhances the credibility and traceability of the data, which is crucial for the operation and decision-making of the power system, because high-quality data is the basis of scientific decision-making. The use of USB keys eliminates the need for users to remember complex passwords or perform cumbersome multi-step verifications. Identity authentication and data upload can be completed with simple physical operations, which not only improves the user's operational convenience, but also reduces the possibility of errors.

[0060] Furthermore, the power data quality detection method is set to evaluate the power data, which refers to the use of statistical analysis to extract the average value g, standard deviation b, and maximum value g of the power data. max and the minimum value g min , and merged into a comprehensive feature vector, the formula is:

[0061] F=[g,b,g max ,g min ],

[0062] Where F is the comprehensive feature vector;

[0063] Collect historical power data and use statistical analysis to extract features to obtain the historical comprehensive vector F ’ , generate training set;

[0064] Construct a convolutional neural network (CNN) model, including input layer, convolution layer, pooling layer, and fully connected output layer;

[0065] Define the input layer as the comprehensive feature vector F;

[0066] The historical comprehensive feature vector F′ output by the convolutional layer is processed as follows:

[0067] C l =ReLU(W l *F′+b l ),

[0068] Among them C l is the output of the convolutional layer at layer l, W l is the weight of the lth layer, b l is the bias of the lth layer, and ReLU is the activation function;

[0069] Using the maximum pooling operation, the formula is:

[0070] P l =max(C l ),

[0071] Where P l is the pooling output of the lth layer, and max is the pooling operation of taking the maximum value of the region;

[0072] The node of the pooling layer is input into the fully connected layer, and the formula is:

[0073] Y=W f *P+b f ,

[0074] Where Y is the evaluation output, W f is the weight of the fully connected layer, b f is the bias term of the fully connected layer;

[0075] Use the training set to train the convolutional neural network (CNN) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters;

[0076] The comprehensive feature vector F is input into the trained convolutional neural network CNN model to obtain the power data evaluation result Y;

[0077] Based on setting the inspection threshold Q according to expert opinions, if the power data evaluation result Y>inspection threshold Q, it is set as advanced power data quality detection, and if the power data evaluation result Y≤inspection threshold Q, it is set as basic power data quality detection.

[0078] Using convolutional layers and pooling layers to process the feature vector F can effectively extract the temporal and spatial features in the power data, which is more efficient and accurate than traditional data processing techniques. Based on the inspection threshold Q set by experts, the power data quality detection results are classified into advanced detection and basic detection, allowing for fine management of data quality issues at different levels. This hierarchical detection strategy allows resources to be allocated more effectively and gives priority to solving more serious quality problems. Through automated data evaluation processes and real-time data processing, the need for manual participation is greatly reduced, and the efficiency and timeliness of data processing are improved.

[0079] S2. Perform power data quality detection based on the set power data quality detection method, and generate a report based on the power data quality detection result for output;

[0080] Specifically, the power data quality detection based on the set power data quality detection method includes:

[0081] Basic power data quality testing, including the use of data quality testing tools to test the integrity, consistency, numerical repeatability and timeliness of power data;

[0082] Define business rules based on voltage standards and use validation tools to check compliance of power data with business rules;

[0083] Advanced power data quality detection, including collecting historical advanced power data and generating action sets;

[0084] Constructing an isolation forest model, including estimating the proportion of outliers using statistical analysis on historical advanced power data, setting a value of the pollution rate according to the proportion of outliers, and randomly selecting an integer as a random state value;

[0085] The Isolation Forest model is trained using the set of operations. Each tree randomly draws a subsample from the dataset and is built independently on the subsample. At the node of each tree, a high-level power data and a split value are randomly selected to split the data until the data is isolated.

[0086] The advanced power data is input into the trained isolation forest model to obtain the outliers of the advanced power data;

[0087] Based on the normal distribution theory, an abnormal threshold is set, the mean and standard deviation of the advanced power data are calculated, and the standard score is calculated using the Z-score method. The advanced power data whose absolute value of the standard score exceeds the abnormal threshold is marked as an abnormal value;

[0088] Based on the theoretical distribution, the expected probability distribution G is set, and the occurrence probability of the advanced power data is calculated to set the actual probability distribution U;

[0089] Use the formula to calculate the KL divergence D between two distributions KL (U‖G), the formula is:

[0090]

[0091] Where U(d) is the probability of the advanced power data d in the actual distribution, and G(d) is the probability of the advanced power data d in the expected distribution.

[0092] By systematically checking the integrity and consistency of the data, we can discover and correct missing and inconsistencies in the data, thereby improving the quality and reliability of the data. Through timeliness testing, we can ensure that the data is updated in a timely manner and avoid using expired or invalid data. By building an isolation forest model and performing unsupervised learning on the data, we can efficiently identify and isolate outliers in the data. By calculating the KL divergence and analyzing the deviation between the actual distribution and the expected distribution of advanced power data, we can help discover potential systemic problems.

[0093] Further, generating a report for outputting the results of the power data quality test means using SQL query to aggregate all power data quality test results, including the completeness, validity, precision, repeatability, timeliness, accuracy and outlier test results of basic and advanced tests;

[0094] Use ReportLab's function to generate the final PDF file and set the automatic save path.

[0095] By using SQL queries, all power data quality test results can be extracted and summarized from the database quickly and efficiently. SQL query statements can be flexibly adjusted according to specific needs to ensure that the extracted data is accurate. Using ReportLab to generate PDF reports can ensure the consistency and professionalism of the report format. ReportLab allows users to customize various parts of the report, such as titles, tables, charts, etc. By setting the automatic save path, the report generation process can be fully automated without manual intervention. The generated PDF report can intuitively and clearly display all power data quality test results, which is convenient for users to view and analyze. By using professional typesetting and charting tools, the report is not only detailed, but also beautiful and easy to read. Through the systematic data aggregation, analysis and report generation process, it is ensured that all aspects of power data quality management can operate efficiently, which not only improves the accuracy and timeliness of data quality detection, but also provides a reliable basis for subsequent data analysis and decision-making.

[0096] S3. Construct a visual interface to display the results of power data quality detection in real time, implement security encryption on the results of power data quality detection, and store, collect and analyze the generated power data;

[0097] Specifically, building a visualization interface to display the results of power data quality detection in real time means using the visualization tool Matplotlib to build a visualization interface, laying out a chart area in the middle of the page, displaying the results of power data quality detection in real time, and adding user controls in the sidebar of the page, including a date selector and a drop-down menu.

[0098] By using Matplotlib to build a visualization interface, the power data quality test results can be updated and displayed in real time. User controls such as date selectors and drop-down menus are added to the visualization interface, allowing users to filter and filter data according to their needs. The visualization interface improves the readability and ease of use of data display through intuitive charts and friendly user controls. Users can quickly obtain the required information and make accurate judgments and decisions through simple operations.

[0099] Furthermore, implementing secure encryption on the results of the power data quality detection means using a strong random number generator to create a 256-bit encryption key, and storing the generated key in the key management service AWS;

[0100] Use AES to encrypt the power data input by the user;

[0101] When evaluating the power data, the decryption key is obtained from the key management service AWS, and the encrypted data is decrypted using the AES decryption algorithm and key to restore the original data.

[0102] A strong random number generator is used to generate a 256-bit encryption key, which ensures the high security and unpredictability of the key. The power data is encrypted using the AES encryption algorithm to prevent the data from being stolen and tampered with during transmission and storage. The generated encryption key is stored in AWS KMS. Its secure key management function can prevent key loss and unauthorized access. The key management service provided by AWS KMS includes key generation, storage, rotation, and access control functions to ensure the security management of the key throughout its life cycle. When evaluating the power data, the decryption key is obtained from AWS KMS, and the original data is restored using the AES decryption algorithm to ensure the integrity and availability of the data when it is in use. By combining encryption and key management to form a complete security system, the overall security and reliability of the system can be significantly improved.

[0103] Furthermore, storing the collected and analyzed power data means storing the collected power data and the power data quality test results generated by the analysis in a cloud database, sorting the data in chronological order and marking the data with corresponding tags, and synchronously backing up the collected power data and the power data quality test results generated by the analysis in the cloud, and regularly performing integrity checks on the collected power data, the power data quality test results generated by the analysis and the backup data.

[0104] The power data is stored in the cloud database. The high scalability and high availability of the cloud database can be used to store and manage large amounts of data. Sorting in chronological order makes data access and analysis more convenient and efficient. Through cloud backup, the data has multiple redundant copies in the cloud storage to ensure that the data will not be lost in the event of hardware failure, network interruption, etc. Regular integrity testing ensures that the data has not been tampered with during transmission and storage, and maintains data consistency. The use of cloud databases and cloud backups not only improves the efficiency of data management, but also enhances data security and reliability. Through automated data storage, backup, and integrity testing, the complexity and error rate of manual operations are reduced.

[0105] Embodiment 2, if the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0107] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0108] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A method for detecting power data quality, Features: include, Input power data through user identity authentication, evaluate the power data and set the power data quality detection method; Perform power data quality detection based on a set power data quality detection method, and generate a report based on the power data quality detection result for output; Build a visual interface to display the results of power data quality detection in real time, implement secure encryption on the results of power data quality detection, and store, collect and analyze the generated power data.

2. The power data quality detection method according to claim 1, Features: The input of power data through user identity authentication refers to the user obtaining a digital certificate through real-name verification, including the user's public key and identity information; The user installs the digital certificate on a USB key and connects it to a computer by inserting the USB key. When the user uploads power detection data, the system verifies the digital certificate based on the digital certificate of the requested electronic key and the pre-stored CA public key. If the verification is successful, the user is reminded to upload the power data. If the verification fails, the user is reminded to obtain a digital certificate through real-name verification for upload.

3. The power data quality detection method according to claim 2, Features: The method for evaluating the power data and setting the power data quality detection method refers to using a statistical analysis method to extract the average value g, standard deviation b, and maximum value g of the power data. max and the minimum value g min , and merged into a comprehensive feature vector, the formula is: F=[g,b,g max ,g min ], Where F is the comprehensive feature vector; Collect historical power data and use statistical analysis to extract features to obtain the historical comprehensive vector F ’ , generate training set; Construct a convolutional neural network (CNN) model, including input layer, convolution layer, pooling layer, and fully connected output layer; Define the input layer as the comprehensive feature vector F; The historical comprehensive feature vector F′ output by the convolutional layer is processed as follows: C l =ReLU(W l *F′+b l ), Among them C l is the output of the convolutional layer at layer l, W l is the weight of the lth layer, b l is the bias of the lth layer, and ReLU is the activation function; Using the maximum pooling operation, the formula is: P l =max(C l ), Where P l is the pooling output of the lth layer, and max is the pooling operation of taking the maximum value of the region; The node of the pooling layer is input into the fully connected layer, and the formula is: Y=W f *P+b f , Where Y is the evaluation output, W f is the weight of the fully connected layer, b f is the bias term of the fully connected layer; Use the training set to train the convolutional neural network (CNN) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters; The comprehensive feature vector F is input into the trained convolutional neural network CNN model to obtain the power data evaluation result Y; Based on setting the inspection threshold Q according to expert opinions, if the power data evaluation result Y>inspection threshold Q, it is set as advanced power data quality detection, and if the power data evaluation result Y≤inspection threshold Q, it is set as basic power data quality detection.

4. The power data quality detection method according to claim 3, Features: The power data quality detection based on the set power data quality detection method includes: Basic power data quality testing, including the use of data quality testing tools to test the integrity, consistency, numerical repeatability and timeliness of power data; Define business rules based on voltage standards and use validation tools to check compliance of power data with business rules; Advanced power data quality detection, including collecting historical advanced power data and generating action sets; Constructing an isolation forest model, including estimating the proportion of outliers using statistical analysis on historical advanced power data, setting a value of the pollution rate according to the proportion of outliers, and randomly selecting an integer as a random state value; The Isolation Forest model is trained using the set of operations. Each tree randomly draws a subsample from the dataset and is built independently on the subsample. At the node of each tree, a high-level power data and a split value are randomly selected to split the data until the data is isolated. The advanced power data is input into the trained isolation forest model to obtain the outliers of the advanced power data; Based on the normal distribution theory, an abnormal threshold is set, the mean and standard deviation of the advanced power data are calculated, and the standard score is calculated using the Z-score method. The advanced power data whose absolute value of the standard score exceeds the abnormal threshold is marked as an abnormal value; Based on the theoretical distribution, the expected probability distribution G is set, and the occurrence probability of the advanced power data is calculated to set the actual probability distribution U; The KL divergence D between two distributions is calculated using the formula KL (U‖G), the formula is: Where U(d) is the probability of the advanced power data d in the actual distribution, and G(d) is the probability of the advanced power data d in the expected distribution.

5. The power data quality detection method according to claim 4, Features: Generating a report from the results of the power data quality test for output means using SQL query to aggregate all power data quality test results, including the integrity, effectiveness, precision, repeatability, timeliness, accuracy and outlier test results of basic and advanced tests; Use ReportLab's function to generate the final PDF file and set the automatic save path.

6. The power data quality detection method according to claim 5, Features: The construction of a visualization interface to display the results of the power data quality detection in real time refers to using the visualization tool Matplotlib to construct a visualization interface, laying out a chart area in the middle of the page, displaying the results of the power data quality detection in real time, and adding user controls in the sidebar of the page, including a date selector and a drop-down menu.

7. The power data quality detection method according to claim 6, Features: The secure encryption of the power data quality detection result refers to using a strong random number generator to create a 256-bit encryption key, and storing the generated key in the key management service AWS; Use AES to encrypt the power data input by the user; When evaluating the power data, the decryption key is obtained from the key management service AWS, and the encrypted data is decrypted using the AES decryption algorithm and key to restore the original data.

8. The power data quality detection method according to claim 7, Features: The storing of the power data collected and generated by analysis refers to storing the collected power data and the power data quality test results generated by the analysis in a cloud database, sorting the data in chronological order and marking the corresponding tags, synchronously backing up the collected power data and the power data quality test results generated by the analysis in the cloud, and regularly performing integrity checks on the collected power data, the power data quality test results generated by the analysis and the backup data.

9. A computer device, include: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of the power data quality detection method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the steps of the power data quality detection method according to any one of claims 1 to 8 are implemented.

Images