Automatic preprocessing method, system and equipment for power data and readable storage medium

By extracting arc features using temporal convolutional neural networks and attention mechanisms, and combining sparse constrained autoencoders and long short-term memory networks to train the model, the problem of time-consuming and laborious manual annotation of arc datasets is solved, achieving efficient and accurate automatic annotation of arc data and model training.

CN119004250AInactive Publication Date: 2024-11-22GUANGDONG SUNFLY ELECTRONICS HLDG CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410955754.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In existing technologies, the formation of electric arc datasets relies on manual annotation, which is time-consuming, labor-intensive, and easily affected by subjective factors, resulting in low dataset quality and low model training efficiency.

Method used

We employ temporal convolutional neural networks and attention mechanisms to extract electric arc features, combine them with sparse-constrained autoencoders to select target features, construct an electric arc dataset, and use long short-term memory networks and transfer learning to train an electric arc data labeling model. We then optimize the model performance through intelligent labeling.

Benefits of technology

This improves the training efficiency and recognition accuracy of the arc data labeling model, reduces manual intervention, and continuously improves the model classification accuracy through iterative learning, ensuring rapid and accurate preprocessing and automatic labeling of fault arc data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004250A_ABST
    Figure CN119004250A_ABST
Patent Text Reader

Abstract

The invention discloses an electric power data automatic preprocessing method, system and device and a readable storage medium, and relates to the technical field of fault diagnosis, and the method comprises the steps: obtaining reference arc data, and carrying out the preprocessing of the reference arc data; initial arc characteristics are extracted from the preprocessed reference arc data, and target arc characteristics are selected from the initial arc characteristics; constructing an arc data set according to the target arc characteristics; according to the arc data set, training a pre-constructed arc data marking model through a transfer learning method; and obtaining target arc data, and intelligently marking the target arc data through the trained arc data marking model. According to the invention, the training efficiency and recognition precision of the arc data marking model can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis technology, and in particular to a method, system, device and readable storage medium for automatic preprocessing of power data. Background Art

[0002] Fault arcs are a common form of fault in power systems and are the main factor causing electrical fires in buildings. Therefore, timely and accurate identification of fault arcs is crucial to ensuring the safety of the power grid.

[0003] Arc datasets are the basis for training arc fault recognition models. The existing method of forming arc datasets is to manually label the data after collecting it, that is, the operator classifies and labels the arc fault signals according to the operating instructions to form a dataset. However, this process is highly dependent on manual operation, which is not only time-consuming and labor-intensive, but also easily affected by subjective factors, which seriously affects the quality of the dataset and the efficiency of model training.

[0004] Therefore, it is particularly urgent to develop a system that can automatically and efficiently complete arc data preprocessing and automatic labeling. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method, system, device and readable storage medium for automatic preprocessing of electric power data, which can improve the training efficiency and recognition accuracy of the arc data marking model.

[0006] In order to solve the above technical problems, the present invention provides a method for automatic preprocessing of power data, comprising: obtaining reference arc data and preprocessing the reference arc data; extracting initial arc features from the preprocessed reference arc data, and selecting target arc features from the initial arc features; constructing an arc data set according to the target arc features; training a pre-constructed arc data labeling model through a transfer learning method based on the arc data set; obtaining target arc data, and intelligently labeling the target arc data through the trained arc data labeling model.

[0007] As an improvement of the above scheme, the steps of extracting initial arc features from the preprocessed benchmark arc data and selecting target arc features from the initial arc features include: capturing the time series features of the preprocessed benchmark arc data through a temporal convolutional neural network; extracting initial arc features through an attention mechanism based on the time series features; and selecting target arc features from the initial arc features through a sparse constrained autoencoder.

[0008] As an improvement of the above-mentioned scheme, the step of constructing an arc data set according to the target arc characteristics includes: according to the target arc characteristics, selecting a preset number of reference arc data for marking processing to mark the selected reference arc data as normal state data and arc state data; dividing the marked reference arc data into a training set and a test set.

[0009] As an improvement of the above-mentioned scheme, the method for constructing the arc data labeling model includes: selecting a power system fault prediction model as the arc data labeling model; adding a long short-term memory network layer to the arc data labeling model according to the arc classification characteristics, and adjusting the output setting requirements of the fully connected layer in the arc data labeling model; constructing a binary cross entropy loss function and an Adam optimizer in the arc data labeling model.

[0010] As an improvement of the above scheme, the automatic preprocessing method for power data also includes: scoring the arc data marking model according to the labeling results, and the scoring indicators include accuracy and precision; optimizing the arc data marking model according to the scoring results.

[0011] As an improvement of the above solution, the automatic preprocessing method for power data also includes: generating a performance report based on the labeling results and the scoring results.

[0012] Correspondingly, the present invention also provides an automatic preprocessing system for electric power data, comprising: a preprocessing module, used to obtain reference arc data and preprocess the reference arc data; a feature extraction module, used to extract initial arc features from the preprocessed reference arc data, and select target arc features from the initial arc features; a data set construction module, used to construct an arc data set according to the target arc features; a model training module, used to train a pre-constructed arc data labeling model through a transfer learning method based on the arc data set; and an intelligent labeling module, used to obtain target arc data and intelligently label the target arc data through a trained arc data labeling model.

[0013] As an improvement of the above scheme, the automatic preprocessing system for electric power data also includes: an evaluation and optimization module, which is used to score the arc data marking model according to the marking results, and optimize the arc data marking model according to the scoring results, and the scoring indicators include accuracy and precision.

[0014] Correspondingly, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the above-mentioned automatic pre-processing method for power data when executing the computer program.

[0015] Correspondingly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above-mentioned method for automatic pre-processing of power data when executed by a processor.

[0016] The implementation of the present invention has the following beneficial effects:

[0017] The present invention improves data preprocessing efficiency and automatic labeling accuracy through intelligent means, thereby optimizing the construction process of arc data sets and systematically improving the training efficiency and recognition accuracy of arc data labeling models. Specifically:

[0018] The present invention uses a combination of a temporal convolutional neural network and an attention mechanism to automatically extract features, which not only greatly reduces the feature dimension and computational complexity, but also significantly improves the expressiveness and discrimination of the features, so that the model can more accurately distinguish different types of fault arcs.

[0019] Based on the long short-term memory network framework, the present invention uses the idea of ​​transfer learning to train a preliminary arc data labeling model in the case of a small amount of arc labeled data, which is used to predict a large amount of unlabeled data later;

[0020] Furthermore, the present invention forms a closed-loop optimization system by performing real-time monitoring of automatically processed and labeled data, dynamically adjusting preprocessing parameters and labeling strategies according to the uncertainty of model prediction and manual review results, and continuously optimizing model performance through online learning or periodic retraining. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a first embodiment of the method for automatic preprocessing of electric power data of the present invention;

[0022] Figure 2 is a flow chart of a second embodiment of the method for automatic preprocessing of electric power data of the present invention;

[0023] Figure 3 It is a schematic diagram of the structure of an embodiment of the automatic preprocessing system for power data of the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0025] See also Figure 1 , Figure 1 The flowchart of the first embodiment of the method for automatic preprocessing of electric power data of the present invention is shown, which includes:

[0026] S101, obtaining reference arc data, and preprocessing the reference arc data;

[0027] The present invention collects reference arc data through an oscilloscope device and saves it in a file in csv format; wherein the oscilloscope has a filtering and denoising function and has a certain noise processing capability for the collected reference arc data to ensure that the cleaned reference arc data is obtained.

[0028] After the power data automatic preprocessing system obtains the benchmark arc data uploaded by the oscilloscope, it converts the benchmark arc data in csv format into json format, thereby realizing data preprocessing (data segmentation, invalid data elimination, etc.) to obtain the preprocessed benchmark arc data, ensuring the validity of the benchmark arc data and facilitating subsequent feature extraction.

[0029] S102, extracting initial arc features from the preprocessed reference arc data, and selecting target arc features from the initial arc features;

[0030] The present invention uses artificial intelligence technology to extract initial arc features from preprocessed reference arc data, and selects the most discriminative target arc features from them, so as to realize automatic extraction and selection of arc features. The specific steps include:

[0031] (1) Capture the time series characteristics of the preprocessed benchmark arc data through a time series convolutional neural network;

[0032] Temporal Convolutional Network (TCN) can capture the time series characteristics of benchmark arc data through multiple layers of one-dimensional convolutional layers.

[0033] (2) Based on the time series features, the initial arc features are extracted through the attention mechanism;

[0034] The attention mechanism can dynamically adjust the weights according to the importance of the features and automatically highlight the key information of the fault arc.

[0035] (3) The target arc features are selected from the initial arc features through a sparse constrained autoencoder.

[0036] The present invention introduces a sparse constrained autoencoder as a feature selection tool, which minimizes the reconstruction error while reducing the number of features, thereby achieving efficient simplification of features.

[0037] Therefore, step S102 uses a combination of a temporal convolutional neural network and an attention mechanism to automatically extract features, which not only greatly reduces the feature dimension and computational complexity, but also significantly improves the expressiveness and discrimination of the features, allowing the model to more accurately distinguish different types of fault arcs.

[0038] S103, constructing an arc data set according to target arc features;

[0039] Specifically, the steps of constructing an arc data set according to target arc features include:

[0040] (1) selecting a preset number of reference arc data for marking processing according to target arc characteristics, so as to mark the selected reference arc data as normal state data and arc state data;

[0041] In practical applications, a small amount of reference arc data can be manually marked manually.

[0042] For example, 80 groups of reference arc data may be randomly selected and manually marked from a large amount of collected reference arc data, and the reference arc data may be marked as two types: normal state data and arc state data.

[0043] (2) Divide the marked reference arc data into a training set and a test set.

[0044] Preferably, the marked reference arc data can be divided into a training set and a test set in a ratio of 8:2, but this is not a limitation and can be adjusted according to actual conditions.

[0045] S104, training a pre-built arc data labeling model by a transfer learning method according to the arc data set;

[0046] Accordingly, the method for constructing the arc data labeling model includes:

[0047] (1) Selecting a power system fault prediction model as an arc data marking model;

[0048] Generally, a power system fault prediction model can be selected as a pre-trained arc data labeling model.

[0049] (2) According to the characteristics of arc classification, a long short-term memory network layer is added to the arc data labeling model, and the output setting requirements of the fully connected layer in the arc data labeling model are adjusted;

[0050] In practical applications, the arc data labeling model can be structurally adjusted according to the arc classification characteristics. The specific adjustments include: adding a long short-term memory network layer (LSTM) and adjusting the output setting requirements in the fully connected layer.

[0051] (3) Construct a binary cross entropy loss function and Adam optimizer in the arc data labeling model.

[0052] Specifically, a binary cross entropy loss function can be defined for the arc data labeling model, and the Adam optimizer can be selected.

[0053] Accordingly, the training set is input into the arc data labeling model, and the training program is executed to obtain a preliminarily trained arc data labeling model; then the test set is input into the preliminarily trained arc data labeling model, and the test program is executed to obtain a basically trained arc data labeling model.

[0054] Therefore, based on the long short-term memory network framework, the present invention uses the idea of ​​transfer learning to train a preliminary arc data labeling model in the case of a small amount of arc labeled data, which is used to subsequently predict a large amount of unlabeled data.

[0055] S105, acquiring target arc data, and intelligently labeling the target arc data using a trained arc data labeling model.

[0056] When working, the unlabeled target arc data is input into the trained arc data labeling model, and the prediction result of the target arc data being "normal state data" or "arc state data" can be output. The prediction result is the label of the target arc data.

[0057] Therefore, the present invention improves the data preprocessing efficiency and automatic labeling accuracy through intelligent means, thereby optimizing the construction process of the arc data set and systematically improving the training efficiency and recognition accuracy of the arc data labeling model.

[0058] See also Figure 2 , Figure 2 The flowchart of the second embodiment of the method for automatic preprocessing of electric power data of the present invention is shown, which includes:

[0059] S201, obtaining reference arc data, and preprocessing the reference arc data;

[0060] S202, extracting initial arc features from the preprocessed reference arc data, and selecting target arc features from the initial arc features;

[0061] S203, constructing an arc data set according to target arc features;

[0062] S204, training a pre-built arc data labeling model by a transfer learning method according to the arc data set;

[0063] S205, acquiring target arc data, and intelligently labeling the target arc data using a trained arc data labeling model.

[0064] S206, scoring the arc data marking model according to the marking results;

[0065] In order to evaluate the accuracy of the annotation results, it is necessary to score the annotation results of the arc data marking model. The present invention uses accuracy and precision as scoring indicators, where:

[0066] Accuracy is the ratio of correctly predicted samples to the total number of samples;

[0067] Precision is the proportion of samples predicted as “arc state data” that are actually arcs.

[0068] S207, optimizing the arc data marking model according to the scoring result.

[0069] S208: Generate a performance report based on the labeling results and the scoring results.

[0070] and Figure 1 What is different from the first embodiment shown is that in this embodiment, the arc data marking model is also scored based on the marking results, thereby optimizing the arc data marking model and generating a performance report including the marking results and the scoring results for recording and displaying the performance of the arc data marking model.

[0071] Furthermore, if the arc data labeling model has a low score, the arc data labeling model needs to be further adjusted or retrained. Specifically, feedback information can be collected based on the labeling results and the scoring results to further optimize the arc data labeling model, and this process may require multiple iterations to continuously improve the prediction performance of the arc data labeling model.

[0072] In addition, for a small amount of data that still has a very low score after the optimization of the arc data labeling model, the labeling results of these data can be manually reviewed to ensure accuracy and reliability.

[0073] Therefore, the present invention forms a closed-loop optimization system by monitoring the automatically processed and labeled data in real time, dynamically adjusting the preprocessing parameters and labeling strategies according to the uncertainty of model prediction and manual review results, and continuously optimizing the model performance through online learning or periodic retraining.

[0074] See also Figure 3 , Figure 3 The specific structure of the automatic preprocessing system 100 of power data of the present invention is shown, which includes a preprocessing module 1, a feature extraction module 2, a data set construction module 3, a model training module 4 and an intelligent labeling module 5. Specifically:

[0075] The preprocessing module 1 is used to obtain reference arc data and preprocess the reference arc data;

[0076] The feature extraction module 2 is used to extract the initial arc features from the preprocessed reference arc data, and select the target arc features from the initial arc features;

[0077] The data set construction module 3 is used to construct an arc data set according to the target arc characteristics;

[0078] The model training module 4 is used to train a pre-built arc data labeling model through a transfer learning method according to the arc data set;

[0079] The intelligent labeling module 5 is used to obtain target arc data and intelligently label the target arc data through a trained arc data labeling model.

[0080] The preprocessing module 1, feature extraction module 2, data set construction module 3, model training module 4 and intelligent labeling module 5 are described in detail below:

[0081] 1. Preprocessing module

[0082] After the preprocessing module 1 obtains the reference arc data uploaded by the oscilloscope, it converts the reference arc data in csv format into json format, thereby realizing data preprocessing (data segmentation, invalid data elimination, etc.) to obtain preprocessed reference arc data, ensuring the validity of the reference arc data and facilitating subsequent feature extraction.

[0083] 2. Feature Extraction Module

[0084] Feature extraction module 2 includes:

[0085] A temporal convolutional neural network unit is used to capture the time series characteristics of the preprocessed benchmark arc data through a temporal convolutional neural network; wherein, the temporal convolutional neural network (TCN, Temporal Convolutional Network) can capture the time series characteristics of the benchmark arc data through multiple layers of one-dimensional convolutional layers.

[0086] The attention mechanism unit is used to extract the initial arc features through the attention mechanism according to the time series features; wherein the attention mechanism can dynamically adjust the weight according to the importance of the features and automatically highlight the key information of the fault arc.

[0087] The autoencoder unit is used to select target arc features from initial arc features through a sparse constrained autoencoder; it should be noted that the present invention introduces a sparse constrained autoencoder as a feature selection tool to achieve efficient simplification of features by minimizing reconstruction errors while reducing the number of features.

[0088] Therefore, the feature extraction module uses a combination of temporal convolutional neural network and attention mechanism to automatically extract features, which not only greatly reduces the feature dimension, reduces the computational complexity, and reduces redundant information, but also significantly improves the expressiveness and discrimination of the features, enabling the model to more accurately distinguish different types of fault arcs.

[0089] 3. Dataset Construction Module

[0090] Dataset building module 3 includes:

[0091] A marking unit is used to select a preset number of reference arc data for marking processing according to the target arc characteristics, so as to mark the selected reference arc data as normal state data and arc state data; for example, 80 groups of reference arc data can be randomly selected and marked from a large amount of collected reference arc data, and the reference arc data can be marked as two types: normal state data and arc state data.

[0092] The division unit is used to divide the reference arc data after the marking process into a training set and a test set. Preferably, the reference arc data after the marking process can be divided into a training set and a test set in a ratio of 8:2, but this is not limited to this and can be adjusted according to actual conditions.

[0093] 4. Model Training Module

[0094] The training set is input into the arc data labeling model, and the training program is executed to obtain a preliminarily trained arc data labeling model; then the test set is input into the preliminarily trained arc data labeling model, and the test program is executed to obtain a basically trained arc data labeling model.

[0095] 5. Intelligent Labeling Module

[0096] When working, the unlabeled target arc data is input into the trained arc data labeling model, and the prediction result of the target arc data being "normal state data" or "arc state data" can be output. The prediction result is the label of the target arc data.

[0097] Therefore, the present invention constructs an arc data labeling model based on semi-supervised learning and transfer learning. First, the arc data labeling model is trained using a small number of manually labeled samples. Then, the trained arc data labeling model is used to autonomously learn and label a large amount of unlabeled data, so as to systematically improve the training efficiency and recognition accuracy of the arc data labeling model.

[0098] Furthermore, the automatic power data preprocessing system 100 also includes an evaluation and optimization module for scoring the arc data labeling model according to the labeling results, and optimizing the arc data labeling model according to the scoring results. The scoring indicators include accuracy and precision.

[0099] In order to evaluate the accuracy of the annotation results, it is necessary to score the annotation results of the arc data marking model. The present invention uses accuracy and precision as scoring indicators, where:

[0100] Accuracy is the ratio of correctly predicted samples to the total number of samples;

[0101] Precision is the proportion of all samples predicted to be “arc state data” that are actually arcs.

[0102] In addition, the automatic power data preprocessing system 100 further includes a feedback optimization module. If the arc data labeling model has a low score, the arc data labeling model is further adjusted or retrained through the feedback optimization module.

[0103] Specifically, the feedback optimization module can collect feedback information based on the labeling results and scoring results to further optimize the arc data labeling model. This process may require multiple iterations to continuously improve the prediction performance of the arc data labeling model.

[0104] Therefore, the present invention ensures continuous improvement of data processing and model training, effectively avoids the accumulation of data quality problems, improves the stability of the overall system and the accuracy of fault identification, and ensures the long-term effectiveness of the automatic preprocessing and automatic labeling system for fault arc data.

[0105] Accordingly, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the above-mentioned automatic preprocessing method for power data when executing the computer program. At the same time, the present invention also discloses a computer-readable storage medium, wherein the computer program is stored thereon, wherein the steps of the above-mentioned automatic preprocessing method for power data are implemented when the computer program is executed by the processor.

[0106] It can be seen from the above that the present invention greatly improves the efficiency of data labeling, reduces manual intervention, and continuously improves the classification accuracy of the model through iterative learning, thereby realizing fast and accurate preprocessing and automatic labeling of fault arc data, significantly improving the efficiency and quality of data set construction, and providing a solid foundation for subsequent fault diagnosis models, which helps to shorten fault response time, enhance the safety and stability of the power system, and improve the speed of production of industry arc identification algorithms and models.

[0107] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for automatic preprocessing of power data, characterized in that: include: Acquiring reference arc data, and preprocessing the reference arc data; Extracting initial arc features from the preprocessed reference arc data, and selecting target arc features from the initial arc features; constructing an arc data set according to the target arc characteristics; According to the arc data set, a pre-built arc data labeling model is trained by a transfer learning method; Target arc data is acquired, and the target arc data is intelligently labeled using a trained arc data labeling model.

2. The method for automatic preprocessing of electric power data according to claim 1, characterized in that: The step of extracting initial arc features from the pre-processed reference arc data and selecting target arc features from the initial arc features comprises: Capturing the time series characteristics of the preprocessed reference arc data through a time series convolutional neural network; According to the time series features, initial arc features are extracted through an attention mechanism; A target arc feature is selected from the initial arc features by a sparse constrained autoencoder.

3. The method for automatic preprocessing of electric power data according to claim 1, characterized in that: The step of constructing an arc data set according to the target arc characteristics comprises: According to the target arc characteristics, a preset number of reference arc data are selected for marking processing, so as to mark the selected reference arc data as normal state data and arc state data; The marked reference arc data are divided into a training set and a test set.

4. The method for automatic preprocessing of electric power data according to claim 1, characterized in that: The method for constructing the arc data marking model includes: The power system fault prediction model is selected as the arc data marking model; According to the arc classification characteristics, a long short-term memory network layer is added to the arc data labeling model, and the output setting requirements of the fully connected layer in the arc data labeling model are adjusted; A binary cross entropy loss function and an Adam optimizer are constructed in the arc data labeling model.

5. The method for automatic preprocessing of electric power data according to claim 1, characterized in that: Also includes: Scoring the arc data marking model according to the marking results, wherein the scoring indicators include accuracy and precision; The arc data marking model is optimized according to the scoring results.

6. The method for automatic preprocessing of electric power data according to claim 5, characterized in that: Also includes: A performance report is generated according to the labeling results and the scoring results.

7. An automatic preprocessing system for power data, characterized in that: include: A preprocessing module, used for acquiring reference arc data and preprocessing the reference arc data; A feature extraction module, used to extract initial arc features from the preprocessed reference arc data, and select target arc features from the initial arc features; A data set construction module, used for constructing an arc data set according to the target arc characteristics; A model training module, used for training a pre-built arc data labeling model by a transfer learning method according to the arc data set; The intelligent labeling module is used to obtain target arc data and intelligently label the target arc data through a trained arc data labeling model.

8. The automatic preprocessing system for electric power data according to claim 7, characterized in that: Also includes: The evaluation and optimization module is used to score the arc data labeling model according to the labeling results, and optimize the arc data labeling model according to the scoring results. The scoring indicators include accuracy and precision.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.