System and method for classifying and sorting electric power data

Abstract

The invention provides a system and method for power data classification and arrangement, and relates to the technical field of power data management, and the method comprises the steps that a semantic fusion module integrates multi-source power archive data, carries out semantic matching and fusion, and generates a to-be-classified data set; the entity recognition module extracts classified data from the database and recognizes key entities by using a text recognition model; the incidence relation obtaining module identifies and corrects the incidence relation between the entities; the knowledge graph construction module constructs an electric power archive knowledge graph based on the entity and the association relationship; the classification model acquisition module trains an intelligent classification model, and the classification and arrangement module performs classification and arrangement on the target data set. According to the method and the device, the problem of low efficiency of power archive classification due to inaccurate entity identification of multi-source data and poor accuracy of the association relationship in the prior art can be solved, the accuracy and efficiency of power archive data classification are improved, and the automation level of power archive classification is improved.

Description

Technical Field

[0001] The present application relates to the technical field of power data management, and in particular to a system and method for classifying and arranging power data. Background Art

[0002] With the development of smart grid and Internet of Things technologies, the power system will generate massive amounts of data during operation, which needs to be effectively stored, processed and analyzed. At present, the existing power archive management system mainly relies on traditional manual classification and retrieval methods, which is incapable of processing massive, multi-source power archive data and is inefficient. At the same time, due to the lack of effective data fusion and standardization methods, there are large differences and inconsistencies between power archive data from different sources.

[0003] In summary, the existing technology has inaccurate entity recognition of multi-source data and poor accuracy of association relationships, resulting in low efficiency in the classification and organization of power archives. Summary of the invention

[0004] The purpose of this application is to provide a system and method for classifying and organizing power data, so as to solve the problem that the prior art has inaccurate entity recognition of multi-source data and poor accuracy of association relationships, resulting in low efficiency in classifying and organizing power archives.

[0005] In view of the above problems, the present application provides a system and method for classifying and organizing power data.

[0006] In a first aspect, the present application provides a system for classifying and arranging power data, the system for classifying and arranging power data includes: a semantic fusion module, which is used to collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion based on the semantic matching results, and generate a target power archive data set to be classified; an entity recognition module, which is used to extract a classified power archive data set from a power data archive data system, perform entity recognition on the classified power archive data set based on a predetermined classification entity through an archive text recognition model, and extract multiple historical target entities, wherein the predetermined classification entity includes a grid-connected dispatching agreement, a power business license, and a project approval, and the multiple historical target entities include a historical grid-connected dispatching agreement, a historical power business license , historical project approvals; an association relationship acquisition module, used to identify a first association relationship set between the historical grid-connected dispatching agreement, the historical electricity business license, and the historical project approvals, perform an accuracy assessment on the first association relationship set, and modify the first association relationship set according to the assessment result to generate a historical target association relationship set; a knowledge graph construction module, used to construct an electric power archive knowledge graph according to the historical grid-connected dispatching agreement, the historical electricity business license, the historical project approvals, and the historical target association relationship set; a classification model acquisition module, used to extract classification features of the electric power archive knowledge graph and train an intelligent classification model for electric power archive data; a classification and sorting module, used to classify and sort the target electric power archive data set to be classified according to the intelligent classification model for electric power archive data.

[0007] In a second aspect, the present application provides a method for classifying and arranging power data, which is implemented by a system for classifying and arranging power data, wherein the method for classifying and arranging power data includes: collecting multi-source power archive data, performing semantic matching on the multi-source power archive data, performing semantic fusion based on the semantic matching results, and generating a target power archive data set to be classified; extracting a classified power archive data set from a power data archive data system, performing entity recognition on the classified power archive data set based on a predetermined classification entity through an archive text recognition model, and extracting multiple historical target entities, wherein the predetermined classification entity includes a grid-connected dispatching agreement, a power business license, a project approval, and a power supply license. The method comprises the steps of: identifying a first association relationship set among the historical grid-connected dispatching agreement, the historical electricity business license, and the historical project approval, evaluating the accuracy of the first association relationship set, and revising the first association relationship set according to the evaluation result to generate a historical target association relationship set; constructing an electric power archive knowledge graph according to the historical grid-connected dispatching agreement, the historical electricity business license, the historical project approval, and the historical target association relationship set; extracting classification features of the electric power archive knowledge graph, and training an intelligent classification model for electric power archive data; and classifying and arranging the target electric power archive data set to be classified according to the intelligent classification model for electric power archive data.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] A semantic fusion module is used to collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion based on the semantic matching results, and generate a target power archive data set to be classified; an entity recognition module is used to extract a classified power archive data set from a power data archive data system, perform entity recognition on the classified power archive data set based on a predetermined classification entity through an archive text recognition model, and extract multiple historical target entities, wherein the predetermined classification entity includes a grid-connected dispatching agreement, a power business license, and a project approval, and the multiple historical target entities include a historical grid-connected dispatching agreement, a historical power business license, and a historical project approval; an association relationship acquisition module is used to identify a first association relationship set between the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, and to identify the first association relationship set. The invention relates to a power archive data set, a power archive data set and a power archive data set. The power archive data set is used for performing accuracy evaluation on the power archive data set, and the first association relationship set is modified according to the evaluation result to generate a historical target association relationship set; a knowledge graph construction module is used to construct a power archive knowledge graph according to the historical grid-connected dispatching agreement, the historical power business license, the historical project approval and the historical target association relationship set; a classification model acquisition module is used to extract the classification features of the power archive knowledge graph and train the power archive data intelligent classification model; a classification and sorting module is used to classify and sort the target power archive data set to be classified according to the power archive data intelligent classification model, which effectively solves the problem of inefficient power archive classification and sorting due to inaccurate entity recognition of multi-source data and poor accuracy of association relationships in the prior art, improves the accuracy and efficiency of power archive data classification, and improves the automation level of power archive classification and sorting.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 This is a schematic diagram of the structure of a system for classifying and arranging power data in this application;

[0013] Figure 2 A flowchart of a method for classifying and arranging power data according to the present application is provided.

[0014] Description of reference numerals:

[0015] Semantic fusion module 11, entity recognition module 12, association relationship acquisition module 13, knowledge graph construction module 14, classification model acquisition module 15, classification sorting module 16. DETAILED DESCRIPTION

[0016] The present application provides a system and method for classifying and organizing power data, thereby solving the problem in the prior art that the classification and organization of power archives is inefficient due to inaccurate entity recognition of multi-source data and poor accuracy of association relationships, improving the accuracy and efficiency of power archive data classification, and improving the level of automation of power archive classification and organization.

[0017] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0018] Embodiment 1: This application provides a system for classifying and arranging power data. Figure 1 , the system for classifying and arranging power data comprises:

[0019] The semantic fusion module 11 is used to collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion according to the semantic matching results, and generate a target power archive data set to be classified.

[0020] Specifically, determine the data source to be collected, including but not limited to the archive management system within the power company, the public data of the data sharing platform, etc. For structured data, such as data tables in the database, you can use database query statements, such as SQL to directly extract. For unstructured or semi-structured data, such as text files, PDF documents, pictures, etc., use crawler technology, OCR recognition technology or document parsing tools to extract. Clean the collected data, remove duplicate, erroneous or irrelevant data, standardize the data, including unifying data formats, encodings, units, etc., and use natural language processing technology to process text data, such as word segmentation, part-of-speech tagging, named entity recognition, etc. Apply semantic similarity algorithms, such as cosine similarity, Jaccard similarity, and similarity calculation based on word vectors, to evaluate the semantic similarity between data items in different data sources. Cross-validate the matching results to ensure the accuracy of the matching, and adjust the semantic matching strategy or parameters based on the evaluation results to optimize the matching effect. Formulate appropriate fusion strategies based on the matching results and data characteristics. For example, for highly similar data items, you can choose to merge; for data items with differences, make decisions based on specific business rules. Use data fusion tools to implement the actual data fusion operation, including steps such as merging similar data items, handling data conflicts, and filling missing values. Perform quality checks on the fused data, including verification of completeness, consistency, accuracy, etc. Based on the fused data, construct the target power archive data set to be classified. This data set contains all power archive data items that need to be classified, and each data item should be accurate, complete, and consistent.

[0021] The entity recognition module 12 is used to extract a classified power archive data set from the power data archive data system, perform entity recognition on the classified power archive data set based on predetermined classification entities through an archive text recognition model, and extract multiple historical target entities, wherein the predetermined classification entities include grid-connected dispatching agreements, power business licenses, and project approvals, and the multiple historical target entities include historical grid-connected dispatching agreements, historical power business licenses, and historical project approvals.

[0022] Specifically, a classified power archive data set is extracted from the power data archive storage database. The classified power archive data set has been classified and stored according to certain rules or standards. Next, the extracted data set is subjected to entity recognition using the archive text recognition model, and the model has the ability to recognize predetermined classified entities. In this process, the predetermined classified entity refers to the specific information that needs to be extracted from the power archive data, such as grid-connected dispatching agreements, power business licenses, project approvals, etc. Through the processing of the archive text recognition model, multiple historical target entities can be extracted from the power archive data. Using SQL query statements or database access interfaces, the classified power archive data set is extracted from the power data archive storage database. The extracted data set is subjected to text preprocessing, including data cleaning, removal of irrelevant information, and unified text format. The labeled data set is used for training, and the model is configured to recognize predetermined classified entities such as historical grid-connected dispatching agreements, historical power business licenses, and historical project approvals. The preprocessed text data is input into the archive text recognition model for entity recognition. The model outputs the recognized entities and their corresponding text fragments. Multiple historical target entities are extracted from the output of the model, including historical grid-connected dispatching agreements, historical power business licenses, and historical project approvals.

[0023] The association relationship acquisition module 13 is used to identify the first association relationship set between the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, perform accuracy evaluation on the first association relationship set, modify the first association relationship set according to the evaluation result, and generate a historical target association relationship set.

[0024] Specifically, text mining or natural language processing technology is used to extract the associations between historical grid-connected dispatching agreements, historical power business licenses, and historical project approvals from power archive data, through methods such as keyword matching, dependency syntax analysis, and semantic role labeling. The extracted associations are organized into a set, namely the first association set, which contains the associations between all identified entities. The first association set is evaluated based on a machine learning model. The evaluation is to determine the reliability and accuracy of the associations. According to the evaluation results, errors or inconsistencies in the first association set are identified, and the erroneous or inconsistent associations are corrected to ensure their accuracy and consistency, including deleting erroneous associations, adding missing associations, modifying inaccurate associations, etc. The corrected associations are integrated into a new set, that is, a historical target association set is generated.

[0025] The knowledge graph construction module 14 is used to construct a power archive knowledge graph based on the historical grid-connected dispatching agreement, the historical power business license, the historical project approval and the historical target association relationship set.

[0026] Specifically, ensure that historical grid-connected dispatching agreements, historical power business licenses, historical project approvals, etc. are correctly identified and extracted, and entities are deduplicated and standardized to ensure the uniqueness of each entity in the knowledge graph. Sort out the historical target association set, ensure that all associations are accurate and consistent, and classify and standardize the associations. In the knowledge graph, each historical target entity will be represented as a node, and the associations between entities will be represented as edges between nodes. Design attributes for each node, such as the name, type, description, etc. of the entity, and design attributes for each edge, such as the type, strength, direction, etc. of the association. Create nodes in the knowledge graph database, assign a unique identifier to each node, and add the attribute information of the entity to the corresponding node. Create edges in the knowledge graph based on the historical target association set, connect related nodes, add attribute information to each edge, describe the relationship between nodes, and perform data verification on the constructed knowledge graph to ensure the correctness of nodes and edges.

[0027] The classification model acquisition module 15 is used to extract the classification features of the power archive knowledge graph and train the power archive data intelligent classification model.

[0028] Specifically, extract features from the nodes of the power archive knowledge graph, such as the type, name, description, etc. of the entity. Extract the features of the edges in the knowledge graph, such as the type, strength, direction, etc. of the association relationship, to represent the relationship and connectivity between nodes. Extract the graph structure features of the knowledge graph, such as the neighbors, paths, subgraphs, etc. of the nodes, which can provide information about the position and importance of the nodes in the graph. Encode the extracted features, and the features can be encoded using methods such as one-hot encoding and embedding representation. Prepare a labeled power archive dataset, which contains classified power archive samples, and each sample has a corresponding label to indicate the category to which it belongs. Based on the extracted features, construct a feature matrix, in which each row represents a sample and each column represents a feature. Divide the labeled dataset into a training set, a validation set, and a test set. Select a machine learning model suitable for the classification of power archive data, such as a support vector machine, a random forest, a deep learning model, etc. Use the training set to train the selected model, adjust the model parameters to optimize the classification performance, use the validation set to validate the trained model, evaluate its classification accuracy and generalization ability, and tune the model based on the validation results, such as adjusting hyperparameters and optimizing feature selection. The classification features of the power archive knowledge graph can be extracted, and an intelligent classification model for power archive data can be trained. This model will be able to automatically classify new power archive data and improve the efficiency and accuracy of power archive management.

[0029] The classification and sorting module 16 is used to classify and sort the target power archive data set to be classified according to the power archive data intelligent classification model.

[0030] Specifically, the target power archive data set to be classified is cleaned to remove irrelevant information, fill missing values, correct erroneous data, etc., and the trained power archive data intelligent classification model is loaded, and the processed power archive data set to be classified is input into the model. The model classifies and predicts the input power archive data and assigns a category label to each sample. The classification result of the model is output to obtain a classification list containing sample IDs and corresponding category labels. According to the classification results, the power archive data set to be classified is sorted into subsets of different categories.

[0031] Furthermore, the entity identification module 12 is also used for:

[0032] According to the grid-connected dispatching agreement, the power business license, and the project approval, the classified power archive data set is randomly labeled to obtain an unlabeled power archive data set and a labeled power archive data set; based on self-training and generative adversarial networks, the archive text recognition model is trained based on the unlabeled power archive data set and the labeled power archive data set.

[0033] Specifically, a part of samples are randomly selected from the classified power archive dataset as the annotation objects. The selected samples are carefully annotated, including text content, key information, category, etc., to generate the annotated power archive dataset, and the remaining unannotated samples are used as the unannotated power archive dataset. The annotated and unannotated power archive datasets are preprocessed, such as text cleaning, word segmentation, and stop word removal. Text features, such as bag-of-words model, TF-IDF, word embedding, etc., are extracted for model training. The training framework of the archive text recognition model is constructed by combining self-training and generative adversarial networks. An initial archive text recognition model is trained using the annotated power archive dataset, and the initial model is used to predict the unlabeled power archive dataset to obtain pseudo-annotations. The pseudo-annotated samples with high confidence are added to the annotated dataset, and the model is retrained using the updated annotated dataset, and this process is iterated. The generator and discriminator are constructed. The generator generates pseudo samples, and the discriminator distinguishes between real samples and pseudo samples. Through adversarial training, the generalization ability and recognition accuracy of the model are improved, and the model is tuned according to the performance of the validation set, such as adjusting hyperparameters and optimizing model structure.

[0034] Furthermore, the entity identification module 12 is also used for:

[0035] A pre-trained text recognition model is loaded, and supervised learning is performed on the annotated power archive data set to obtain an initial text recognition model; the initial text recognition model is used to predict the unlabeled power archive data set to generate pseudo labels, and the pseudo labels are screened to obtain target pseudo labels; the target pseudo labels and the annotated power archive data set are merged to obtain a target training data set, and the text recognition model is trained to obtain the archive text recognition model.

[0036] Specifically, the pre-trained model is further trained using the annotated power archive dataset, and the model parameters are adjusted through supervised learning to make it more suitable for the recognition task of power archive text, thereby obtaining the initial text recognition model. The initial text recognition model is used to predict the unlabeled power archive dataset, and a predicted label, i.e., a pseudo-label, is assigned to each unlabeled sample. The quality of the generated pseudo-labels is evaluated, and pseudo-labels with high confidence and accurate prediction are selected as target pseudo-labels. The screened target pseudo-labels are merged with the original annotated power archive dataset to obtain a new, larger target training dataset. The initial text recognition model is further trained using the target training dataset, and the model parameters are iteratively optimized to improve the performance of the model on the power archive text recognition task, and finally the archive text recognition model is obtained.

[0037] Furthermore, the entity identification module 12 is also used for:

[0038] Construct a dual generator of target training data and a multi-level target training data discriminator, wherein the dual generator of target training data includes a first generator and a second generator, and the first generator and the second generator have the same structure; use the first generator to generate initial synthetic data, perform authenticity evaluation on the initial synthetic data, adjust parameters of the second generator according to the evaluation result to obtain a second optimized generator; optimize the initial synthetic data according to the second optimized generator to obtain target synthetic data; evaluate the target synthetic data through the multi-level target training data discriminator, and add the target synthetic data to the target training data set according to the evaluation result.

[0039] Specifically, the goal of the first generator is to generate initial synthetic data. New electric power archive text data similar to the annotated electric power archive data set can be generated based on the characteristics and distribution of the annotated electric power archive data set. The first generator and the second generator have the same structure, which is a multi-layer neural network that maps the input random noise to the data space through multi-layer nonlinear transformation. Using the first generator, a batch of initial synthetic data is generated based on the characteristics and distribution of the annotated electric power archive data set. Then the reward function is used to calculate the reward value of the initial synthetic data. With the goal of maximizing the reward, the gradient descent method is used for optimization, and the negative reward value is used as the loss function. The goal of the generator is to minimize the negative reward value, that is, to maximize the reward. The gradient of the loss function to the generator parameters is calculated by the back propagation algorithm. The gradient reflects the direction and degree of adjustment of the parameters. The parameters of the second generator are updated according to the calculated gradient to obtain the second optimized generator. The initial synthetic data is input into the second optimized generator for further optimization and adjustment to obtain higher quality target synthetic data. The discriminator is a multi-level network structure that can capture the complex characteristics of the electric power archive text data, including multiple convolutional layers, pooling layers, fully connected layers, etc., to achieve accurate evaluation of synthetic data. The discriminator is trained using the labeled power archive dataset as positive samples and the initial synthetic data as negative samples, so that the discriminator can accurately distinguish between real data and synthetic data. The target synthetic data is evaluated using the trained multi-level target training data discriminator, including the confidence of the discriminator output, the similarity between synthetic data and real data, etc. According to the evaluation results, the target synthetic data with higher quality is selected and added to the target training dataset, which can expand the scale of the training dataset and improve the generalization ability and performance of the archive text recognition model.

[0040] Furthermore, the association relationship acquisition module 13 is also used for:

[0041] Read a predetermined feature picking strategy, and based on the predetermined feature picking strategy, perform feature picking on the historical grid-connected dispatching agreement, historical electricity business license, and historical project approval to obtain historical agreement feature information, historical license feature information, and historical approval feature information, respectively; perform association analysis based on the historical agreement feature information, historical license feature information, and historical approval feature information to obtain the first association relationship; introduce an association relationship evaluation function to perform accuracy evaluation on the first association relationship to obtain a first accuracy; when the first accuracy does not meet the predetermined constraints, modify the first association relationship to obtain the historical target association relationship.

[0042] Specifically, for historical grid-connected dispatching agreements, key features such as agreement number, signing date, grid-connected capacity, dispatching method, etc. are extracted according to the feature picking strategy to obtain historical agreement feature information. For historical power business licenses, features such as license number, issuing authority, validity period, business scope, etc. are also extracted according to the strategy to obtain historical license feature information. For historical project approvals, features such as approval number, approval date, project name, investment scale, etc. are extracted to obtain historical approval feature information. The extracted historical agreement feature information, historical license feature information, and historical approval feature information are analyzed for correlation. Find out the inherent connections and rules between these feature information. For example, it can be analyzed whether a certain grid-connected dispatching agreement corresponds to a specific power business license, or whether a project approval is associated with a specific grid-connected dispatching agreement. An association relationship evaluation function is introduced to evaluate the accuracy of the first association relationship obtained by the analysis. This evaluation function is based on a machine learning model or a statistical method. The purpose of the evaluation is to determine the reliability and accuracy of the association relationship. If the evaluation result shows that the accuracy of the first association relationship does not meet the predetermined constraints, for example, the accuracy is lower than the preset accuracy threshold, which is set by a person skilled in the art, the association relationship is corrected, including re-analyzing the feature information, adjusting the association rules, introducing more feature information, etc. The corrected association relationship is called the historical target association relationship.

[0043] Furthermore, the association relationship acquisition module 13 is also used for:

[0044] The historical protocol feature information, historical license feature information, and historical approval feature information are converted into an association analysis data set, wherein each data in the association analysis data set represents a feature information; the association analysis data set is traversed to calculate the occurrence frequency of the first data, and when the occurrence frequency is greater than a preset minimum frequency threshold, the first data is added into a descending frequent association data set; the association analysis data set is sorted according to the descending frequent association data set to obtain a descending association analysis data set; a frequent association tree is constructed according to the descending association analysis data set, and a frequent head node is constructed at the same time; data is randomly extracted from the descending association analysis data set, and added into the frequent head node; the frequent association tree is mined according to the frequent head node to generate multiple conditional association pattern bases; multiple conditional frequent association trees are constructed according to the multiple conditional association pattern bases, and the multiple conditional frequent association trees are recursively mined respectively to generate the first association relationship set.

[0045] Specifically, the historical protocol feature information, historical license feature information, and historical approval feature information are converted into an association analysis data set. In this data set, each data item represents a feature information, such as an agreement number, a license issuing authority, or a project approval date. The association analysis data set is traversed to calculate the frequency of occurrence of each feature information. When the frequency of occurrence of a feature information is greater than the preset minimum frequency threshold, the feature information is added to the descending frequent association data set, and the association analysis data set is sorted according to the descending frequent association data set to obtain the descending association analysis data set. A frequent association tree is constructed according to the descending association analysis data set. This tree will be used to store the association relationship between feature information, and at the same time, frequent head nodes will be constructed, which will serve as the starting point for mining association patterns. Data are randomly extracted from the descending association analysis data set and added to the frequent head nodes. The frequent association tree is mined according to the frequent head nodes to generate multiple conditional association pattern bases, which represent the potential association relationship between feature information. Each conditional association pattern base is further analyzed and processed to extract useful association rules. Multiple conditional frequent association trees are constructed according to multiple association pattern bases to store more specific association relationships. Recursively mine each conditional frequent association tree to generate a first association relationship set, which includes all useful association relationships mined from historical protocol feature information, historical license feature information, and historical approval feature information. Finally, the first association relationship set is output as the final result.

[0046] Furthermore, the classification and sorting module 16 is also used for:

[0047] Collect user feedback data in real time; identify misclassified archive data based on the user feedback data, and calculate the error frequency of the misclassified archive data; and optimize the intelligent classification model of power archive data based on the error frequency adjustment.

[0048] Specifically, user feedback on the classification results of power archival data is collected in real time. The collected user feedback data is analyzed in depth to identify the misclassified archival data by comparing user feedback with the model classification results. The error frequency of each misclassified archival data is calculated, that is, the ratio of the number of times the archival data is misclassified to the total number of classifications. According to the error frequency of the misclassified archival data, the weight of the intelligent classification model of power archival data is adjusted. The weight of the misclassified archival data in the model training is increased to reduce the possibility of its misclassification. The model is retrained and the classification performance of the model is improved using an extended data set containing misclassified archival data. This can be achieved through incremental learning or online learning techniques in machine learning algorithms. The performance of the optimized model is verified. An independent test data set is used to evaluate the classification accuracy of the model to ensure that the optimization measures have effectively improved the performance of the model.

[0049] For Example 2, please refer to the attached Figure 2 The present application provides a method for classifying and arranging power data, wherein the method for classifying and arranging power data is applied to a system for classifying and arranging power data, and the method for classifying and arranging power data specifically includes the following steps:

[0050] S1: Collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion based on the semantic matching results, and generate a target power archive data set to be classified.

[0051] S2: extracting a classified power archive data set from a power data archive system, performing entity recognition on the classified power archive data set based on predetermined classification entities through an archive text recognition model, and extracting multiple historical target entities, wherein the predetermined classification entities include a grid-connected dispatching agreement, a power business license, and a project approval, and the multiple historical target entities include a historical grid-connected dispatching agreement, a historical power business license, and a historical project approval.

[0052] S3: Identify a first association relationship set among the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, evaluate the accuracy of the first association relationship set, modify the first association relationship set according to the evaluation result, and generate a historical target association relationship set.

[0053] S4: Constructing a power archive knowledge graph based on the historical grid-connected dispatching agreement, the historical power business license, the historical project approval and the historical target association relationship set.

[0054] S5: Extract the classification features of the power archive knowledge graph and train the power archive data intelligent classification model.

[0055] S6: Classify and sort the target power archive data set to be classified according to the power archive data intelligent classification model.

[0056] Furthermore, step S2 of the present application also includes:

[0057] According to the grid-connected dispatching agreement, the power business license, and the project approval, the classified power archive data set is randomly labeled to obtain an unlabeled power archive data set and a labeled power archive data set; based on self-training and generative adversarial networks, the archive text recognition model is trained based on the unlabeled power archive data set and the labeled power archive data set.

[0058] Furthermore, step S2 of the present application also includes:

[0059] A pre-trained text recognition model is loaded, and supervised learning is performed on the annotated power archive data set to obtain an initial text recognition model; the initial text recognition model is used to predict the unlabeled power archive data set to generate pseudo labels, and the pseudo labels are screened to obtain target pseudo labels; the target pseudo labels and the annotated power archive data set are merged to obtain a target training data set, and the text recognition model is trained to obtain the archive text recognition model.

[0060] Furthermore, step S2 of the present application also includes:

[0061] Construct a dual generator of target training data and a multi-level target training data discriminator, wherein the dual generator of target training data includes a first generator and a second generator, and the first generator and the second generator have the same structure; use the first generator to generate initial synthetic data, perform authenticity evaluation on the initial synthetic data, adjust parameters of the second generator according to the evaluation result to obtain a second optimized generator; optimize the initial synthetic data according to the second optimized generator to obtain target synthetic data; evaluate the target synthetic data through the multi-level target training data discriminator, and add the target synthetic data to the target training data set according to the evaluation result.

[0062] Furthermore, step S3 of the present application also includes:

[0063] Read a predetermined feature picking strategy, and perform feature picking on the multiple historical target entities based on the predetermined feature picking strategy, the multiple target entities including historical grid-connected dispatching agreements, historical electricity business licenses, and historical project approvals, and obtain historical agreement feature information, historical license feature information, and historical approval feature information, respectively; perform association analysis based on the historical agreement feature information, historical license feature information, and historical approval feature information to obtain the first association relationship; introduce an association relationship evaluation function to perform accuracy evaluation on the first association relationship to obtain a first accuracy; when the first accuracy does not meet the predetermined constraints, modify the first association relationship to obtain the historical target association relationship.

[0064] Furthermore, step S3 of the present application also includes:

[0065] The historical protocol feature information, historical license feature information, and historical approval feature information are converted into an association analysis data set, wherein each data in the association analysis data set represents a feature information; the association analysis data set is traversed to calculate the occurrence frequency of the first data, and when the occurrence frequency is greater than a preset minimum frequency threshold, the first data is added into a descending frequent association data set; the association analysis data set is sorted according to the descending frequent association data set to obtain a descending association analysis data set; a frequent association tree is constructed according to the descending association analysis data set, and a frequent head node is constructed at the same time; data is randomly extracted from the descending association analysis data set, and added into the frequent head node; the frequent association tree is mined according to the frequent head node to generate multiple conditional association pattern bases; multiple conditional frequent association trees are constructed according to the multiple conditional association pattern bases, and the multiple conditional frequent association trees are recursively mined respectively to generate the first association relationship set.

[0066] Furthermore, step S6 of the present application also includes:

[0067] Collect user feedback data in real time; identify misclassified archive data based on the user feedback data, and calculate the error frequency of the misclassified archive data; and optimize the intelligent classification model of power archive data based on the error frequency adjustment.

[0068] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0069] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A system for classifying and arranging power data, characterized in that: include: A semantic fusion module is used to collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion based on the semantic matching results, and generate a target power archive data set to be classified; An entity recognition module is used to extract a classified power archive data set from a power data archive data system, perform entity recognition on the classified power archive data set based on a predetermined classification entity through an archive text recognition model, and extract multiple historical target entities, wherein the predetermined classification entity includes a grid-connected dispatching agreement, a power business license, and a project approval, and the multiple historical target entities include a historical grid-connected dispatching agreement, a historical power business license, and a historical project approval; An association relationship acquisition module is used to identify a first association relationship set between the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, perform an accuracy assessment on the first association relationship set, and modify the first association relationship set according to the assessment result to generate a historical target association relationship set; A knowledge graph construction module, used to construct a power archive knowledge graph based on the historical grid-connected dispatching agreement, the historical power business license, the historical project approval and the historical target association relationship set; A classification model acquisition module, used to extract classification features of the power archive knowledge graph and train an intelligent classification model for power archive data; The classification and sorting module is used to classify and sort the target power archive data set to be classified according to the power archive data intelligent classification model.

2. A system for classifying and arranging power data according to claim 1, characterized in that: The entity recognition module is also used for: Randomly label the classified power archive data set according to the grid-connected dispatching agreement, the power business license, and the project approval to obtain an unlabeled power archive data set and a labeled power archive data set; Based on self-training and generative adversarial networks, the archive text recognition model is obtained by training according to the unlabeled power archive data set and the labeled power archive data set.

3. A system for classifying and arranging power data according to claim 2, characterized in that: The entity recognition module is also used for: Loading a pre-trained text recognition model, performing supervised learning on the annotated power archive data set, and obtaining an initial text recognition model; Use the initial text recognition model to predict the unlabeled power archive data set, generate pseudo labels, and filter the pseudo labels to obtain target pseudo labels; The target pseudo-label and the annotated power archive data set are merged to obtain a target training data set, and the text recognition model is trained to obtain the archive text recognition model.

4. A system for classifying and arranging power data according to claim 3, characterized in that: The entity recognition module is also used for: Constructing a dual generator of target training data and a multi-level target training data discriminator, wherein the dual generator of target training data includes a first generator and a second generator, and the first generator and the second generator have the same structure; Use the first generator to generate initial synthetic data, perform authenticity evaluation on the initial synthetic data, and adjust parameters of the second generator according to the evaluation result to obtain a second optimized generator; Optimizing the initial synthetic data according to the second optimization generator to obtain target synthetic data; The target synthetic data is evaluated by the multi-level target training data discriminator, and the target synthetic data is added to the target training data set according to the evaluation result.

5. A system for classifying and arranging power data as claimed in claim 1, characterized in that: The association relationship acquisition module is also used for: Reading a predetermined feature picking strategy, and based on the predetermined feature picking strategy, performing feature picking on the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, to obtain historical agreement feature information, historical license feature information, and historical approval feature information, respectively; Perform association analysis based on the historical agreement feature information, the historical license feature information, and the historical approval feature information to obtain the first association relationship; Introducing an association relationship evaluation function to evaluate the accuracy of the first association relationship to obtain a first accuracy; When the first accuracy does not meet the predetermined constraint, the first association relationship is modified to obtain the historical target association relationship.

6. A system for classifying and arranging power data as claimed in claim 5, characterized in that: The association relationship acquisition module is also used for: Converting the historical agreement feature information, the historical license feature information, and the historical approval feature information into a correlation analysis data set, wherein each data in the correlation analysis data set represents a feature information; Traversing the association analysis data set, calculating the occurrence frequency of the first data, and when the occurrence frequency is greater than a preset minimum frequency threshold, adding the first data into the descending frequent association data set; Sorting the association analysis data set according to the descending frequently associated data set to obtain a descending association analysis data set; Constructing a frequent association tree according to the descending association analysis data set, and constructing a frequent head node at the same time; Randomly extract data from the descending association analysis data set and add it to the frequent head node; Mining the frequent association tree according to the frequent head nodes to generate multiple conditional association pattern bases; A plurality of conditional frequent association trees are constructed based on the plurality of conditional association pattern bases, and the plurality of conditional frequent association trees are respectively mined recursively to generate the first association relationship set.

7. A system for classifying and arranging power data as claimed in claim 1, characterized in that: The classification and sorting module is also used for: Collect user feedback data in real time; According to the user feedback data, identifying misclassified archive data, and calculating the error frequency of the misclassified archive data; The intelligent classification model of the power archive data is optimized according to the error frequency adjustment.

8. A method for classifying and arranging power data, characterized in that: The method for classifying and arranging power data is implemented by a system for classifying and arranging power data according to any one of claims 1 to 7, wherein the method for classifying and arranging power data comprises: Collect multi-source power archive data, perform semantic matching on the multi-source power archive data, perform semantic fusion according to the semantic matching results, and generate a target power archive data set to be classified; Extracting a classified power archive data set from a power data archive data system, performing entity recognition on the classified power archive data set based on a predetermined classification entity through an archive text recognition model, and extracting a plurality of historical target entities, wherein the predetermined classification entity includes a grid-connected dispatching agreement, a power business license, and a project approval, and the plurality of historical target entities include a historical grid-connected dispatching agreement, a historical power business license, and a historical project approval; Identify a first association relationship set between the historical grid-connected dispatching agreement, the historical power business license, and the historical project approval, evaluate the accuracy of the first association relationship set, modify the first association relationship set according to the evaluation result, and generate a historical target association relationship set; Constructing a power archive knowledge graph based on the historical grid-connected dispatching agreement, the historical power business license, the historical project approval and the historical target association relationship set; Extracting classification features of the power archive knowledge graph and training an intelligent classification model for power archive data; The target power archive data set to be classified is classified and sorted according to the power archive data intelligent classification model.

Images