Power grid data processing method and device

Abstract

The invention discloses a power grid data processing method and device, and relates to the field of power grids. The method comprises the steps that power grid data collected from different data sources are classified, original power grid data are obtained, and the types of the original power grid data comprise character type data, image type data and digital type data; screening the original power grid data through a quality scoring method to obtain data after primary screening; based on the feature vector of the data after primary screening, screening the data after primary screening again to obtain data after secondary screening; and updating a power grid data knowledge graph stored in a graph database based on the data after secondary screening, the power grid data knowledge graph being used for realizing distribution network scheduling. The technical problem that the updating accuracy of the power grid knowledge graph is low is solved.

Description

Technical Field

[0001] The present invention relates to the field of power grids, and in particular to a method and device for processing power grid data. Background Art

[0002] With the development of smart grid technology, the power system is becoming more and more complex, including not only traditional power plants, substations and other infrastructure, but also distributed energy such as solar and wind energy, electric vehicle charging stations and other new elements. In the face of such a dynamic and complex environment, the power grid knowledge graph can be used to store and manage power grid data related to distribution network scheduling.

[0003] The power grid knowledge graph needs to update the stored power grid data as the power grid operates, but the sources of power grid data are diverse, and the data formats, collection methods and accuracy vary. There is a large amount of noise and inconsistent information, which will make the update accuracy of the power grid knowledge graph low.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for processing power grid data, so as to at least solve the technical problem of low updating accuracy of the power grid knowledge graph.

[0006] According to one aspect of an embodiment of the present invention, a method for processing power grid data is provided, comprising: classifying power grid data collected from different data sources to obtain original power grid data, wherein the original power grid data comprises: text type data, image type data and digital type data; filtering the original power grid data by a quality scoring method to obtain primary filtered data; based on a feature vector of the primary filtered data, filtering the primary filtered data again to obtain secondary filtered data; and updating a power grid data knowledge graph stored in a graph database based on the secondary filtered data, wherein the power grid data knowledge graph is used for distribution network scheduling.

[0007] In one embodiment of the present application, based on the feature vector of the data after the first screening, the data after the first screening is screened again to obtain the data after the second screening, including: extracting features from the data after the first screening to obtain the feature vector; determining the Euclidean norm of the data after the first screening based on the feature vector; determining the first comprehensive threshold of the data after the first screening based on the Euclidean norm; and screening the data after the first screening according to the comparison result between the Euclidean norm and the first comprehensive threshold to obtain the data after the second screening.

[0008] In one embodiment of the present application, in response to the type of the original power grid data being text type data, feature extraction is performed on the data after a filter to obtain a feature vector of the data after a filter, including: feature extraction is performed on the data after a filter to obtain a word frequency and an inverse document frequency of the data after a filter; a feature matrix is ​​constructed based on the word frequency and the inverse document frequency; and the feature matrix is ​​converted into a feature vector.

[0009] In one embodiment of the present application, when the type of the original power grid data is image type data, feature extraction is performed on the data after a screening to obtain a feature vector of the data after a screening, including: feature extraction is performed on the data after a screening to obtain key visual features, the key visual features include at least one of: key points in the image, descriptors in the image, and histograms of gradient directions in the image; an image feature space is constructed based on the key visual features; and the feature values ​​in the image feature space are combined to obtain a feature vector.

[0010] In one embodiment of the present application, when the type of the original power grid data is digital type data, feature extraction is performed on the data after a screening to obtain a feature vector of the data after a screening, including: feature extraction is performed on the data after a screening to obtain key data features, the key data features include statistical features and correlation indicators; a data feature list is constructed based on the key data features; and the feature values ​​in the data feature list are combined to obtain a feature vector.

[0011] In one embodiment of the present application, determining a first comprehensive threshold of the data after a single screening based on the Euclidean norm includes: determining a first weight of each data after a single screening in the data after a single screening; determining the mean of the Euclidean norm of the data after a single screening as the first threshold; and determining a first comprehensive threshold based on the first weight and the first threshold.

[0012] In one embodiment of the present application, the data after the first screening is screened according to the comparison result of the Euclidean norm and the first comprehensive threshold value to obtain the data after the second screening, including: when the Euclidean norm is greater than or equal to the first comprehensive threshold value, determining the data after the first screening corresponding to the Euclidean norm as the data after the second screening.

[0013] In one embodiment of the present application, the original power grid data is screened once by a quality scoring method to obtain the data after the first screening, including: performing a preprocessing operation on the original power grid data to obtain the data after the preprocessing operation; performing a quality scoring on the data after the preprocessing operation to obtain an evaluation score of the data after the preprocessing operation; when the evaluation score is greater than a preset score, determining that the data after the preprocessing operation is the data after the first screening.

[0014] In one embodiment of the present application, the method further includes: updating the power grid data knowledge graph based on the data after secondary screening to obtain an updated power grid data knowledge graph; and storing the updated power grid data knowledge graph in a graph database.

[0015] According to another aspect of an embodiment of the present invention, a power grid data processing device is also provided, including: a classification module, used to classify the power grid data collected from different data sources to obtain original power grid data, wherein the types of original power grid data include: text type data, image type data and digital type data; a first screening module, used to screen the original power grid data by a quality scoring method to obtain data after a first screening; a second screening module, used to screen the data after a first screening again based on the feature vector of the data after a first screening to obtain data after a second screening; an updating module, used to update the power grid data knowledge graph stored in the graph database based on the data after the second screening, wherein the power grid data knowledge graph is used to realize distribution network scheduling.

[0016] In an embodiment of the present invention, the power grid data collected from different data sources are classified to obtain original power grid data; the original power grid data is screened by a quality scoring method to obtain data after a primary screening; based on the feature vector of the data after the primary screening, the data after the primary screening is screened again to obtain data after a secondary screening; based on the data after the secondary screening, the power grid data knowledge graph stored in the graph database is updated, wherein the power grid data knowledge graph is used to implement distribution network scheduling. It is easy to notice that the original power grid data from multiple data sources is screened once based on the quality score and once based on the feature vector, which improves the quality of the data after the secondary screening, and the power grid data knowledge graph stored in the graph database is updated based on the data after the secondary screening, achieving the technical effect of improving the update accuracy of the power grid knowledge graph, thereby solving the technical problem of low update accuracy of the power grid knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flow chart of a method for processing power grid data according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of a device for processing power grid data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Example 1

[0023] According to an embodiment of the present invention, an embodiment of a method for processing power grid data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of a method for processing power grid data according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0025] Step S102 , classifying the power grid data collected from different data sources to obtain original power grid data, wherein the original power grid data includes: text type data, image type data and digital type data.

[0026] The different data sources in the above steps include but are not limited to the internal system of the power company, external data providers, sensors and monitoring equipment, and public databases. Among them, the internal system of the power company may include but is not limited to: SCADA (Supervisory Control and Data Acquisition) system, GIS (Geographic Information System) system and maintenance management system, etc. The SCADA system can provide real-time data of power grid operation, such as voltage, current, power, etc. The GIS system can provide geographic location information, such as the location of the substation, line layout, etc. The maintenance management system can provide equipment maintenance records, fault reports, etc. External data providers may include but are not limited to: weather stations, social media platforms, news websites, etc. Weather stations can provide weather forecast data, such as temperature, humidity, wind speed, etc. Social media platforms can provide user feedback, power outage reports, etc. News websites can provide related event reports, policy changes, etc. Sensors and monitoring equipment may include but are not limited to: voltage sensors, current sensors, temperature sensors, etc. Voltage sensors can monitor voltage changes in the power grid, among which current sensors can monitor current changes in the power grid, and temperature sensors can monitor the operating temperature of the equipment. Public databases include but are not limited to: statistical data released by public platforms, open data platforms, etc. The statistical data released by public platforms can provide energy consumption statistics, population distribution, etc. The open data platform can be an OpenStreetMap (Open Street Map) platform that can provide geographic information data, etc.

[0027] The original power grid data in the above steps is classified data, including text type data, image type data and digital type data.

[0028] The text-type data in the above steps refers to power grid data containing text information, such as user feedback, operation logs, maintenance records, policy announcements, etc. This type of data usually exists in the form of natural language, and useful information can be extracted through text analysis and natural language processing technology.

[0029] The image type data in the above steps is power grid data containing graphic or picture information, which may include but is not limited to on-site photos of equipment status, map images in GIS systems, thermal maps under fault conditions, etc. Image data can be parsed and features extracted through image recognition and processing technology.

[0030] The digital type data in the above steps are power grid data with specific values, such as voltage, current, power, temperature, humidity, wind speed, longitude and latitude coordinates, time series data, etc.

[0031] In an optional embodiment, the power grid data collected from different data sources include but are not limited to the following: user feedback and complaint reports, including user text descriptions and opinions; news reports and announcements, including text information about power grid operations and policy changes; maintenance records and fault reports, including detailed text descriptions and processing procedures; on-site photos, such as photos of equipment status, photos of fault sites, etc.; map images, such as map images in GIS systems, layout images showing substations and lines, etc.; charts and graphs, such as trend charts, heat maps, etc.; power grid operation data, such as voltage, current, power and other values; weather data, such as temperature, humidity, wind speed, etc.; location data, such as longitude and latitude coordinates, etc.; time series data, such as hourly electricity consumption and power generation, etc.

[0032] The power grid data collected from different data sources are classified according to data characteristics, wherein the data characteristics may include structured, unstructured, semi-structured, and pixel-based. In the classification process, based on the unstructured or semi-structured data characteristics, the above-mentioned user feedback reports, news reports, maintenance records and other data are classified as text-type data, which contains information described in natural language. Based on the structured data characteristics, voltage, current, power, temperature, humidity, wind speed, longitude and latitude coordinates, time series and other data are classified as digital-type data, which can be mathematically operated and statistically analyzed. Based on the data characteristics in pixels, on-site photos, map images, charts and graphics and other data are classified as image-type data, which usually contains visual information. After the power grid data collected from different data sources are classified, the original power grid data containing text-type data, image-type data and digital-type data is obtained.

[0033] Step S104, screening the original power grid data by a quality scoring method to obtain screened data.

[0034] The data after the first screening in the above steps is a relatively accurate data set that meets the preliminary usability standards after preliminary screening through the quality scoring method from the original power grid data.

[0035] In an optional embodiment, a quality scoring index for evaluating data is preset. Then, for each piece of original power grid data, a score is given according to the designed quality scoring index. The scoring method for scoring the original power grid data can be to use a trained machine learning model to evaluate the performance of the data on the above indicators and give a comprehensive score value, but is not limited thereto.

[0036] Then, it is determined whether the original power grid data is the data after one screening according to the comprehensive score value of the original power grid data. For example, the average of the comprehensive score values ​​of all the original power grid data can be calculated, and the original power grid data corresponding to the comprehensive score value greater than the average can be determined as the data after one screening. It is also possible to pre-set a threshold value, and determine the original power grid data corresponding to the comprehensive score value greater than the threshold value as the data after one screening.

[0037] Step S106, based on the feature vector of the data after the first screening, the data after the first screening is screened again to obtain the data after the second screening.

[0038] The feature vector in the above steps is a vector used to characterize data features. For example, the feature vector of digital data in the data after a single screening can be represented by the statistical features of digital data, such as mean, standard deviation, maximum, minimum, etc. The feature vector of text data in the data after a single screening needs to be converted into numerical features, and the feature vector is constructed based on the frequency of occurrence of words. The feature vector of image data in the data after a single screening needs to identify the features of the image, and the feature vector is composed of the numerical description of these features.

[0039] In an optional embodiment, a feature extraction operation can be performed on the data after the first screening to obtain a feature vector. Then, the Euclidean norm of each feature vector is calculated, and the feature strength of the data after the first screening is characterized by the Euclidean norm. Then, based on the Euclidean norm distribution corresponding to the data after the first screening, a threshold value as a standard for secondary screening is set, wherein the threshold value can be the mean of the Euclidean norm or the median of the Euclidean norm, but is not limited thereto. After setting the threshold value, the Euclidean norm of each feature vector is compared with the set threshold value. If the Euclidean norm is higher than the threshold value, it means that the data after the first screening corresponding to the Euclidean norm contains more key information and has a higher feature strength. At this time, the data after the first screening corresponding to the Euclidean norm is retained, and the data after the first screening corresponding to the Euclidean norm is determined as the data after the second screening.

[0040] Step S108, based on the data after secondary screening, the power grid data knowledge graph stored in the graph database is updated, wherein the power grid data knowledge graph is used for distribution network scheduling.

[0041] The graph database in the above steps is a database used to store the knowledge graph of power grid data.

[0042] The power grid data knowledge graph in the above steps is a data model built in the form of a graph database, which is used to describe and store various entities in the power system and the relationships and attributes between entities. The core components of the power grid data knowledge graph include: entities, relationships, and attributes. Among them, entities represent various objects in the power grid, such as substations, lines, switches, users, weather conditions, etc. Attributes are specific information attached to entities or relationships to describe their status, characteristics, or behaviors. Each entity has specific attributes, such as location, voltage level, current, power, status, etc. Relationships represent connections or associations between entities, such as power supply relationships, geographical proximity, fault impact, maintenance records, etc. Relationships can be directed or undirected, depending on the nature of the relationship between entities.

[0043] In an optional embodiment, the data after the secondary screening is analyzed in detail to identify the entities and entity relationships contained in the data. If the data after the secondary screening contains new entities, such as new equipment, new users, etc., the corresponding nodes are added to the power grid data knowledge graph. For the updated information of known entities, such as changes in equipment status, updates to user power consumption, etc., the attributes of existing nodes in the power grid data knowledge graph are updated. According to the entity relationships parsed from the data after the secondary screening, the edges in the power grid data knowledge graph are added or updated to reflect new connections or relationship changes between entities, such as power supply relationships between devices, associations between users and devices, etc. By updating the nodes, node attributes and edges in the power grid data knowledge graph, the update of the power grid data knowledge graph stored in the graph database is completed.

[0044] In an embodiment of the present invention, the power grid data collected from different data sources are classified to obtain original power grid data; the original power grid data is screened by a quality scoring method to obtain data after a primary screening; based on the feature vector of the data after the primary screening, the data after the primary screening is screened again to obtain data after a secondary screening; based on the data after the secondary screening, the power grid data knowledge graph stored in the graph database is updated, wherein the power grid data knowledge graph is used to implement distribution network scheduling. It is easy to notice that the original power grid data from multiple data sources is screened once based on the quality score and once based on the feature vector, which improves the quality of the data after the secondary screening, and the power grid data knowledge graph stored in the graph database is updated based on the data after the secondary screening, achieving the technical effect of improving the update accuracy of the power grid knowledge graph, thereby solving the technical problem of low update accuracy of the power grid knowledge graph.

[0045] In one embodiment of the present application, based on the feature vector of the data after the first screening, the data after the first screening is screened again to obtain the data after the second screening, including: extracting features from the data after the first screening to obtain the feature vector; determining the Euclidean norm of the data after the first screening based on the feature vector; determining the first comprehensive threshold of the data after the first screening based on the Euclidean norm; and screening the data after the first screening according to the comparison result between the Euclidean norm and the first comprehensive threshold to obtain the data after the second screening.

[0046] The Euclidean norm in the above steps is a mathematical measure of the feature vector, which is used to quantify the size of the feature vector of the data after a screening, so as to evaluate the complexity or information richness of the data corresponding to the feature vector.

[0047] The first comprehensive threshold in the above step is determined in the secondary screening process based on the Euclidean norm of the data after the primary screening, and is a critical value used to determine whether the data should be retained.

[0048] In an optional embodiment, for the digital type data in the data after the first screening, the statistical features, time series features and frequency domain features of the data are extracted. For the text type data in the data after the first screening, the word frequency features, inverse document frequency features and word embedding features of the data are extracted. For the image type data in the data after the first screening, the color features, texture features and edge and contour features of the image are extracted. Then, the extracted features are scale-normalized, and the normalized features are placed in a vector in a predetermined order to form a feature vector.

[0049] Based on the eigenvector, the Euclidean norm of the filtered data is determined, the sum of the squares of the elements in the eigenvector is calculated, and then the square root of the sum of the squares is taken to obtain the Euclidean norm of the filtered data. m ], the calculation formula of the Euclidean norm is:

[0050]

[0051] Where L2(u) is the Euclidean norm, m is the number of elements in the eigenvector, and u j Represents the jth element in the feature vector.

[0052] Then, a weight is assigned to each of the first-screened data, and then the mean of the Euclidean norms corresponding to the first-screened data is calculated. The first comprehensive threshold is determined based on the weight of each of the first-screened data and the mean of the Euclidean norm. Then, the Euclidean norm is compared with the first comprehensive threshold, and according to the comparison result, it is determined whether the first-screened data corresponding to the Euclidean norm is the second-screened data.

[0053] In one embodiment of the present application, in response to the type of the original power grid data being text type data, feature extraction is performed on the data after a filter to obtain a feature vector of the data after a filter, including: feature extraction is performed on the data after a filter to obtain a word frequency and an inverse document frequency of the data after a filter; a feature matrix is ​​constructed based on the word frequency and the inverse document frequency; and the feature matrix is ​​converted into a feature vector.

[0054] The term frequency (TF) in the above step refers to the frequency of a word appearing in a document, which can be used as a basis for judging the importance of a word in a document.

[0055] The Inverse Document Frequency (IDF) in the above steps is intended to measure the prevalence and importance of a word. If a word appears in many documents, it is considered that the word has a low degree of discrimination for the document, so its IDF value will be relatively small. On the contrary, if a word appears in fewer documents, it is considered that the word has a high degree of discrimination for the document, and the IDF value will be relatively large.

[0056] The feature matrix in the above steps is a data structure used when converting multiple documents into numerical features. It is a two-dimensional matrix in which rows represent different documents and columns represent different features. The elements in the matrix can represent the value of a feature in a certain document.

[0057] In an optional embodiment, in the process of extracting features from the data after the first screening, for the text type data in the data after the first screening, the word frequency and inverse document frequency of the words in the text type data are extracted.

[0058] The formula for calculating word frequency is:

[0059]

[0060] Among them, TF(t,d) is the word frequency.

[0061] The calculation formula for inverse document frequency is:

[0062]

[0063] Among them, IDF(t,d) is the inverse document frequency.

[0064] Then calculate TF-IDF (Term Frequency-Inverse Document Frequency). TF-IDF is the product of term frequency and inverse document frequency. It is used to evaluate the importance of a word in a document. The calculation formula is:

[0065] TF-IDF(t, d, D) = TF(t, d)·IDF(t, d);

[0066] Wherein, TF-IDF(t, d, D) is the term frequency-inverse document frequency, TF(t, d) is the term frequency, and IDF(t, d) is the inverse document frequency.

[0067] Then, define a vocabulary that contains the features of all text type data. Specifically, collect all the documents in the data after the first screening: aggregate the text type data after the first screening together, perform word segmentation on each document, split the text into words or phrases, remove common stop words such as "de", "shi", "zai", etc., restore the words to their basic forms to reduce the lexical diversity, and then gather all the processed words or phrases together to form a unique vocabulary.

[0068] In the process of constructing the feature matrix based on the term frequency and the inverse document frequency, taking the construction of the term frequency matrix as an example for illustration, create an empty two-dimensional array and use it as the term frequency matrix. Then initialize the term frequency matrix so that the number of rows of the term frequency matrix is equal to the number of documents, and the number of columns is equal to the size of the vocabulary. Subsequently, traverse each document based on the vocabulary. For each document, count the term frequency corresponding to each word in the vocabulary in this document, and fill the statistical result into the corresponding position of the term frequency matrix. Specifically, if a certain word appears k times in the i-th document, then fill k in the i-th row and the column corresponding to this word in the term frequency matrix. The process of constructing the TF-IDF matrix is similar and will not be elaborated here.

[0069] After that, convert the term frequency matrix and the TF-IDF matrix, which are used as the feature matrix, into vector forms to obtain the feature vectors.

[0070] In an implementation manner of this application, when the type of the original power grid data is image type data, perform feature extraction on the data after the first screening to obtain the feature vectors of the data after the first screening, including: performing feature extraction on the data after the first screening to obtain key visual features, and the key visual features include at least one of the following: key points in the image, descriptors in the image, histograms of gradient directions in the image; constructing an image feature space based on the key visual features; combining the eigenvalues in the image feature space to obtain the feature vectors.

[0071] The key visual features in the above steps refer to the information that can describe the local or overall features of the image extracted from the image.

[0072] The key points in the image in the above steps, also called feature points or interest points, are local areas in the image that have unique structures or are easy to detect, and usually have high descriptiveness and stability. Key point detection algorithms may include but are not limited to SIFT (Scale-Invariant Feature Transform) algorithm, SURF (Speeded-Up Robust Features) algorithm, etc.

[0073] The descriptor in the image in the above step is a numerical representation used to describe the local image features around the key point. The descriptor is usually a vector containing multiple numerical values ​​that can quantify the texture, color, shape and other features around the key point.

[0074] The histogram of gradient directions in the image in the above step is a histogram of gradient directions of each small area in the image, which is used to capture the edge and contour information of the image.

[0075] The image feature space in the above steps is a multidimensional space composed of image feature vectors.

[0076] In an optional embodiment, the SIFT algorithm can be used to extract key points and descriptors in the image, and the HOG (Histogram of Oriented Gradients) algorithm can be used to extract the histogram of the gradient direction in the image to obtain the key visual features. Then, an image feature space containing the key visual features is defined, and the key visual features are collected using the image feature space, which includes the SIFT feature space and the HOG feature space.

[0077] For each image, calculate the value corresponding to each key visual feature of the image in the feature space. Then combine the eigenvalues ​​in the image feature space. Specifically, it is necessary to obtain the image feature space and construct an eigenvalue matrix based on the eigenvalues ​​in the image feature space. When constructing the eigenvalue matrix, it is necessary to create an empty two-dimensional array with the number of rows equal to the number of images and the number of columns equal to the size of the feature space, and then fill the calculated eigenvalues ​​into the corresponding positions of the feature matrix. For each image, convert the corresponding row of the image in the eigenvalue matrix into an eigenvector. This eigenvector is a multidimensional vector with a dimension equal to the size of the feature space. Each element in the vector represents the value of the corresponding feature in the feature space.

[0078] In one embodiment of the present application, when the type of the original power grid data is digital type data, feature extraction is performed on the data after a screening to obtain a feature vector of the data after a screening, including: feature extraction is performed on the data after a screening to obtain key data features, the key data features include statistical features and correlation indicators; a data feature list is constructed based on the key data features; and the feature values ​​in the data feature list are combined to obtain a feature vector.

[0079] The key data features in the above steps refer to the information extracted from the digital data in the data after a screening, which is of great value for subsequent processing and can reflect the basic properties of the data and the potential connections between the data.

[0080] The statistical features in the above steps are descriptive features obtained by performing mathematical statistical analysis on a set of data, mainly including mean, median, mode, standard deviation, variance, skewness, kurtosis, etc.

[0081] The correlation indicators in the above steps are used to measure the strength and type of relationships between different data variables.

[0082] The data feature list in the above steps is used to represent the listing and summary of all key data features obtained in the feature extraction process.

[0083] In an optional embodiment, when extracting key data features, statistical features can be obtained by calculating the mean, median, mode, standard deviation, variance, minimum and maximum values, skewness and kurtosis of the digital data in the filtered data. The correlation index can be determined by calculating the correlation coefficient of the digital data or using a cross-correlation function.

[0084] Then define a feature list containing key data features, calculate the eigenvalues ​​of the digital data in the filtered data on the key data features, fill the calculated eigenvalues ​​into the corresponding positions of the feature list, and complete the construction of the feature list. Combine the eigenvalues ​​of each feature in the data feature list to obtain a feature vector.

[0085] In one embodiment of the present application, determining a first comprehensive threshold of the data after a single screening based on the Euclidean norm includes: determining a first weight of each data after a single screening in the data after a single screening; determining the mean of the Euclidean norm of the data after a single screening as the first threshold; and determining a first comprehensive threshold based on the first weight and the first threshold.

[0086] The first weight in the above step is the value assigned to the data point according to the importance of each data point relative to the entire data set when the data after the first screening is screened for the second time.

[0087] The first threshold in the above step is the mean of the Euclidean norm of the data after one screening.

[0088] In an optional embodiment, a weight is assigned to each data after a single screening, and then the proportion of the weight is determined according to the quality score result corresponding to the data after a single screening, wherein the higher the quality score, the higher the proportion of the weight. Then, the weight is weighted and summed based on the proportion of the weight to obtain a first comprehensive weight. The mean of the Euclidean norm corresponding to the data after a single screening is calculated as the first threshold. The first comprehensive weight and the first threshold are normalized, and the product of the normalized first comprehensive weight and the planned first threshold is calculated to obtain the first comprehensive threshold.

[0089] In one embodiment of the present application, the data after the first screening is screened according to the comparison result of the Euclidean norm and the first comprehensive threshold value to obtain the data after the second screening, including: when the Euclidean norm is greater than or equal to the first comprehensive threshold value, determining the data after the first screening corresponding to the Euclidean norm as the data after the second screening.

[0090] In an optional embodiment, the Euclidean norm reflects the comprehensive size of each eigenvalue in the eigenvector, that is, the length or scale of the secondary filtered data corresponding to the eigenvector in the feature space. A higher Euclidean norm means that the data has higher values ​​on multiple features, indicating that the data carries more information and details, and is more valuable for constructing a knowledge graph of power grid data. Therefore, when the Euclidean norm is greater than or equal to the first comprehensive threshold, the primary filtered data corresponding to the Euclidean norm is determined to be the secondary filtered data.

[0091] In one embodiment of the present application, the original power grid data is screened once by a quality scoring method to obtain the data after the first screening, including: performing a preprocessing operation on the original power grid data to obtain the data after the preprocessing operation; performing a quality scoring on the data after the preprocessing operation to obtain an evaluation score of the data after the preprocessing operation; when the evaluation score is greater than a preset score, determining that the data after the preprocessing operation is the data after the first screening.

[0092] The preset score in the above steps is a pre-set score, which can be preset according to actual application conditions, and the specific value of the preset score is not limited here.

[0093] In an optional embodiment, the original power grid data is preprocessed, and the preprocessing steps for text type data may include but are not limited to: removing punctuation marks and stop words, standardizing text format, segmenting words and performing stem extraction or word form restoration, etc. The preprocessing steps for image type data may include but are not limited to: adjusting image size, denoising, enhancing contrast, etc. The preprocessing steps for image type data may include but are not limited to: correcting format problems, filling missing values, deleting duplicate records, etc. After the original power grid data is preprocessed, the preprocessed data is obtained.

[0094] Then, a scoring index is constructed. For example, for digital type data in the data after preprocessing operations, the quality scoring index can be completeness and accuracy, where completeness is used to characterize whether there are missing values, and accuracy is used to characterize whether it is consistent with the actual measured value. For text type data in the data after preprocessing operations, the quality scoring index can be information richness, language clarity, and relevance, where information richness is used to characterize whether it contains key information, language clarity is used to characterize whether it is clearly expressed, and relevance is used to characterize whether it is directly related to the operation of the power grid. For image type data in the data after preprocessing operations, the quality scoring index can be clarity, content relevance, and completeness, where clarity is used to characterize whether the image is clear, content relevance is used to characterize whether the image contains valuable information, and completeness is used to characterize whether the image fully displays the required information.

[0095] Then, the quality of the data after the preprocessing operation is scored based on the above-mentioned scoring indicators. Then, a preset score is set, and the evaluation score is compared with the preset score. When the evaluation score is greater than the preset score, the data after the preprocessing operation is determined to be the data after a screening.

[0096] In one embodiment of the present application, the method further includes: updating the power grid data knowledge graph based on the data after secondary screening to obtain an updated power grid data knowledge graph; and storing the updated power grid data knowledge graph in a graph database.

[0097] In an optional embodiment, the data after the secondary screening is parsed, and entities and relationships are identified from the data after the secondary screening. For digital type data, the status information of the device can be directly identified; for text type data, NLP (Natural Language Processing) technology such as entity recognition and relationship extraction is used to extract entities and relationships; for image type data, image recognition technology is used to identify entities and states.

[0098] The identified entities and relationships are then compared with the existing knowledge graph to determine the parts that need to be updated. If entities that do not exist in the original power grid data knowledge graph appear in the data after the secondary screening, a new node is created in the power grid data knowledge graph of the graph database. If the data provides updated information about existing entities, such as equipment status, maintenance records, or real-time monitoring data, the attributes of the corresponding nodes in the power grid data knowledge graph are updated. If new relationships are identified from the data after the secondary screening, the original power grid data knowledge graph is updated with new relationship conditions or original relationships. During the update process, the update operation can be performed using the interface or query language of the graph database, and the updated power grid data knowledge graph is stored in the graph database.

[0099] Example 2

[0100] According to an embodiment of the present invention, an embodiment of a power grid resource allocation device is provided, and the device can execute the power grid resource allocation method provided in the above-mentioned embodiment 1. The specific implementation method and preferred application scenario are the same as those of the above-mentioned embodiment 1, and will not be repeated here.

[0101] Figure 2 is a schematic diagram of a power grid data processing device according to an embodiment of the present invention. Figure 2 As shown, the processing device of power grid data includes:

[0102] The classification module 20 is used to classify the power grid data collected from different data sources to obtain original power grid data, wherein the original power grid data includes: text type data, image type data and digital type data;

[0103] The first screening module 22 is used to classify the power grid data collected from different data sources to obtain original power grid data, wherein the types of the original power grid data include: text type data, image type data and digital type data;

[0104] The second screening module 24 is used to screen the data after the first screening again based on the feature vector of the data after the first screening to obtain the data after the second screening;

[0105] The updating module 26 is used to update the power grid data knowledge graph stored in the graph database based on the data after secondary screening, wherein the power grid data knowledge graph is used for distribution network scheduling.

[0106] The second screening module includes: a feature extraction unit, used to extract features from the data after the first screening to obtain a feature vector; a first determination unit, used to determine the Euclidean norm of the data after the first screening based on the feature vector; a second determination unit, used to determine the first comprehensive threshold of the data after the first screening based on the Euclidean norm; a screening unit, used to screen the data after the first screening according to the comparison result between the Euclidean norm and the first comprehensive threshold to obtain the data after the second screening.

[0107] The feature extraction unit is also used to extract features from the data after a screening, to obtain the word frequency and inverse document frequency of the data after a screening; to construct a feature matrix based on the word frequency and inverse document frequency; and to convert the feature matrix into a feature vector.

[0108] The feature extraction unit is also used to extract features from the data after a screening to obtain key visual features, which include at least one of: key points in the image, descriptors in the image, and histograms of gradient directions in the image; construct an image feature space based on the key visual features; and combine the eigenvalues ​​in the image feature space to obtain a feature vector.

[0109] The feature extraction unit is also used to extract features from the data after a screening to obtain key data features, which include statistical features and correlation indicators; construct a data feature list based on the key data features; and combine the feature values ​​in the data feature list to obtain a feature vector.

[0110] The second determination unit is also used to determine the first weight of each filtered data in the filtered data; determine the mean of the Euclidean norm of the filtered data as the first threshold; and determine the first comprehensive threshold based on the first weight and the first threshold.

[0111] The screening unit is also used to determine that the once-screened data corresponding to the Euclidean norm is the twice-screened data when the Euclidean norm is greater than or equal to the first comprehensive threshold.

[0112] The first screening module includes: a preprocessing unit, which is used to perform preprocessing operations on the original power grid data to obtain data after the preprocessing operation; a quality scoring unit, which is used to perform quality scoring on the data after the preprocessing operation to obtain an evaluation score for the data after the preprocessing operation; and a third determination unit, which is used to determine that the data after the preprocessing operation is data after a screening operation when the evaluation score is greater than a preset score.

[0113] The update module includes: an update unit, which is used to update the power grid data knowledge graph based on the data after secondary screening to obtain an updated power grid data knowledge graph; and a storage unit, which is used to store the updated power grid data knowledge graph in a graph database.

[0114] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0118] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0120] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing power grid data, characterized in that: include: Classifying the power grid data collected from different data sources to obtain original power grid data, wherein the original power grid data includes: text type data, image type data and digital type data; The original power grid data is screened by a quality scoring method to obtain screened data; Based on the feature vector of the data after the first screening, the data after the first screening is screened again to obtain data after the second screening; The power grid data knowledge graph stored in the graph database is updated based on the data after the secondary screening, wherein the power grid data knowledge graph is used for distribution network scheduling.

2. The method for processing power grid data according to claim 1, characterized in that: Based on the feature vector of the data after the first screening, the data after the first screening is screened again to obtain the data after the second screening, including: Performing feature extraction on the data after the first screening to obtain the feature vector; Determine the Euclidean norm of the once filtered data based on the feature vector; Determine a first comprehensive threshold of the once-screened data based on the Euclidean norm; According to the comparison result between the Euclidean norm and the first comprehensive threshold, the data after the first screening is screened to obtain the data after the second screening.

3. The method for processing power grid data according to claim 2, characterized in that: In response to the type of the original power grid data being text type data, feature extraction is performed on the filtered data to obtain a feature vector of the filtered data, including: Performing feature extraction on the data after the first screening to obtain the word frequency and inverse document frequency of the data after the first screening; Constructing a feature matrix based on the word frequency and the inverse document frequency; The feature matrix is ​​converted into the feature vector.

4. The method for processing power grid data according to claim 2, characterized in that: When the type of the original power grid data is image type data, feature extraction is performed on the data after the primary screening to obtain a feature vector of the data after the primary screening, including: Performing feature extraction on the filtered data to obtain key visual features, wherein the key visual features include at least one of the following: key points in the image, descriptors in the image, and a histogram of gradient directions in the image; constructing an image feature space based on the key visual features; The eigenvalues ​​in the image feature space are combined to obtain the eigenvector.

5. The method for processing power grid data according to claim 2, characterized in that: When the type of the original power grid data is digital data, feature extraction is performed on the filtered data to obtain a feature vector of the filtered data, including: Extracting features from the data after the first screening to obtain key data features, wherein the key data features include statistical features and correlation indicators; Building a data feature list based on the key data features; The feature values ​​in the data feature list are combined to obtain the feature vector.

6. The method for processing power grid data according to claim 2, characterized in that: Determining a first comprehensive threshold of the filtered data based on the Euclidean norm includes: Determine a first weight of each piece of data after the first screening in the data after the first screening; Determine the mean value of the Euclidean norm of the data after the primary screening as a first threshold; The first comprehensive threshold is determined based on the first weight and the first threshold.

7. The method for processing power grid data according to claim 2, characterized in that: According to the comparison result of the Euclidean norm and the first comprehensive threshold, the data after the primary screening is screened to obtain the data after the secondary screening, including: When the Euclidean norm is greater than or equal to the first comprehensive threshold, the first-screened data corresponding to the Euclidean norm is determined to be the second-screened data.

8. The method for processing power grid data according to claim 1, characterized in that: The original power grid data is screened once by a quality scoring method to obtain screened data, including: Performing a preprocessing operation on the original power grid data to obtain preprocessed data; Performing a quality score on the data after the preprocessing operation to obtain an evaluation score for the data after the preprocessing operation; In the case where the evaluation score is greater than a preset score, the data after the preprocessing operation is determined to be the data after the first screening.

9. The method for processing power grid data according to claim 1, characterized in that: The method further comprises: Based on the data after the secondary screening, the power grid data knowledge graph is updated to obtain an updated power grid data knowledge graph; The updated power grid data knowledge graph is stored in the graph database.

10. A device for processing power grid data, characterized in that: include: A classification module, used to classify the power grid data collected from different data sources to obtain original power grid data, wherein the types of the original power grid data include: text type data, image type data and digital type data; A first screening module is used to screen the original power grid data by a quality scoring method to obtain screened data; A second screening module is used to screen the data after the first screening again based on the feature vector of the data after the first screening to obtain the data after the second screening; An updating module is used to update the power grid data knowledge graph stored in the graph database based on the data after the secondary screening, wherein the power grid data knowledge graph is used to realize distribution network scheduling.

Images