Data quality evaluation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of regional energy data management, and discloses a data quality evaluation method, device and equipment and a storage medium, and the method comprises the steps: analyzing the received to-be-evaluated energy data, and determining the current data type of the to-be-evaluated energy data; based on the current data type, determining a remaining data type lacking during quality evaluation, and obtaining target energy data corresponding to the remaining data type from the historical energy data; and generating simulated energy data corresponding to the remaining data type according to a preset energy data knowledge graph and the target energy data. The method can generate the simulated energy data according to the preset energy data knowledge graph and the target energy data, carries out the quality evaluation of the to-be-evaluated energy data based on the simulated energy data, does not need to wait for the receiving of the energy data of all data types and then carries out the batch evaluation, and guarantees the real-time performance of the data quality evaluation.

Description

Technical Field

[0001] This application relates to the technical field of regional energy data governance, and particularly to a data quality evaluation method, device, equipment, and storage medium. Background Art

[0002] In the field of regional energy data processing, there are usually various types of energy data, which have different sources, formats, and characteristics. Due to the certain interrelationships between different energy data, for example, the power generation of renewable energy such as wind energy and solar energy may be closely related to meteorological data (such as wind speed and light intensity), when conducting data quality evaluation, it is necessary to comprehensively consider these interrelationships to ensure the accuracy and reliability of the data in practical applications.

[0003] When the existing method conducts data quality evaluation on energy data, due to the certain interrelationships between different types of energy data, generally, after receiving all types of related energy data, batch evaluation is carried out to ensure the accuracy of the quality evaluation. However, because the existing method needs to wait for the device to receive all types of data before batch processing, the real-time performance is poor. Summary of the Invention

[0004] The main purpose of this application is to provide a data quality evaluation method, aiming to solve the technical problem that the existing technology has poor real-time performance due to batch processing after receiving all types of data.

[0005] To achieve the above purpose, this application proposes a data quality evaluation method, and the method includes: Analyze the received energy data to be evaluated, and determine the current data type of the energy data to be evaluated; Based on the current data type, determine the remaining data types missing during quality evaluation, and obtain the target energy data corresponding to the remaining data types from historical energy data; Generate simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, and the preset energy data knowledge graph is generated by the historical energy data; Conduct quality evaluation on the energy data to be evaluated based on the simulated energy data.

[0006] In one embodiment, the step of generating simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data includes: Conduct time series analysis on the target energy data to obtain the time series characteristics corresponding to the target energy data; Obtain the data type associations between various data types based on a preset energy data knowledge graph; Classify the target energy data according to the time series characteristics, and generate simulated energy data corresponding to the remaining data types according to the data type associations and the classified target energy data.

[0007] In one embodiment, before the step of analyzing the received energy data to be evaluated, it further includes: Perform entity analysis based on historical energy data to determine the data entities corresponding to each data type, and perform association analysis based on the historical energy data to determine the data type associations between various data types; Generate data nodes according to the data entities, and generate data edges according to the data type associations; Construct a preset energy data knowledge graph based on the data nodes and the data edges.

[0008] In one embodiment, the step of constructing a preset energy data knowledge graph based on the data nodes and the data edges includes: Construct an initial energy knowledge graph based on the data nodes and the data edges; Obtain the mean square error and the mean absolute error according to the data in the initial energy knowledge graph, and perform quality evaluation on the initial energy knowledge graph according to the mean square error and the mean absolute error; When the quality evaluation result meets the preset quality requirements, use the initial energy knowledge graph as the preset energy data knowledge graph.

[0009] In one embodiment, the step of performing entity analysis based on historical energy data to determine the data entities corresponding to each data type further includes: Perform time series analysis based on historical energy data to obtain the historical time series characteristics corresponding to the historical energy data; Generate a time series graph according to the historical time series characteristics and the historical energy data, and analyze the time series graph; When the analysis result shows that there are missing values, obtain the average value of the historical energy data based on the historical energy data, and fill in the historical energy data based on the average value of the historical energy data to obtain new historical energy data; Perform entity analysis based on the new historical energy data to determine the data entities corresponding to each data type.

[0010] In one embodiment, the step of performing quality evaluation on the energy data to be evaluated based on the simulated energy data includes: Determine whether the simulated energy data meets the preset business rules; When the simulated energy data does not meet the preset business rules, perform a regression analysis on the simulated energy data according to the historical energy data; Correct the simulated energy data based on the regression analysis result, and perform a quality evaluation on the energy data to be evaluated based on the corrected simulated energy data.

[0011] In one embodiment, the step of determining the remaining data types missing when performing a quality evaluation based on the current data type includes: Determine the target evaluation rule corresponding to the current data type based on the current data type; Determine the complete data types required by the target evaluation rule according to the target evaluation rule; Determine the remaining data types missing when performing a quality evaluation based on the complete data types and the current data type.

[0012] In addition, to achieve the above object, the present application also proposes a data quality evaluation device, and the device includes: An analysis module, configured to analyze the energy data to be evaluated received, and determine the current data type of the energy data to be evaluated; A selection module, configured to determine the remaining data types missing when performing a quality evaluation based on the current data type, and obtain the target energy data corresponding to the remaining data types from the historical energy data; A simulation module, configured to generate simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, and the preset energy data knowledge graph is generated by the historical energy data; An evaluation module, configured to perform a quality evaluation on the energy data to be evaluated based on the simulated energy data.

[0013] In addition, to achieve the above object, the present application also proposes a data quality evaluation device, and the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the data quality evaluation method as described above.

[0014] In addition, to achieve the above object, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the data quality evaluation method as described above are implemented.

[0015] The present application provides a method, apparatus, device, and storage medium for evaluating data quality. The method includes: analyzing the received energy data to be evaluated to determine the current data type of the energy data to be evaluated; determining the remaining data types missing during quality evaluation based on the current data type, and obtaining the target energy data corresponding to the remaining data types from historical energy data; generating simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, where the preset energy data knowledge graph is generated through the historical energy data; and evaluating the quality of the energy data to be evaluated based on the simulated energy data. Since the present application can generate simulated energy data according to a preset energy data knowledge graph and target energy data, and evaluate the quality of the energy data to be evaluated based on the simulated energy data, there is no need to wait to receive energy data of all data types and then perform batch evaluation, ensuring the real-time nature of data quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] To more clearly illustrate the technical solutions in the embodiments or the prior art, the following briefly introduces the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0018] Figure 1 Flowchart of the first embodiment of the data quality evaluation method proposed by the embodiment of the present application; Figure 2 Flowchart of the second embodiment of the data quality evaluation method proposed by the embodiment of the present application; Figure 3 Periodic time sequence node matching diagram in the data quality evaluation method proposed by the embodiment of the present application; Figure 4 Flowchart of the third embodiment of the data quality evaluation method proposed by the embodiment of the present application; Figure 5 Schematic diagram of the preset energy data knowledge graph in the data quality evaluation method proposed by the embodiment of the present application; Figure 6 Diagram of the data quality evaluation apparatus provided in this embodiment; Figure 7 Structural schematic diagram of the data quality evaluation device suitable for implementing this embodiment.

[0019] The implementation, functional features, and advantages of the objectives of the present application will be further described in conjunction with the embodiments with reference to the drawings. Detailed implementation manners

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0022] It should be noted that all directional indications (such as up, down, left, right, front, back...) in this embodiment are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0023] It can be understood that in the field of energy data processing, there are usually various different types of energy data, and these data have different sources, formats, and characteristics. Since there is a certain correlation between different energy data, for example, the power generation of renewable energy such as wind energy and solar energy may be closely related to meteorological data (such as wind speed, light intensity), when evaluating data quality, it is necessary to comprehensively consider these correlations to ensure the accuracy and reliability of the data in practical applications.

[0024] When evaluating the data quality of existing energy data, due to the certain correlation between different types of energy data, generally, after receiving all the energy data of the related data types, batch evaluation is performed to ensure the accuracy of the quality evaluation. However, since the existing method needs to wait for the device to receive all types of data before batch processing, the real-time performance is poor.

[0025] Therefore, in order to solve the technical problem in the prior art that batch processing is performed after receiving all types of data, resulting in poor real-time performance, this embodiment proposes a data quality evaluation method, which includes: analyzing the received energy data to be evaluated to determine the current data type of the energy data to be evaluated; determining the remaining data types missing during quality evaluation based on the current data type, and obtaining the target energy data corresponding to the remaining data types from historical energy data; generating simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, where the preset energy data knowledge graph is generated from historical energy data; and performing quality evaluation on the energy data to be evaluated based on the simulated energy data. Since this embodiment can generate simulated energy data according to the preset energy data knowledge graph and the target energy data, and perform quality evaluation on the energy data to be evaluated based on the simulated energy data, there is no need to wait to receive energy data of all data types and then perform batch evaluation, ensuring the real-time performance of data quality evaluation.

[0026] For ease of understanding, the following specifically introduces Figures 1 to 7 the data quality evaluation method provided in this embodiment and the data quality evaluation methods, devices, equipment, and storage media provided in the following embodiments.

[0027] This embodiment provides a data quality evaluation method. Refer to Figure 1 , Figure 1 which is the flowchart of the first embodiment of the data quality evaluation method proposed in the embodiments of this application.

[0028] As Figure 1 shown, the method includes: Step S10: Analyze the received energy data to be evaluated to determine the current data type of the energy data to be evaluated.

[0029] It should be noted that the execution subject of this embodiment can be a multifunctional machine device with data quality evaluation, such as a data quality evaluation device, or a device capable of implementing the above functions. This embodiment uses a data quality evaluation device (hereinafter referred to as the device) for illustration.

[0030] It should also be noted that the above energy data to be evaluated can be the energy data of a preset area received by the above device, such as wind energy data, solar energy data, and electric energy in a certain industrial park. The above current data type can be the type of energy, environmental factors, and data source corresponding to the above energy data to be evaluated. For example, wind energy data type, electric power data type, solar energy data type, power generation equipment data type, meteorological data type, etc. The above device can analyze the received energy data through part-of-speech analysis, a preset mapping relationship table, etc. This embodiment uses part-of-speech analysis for explanation, but does not specifically limit this embodiment.

[0031] In specific implementation, when the above device receives the energy data of the preset area, it can perform part-of-speech analysis on the received energy data, distinguish nouns, quantifiers, unit words, etc. in the energy data, and determine whether the nouns in the energy data conform to the preset data type. If they conform, the data type corresponding to the noun is used as the current data type. Among them, the above preset data type can be the data type set in advance by the user according to the data types existing in the preset area.

[0032] For ease of understanding, the following is illustrated by way of example, but does not specifically limit this embodiment. Assume that the above device receives an energy data "The wind speed of a certain wind farm is 10 m / s and the power generation is 500 kW", then through natural language processing technologies such as part-of-speech analysis, the noun "wind speed" is identified, and by comparing "wind speed" with the preset data type, it is determined that it belongs to the wind energy category.

[0033] Step S20: Determine the remaining data types lacking when performing quality evaluation based on the current data type, and obtain the target energy data corresponding to the remaining data types from the historical energy data.

[0034] It should be noted that the above remaining data types can be other types of data required when performing quality evaluation on the energy data to be evaluated. For example, when analyzing the complementarity of wind energy and solar energy to optimize regional energy distribution, assume that the device in the regional energy system receives the wind speed data of a certain industrial park, and determines the current data type as wind energy data through part-of-speech analysis. At this time, in order to comprehensively evaluate the quality of the wind energy data, solar energy data is also required for comprehensive analysis. Therefore, the remaining data type is solar energy data. The above historical energy data can be the energy data accumulated in the preset area during past operations. The above target energy data can be the energy data corresponding to the remaining data types obtained from the historical energy data.

[0035] Furthermore, in order to determine the remaining data types, the step of determining the remaining data types lacking when performing quality evaluation based on the current data type includes: Step S21: Determine the target evaluation rule corresponding to the current data type based on the current data type; Step S22: Determine the complete data types required by the target evaluation rule according to the target evaluation rule; Step S23: Determine the remaining data types missing during quality evaluation based on the complete data types and the current data type.

[0036] It should be noted that the above target evaluation rule can be a preset rule for guiding the device on how to use different data types for quality evaluation. The above complete data types can be all the data types required for quality evaluation to meet the requirements of the target evaluation rule.

[0037] In specific implementation, after the device confirms the current data type of the energy data to be evaluated, it can select the target evaluation rule corresponding to the current data type from the built-in preset evaluation rule library through the current data type. Among them, the device can select the target evaluation rule by looking up the preset mapping relation table through the current data type, and the data types and evaluation rules in the above preset mapping relation table correspond to each other.

[0038] After determining the target evaluation rule, the device will analyze these rules, confirm the complete data types required for quality evaluation, and finally determine the remaining data types in the complete data types except the current data type according to the complete data types and the current data type.

[0039] For ease of understanding, the following is illustrated by way of example, but the present embodiment is not specifically limited. Suppose the device receives the wind speed data of a certain wind farm and determines that the current data type is wind speed data. The device identifies the target rules for wind energy quality evaluation according to the preset evaluation rule library, including two key indicators: wind speed and power generation. The analysis shows that the complete data types should include wind energy data types and power data types. After comparison, it is found that the data of the power data type has not been received, so the power data type is determined as the remaining data type.

[0040] Step S30: Generate simulated energy data corresponding to the remaining data type according to the preset energy data knowledge graph and the target energy data, and the preset energy data knowledge graph is generated by the historical energy data.

[0041] It should be noted that the above preset energy data knowledge graph can be a knowledge graph containing different data types and their relationships generated by historical energy data. The above simulated energy data can be simulated data generated based on the knowledge graph and the target data, and is used to supplement the missing data types.

[0042] In a specific implementation, the above-mentioned device analyzes the part of the knowledge graph related to the target energy data, extracts the rules and parameters required to generate the simulated energy data, and uses the above-mentioned rules and parameters, combined with the obtained target energy data, to generate the simulated energy data corresponding to the remaining data types through a preset algorithm and model (such as time series analysis or machine learning model).

[0043] For ease of understanding, the following is illustrated by way of example, but the present embodiment is not specifically limited.

[0044] Suppose there is such a scenario where the device has received wind energy data but still needs solar energy data to complete the quality evaluation. The device uses a preset knowledge graph of energy data, which contains the historical relationship between wind energy and solar energy (such as the change trend of both under specific weather conditions). By analyzing these relationships and combining the current wind energy data, the above-mentioned device uses time series analysis or machine learning model to generate the corresponding simulated solar energy data. Even in the absence of actual solar energy data, the above-mentioned device can perform a complete quality evaluation through the simulated energy data.

[0045] Step S40: Perform a quality evaluation on the energy data to be evaluated based on the simulated energy data.

[0046] In a specific implementation, after the above-mentioned device generates the simulated energy data corresponding to the remaining data types, it unifies these simulated energy data and the energy data to be evaluated into a unified batch, and uses the above-mentioned target evaluation rules to comprehensively evaluate the quality of the energy data to be evaluated, and evaluate indicators such as the integrity, accuracy, consistency, and timeliness of the data.

[0047] Further, in order to ensure the accuracy of the simulated data and thus improve the accuracy of the quality evaluation, the step of performing a quality evaluation on the energy data to be evaluated based on the simulated energy data includes: Step S41: Determine whether the simulated energy data meets the preset business rules; It should be noted that the above-mentioned preset business rules can be rules preset by the user according to business requirements for verifying the integrity and accuracy of the simulated energy data. In the present embodiment, the ratio range between the simulated energy data and the limited value of the data can be used as the preset business rule. For example, in an electric field, assume that the simulated data is the power generation power of a generator, the simulated power is y, and the rated power is , the preset business rule:[[]] ; When the ratio between y and is within the above range, it is determined that the simulated power meets the preset business rules, otherwise it is determined that the simulated power does not meet the preset business rules.

[0048] Step S42: When the simulated energy data does not satisfy the preset business rule, performing regression analysis on the simulated energy data according to the historical energy data; It should be noted that regression analysis is an operation that can construct a model that best fits the observed data to analyze quantitative relationships.

[0049] Step S43: correcting the simulated energy data based on the regression analysis result, and performing quality evaluation on the energy data to be evaluated based on the corrected simulated energy data.

[0050] In a specific implementation, when the above device performs a quality evaluation on the energy data to be evaluated based on the simulated energy data, it will first determine whether the simulated energy data meets the preset business rules. If the simulated energy data does not meet the preset business rules, the above device will use the historical energy data to perform a regression analysis on the simulated energy data. Based on the results of the regression analysis, the above device will correct the simulated energy data. Finally, the above device will perform a comprehensive quality evaluation on the energy data to be evaluated based on the corrected simulated energy data.

[0051] Furthermore, the step of performing regression analysis on the simulated energy data according to the historical energy data includes: Fitting a preset regression model based on the historical energy data to obtain a target regression coefficient; The simulated energy data is subjected to regression analysis according to the target regression coefficient and the preset regression model.

[0052] The step of correcting the simulated energy data based on the regression analysis result includes: The simulated energy data and the target regression coefficient are input into the preset regression model to obtain the corrected simulated energy data.

[0053] It should be noted that the preset regression model can be a mathematical model pre-selected according to the characteristics of energy data, used to describe the relationship between energy data, such as a linear regression model, a polynomial regression model, etc. The target regression coefficient can be a model parameter obtained through regression analysis, used to quantify the relationship between different energy data.

[0054] In a specific implementation, the above device uses a regression analysis method, such as linear regression, to establish a mathematical model that reflects the internal relationship between energy data as a preset regression model, and fits the preset regression model based on historical energy data to obtain a target regression coefficient. The above device uses the target regression coefficient and the preset regression model to perform regression analysis on the simulated energy data, that is, the simulated energy data and the target regression coefficient are input into the regression model to obtain the corrected simulated energy data.

[0055] In addition, in this embodiment, in order to fit a preset regression model using historical energy data, it is first necessary to analyze the preset regression model. In this embodiment, is used as the preset regression model for explanation, but it does not specifically limit this embodiment. Among them, y is the target variable (such as power generation), X is the feature matrix (such as wind speed, temperature, etc.), β is the regression coefficient to be determined, is the error term. And the least squares method is used to determine the target fitting formula: . Through the fitting of the preset regression model using historical energy data and the target fitting formula, β is determined. Subsequently, the preset correction formula: is used to correct the simulated energy data, where is the mean of the historical energy data, is the simulated feature matrix, is the simulated energy data after correction, is the regression coefficient vector, which is an estimate of the true regression coefficient , is the correction parameter, which is used to control the weight between the mean of the historical energy data and the simulation result. Its value range is usually between 0 and 1. When = 0, the corrected result is completely based on the simulation model; when = 1, the corrected result is completely based on the mean of the historical data.

[0056] For the sake of easy understanding, the following is illustrated by way of example, but it does not specifically limit this embodiment. Suppose the above device collects historical wind speed and power generation data when analyzing wind energy data. Through the linear regression model, the device fits the relationship between the wind speed (independent variable) and the power generation (dependent variable), and obtains the target regression coefficient: power generation = 20 × wind speed + 50. This means that when the wind speed increases by 1 m / s, the power generation is expected to increase by 20 kW.

[0057] When the device receives new simulated energy data, such as simulated wind speed data, it will input these data into the preset regression model. The model will calculate the expected power generation according to the target regression coefficient. If there is a significant difference between the power generation predicted by the simulated data and the result calculated by the model, the device will identify the possible error in the simulated data and make adjustments accordingly to improve the accuracy and reliability of the data.

[0058] This embodiment generates simulated energy data according to the preset energy data knowledge graph and the target energy data, and performs quality evaluation on the energy data to be evaluated based on the simulated energy data, without waiting to receive all types of energy data and then performing batch evaluation, ensuring the real-time nature of the data quality evaluation.

[0059] Based on the first embodiment, in the second embodiment, for the same or similar content as in the above-mentioned first embodiment, reference can be made to the above introduction and will not be elaborated hereinafter. On this basis, please refer to Figure 2 , Figure 2 which is a flowchart of the second embodiment of the data quality evaluation method proposed in the embodiments of the present application. Further, the step of generating the simulated energy data corresponding to the remaining data types according to the preset energy data knowledge graph and the target energy data includes: Step S24: Perform time series analysis on the target energy data to obtain the time series characteristics corresponding to the target energy data; It should be noted that time series analysis is a means of analyzing the time series in which the above-mentioned target energy data is located. When performing time series analysis, historical data needs to be collected first to ensure that the data is arranged in chronological order, missing values and outliers are processed, and then feature extraction is performed on the processed data. Among them, feature extraction can include: trend analysis, that is, identifying the long-term trend in the data, such as linear growth or decay; seasonal analysis, that is, detecting the periodic pattern in the data, such as daily, weekly, and monthly periodic changes; periodic analysis, that is, identifying the long-term periodic changes in the data, such as annual cycles. Then, a suitable time series model is selected according to the characteristics of the data, and the model is applied to the historical data to learn the patterns in the data.

[0060] In addition, it should also be noted that the above time series characteristics can be statistical characteristics exhibited in time series data, such as trends, seasonality, and periodicity. In a specific implementation, after receiving the target energy data, the above device first preprocesses the data to remove noise and outliers. Then, the above device uses statistical methods and visualization tools to identify the long-term trends, short-term fluctuations, and periodic patterns in the data, and quantifies these characteristics into specific parameters, namely time series characteristics.

[0061] For ease of understanding, the following is an example for illustration, but the present embodiment is not specifically limited. Assume that the above device is processing wind speed data. First, the device preprocesses the data, such as filling in missing values and removing noise. Then, the moving average method is used to smooth the data to make the trend more obvious. By plotting the time series graph, the device observes that the wind speed has obvious periodic changes in a day: the wind speed is higher during the day and lower at night. Further analysis reveals that this daily periodicity can be described by a 24-hour cycle. In addition, the device also identifies seasonal variations, that is, the wind speed is generally higher in spring and autumn, while relatively lower in summer and winter. These time series characteristics will be quantified and used for subsequent generation of simulated energy data.

[0062] Step S25: Obtain the data type associations between various data types based on the preset energy data knowledge graph.

[0063] It should be noted that the above data type associations can be relationships between different data types, such as the positive correlation between wind speed and power generation. In specific implementation, the above device first loads the atlas and identifies data entities and relationship nodes. For example, in the scenario of wind power and grid load, the device identifies entities such as wind speed and power generation. Then, by querying the association relationships defined in the atlas, the above device obtains data type associations such as the positive correlation between wind speed and power generation.

[0064] Step S26: Classify the target energy data according to the time series characteristics, and generate simulated energy data corresponding to the remaining data types according to the data type association and the target energy data after data classification.

[0065] In specific implementation, after obtaining the time series characteristics, the above device classifies the data according to the time series characteristics. For example, data with similar periodicity and trends are classified into one category. Combining the classified target energy data and the data type association, the above device generates simulated energy data corresponding to the remaining data types.

[0066] Further, in order to generate simulated energy data suitable for the current energy data to be evaluated, the step of generating simulated energy data corresponding to the remaining data types according to the data type association and the target energy data after data classification includes: Perform time series node analysis on the energy data to be evaluated to determine the current time series node where the energy data to be evaluated is located; Based on the current time series node, screen out target time series data from the target energy data after data classification, and select the target data type association between the target time series data and the energy data to be evaluated from the data type association according to the target time series data and the energy data to be evaluated; Generate a target association expression according to the target data type association, and generate simulated energy data corresponding to the remaining data types according to the target time series data and the target association expression.

[0067] It should be noted that the above current time series node can be the position of the time point corresponding to the energy data to be evaluated in the time series. For example, when the position of T / 2 in the data with a period of T is the current time series node, or the season in which the energy data to be evaluated is located is spring, etc. The above target time series data can be target energy data in the historical period, season or trend that is in the same relative position as the current time series node. Refer to Figure 3 , Figure 3 is the periodic time series node matching diagram in the data quality evaluation method proposed in the embodiment of the present application. For example, when the period is T (i.e., Figure 3In the periodic timing with T = 24h), if the current timing node (i.e., Figure 3 the current timing node in Figure 3 is at the T / 2 position of the current cycle (i.e., Figure 3 the current cycle in Figure 3 ), the target timing data can be the target energy data at the T / 2 position (i.e.,

[0068] the target timing data point (T / 2) in

[0069] the historical cycles 1 and 2 in

[0070] the past cycles (i.e.,

[0071] the historical cycles in

[0072] Based on the first and second embodiments, in the third embodiment, the same or similar content as in the first and second embodiments can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 4 , Figure 4 which is the flowchart of the third embodiment of the data quality evaluation method proposed in this application embodiment. Further, in order to obtain the preset energy data knowledge graph, before the step of analyzing the received energy data to be evaluated, it further includes: Step S01: Perform entity analysis based on historical energy data to determine the data entities corresponding to each data type, and perform correlation analysis based on the historical energy data to determine the data type associations between the data types.

[0073] It should be noted that the above entity analysis can be a process of identifying and extracting entities with clear boundaries from a large amount of data. For example, objects, people, locations, organizations, events, etc. In this embodiment, the above device can identify key data objects from historical energy data through entity analysis, such as power generation equipment, substations, users, sensors, etc. In specific implementation, a series of rules can be formulated according to domain knowledge and data characteristics to identify entities. For example, in energy data, it is stipulated that device numbers, geographical location information, etc. that conform to a specific format correspond to specific data entities. For example, numbers starting with "wind turbine-" correspond to specific wind turbine equipment.

[0074] The above correlation analysis can be a process of studying the relationships, their strengths, and laws existing between different entities. In this embodiment, the above device can find the associations between different data entities through correlation analysis. For example, correlation, dependence, or causal relationships, etc. In specific implementation, the correlation coefficients between different data entities can be calculated, such as Pearson correlation coefficient, Spearman rank correlation coefficient, etc., to determine the linear or non-linear relationships between them.

[0075] In addition, it should also be noted that the above data entities can be the specific objects described by the data, such as power generation equipment, substations, users, etc. For example, in a power plant, each generator set, each transformer, and each transmission line can be a data entity, which are the basic carriers of energy data, carrying specific attributes and information. The above data type associations can be the relationships between different data types, such as the positive correlation between wind speed and power generation.

[0076] In specific implementation, the above device can identify the historical energy data through preset identification rules to determine the data entities corresponding to each data type in the historical energy data, and confirm the data type associations between the data types by calculating the correlation coefficients between different data entities.

[0077] Furthermore, in order to ensure the accuracy of historical energy data, the step of performing entity analysis based on historical energy data to determine the data entities corresponding to each data type further includes: Step S011: Perform time series analysis based on historical energy data to obtain the historical time series characteristics corresponding to the historical energy data; Step S012: Generate a time series graph according to the historical time series characteristics and the historical energy data, and analyze the time series graph; Step S013: When the analysis result indicates the existence of missing values, obtain the average value of the historical energy data based on the historical energy data, and fill in the missing values in the historical energy data based on the average value of the historical energy data to obtain new historical energy data; Step S014: Based on the new historical energy data, perform entity analysis to determine the data entities corresponding to each data type.

[0078] It should be noted that the above time series graph can be a graph that shows the change trend and distribution of data based on time and data values. The above missing values can be the data points with gaps in the historical energy data. The above historical time series features can be the data features extracted by analyzing the time series formed by arranging the historical energy data in chronological order, such as trend features, seasonal features, periodic features, volatility features, and mutation features, etc. The above average value of the historical energy data can be the average value of the data calculated separately for each data type.

[0079] In specific implementation, the above device performs time series analysis based on the historical energy data to obtain historical time series features. For example, when analyzing the historical data of a wind farm, the device can identify the change trend of the power generation output over time and the fluctuation patterns in different time periods.

[0080] Then, the above device generates a time series graph, such as a line graph or a scatter plot, based on the historical time series features and the historical energy data to visually display the change trend and distribution of the data. Then, analyze the graph to determine whether there are abnormal situations such as missing values. If missing values are found, the above device will calculate the average value based on the historical energy data and use this average value to fill in the missing data to generate new historical energy data.

[0081] Step S02: Generate data nodes according to the data entities, and generate data edges according to the association of the data types.

[0082] Step S03: Construct a preset energy data knowledge graph based on the data nodes and the data edges.

[0083] It should be noted that the above data nodes can be the basic units that represent data entities in the knowledge graph and constitute the knowledge graph. Each node represents a specific entity, such as a specific power generation device or a substation, and the node can contain the relevant attributes and information of the entity. The above data edges can be the lines between the nodes that represent the relationships between data entities in the knowledge graph. For example, in energy data, the power transmission relationship between a power generation device and a substation can be represented by an edge, and the type and attributes of the edge can reflect the nature and strength of the relationship.

[0084] Reference Figure 5 ,Figure 5 This is a schematic diagram of a preset energy data knowledge graph in the data quality evaluation method proposed in the embodiment of the present application. In specific implementation, the above device generates data nodes according to data entities (i.e., Figure 5 the fan #001 IDWT23, the substation X capacity: 50MW, the wind speed sensor #01, the industrial area A, and the meteorological database in Figure 5 ), and then generates data edges according to data type association (i.e.,

[0085] the power generation transmission voltage: 35kV, the data dependency update frequency: 1Hz, the power supply coverage distance: 5km, and the data synchronization delay: <50ms in ). The above edges connect relevant nodes. For example, there will be an edge between the power generation equipment node and the substation node, indicating the power transmission relationship. Finally, the above device constructs a preset energy data knowledge graph through these nodes and edges. Further, in order to ensure the accuracy of the preset energy data knowledge graph, the step of constructing the preset energy data knowledge graph based on the data nodes and the data edges includes: Step S031: Construct an initial energy knowledge graph based on the data nodes and the data edges;

[0086] It should be noted that the above initial energy knowledge graph can be a knowledge graph constructed based on data nodes and edges, representing the entities and their mutual relationships in the energy system. The above mean square error can be the average value of the squared differences between the predicted value and the true value, used to evaluate the prediction accuracy of the model. The above mean absolute error can be the average value of the absolute differences between the predicted value and the true value, intuitively reflecting the average error size. The above preset quality requirement can be a quality standard preset during the construction of the energy data knowledge graph. In practical applications, the mean square error threshold, the mean absolute error threshold, and the completeness of the knowledge graph can be used as the preset quality requirements. In this embodiment, the preset mean square error threshold and the preset mean absolute error threshold are used as the preset quality requirements for explanation, but the specific limitations of this embodiment are not made.

[0087] In a specific implementation, the above-mentioned device first constructs an initial energy knowledge graph based on data nodes and edges. The data nodes represent entities in the energy system, such as power generation devices, substations, users, etc.; the data edges represent the relationships between entities, such as power transmission, data collection, etc. For example, in a wind farm, the above-mentioned device identifies entities such as wind turbines, wind speed sensors, power generation monitoring devices, etc. and generates nodes, and then generates edges according to the relationships between the entities to construct an initial energy knowledge graph.

[0088] Next, the above-mentioned device calculates the mean square error and mean absolute error based on the data in the initial energy knowledge graph. The mean square error measures the average of the squared differences between the predicted values and the true values, and is used to evaluate the prediction accuracy of the model; the mean absolute error measures the average of the absolute differences between the predicted values and the true values, and intuitively reflects the size of the average error. For example, the above-mentioned device can compare the difference between the predicted power generation based on the knowledge graph and the actual power generation, and calculate the mean square error and mean absolute error. For example, the preset quality requirement is that the mean square error is less than 0.1 and the mean absolute error is less than 0.05.

[0089] If the quality evaluation result of the initial energy knowledge graph meets the preset quality requirements, the above-mentioned device determines the initial energy knowledge graph as the preset energy data knowledge graph. If the quality evaluation result does not meet the requirements, the above-mentioned device will optimize and adjust the knowledge graph, such as correcting incorrect entity attributes or relationships, supplementing missing data, etc., and then re-perform the quality evaluation until the preset quality requirements are met.

[0090] This embodiment also provides a first embodiment of a data quality evaluation device. Please refer to Figure 6 , Figure 6 which is the data quality evaluation device diagram provided by this embodiment. The data quality evaluation device includes: An analysis module for analyzing the received energy data to be evaluated and determining the current data type of the energy data to be evaluated; A selection module for determining the remaining data types missing during quality evaluation based on the current data type, and obtaining the target energy data corresponding to the remaining data types from historical energy data; A simulation module for generating simulated energy data corresponding to the remaining data types according to the preset energy data knowledge graph and the target energy data, where the preset energy data knowledge graph is generated by the historical energy data; An evaluation module for performing a quality evaluation of the energy data to be evaluated based on the simulated energy data; The evaluation module is further configured to determine whether the simulated energy data meets the preset service rules; when the simulated energy data does not meet the preset service rules, perform regression analysis on the simulated energy data according to the historical energy data; correct the simulated energy data based on the regression analysis result, and perform quality evaluation on the energy data to be evaluated based on the corrected simulated energy data; The selection module is further configured to determine the target evaluation rule corresponding to the current data type based on the current data type; determine the complete data types required by the target evaluation rule according to the target evaluation rule; determine the remaining data types missing during quality evaluation based on the complete data types and the current data type.

[0091] Referring to the first embodiment of the data quality evaluation device, this embodiment also proposes a second embodiment of the data quality evaluation device. For the same or similar content as the first embodiment of the data quality evaluation device, reference can be made to the above introduction and will not be repeated hereinafter.

[0092] The selection module is further configured to perform time series analysis on the target energy data to obtain the time series characteristics corresponding to the target energy data; obtain the data type associations between various data types based on the preset energy data knowledge graph; classify the target energy data according to the time series characteristics, and generate the simulated energy data corresponding to the remaining data types according to the data type associations and the classified target energy data; The selection module is further configured to perform time series node analysis on the energy data to be evaluated to determine the current time series node where the energy data to be evaluated is located; filter out the target time series data from the classified target energy data based on the current time series node, and select the target data type association between the target time series data and the energy data to be evaluated from the data type associations according to the target time series data and the energy data to be evaluated; generate a target association expression according to the target data type association, and generate the simulated energy data corresponding to the remaining data types according to the target time series data and the target association expression.

[0093] Referring to the first embodiment of the data quality evaluation device and the second embodiment of the data quality evaluation device, this embodiment also proposes a third embodiment of the data quality evaluation device. For the same or similar content as the first embodiment of the data quality evaluation device and the second embodiment of the data quality evaluation device, reference can be made to the above introduction and will not be repeated hereinafter.

[0094] The analysis module is further configured to perform entity analysis based on historical energy data to determine data entities corresponding to each data type, and perform association analysis based on the historical energy data to determine data type associations between each data type; generate data nodes according to the data entities, and generate data edges according to the data type associations; construct a preset energy data knowledge graph based on the data nodes and the data edges; The analysis module is further configured to construct an initial energy knowledge graph based on the data nodes and the data edges; obtain the mean square error and the mean absolute error according to the data in the initial energy knowledge graph, and perform quality evaluation on the initial energy knowledge graph according to the mean square error and the mean absolute error; when the quality evaluation result meets the preset quality requirement, use the initial energy knowledge graph as the preset energy data knowledge graph; The analysis module is further configured to perform time series analysis based on historical energy data to obtain historical time series features corresponding to the historical energy data; generate a time series graph according to the historical time series features and the historical energy data, and analyze the time series graph; when the analysis result indicates the existence of missing values, obtain the average value of the historical energy data based on the historical energy data, and fill the historical energy data based on the average value of the historical energy data to obtain new historical energy data; perform entity analysis based on the new historical energy data to determine data entities corresponding to each data type.

[0095] The data quality evaluation device provided in this embodiment adopts the data quality evaluation method in the above embodiment, and can solve the technical problem that the prior art has poor real-time performance due to batch processing after receiving all types of data. Compared with the prior art, the beneficial effects of the data quality evaluation device provided in this embodiment are the same as those of the data quality evaluation method provided in the above embodiment, and other technical features in the data quality evaluation device are the same as the features disclosed in the method of the above embodiment, and will not be elaborated here.

[0096] This embodiment provides a data quality evaluation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data quality evaluation method in the first embodiment above.

[0097] Next, refer to Figure 7 , Figure 7It is a schematic structural diagram of a data quality evaluation device suitable for implementing the data quality evaluation device of this embodiment. The data quality evaluation device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The shown data quality evaluation device is merely an example and should not impose any limitations on the functions and scope of use of this embodiment.

[0098] As Figure 7 shown, the data quality evaluation device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the data quality evaluation device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data quality evaluation device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a data quality evaluation device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0099] In particular, according to this embodiment, the process described above with reference to the flowchart can be implemented as a computer software program. For example, this embodiment includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the method of the disclosed embodiment of this embodiment are executed.

[0100] The data quality evaluation device provided in this embodiment adopts the data quality evaluation method in the above embodiment, and can solve the technical problem that the prior art has poor real-time performance due to batch processing after receiving all types of data. Compared with the prior art, the beneficial effects of the data quality evaluation device provided in this embodiment are the same as those of the data quality evaluation method provided in the above embodiment, and other technical features in this data quality evaluation device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0101] It should be understood that each part disclosed in this embodiment can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0102] As described above, this is only the specific implementation manner of this embodiment, but the protection scope of this embodiment is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this embodiment, and all of them should be covered by the protection scope of this embodiment. Therefore, the protection scope of this embodiment should be subject to the protection scope of the claims.

[0103] This embodiment provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the data quality evaluation method in the above embodiment.

[0104] The computer-readable storage medium provided in this embodiment may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0105] The above computer-readable storage medium may be included in the data quality evaluation device; or it may exist separately without being assembled into the data quality evaluation device.

[0106] The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the data quality evaluation device, the data quality evaluation device is caused to: perform data quality evaluation.

[0107] Computer program code for performing the operations of this embodiment may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present embodiment. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0109] The modules described in this embodiment can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0110] The readable storage medium provided in this embodiment is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above data quality evaluation method, and can solve the technical problem that the prior art has poor real-time performance due to batch processing after receiving all types of data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as those of the data quality evaluation method provided in the above embodiment, and will not be elaborated here.

[0111] The above are only some embodiments, and thus do not limit the patent scope of this embodiment. Any equivalent structural transformation made by using the description of this application and the content of the accompanying drawings under the technical concept of this application, or any direct / indirect application in other related technical fields is included in the patent protection scope of this application.

Claims

1. A data quality evaluation method, characterized in that: The method comprises: Analyze the received energy data to be evaluated to determine the current data type of the energy data to be evaluated; Determine the remaining data types that are missing when performing quality evaluation based on the current data type, and obtain target energy data corresponding to the remaining data types from historical energy data; Generate simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, wherein the preset energy data knowledge graph is generated by the historical energy data; The quality of the energy data to be evaluated is evaluated based on the simulated energy data.

2. The method according to claim 1, characterized in that The step of generating simulated energy data corresponding to the remaining data types according to the preset energy data knowledge graph and the target energy data includes: Performing time series analysis on the target energy data to obtain time series characteristics corresponding to the target energy data; Obtain data type associations between various data types based on a preset energy data knowledge graph; The target energy data is classified according to the time series characteristics, and the simulated energy data corresponding to the remaining data types is generated according to the data type association and the classified target energy data.

3. The method according to claim 1, characterized in that Before the step of analyzing the received energy data to be evaluated, the method further includes: Performing entity analysis based on historical energy data to determine data entities corresponding to each data type, and performing association analysis based on the historical energy data to determine data type associations between the data types; Generate data nodes according to the data entities, and generate data edges according to the data type associations; A preset energy data knowledge graph is constructed based on the data nodes and the data edges.

4. The method according to claim 3, characterized in that The step of constructing a preset energy data knowledge graph based on the data nodes and the data edges includes: Constructing an initial energy knowledge graph based on the data nodes and the data edges; Obtaining a mean square error and a mean absolute error according to the data in the initial energy knowledge graph, and performing a quality assessment on the initial energy knowledge graph according to the mean square error and the mean absolute error; When the quality assessment result meets the preset quality requirements, the initial energy knowledge graph is used as the preset energy data knowledge graph.

5. The method according to claim 3, characterized in that The step of performing entity analysis based on historical energy data to determine data entities corresponding to each data type includes: Performing time series analysis based on historical energy data to obtain historical time series characteristics corresponding to the historical energy data; Generating a time series graph according to the historical time series characteristics and the historical energy data, and analyzing the time series graph; When the analysis result shows that there are missing values, an average value of the historical energy data is obtained based on the historical energy data, and the historical energy data is filled based on the average value of the historical energy data to obtain new historical energy data; An entity analysis is performed based on the new historical energy data to determine data entities corresponding to each data type.

6. The method according to claim 1, characterized in that The step of performing quality evaluation on the energy data to be evaluated based on the simulated energy data comprises: Determining whether the simulated energy data meets preset business rules; When the simulated energy data does not satisfy the preset business rule, performing regression analysis on the simulated energy data according to the historical energy data; The simulated energy data is corrected based on the regression analysis result, and the quality of the energy data to be evaluated is evaluated based on the corrected simulated energy data.

7. The method according to claim 1, characterized in that The step of determining the remaining data types that are missing when performing quality evaluation based on the current data type includes: Determine a target evaluation rule corresponding to the current data type based on the current data type; Determining the complete data type required by the target evaluation rule according to the target evaluation rule; The remaining data types that are missing when performing quality evaluation are determined based on the complete data type and the current data type.

8. A data quality evaluation device, characterized in that: The device comprises: An analysis module, used for analyzing the received energy data to be evaluated, and determining the current data type of the energy data to be evaluated; A selection module, configured to determine the remaining data types that are missing when performing quality evaluation based on the current data type, and obtain target energy data corresponding to the remaining data types from historical energy data; A simulation module, configured to generate simulated energy data corresponding to the remaining data types according to a preset energy data knowledge graph and the target energy data, wherein the preset energy data knowledge graph is generated by the historical energy data; An evaluation module is used to perform quality evaluation on the energy data to be evaluated based on the simulated energy data.

9. A data quality evaluation device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data quality assessment method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data quality assessment method according to any one of claims 1 to 7 are implemented.

Images