Fuel gas data grading evaluation method based on intelligent fuel gas Internet of Things

Abstract

The invention discloses a fuel gas data grading evaluation method based on an intelligent fuel gas Internet of Things, and relates to the technical field of data analysis, and the method comprises the following steps: recognizing heterogeneous system data sources participating in evaluation, collecting original fuel gas data from a plurality of heterogeneous system data sources in the intelligent fuel gas Internet of Things, and storing the collected original fuel gas data in a database; carrying out data processing on the collected original gas data; and integrating the processed original gas data, and performing quality evaluation on the original gas data. According to the method, the sensitivity and importance of the data are accurately identified through grading evaluation of the gas data, so that more strict safety protection measures are taken for the data, reasonable resource allocation is facilitated, cost benefit maximization is realized, high-value data can be preferentially processed and maintained by evaluating the service influence and access frequency of the data, and the method is suitable for popularization and application. And meanwhile, the low-value data is filed or rejected, so that the real-time performance and accuracy of the data are ensured, and the decision quality and the market change response capability are improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a gas data grading evaluation method based on a smart gas Internet of Things. Background Art

[0002] As one of the cleanest and most efficient energy sources, gas is widely used in residents' lives, industrial production and commercial fields. With the continuous development and progress of science and technology, the construction of smart gas has become one of the important directions to promote the transformation and upgrading of the energy industry. Traditional gas systems face challenges in safety management, supply efficiency and energy utilization, and need to rely on advanced technical means for intelligent transformation and management. The introduction of smart gas Internet of Things provides the possibility for the hierarchical evaluation of gas data, thereby promoting the overall upgrading of the gas industry.

[0003] In the existing technology, smart gas systems often involve the integration and fusion of multiple heterogeneous systems, and the gas data indicators associated with different systems are different, which increases the difficulty of data integration. In addition, the importance of data is different, making it difficult to use them in a hierarchical manner, which will lead to limited data utilization. Therefore, how to analyze the quality of heterogeneous system data to improve the accuracy of gas data hierarchical evaluation is the problem we need to solve. To this end, a gas data hierarchical evaluation method based on a smart gas Internet of Things is proposed. Summary of the invention

[0004] The purpose of the present invention is to provide a gas data grading evaluation method based on a smart gas Internet of Things to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A gas data grading evaluation method based on a smart gas Internet of Things comprises the following steps:

[0007] Step 1: Identify the heterogeneous system data sources involved in the evaluation, collect raw gas data from multiple heterogeneous system data sources in the smart gas Internet of Things, and process the collected raw gas data;

[0008] Step 2: Integrate the processed raw gas data, conduct quality assessment on the raw gas data, determine its data quality level, identify and mark abnormal data, establish data mapping relationships, and uniformly store them in the data warehouse;

[0009] Step 3: extract gas data evaluation features from the original gas data, construct a gas data evaluation index system, and divide the evaluation levels based on the importance of the gas data in combination with the original gas data;

[0010] Step 4: Based on the gas data evaluation features and the original gas data, the gas data is evaluated according to the importance, sensitivity and timeliness factors of the gas data, and the gas data classification coefficient is obtained;

[0011] Step 5, set the corresponding threshold of the evaluation level, and determine the specific evaluation level of the gas data according to the result of the gas data classification coefficient, and implement corresponding management and decision-making according to the evaluation result.

[0012] A further improvement of the technical solution of the present invention is that in step 1, the process of collecting the original gas data is:

[0013] Step 101, identifying heterogeneous system data sources containing required gas data, including a gas meter reading system, a gas pipeline monitoring system, a safety alarm system, a user management system, and a meteorological data system;

[0014] Step 102, collect data according to the data interface corresponding to each data source, obtain the original gas data, verify the collected data to ensure the integrity and accuracy of the data, and store the verified data in the data warehouse for subsequent processing and analysis;

[0015] Step 103, preprocessing operations are performed on the original gas data, including data cleaning, data integration and data conversion. Data cleaning is used to remove duplicate data, correct erroneous data, process missing values, etc. to ensure data quality. Data integration is used to integrate data from different data sources according to certain rules to form a unified data format and view. Data conversion is used to convert the data format.

[0016] A further improvement of the technical solution of the present invention is that in step 2, the process of quality assessment of the original gas data is:

[0017] Step 201, matching and integrating gas data from different sources and types according to key fields such as timestamp, user ID, and location, so that data from different data sources correspond correctly;

[0018] Step 202, associating similar data items in different data sources, establishing a data mapping relationship, and storing the integrated and mapped data in a data warehouse, establishing an index to improve query efficiency;

[0019] Step 203, determine the data quality assessment indicators, which are accuracy, completeness, consistency, timeliness and reliability, obtain the data accuracy assessment index, data completeness assessment index, data consistency assessment index, data timeliness assessment index and data reliability assessment index, and evaluate the data quality;

[0020] Step 204: further analyze the identified abnormal data according to the abnormal value detection result of the data to determine the cause, correct or delete the obvious erroneous data, and mark and retain the abnormal but unidentified data for further analysis or verification;

[0021] Step 205 , the data quality assessment indicators are combined and calculated to obtain a comprehensive assessment index, which comprehensively measures the data quality and divides the data into different quality levels to quantify the range of data quality.

[0022] A further improvement of the technical solution of the present invention is that the accuracy is obtained by calculating the absolute deviation of each data point from the standard value, finding the average value, analyzing the degree of deviation between the data point and the standard value, and obtaining a data accuracy evaluation index to evaluate the accuracy of the data quality, and the expression is:

[0023]

[0024] Among them, A X is a data accuracy evaluation index used to measure the deviation of data from the standard value, n is the total number of data points in the data set, and x i is a data point in the data set, t i is the corresponding standard value. It should be noted that A X The smaller the value, the more accurate the data;

[0025] The integrity is obtained by calculating the ratio of the number of non-missing values ​​to the size of the initial data set, analyzing the ratio of non-missing values ​​in the data set, and obtaining a data integrity evaluation index to evaluate the integrity of the data quality. The expression is:

[0026]

[0027] Among them, C X is the data integrity assessment index, n on is the number of non-missing values, n in is the size of the initial data set. It should be noted that C X The closer the value is to 1, the more complete the data is;

[0028] The consistency is calculated by calculating the ratio of the number of consistent data items to the number of all compared data items, analyzing the ratio of consistent data items in different data sources in the data set, and obtaining a data consistency evaluation index to evaluate the consistency of data quality. The expression is:

[0029]

[0030] Among them, Y X is the data consistency assessment index, n csis the number of data items that are consistent across all data sources, n to is the number of all compared data items. It should be noted that Y X The larger the value of, the more consistent the data;

[0031] The timeliness is calculated by dividing the difference between the current time and the last update time of the data by the expected update frequency, analyzing the delay time between the data update and the current time, and obtaining the data timeliness evaluation index to evaluate the timeliness of the data quality. The expression is:

[0032]

[0033] Among them, E X is the data timeliness evaluation index, T c is the current time, T u is the time when the data was last updated, T ex is the expected update frequency. It should be noted that E X The smaller the value, the more timely the data update;

[0034] The reliability is evaluated by calculating the difference between each data point and the average value, dividing it by the standard deviation, analyzing the degree of deviation between a single data point and the average value of the data set, so as to evaluate the reliability of the data quality and obtain a data reliability evaluation index, which is expressed as:

[0035]

[0036] Among them, K X is the data reliability evaluation index, x i is a single data point, is the mean value of the data, σ is the standard deviation of the data. It should be noted that K X The larger the absolute value, the more likely the data point is an outlier.

[0037] A further improvement of the technical solution of the present invention is that the calculation formula of the comprehensive evaluation index is:

[0038]

[0039] Among them, D Z is a comprehensive evaluation index, A X is the data accuracy evaluation index, C X is the data integrity assessment index, Y X is the data consistency assessment index, E X is the data timeliness evaluation index, K X is the data reliability evaluation index, n is the total number of data points in the data set, μ(E X ) is the mean value of the data timeliness evaluation index, is the sum of the data reliability evaluation indexes of all data points in the data set, w1+w2+w3=1. It should be noted that D Z The higher the value, the better the data quality.

[0040] A further improvement of the technical solution of the present invention is that the data quality level classification process is as follows:

[0041] Step 2051, setting the data quality level, which are respectively excellent data quality level, good data quality level, medium data quality level, poor data quality level, and extremely poor data quality level;

[0042] Step 2052, for each data quality level, assign a corresponding data quality assessment threshold;

[0043] Step 2053, using the comprehensive data evaluation index to score the data set and determine the data quality level of the data set;

[0044] The plurality of data quality levels correspond to the plurality of data quality assessment thresholds, wherein the data quality assessment thresholds include an upper threshold and a lower threshold;

[0045] The multiple data quality levels and the multiple data quality assessment thresholds satisfy the following relationship:

[0046] Excellent data quality level D ZY ≤D Z <1;

[0047] Good data quality level D ZL ≤D Z <D ZY ;

[0048] Medium data quality level D ZM ≤D Z <D ZL ;

[0049] Poor data quality level D ZC ≤D Z <D ZM ;

[0050] Very poor data quality level 0 <D Z <D ZC ;

[0051] Among them, D Z is the comprehensive evaluation index, D ZY is the lower threshold corresponding to the excellent data quality level and the upper threshold corresponding to the good data quality level, D ZL is the lower threshold corresponding to the good data quality level and the upper threshold corresponding to the medium data quality level, D ZMis the lower threshold corresponding to the medium data quality level and the upper threshold corresponding to the poor data quality level, D ZC It is the lower threshold corresponding to the poor data quality level and the upper threshold corresponding to the extremely poor data quality level.

[0052] A further improvement of the technical solution of the present invention is that in step 3, the construction of the gas data evaluation index system and the division of the evaluation levels are as follows:

[0053] Step 301, extracting the service characteristics and user behavior characteristics of gas usage, wherein the service characteristics of gas usage include daily gas usage, monthly gas usage, gas usage peak time, gas usage valley time, and user behavior characteristics include user type and user historical usage trend;

[0054] Step 302: Determine the dimensions for evaluating gas data based on the business characteristics and user behavior characteristics of gas use, namely data quality, data security, and data value, and build a gas data evaluation index system in each dimension in combination with the comprehensive data evaluation index to quantitatively evaluate the quality of gas data in each dimension.

[0055] Step 303, combining the original gas data with the constructed gas data evaluation index system, and dividing the gas data into different evaluation levels based on the importance of the gas data, namely, the first evaluation level, the second evaluation level and the third evaluation level, wherein the importance of the evaluation level decreases from the first level to the third level.

[0056] A further improvement of the technical solution of the present invention is that in step 4, the process of obtaining the gas data classification coefficient is:

[0057] Step 401, collect and integrate original gas data, including business feature data and user behavior feature data, define evaluation factors, namely, importance, sensitivity and timeliness of gas data, and extract feature data related to importance, sensitivity and timeliness of gas data;

[0058] Step 402, determining corresponding quantitative indicators for each factor, which are importance indicator, sensitivity indicator and timeliness indicator, and calculating scores for each indicator to convert qualitative factors into quantitative scores;

[0059] Step 403, combining the calculated score of each indicator with the comprehensive data evaluation index to obtain a gas data classification coefficient;

[0060] Step 404, calculate the gas data classification coefficient value of each data set, and classify the data set according to the result of the gas data classification coefficient, and perform a classification evaluation on the gas data.

[0061] A further improvement of the technical solution of the present invention is that the calculation formula of the gas data classification coefficient is:

[0062]

[0063]

[0064] Among them, GC is the gas data classification coefficient, Im(I) is the impact score of data importance, which quantifies the importance of data to business goals and operations, R(S) is the data sensitivity risk score, which quantifies the potential risk of data leakage or misuse, U(T) is the data timeliness urgency score, which quantifies the matching degree between data update frequency and business needs, I is the weighted importance score, S is the weighted sensitivity score, T is the weighted timeliness score, and w I is the importance weight, w S is the sensitivity weight, w T is the timeliness weight, w I +w S +w T =1, i is the sum of the data reliability evaluation indexes of all data points in the data set, measuring the number of standard deviations of a data point relative to the mean of the data set, n is the total number of data points in the data set, and k I is the relative coefficient of the importance index, which is used to indicate the relative weight of the importance of the data. I is the benchmark coefficient of the importance index, which is used to indicate the benchmark level of data importance. S is the relative coefficient of the sensitivity index, which is used to indicate the relative risk of data sensitivity. S is the base coefficient of the sensitivity index, which is used to indicate the base risk of data sensitivity. id is the benchmark data update frequency, T max It is the maximum data update frequency.

[0065] A further improvement of the technical solution of the present invention is that in step 5, the process of determining the specific evaluation level of the gas data is:

[0066] Step 501, according to different evaluation levels set, combined with the threshold interval corresponding to the gas data classification coefficient, the calculated gas data classification coefficient result is compared with the threshold value, and the data set is divided into different evaluation levels;

[0067] Step 502, sharing the gas data classification coefficient results and corresponding evaluation levels of each data set to identify the strengths and weaknesses of the data;

[0068] Step 503, implement corresponding management and decision-making measures according to the evaluation level of the data set, wherein, for data of the first-level evaluation level, implement the most stringent protection measures, and give priority to using the data for key business decisions; for data of the second-level evaluation level, perform routine monitoring and protection, and use it appropriately for business decision-making; for data of the third-level evaluation level, develop improvement plans to improve data quality or consider eliminating low-value data.

[0069] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art:

[0070] 1. The present invention provides a gas data hierarchical evaluation method based on a smart gas Internet of Things. Through the hierarchical evaluation of gas data, the sensitivity and importance of the data can be accurately identified, so that more stringent security protection measures can be implemented on the data, and it is helpful to reasonably allocate resources and maximize cost-effectiveness. By evaluating the business impact and access frequency of the data, high-value data can be processed and maintained first, while low-value data can be archived or eliminated, thereby ensuring the real-time and accuracy of the data, thereby improving the quality of decision-making and the ability to respond to market changes.

[0071] 2. The present invention provides a gas data hierarchical evaluation method based on a smart gas Internet of Things. By performing hierarchical evaluation on gas data, resource allocation can be optimized according to the value and risk of the data. High-value data requires more storage and processing resources, while low-value data can appropriately reduce resource input or be cleaned up, which helps to achieve optimal resource allocation within a limited budget and reduce unnecessary costs. For low-value data that is not frequently accessed, a lower-cost storage solution can be selected, or data can be cleaned up regularly to free up storage space and computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0073] Figure 1 is a flow chart of the method of the present invention;

[0074] Figure 2 A flow chart of quality assessment of raw gas data according to the present invention;

[0075] Figure 3 This is a flow chart for obtaining the gas data classification coefficient of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Embodiment 1, as Figure 1 , Figure 2 As shown, the present invention provides a gas data classification evaluation method based on a smart gas Internet of Things, comprising the following steps:

[0078] Step 1: Identify the heterogeneous system data sources involved in the assessment, collect raw gas data from multiple heterogeneous system data sources in the smart gas Internet of Things, and process the collected raw gas data. Identify the heterogeneous system data sources that contain the required gas data, including the gas meter reading system, gas pipeline monitoring system, safety alarm system, user management system, and meteorological data system. Collect data according to the data interface corresponding to each data source, obtain the raw gas data, and verify the collected data to ensure the integrity and accuracy of the data. Store the verified data in the data warehouse for subsequent processing and analysis. Perform preprocessing operations on the raw gas data, including data cleaning, data integration, and data conversion. Data cleaning is used to remove duplicate data, correct erroneous data, process missing values, etc. to ensure data quality. Data integration is used to integrate data from different data sources according to certain rules to form a unified data format and view. Data conversion is used to convert the data format.

[0079] Step 2: Integrate the processed raw gas data, conduct a quality assessment on the raw gas data, determine its data quality level, identify and mark abnormal data, establish a data mapping relationship, and uniformly store them in the data warehouse. Match and integrate gas data from different sources and types according to the key fields of timestamp, user ID, and location, so that the data between different data sources correspond correctly, associate similar data items in different data sources, establish a data mapping relationship, and store the integrated and mapped data in the data warehouse. Establish an index to improve query efficiency, determine data quality assessment indicators, namely accuracy, completeness, consistency, timeliness, and reliability, and obtain a data accuracy assessment index, a data completeness assessment index, a data consistency assessment index, a data timeliness assessment index, and a data reliability assessment index. Evaluate the data quality. According to the data outlier detection results, further analyze the identified abnormal data to determine its cause. Correct or delete obvious erroneous data. Mark and retain abnormal but undetermined data for subsequent further analysis or verification. Combine and calculate the data quality assessment indicators to obtain a comprehensive assessment index, comprehensively measure the data quality, and divide different data quality levels to quantify the range of data quality.

[0080] Furthermore, the accuracy is evaluated by calculating the absolute deviation of each data point from the standard value, finding the average value, analyzing the degree of deviation between the data point and the standard value, and obtaining the data accuracy evaluation index, which is expressed as follows:

[0081]

[0082] Among them, A X is a data accuracy evaluation index used to measure the deviation of data from the standard value, n is the total number of data points in the data set, and x i is a data point in the data set, t i is the corresponding standard value. It should be noted that A X The smaller the value, the more accurate the data;

[0083] The integrity is evaluated by calculating the ratio of the number of non-missing values ​​to the size of the initial data set, analyzing the ratio of non-missing values ​​in the data set, and obtaining the data integrity evaluation index to evaluate the integrity of the data quality. The expression is:

[0084]

[0085] Among them, C X is the data integrity assessment index, n on is the number of non-missing values, n in is the size of the initial data set. It should be noted that C XThe closer the value is to 1, the more complete the data is;

[0086] Consistency is calculated by calculating the ratio of the number of consistent data items to the number of all compared data items, analyzing the ratio of consistent data items in different data sources in the data set, and obtaining the data consistency evaluation index to evaluate the consistency of data quality. The expression is:

[0087]

[0088] Among them, Y X is the data consistency assessment index, n cs is the number of data items that are consistent across all data sources, n to is the number of all compared data items. It should be noted that Y X The larger the value of, the more consistent the data;

[0089] Timeliness is achieved by calculating the difference between the current time and the last update time of the data, dividing it by the expected update frequency, analyzing the delay between the data update and the current time, and obtaining the data timeliness evaluation index to evaluate the timeliness of data quality. The expression is:

[0090]

[0091] Among them, E X is the data timeliness evaluation index, T c is the current time, T u is the time when the data was last updated, T ex is the expected update frequency. It should be noted that E X The smaller the value, the more timely the data update;

[0092] Reliability is calculated by calculating the difference between each data point and the mean value, dividing it by the standard deviation, analyzing the degree of deviation between a single data point and the mean value of the data set to evaluate the reliability of data quality and obtain the data reliability evaluation index, which is expressed as:

[0093]

[0094] Among them, K X is the data reliability evaluation index, x i is a single data point, is the mean value of the data, σ is the standard deviation of the data. It should be noted that K X The larger the absolute number of the value, the more likely the data point is an outlier;

[0095] Furthermore, the calculation formula of the comprehensive evaluation index is:

[0096]

[0097] Among them, D Z is a comprehensive evaluation index, A X is the data accuracy evaluation index, C X is the data integrity assessment index, Y X is the data consistency assessment index, E X is the data timeliness evaluation index, K X is the data reliability evaluation index, n is the total number of data points in the data set, μ(E X ) is the mean value of the data timeliness evaluation index, is the sum of the data reliability evaluation indexes of all data points in the data set, w1+w2+w3=1. It should be noted that D Z The higher the value of, the better the data quality;

[0098] Furthermore, the data quality level classification process is as follows:

[0099] Set data quality levels, namely excellent data quality level, good data quality level, medium data quality level, poor data quality level, and extremely poor data quality level. For each data quality level, divide the corresponding data quality assessment threshold, use the comprehensive data assessment index to score the data set, and determine the data quality level of the data set. Multiple data quality levels correspond to multiple data quality assessment thresholds, where the data quality assessment threshold includes an upper threshold and a lower threshold;

[0100] The multiple data quality levels and the multiple data quality assessment thresholds satisfy the following relationship:

[0101] Excellent data quality level D ZY ≤D Z <1;

[0102] Good data quality level D ZL ≤D Z <D ZY ;

[0103] Medium data quality level D ZM ≤D Z <D ZL ;

[0104] Poor data quality level D ZC ≤D Z <D ZM ;

[0105] Very poor data quality level 0 <D Z <D ZC ;

[0106] Among them, D Z is the comprehensive evaluation index, D ZYis the lower threshold corresponding to the excellent data quality level and the upper threshold corresponding to the good data quality level, D ZL is the lower threshold corresponding to the good data quality level and the upper threshold corresponding to the medium data quality level, D ZM is the lower threshold corresponding to the medium data quality level and the upper threshold corresponding to the poor data quality level, D ZC The lower threshold corresponding to the poor data quality level and the upper threshold corresponding to the extremely poor data quality level;

[0107] Step 3: extract gas data evaluation features from the original gas data, construct a gas data evaluation index system, and divide the evaluation levels based on the importance of the gas data in combination with the original gas data;

[0108] Step 4: Based on the gas data evaluation features and the original gas data, the gas data is evaluated according to the importance, sensitivity and timeliness factors of the gas data, and the gas data classification coefficient is obtained;

[0109] Step 5, set the corresponding threshold of the evaluation level, and determine the specific evaluation level of the gas data according to the result of the gas data classification coefficient, and implement corresponding management and decision-making according to the evaluation result.

[0110] Embodiment 2, as Figure 3 As shown, based on Example 1, the present invention provides a technical solution: Preferably, in step 3, the construction of the gas data evaluation index system and the division of the evaluation levels are as follows:

[0111] Extract the business characteristics and user behavior characteristics of gas use, where the business characteristics of gas use include daily gas usage, monthly gas usage, gas usage peak time, and gas usage trough time; user behavior characteristics include user type and user historical usage trend; determine the dimensions for evaluating gas data based on the business characteristics and user behavior characteristics of gas use, which are data quality, data security, and data value; and build a gas data evaluation index system in each dimension in combination with the comprehensive data evaluation index, quantitatively evaluate the gas data quality in each dimension, combine the original gas data with the constructed gas data evaluation index system, and divide the gas data into different evaluation levels based on their importance, which are the first-level evaluation level, the second-level evaluation level, and the third-level evaluation level, where the evaluation level decreases in importance from the first level to the third level;

[0112] In step 4, the process of obtaining the gas data classification coefficient is as follows:

[0113] Collect and integrate original gas data, including business characteristic data and user behavior characteristic data, define evaluation factors, namely, gas data importance, sensitivity and timeliness, and extract characteristic data related to gas data importance, sensitivity and timeliness, determine corresponding quantitative indicators for each factor, namely, importance indicator, sensitivity indicator and timeliness indicator, and calculate scores for each indicator, convert qualitative factors into quantitative scores, combine the calculated scores of each indicator with the comprehensive data evaluation index to obtain the gas data classification coefficient, calculate the gas data classification coefficient value of each data set, and classify the data sets according to the results of the gas data classification coefficient, and conduct a graded evaluation of the gas data;

[0114] Furthermore, the calculation formula of the gas data classification coefficient is:

[0115]

[0116]

[0117] b I =min(w i , im);

[0118]

[0119] b S =min(w i ,se);

[0120] Among them, GC is the gas data classification coefficient, Im(I) is the impact score of data importance, which quantifies the importance of data to business goals and operations, R(S) is the data sensitivity risk score, which quantifies the potential risk of data leakage or misuse, U(T) is the data timeliness urgency score, which quantifies the matching degree between data update frequency and business needs, I is the weighted importance score, S is the weighted sensitivity score, T is the weighted timeliness score, and w I is the importance weight, w S is the sensitivity weight, w T is the timeliness weight, w I +w S +w T =1, is the sum of the data reliability evaluation indexes of all data points in the data set, measuring the number of standard deviations of a data point relative to the mean of the data set, n is the total number of data points in the data set, and k I is the relative coefficient of the importance index, which is used to indicate the relative weight of the importance of the data. I is the benchmark coefficient of the importance index, which is used to indicate the benchmark level of data importance. i , im is the importance weight of the i-th data, max(w i, im) is the maximum value of the importance weights of all data points in the data set, min(w i , im) is the minimum value of the importance weights of all data points in the data set, k S is the relative coefficient of the sensitivity index, which is used to indicate the relative risk of data sensitivity. S is the base coefficient of the sensitivity index, which is used to indicate the base risk of data sensitivity. i , se is the sensitivity weight of the i-th data, max(w i , se) is the maximum value of the sensitivity weights of all data points in the data set, min(w i ,se) is the minimum value of the sensitivity weights of all data points in the data set, T id is the benchmark data update frequency, T max is the maximum data update frequency. It should be noted that the value range of Im(I) is [0,1]. The higher the value of Im(I), the greater the importance of the data. The value range of R(S) is [0,1]. The lower the value of R(S), the smaller the sensitivity risk of the data. The value range of U(T) is [0,1]. The higher the value of U(T), the stronger the timeliness of the data, that is, the higher the matching degree between the data update frequency and the business demand. The value range of GC is [0,1]. The closer the value of GC is to 1, the higher the overall classification coefficient of the gas data, that is, the better the comprehensive quality of the data. The value of GC increases with the increase of the data importance score, and the value of GC increases with the decrease of the data sensitivity risk score.

[0121] In step 5, the process of determining the specific evaluation level of gas data is as follows:

[0122] According to the different set assessment levels, combined with the corresponding threshold intervals divided by the gas data classification coefficient, the calculated gas data classification coefficient results are compared with the threshold, the data sets are divided into different assessment levels, the gas data classification coefficient results and the corresponding assessment levels of each data set are shared, the advantages and disadvantages of the data are identified, and according to the assessment level of the data set, corresponding management and decision-making measures are implemented. Among them, for data at the first-level assessment level, the most stringent protection measures are implemented, and the data is used for key business decisions in priority; for data at the second-level assessment level, routine monitoring and protection are carried out, and it is used appropriately for business decision-making; for data at the third-level assessment level, improvement plans are carried out to improve data quality or consider eliminating low-value data.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A gas data grading evaluation method based on smart gas Internet of Things, characterized by: The following steps are involved: Step 1: Identify the heterogeneous system data sources involved in the evaluation, collect raw gas data from multiple heterogeneous system data sources in the smart gas Internet of Things, and process the collected raw gas data; Step 2: Integrate the processed raw gas data, conduct quality assessment on the raw gas data, determine its data quality level, identify and mark abnormal data, establish data mapping relationships, and uniformly store them in the data warehouse; Step 3: extract gas data evaluation features from the original gas data, construct a gas data evaluation index system, and divide the evaluation levels based on the importance of the gas data in combination with the original gas data; Step 4: Based on the gas data evaluation features and the original gas data, the gas data is evaluated according to the importance, sensitivity and timeliness factors of the gas data, and the gas data classification coefficient is obtained; Step 5, set the corresponding threshold of the evaluation level, and determine the specific evaluation level of the gas data according to the result of the gas data classification coefficient, and implement corresponding management and decision-making according to the evaluation result.

2. According to claim 1, a gas data grading evaluation method based on smart gas Internet of Things is characterized by: In step 1, the process of collecting raw gas data is as follows: Step 101, identifying heterogeneous system data sources containing required gas data, including a gas meter reading system, a gas pipeline monitoring system, a safety alarm system, a user management system, and a meteorological data system; Step 102, collect data according to the data interface corresponding to each data source, obtain the original gas data, verify the collected data, and store the verified data in the data warehouse; Step 103, preprocessing the raw gas data, including data cleaning, data integration and data conversion.

3. The gas data grading evaluation method based on the smart gas Internet of Things according to claim 2 is characterized by: In step 2, the process of quality assessment of the original gas data is as follows: Step 201, matching and integrating gas data from different sources and types according to key fields such as timestamp, user ID, and location, so that data between different data sources correspond correctly; Step 202, associating similar data items in different data sources, establishing a data mapping relationship, and storing the integrated and mapped data in a data warehouse, and establishing an index; Step 203, determine the data quality assessment indicators, which are accuracy, completeness, consistency, timeliness and reliability, obtain the data accuracy assessment index, data completeness assessment index, data consistency assessment index, data timeliness assessment index and data reliability assessment index, and evaluate the data quality; Step 204: further analyze the identified abnormal data according to the abnormal value detection result of the data to determine the cause, correct or delete the obvious erroneous data, and mark and retain the abnormal but unidentified data; Step 205 , the data quality assessment indicators are combined and calculated to obtain a comprehensive assessment index, which comprehensively measures the data quality and divides the data into different quality levels to quantify the range of data quality.

4. A gas data grading evaluation method based on smart gas Internet of Things according to claim 3, characterized in that: The accuracy is evaluated by calculating the absolute deviation of each data point from the standard value, finding the average value, analyzing the degree of deviation between the data point and the standard value, and obtaining the data accuracy evaluation index to evaluate the accuracy of the data quality. The expression is: Among them, A X is the data accuracy evaluation index, n is the total number of data points in the data set, and x i is a data point in the data set, t i is the corresponding standard value; The integrity is obtained by calculating the ratio of the number of non-missing values ​​to the size of the initial data set, analyzing the ratio of non-missing values ​​in the data set, and obtaining a data integrity evaluation index to evaluate the integrity of the data quality. The expression is: Among them, C X is the data integrity assessment index, n on is the number of non-missing values, n in is the size of the initial dataset; The consistency is calculated by calculating the ratio of the number of consistent data items to the number of all compared data items, analyzing the ratio of consistent data items in different data sources in the data set, and obtaining a data consistency evaluation index to evaluate the consistency of data quality. The expression is: Among them, Y X is the data consistency assessment index, n cs is the number of data items that are consistent across all data sources, n to is the number of all data items being compared The timeliness is calculated by dividing the difference between the current time and the last update time of the data by the expected update frequency, analyzing the delay time between the data update and the current time, and obtaining the data timeliness evaluation index to evaluate the timeliness of the data quality. The expression is: Among them, E X is the data timeliness evaluation index, T c is the current time, T u is the time when the data was last updated, T ex is the desired update frequency; The reliability is evaluated by calculating the difference between each data point and the average value, dividing it by the standard deviation, analyzing the degree of deviation between a single data point and the average value of the data set, so as to evaluate the reliability of the data quality and obtain a data reliability evaluation index, which is expressed as: Among them, K X is the data reliability evaluation index, x i is a single data point, is the mean of the data and σ is the standard deviation of the data.

5. A gas data grading evaluation method based on smart gas Internet of Things according to claim 4, characterized in that: The calculation formula of the comprehensive evaluation index is: Among them, D Z is a comprehensive evaluation index, A X is the data accuracy evaluation index, C X is the data integrity assessment index, Y X is the data consistency assessment index, E X is the data timeliness evaluation index, K X is the data reliability evaluation index, n is the total number of data points in the data set, μ(E X ) is the mean value of the data timeliness evaluation index, It is the sum of the data reliability evaluation indexes of all data points in the data set, w1+w2+w3=1.

6. A gas data grading evaluation method based on smart gas Internet of Things according to claim 5, characterized in that: The data quality level classification process is as follows: Step 2051, setting the data quality level, which are respectively excellent data quality level, good data quality level, medium data quality level, poor data quality level, and extremely poor data quality level; Step 2052, for each data quality level, assign a corresponding data quality assessment threshold; Step 2053, using the comprehensive data evaluation index to score the data set and determine the data quality level of the data set; The plurality of data quality levels correspond to the plurality of data quality assessment thresholds, wherein the data quality assessment thresholds include an upper threshold and a lower threshold; The multiple data quality levels and the multiple data quality assessment thresholds satisfy the following relationship: Excellent data quality level D ZY ≤D Z <1; Good data quality level D ZL ≤D Z <D ZY ; Medium data quality level D ZM ≤D Z <D ZL ; Poor data quality level D ZC ≤D Z <D ZM ; Very poor data quality level 0 <D Z <D ZC ; Among them, D Z is the comprehensive evaluation index, D ZY is the lower threshold corresponding to the excellent data quality level and the upper threshold corresponding to the good data quality level, D ZL is the lower threshold corresponding to the good data quality level and the upper threshold corresponding to the medium data quality level, D ZM is the lower threshold corresponding to the medium data quality level and the upper threshold corresponding to the poor data quality level, D ZC It is the lower threshold corresponding to the poor data quality level and the upper threshold corresponding to the extremely poor data quality level.

7. A gas data grading evaluation method based on smart gas Internet of Things according to claim 6, characterized in that: In step 3, the process of constructing the gas data evaluation index system and dividing the evaluation levels is as follows: Step 301, extracting the service characteristics and user behavior characteristics of gas usage, wherein the service characteristics of gas usage include daily gas usage, monthly gas usage, gas usage peak time, gas usage valley time, and user behavior characteristics include user type and user historical usage trend; Step 302: Determine the dimensions for evaluating gas data based on the business characteristics and user behavior characteristics of gas use, namely data quality, data security, and data value, and build a gas data evaluation index system in each dimension in combination with the comprehensive data evaluation index to quantitatively evaluate the quality of gas data in each dimension. Step 303, combining the original gas data with the constructed gas data evaluation index system, and dividing the gas data into different evaluation levels based on the importance of the gas data, namely, the first evaluation level, the second evaluation level and the third evaluation level, wherein the importance of the evaluation level decreases from the first level to the third level.

8. The gas data grading evaluation method based on the smart gas Internet of Things according to claim 7 is characterized by: In step 4, the process of obtaining the gas data classification coefficient is as follows: Step 401, collect and integrate original gas data, including business feature data and user behavior feature data, define evaluation factors, namely, importance, sensitivity and timeliness of gas data, and extract feature data related to importance, sensitivity and timeliness of gas data; Step 402, determining corresponding quantitative indicators for each factor, which are importance indicator, sensitivity indicator and timeliness indicator, and calculating scores for each indicator to convert qualitative factors into quantitative scores; Step 403, combining the calculated score of each indicator with the comprehensive data evaluation index to obtain a gas data classification coefficient; Step 404, calculate the gas data classification coefficient value of each data set, and classify the data set according to the result of the gas data classification coefficient, and perform a classification evaluation on the gas data.

9. The gas data grading evaluation method based on the smart gas Internet of Things according to claim 8 is characterized by: The calculation formula of the gas data classification coefficient is: Among them, GC is the gas data classification coefficient, Im(I) is the impact score of data importance, R(S) is the data sensitivity risk score, U(T) is the data timeliness urgency score, I is the weighted importance score, S is the weighted sensitivity score, T is the weighted timeliness score, and w I is the importance weight, w S is the sensitivity weight, w T is the timeliness weight, w I +w S +w T =1, is the sum of the data reliability evaluation indexes of all data points in the data set, n is the total number of data points in the data set, and k I is the relative coefficient of the importance index, which is used to indicate the relative weight of the importance of the data. I is the benchmark coefficient of the importance index, which is used to indicate the benchmark level of data importance. S is the relative coefficient of the sensitivity index, which is used to indicate the relative risk of data sensitivity. S is the base coefficient of the sensitivity index, which is used to indicate the base risk of data sensitivity. id is the benchmark data update frequency, T max It is the maximum data update frequency.

10. A gas data grading evaluation method based on smart gas Internet of Things according to claim 9, characterized in that: In step 5, the process of determining the specific evaluation level of the gas data is as follows: Step 501, according to different evaluation levels set, combined with the threshold interval corresponding to the gas data classification coefficient, the calculated gas data classification coefficient result is compared with the threshold value, and the data set is divided into different evaluation levels; Step 502, sharing the gas data classification coefficient results and corresponding evaluation levels of each data set to identify the strengths and weaknesses of the data; Step 503: Implement corresponding management and decision-making measures according to the evaluation level of the data set.

Images