Standard management method and system for gas data, terminal and storage medium

Abstract

The invention discloses a standard management method and system for fuel gas data, a terminal and a storage medium, and the method comprises the steps: obtaining original fuel gas data, and carrying out the preprocessing of the original fuel gas data, and obtaining target fuel gas data; determining a fuel gas quality demand, and performing data quality analysis on the target fuel gas data according to the fuel gas quality demand to obtain a fuel gas data quality analysis result; and determining abnormal gas data in the gas data quality analysis result, and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data meeting the gas quality requirement. According to the method, the multi-source gas data is acquired, and the data quality analysis is performed on the multi-source gas data according to the gas quality requirement, so that the abnormal gas data in the multi-source gas data is detected for decision processing, the standard management and the data quality of the gas data can be effectively improved, and meanwhile, the detection efficiency and the processing efficiency of the abnormal gas data can also be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a standard management method, system, terminal and computer-readable storage medium for gas data. Background Art

[0002] At present, all gas companies have basically built their own multiple business systems and data middle platforms. The data middle platform can manage and call the business data of multiple business systems. However, the data types and data formats of these business data are different, which makes it impossible for the data middle platform to achieve unified standard management of data, nor to detect abnormal data for these integrated business data, and cannot meet user needs.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of the present invention is to provide a standard management method, system, terminal and computer-readable storage medium for gas data, aiming to solve the problem in the prior art that the data types and data formats of the gas business data collected by the gas data middle-end system are different, resulting in the inability to achieve unified standard management of gas business data to detect abnormal data.

[0005] To achieve the above object, the present invention provides a standard management method for gas data, which comprises the following steps:

[0006] Acquire raw gas data, and preprocess the raw gas data to obtain target gas data;

[0007] Determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result;

[0008] Abnormal gas data in the gas data quality analysis result is determined, and intelligent decision processing is performed on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

[0009] Optionally, the standard management method for gas data, wherein the acquiring of raw gas data and preprocessing of the raw gas data to obtain target gas data, specifically includes:

[0010] Determine the target gas data center, and obtain the original gas data of each subsystem module in the target gas data center;

[0011] The raw gas data is preprocessed to obtain target gas data, wherein the preprocessing includes data cleaning processing, outlier removal processing, missing value filling processing, data format conversion processing and normalization processing.

[0012] Optionally, the standard management method for gas data, wherein the determining of the gas quality requirement and performing data quality analysis on the target gas data according to the gas quality requirement to obtain the gas data quality analysis result, specifically includes:

[0013] Determining gas quality requirements, wherein the gas quality requirements include preset data quality analysis dimensions, preset data quality analysis rules, and preset data quality analysis indicators;

[0014] A gas data analysis model is determined according to the gas quality requirement, and data quality analysis is performed on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result.

[0015] Optionally, the standard management method for gas data, wherein the step of performing data quality analysis on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result, specifically includes:

[0016] Acquire historical gas data, and construct a gas data training set based on the historical gas data;

[0017] Determine a preset neural network model, and perform model training and model fine-tuning on the preset neural network model according to the gas data training set to obtain a gas data analysis model;

[0018] The target gas data is input into the gas data analysis model, and a gas data quality analysis result is output.

[0019] Optionally, the standard management method for gas data, wherein the determining of abnormal gas data in the gas data quality analysis result and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement, specifically includes:

[0020] Determining abnormal gas data in the gas data quality analysis result and determining a decision database;

[0021] Searching the decision database according to the abnormal gas data to obtain an intelligent decision method corresponding to the abnormal gas data;

[0022] The abnormal gas data is processed by intelligent decision-making according to the intelligent decision-making method to obtain target quality gas data that meets the gas quality requirement.

[0023] Optionally, the standard management method for gas data, wherein the abnormal gas data in the gas data quality analysis result is determined, and the abnormal gas data is intelligently decided and processed to obtain a target quality gas number that meets the gas quality requirement, further comprising:

[0024] Setting a data quality detection cycle, and monitoring and processing target quality gas data according to the data quality detection cycle and the preset data quality analysis index;

[0025] When the target quality gas data is abnormal, intelligent decision-making is performed.

[0026] Optionally, the standard management method for gas data, wherein the abnormal gas data in the gas data quality analysis result is determined, and the abnormal gas data is intelligently decided and processed to obtain a target quality gas number that meets the gas quality requirement, further comprising:

[0027] The target quality gas data is visualized to obtain a visualization chart, wherein the visualization chart includes a conventional chart, a special-shaped chart, a dashboard and a map.

[0028] In addition, to achieve the above-mentioned purpose, the present invention also provides a standard management system for gas data, wherein the standard management system for gas data comprises:

[0029] A data preprocessing module, used to obtain raw gas data and preprocess the raw gas data to obtain target gas data;

[0030] A data quality analysis module, used to determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result;

[0031] The intelligent decision processing module is used to determine the abnormal gas data in the gas data quality analysis result, and perform intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a standard management program for gas data stored in the memory and executable on the processor, and when the standard management program for gas data is executed by the processor, the steps of the standard management method for gas data as described above are implemented.

[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a standard management program for gas data, and when the standard management program for gas data is executed by a processor, the steps of the standard management method for gas data as described above are implemented.

[0034] In the present invention, raw gas data is obtained, and the raw gas data is preprocessed to obtain target gas data; gas quality requirements are determined, and data quality analysis is performed on the target gas data according to the gas quality requirements to obtain gas data quality analysis results; abnormal gas data in the gas data quality analysis results is determined, and intelligent decision processing is performed on the abnormal gas data to obtain target quality gas data that meets the gas quality requirements. By collecting multi-source gas data and performing data quality analysis on the multi-source gas data according to gas quality requirements, the present invention can not only perform standardized management on the data, but also identify abnormal data in the multi-source gas data. At the same time, by determining the corresponding intelligent decision, the processing efficiency of abnormal data can be effectively improved to ensure the data quality of the gas data. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a preferred embodiment of the standard management method of gas data of the present invention;

[0036] Figure 2 It is a structural diagram of a preferred embodiment of the standard management system for gas data of the present invention;

[0037] Figure 3 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] At present, all gas companies have basically built their own multiple business systems and data middle platforms. The data middle platform can manage and call the business data of multiple business systems. However, the data types and data formats of these business data are different, which makes it impossible for the data middle platform to achieve unified standard management of business data, nor to detect abnormal data for these integrated business data, and cannot meet user needs.

[0040] There are usually problems of non-standard, incomplete, repeated, erroneous and inconsistent data between multi-source heterogeneous data of gas. These problems lead to prominent data quality problems in business sedimentation and make it difficult to form effective data resources to support applications. The present invention aims to improve data quality and propose improved decisions. After achieving data quality improvement, it can support the work of business departments and provide faster, more accurate and effective data.

[0041] The standard management method of gas data described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the standard management method of gas data includes the following steps:

[0042] Step S10: acquiring original gas data, and preprocessing the original gas data to obtain target gas data.

[0043] The present invention collects and defines data requirements: In the data operation process, it is first necessary to clarify the data requirements, including the goals and requirements of data collection, processing and analysis. An effective communication mechanism should be established between the business systems through the data middle platform to collect and clarify data requirements so that subsequent data work can be carried out in a targeted manner.

[0044] For data collection and cleaning: it is necessary to formulate a corresponding data collection plan according to data requirements and ensure that the collected data is accurate, complete and consistent. At the same time, the collected raw data is cleaned and preprocessed as necessary, where preprocessing includes removing outliers, filling missing values, etc., to improve the quality of the data.

[0045] Specifically, determine the target gas data center, and obtain the original gas data of each subsystem module in the target gas data center; preprocess the original gas data to obtain the target gas data, wherein the preprocessing includes data cleaning processing, outlier removal processing, missing value filling processing, data format conversion processing and normalization processing.

[0046] For data processing and analysis: In the data operation process, data processing and analysis are the core links. According to data requirements and analysis objectives, select appropriate data processing methods and tools to perform data conversion, integration and calculation operations to obtain valuable insights and results.

[0047] The present invention is provided with a data acquisition module for collecting various data of gas: the data acquisition module is the source of basic data of the entire smart gas system, so it is designed at the bottom layer, which includes various Internet of Things monitoring data of the SCADA system, basic pipe network data of the GIS system, on-site situation data of the production operation management system, purchase and sales data of the marketing system, production and sales data of the CNG business system, emergency system data, etc. (that is, the original gas data of each subsystem module in the present invention).

[0048] The present invention also provides a data transmission module to transmit the collected various gas data. The data transmission module is the channel part of the entire data transmission, which is divided into two types: wired transmission and wireless transmission. Taking into account various situations of the data flow of the smart gas system, the data transmission channel is designed by combining wired and wireless methods, including signal cable transmission, optical fiber broadband transmission, 2 / 3 / 4 / 5G mobile network transmission, NB-IOT narrowband Internet of Things transmission and other specific means.

[0049] Among them, the process of preprocessing data, in addition to removing outliers and filling missing values, can also include data filling, data denoising, data sampling, type conversion, data normalization and other processing methods. Among them, data filling uses mean filling and simple formula calculation combined with business logic, and the calculation result is used as filling data; data denoising combines specific algorithms to detect outlier data points and then selectively deletes them; data sampling adopts random sampling and combines specific algorithms to solve the problems of overfitting and data loss; type conversion combines various encoding methods to convert non-numeric data into numerical data; data normalization uses linear function normalization to eliminate the influence between data features.

[0050] Furthermore, for the data conversion and standardization process, the present invention sets a data acquisition module, which provides flexible and configurable data conversion and standardization functions, and can convert the data in different data sources into the standard format required by the data middle platform system. In addition, in order to facilitate subsequent analysis and calculation, unified unit conversion, timestamp adjustment and other operations can also be performed on the data.

[0051] In addition, the data collection module is scalable and customizable, allowing configuration and customization according to specific business needs. Users can expand the data collection module according to their own requirements to adapt to different data sources and collection needs. At the same time, the data collection module also provides flexible configuration options. Users can customize parameters such as collection frequency, data fields, and collection strategies to meet specific operation and maintenance requirements.

[0052] In addition, suggestions for data collection and supplementary recording are as follows: During project implementation, if the source systems cannot provide corresponding business data, the gas business department will need to manually collect relevant data and establish a data supplementary recording platform to load the collected supplementary business data into the data platform.

[0053] A feasible solution for data collection and supplementary recording is that data collection is performed using a data collection template in Excel format. The user uploads the Excel data to the Web server through a Web page, and the verification module reads the data and performs a standardization and integrity check on it. The data that passes the check is handed over to the loading module to be loaded into the data platform; the data that fails the check is returned to the user through the Web page for modification. The system implementation of data collection in the present invention includes a user Web application for data upload, download, and error verification, as well as the implementation of a batch supplementary recording and loading system for data.

[0054] Step S20: determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result.

[0055] Specifically, the gas quality requirements are determined, wherein the gas quality requirements include preset data quality analysis dimensions, preset data quality analysis rules, and preset data quality analysis indicators; a gas data analysis model is determined according to the gas quality requirements, and historical gas data is obtained, and a gas data training set is constructed according to the historical gas data; a preset neural network model is determined, and model training and model fine-tuning are performed on the preset neural network model according to the gas data training set to obtain a gas data analysis model; the target gas data is input into the gas data analysis model, and a gas data quality analysis result is output.

[0056] The present invention is provided with a data analysis and prediction module, which is the core component of the data middle platform system in the present invention. It provides evaluation and optimization suggestions for digital operation processes and system performance by conducting in-depth analysis and prediction of the collected data. The data analysis and prediction module also includes the following sub-modules: data analysis algorithms and models. The data analysis and prediction module has built-in various data analysis algorithms and models to adapt to different types of data and business scenarios, including but not limited to statistical analysis, machine learning, data mining and other technologies, which are used to discover the correlation, trend, outliers, etc. between data, and can analyze historical data and real-time data to extract valuable information and insights.

[0057] The data analysis and prediction module provided in the present invention includes a preset neural network model, which may be a perceptron, a convolutional neural network, an autoencoder, etc. The preset neural network model can be trained and fine-tuned with historical business data to analyze the gas data, thereby obtaining a gas data quality analysis result.

[0058] For data quality assessment: the present invention targets multiple business systems such as gas CIS system, GIS system, PS system, pipeline integrity system and school system for nine categories (uniqueness, accuracy, relevance, consistency, integrity, validity, timeliness, accessibility and compliance) of data quality issues, customizes business rules and verification SQL, completes non-null check, foreign key constraint check, business rule check, uniqueness check, value range check, etc. for business systems, and analyzes indicators such as the number of verified records, the number of problem data records and their proportion, to form a report on the distribution of data quality issues and the distribution and proportion of data quality issue categories, etc.

[0059] According to the reference of data standard information items, sort out the usage coverage of data standard information items in each business system, summarize the information items commonly used by each business system, formulate the merging principles, merging methods, merging rules and merging strategies for the same information item fields of different business systems, and establish information coverage principles, mainly including the following: 1. Take a certain system as the standard, and only take the information items of this business system; 2. Take a certain system as the standard, and other systems will supplement with empty values, and subsequent updates will be based on this system; 3. Take a certain system as the standard, and other systems will supplement and update subsequently, and the information entering the system later will be the standard.

[0060] As for the data quality analysis execution method, the present invention divides the data quality analysis method into four steps: 1. Definition: Determine the data quality requirements and determine the success measurement indicators that are consistent with the business requirements for the project, that is, determine the dimensions, rules, evaluation indicators, etc. of the data quality analysis, and provide standards and basis for data quality analysis. 2. Inspection: Check the selected data according to the defined analysis dimensions, rules, etc. 3. Analysis: Based on the results of the inspection, with the assistance of various business systems, conduct an evaluation and analysis to find out the existing data problems and the main causes of the problems, and generate and submit a data quality evaluation report. 4. Improvement: Based on the results of the data quality cause analysis, plan a data quality improvement plan to improve data quality.

[0061] Furthermore, the present invention defines the tasks to be performed in the stages, including defining data quality management requirements and defining data quality management dimensions: 1. Define data quality management requirements: Based on compliance requirements and the application function of compliance reports, data quality requirements are proposed as the information basis for determining the scope of data quality management, designing data quality rules, selecting key quality assessment indicators, determining data quality improvement plans and continuous quality control. 2. Define data quality management dimensions: Based on data quality management requirements, select data quality management dimensions that match the data quality management requirements of the field and the calculation method of measurement indicators for each key data field.

[0062] Among them, the most commonly used data quality dimensions include: 1. Data validity (Valid): Determine whether the data conforms to the business definition in the data standard. 2. Data consistency (Consistent): When the same data item is recorded multiple times in different systems or different tables in the same system, whether multiple data values ​​are the same. 3. Data accuracy (Precise): Whether the data conforms to the business definition in the data standard and reflects the actual business situation. 4. Data uniqueness (Unique): Whether a business-unique key data item value combination corresponds to only one record. 5. Data completeness (Complete): Whether the key data items required for business needs are defined in the system, or whether all key data items have been collected, such as whether there are unfilled data records for the contract validity date.

[0063] Step S30: determining abnormal gas data in the gas data quality analysis result, and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

[0064] First, define the data quality measurement benchmark, define the evaluation index for each measurement rule, and set an expected data quality threshold for the evaluation index. By comparing the gap between the actual measurement result and the baseline, analyze the actual state of data quality and the size of the gap. In the present invention, the gas data quality analysis result is compared with the preset quality result to determine the abnormal gas data in the gas data quality analysis result.

[0065] Specifically, abnormal gas data in the gas data quality analysis result is determined, and a decision database is determined; the decision database is searched according to the abnormal gas data to obtain an intelligent decision method corresponding to the abnormal gas data; and intelligent decision processing is performed on the abnormal gas data according to the intelligent decision method to obtain target quality gas data that meets the gas quality requirements.

[0066] Among them, the decision database set up in the present invention stores the causes of abnormal data corresponding to various abnormal gas data and the corresponding processing decision methods. When abnormal gas data occurs, it can be quickly searched through the decision database to determine the intelligent decision-making method for solving the abnormal gas data and call it, which effectively improves the processing efficiency of abnormal gas data.

[0067] It can be understood that the present invention sets the tasks to be performed in the inspection phase: 1. Formulate data quality measurement rules: design measurement rules for data quality analysis based on data quality requirements and the determined data quality analysis scope, combined with data quality dimensions. 2. Write data quality measurement scripts: convert data quality measurement rules into SQL statements used by data quality measurement tools on the data governance platform. 3. Measure data quality: execute the measurement script to obtain the quality indicator value of each measurement rule and the problem data sample file. 4. Analyze the causes of data quality problems: discover the causes of data quality problems as the basis for defining data quality improvement points. The abnormal data obtained in the measurement phase is the main analysis object. Representative abnormal data records are selected from them. The causes of the data are analyzed through interviews or surveys with business departments and IT technicians, and the causes of the problems are classified as information basis for identifying and defining data quality improvement points. The causes of data quality can be analyzed from four aspects: data standards / business requirements, processes, technical means, and personnel responsibilities, and the cause types can be subdivided under each category.

[0068] Furthermore, a data quality detection cycle is set, and the target quality gas data is monitored and processed according to the data quality detection cycle and the preset data quality analysis index; when the target quality gas data is abnormal, intelligent decision-making processing is performed.

[0069] Regarding data quality management: Data operation process management methods should focus on data quality management, including establishing data quality inspection and monitoring mechanisms to promptly discover and resolve data quality issues. At the same time, data quality assessment indicators and assessment cycles should be established to continuously improve data quality and enhance data accuracy and credibility. By implementing data quality management, enterprises can ensure the orderly progress of data work, improve data quality and efficiency, and provide high-quality data services and decision support.

[0070] Furthermore, the target quality gas data is visualized to obtain a visualization chart, wherein the visualization chart includes a conventional chart, a special-shaped chart, a dashboard and a map.

[0071] The present invention visualizes the data and adopts feature selection when performing specific data analysis, that is, selecting the most relevant data attributes from a large number of data features.

[0072] The present invention provides for data reporting and presentation: in the data operation process, data reports and visual presentations are submitted to relevant departments and positions in a timely and effective manner to support business decisions and promote business development. Data reports and visual presentations should be concise, intuitive and easy to understand, and customized according to the needs of the audience.

[0073] At the same time, the present invention supports intelligent decision-making processing: the data analysis and prediction module provided in the present invention can provide an intuitive data visualization interface, display the analysis results in the form of charts, reports, etc., so that managers can quickly understand the meaning and trend of the data. In addition, the module also supports intelligent decision support functions, and provides an intelligent decision support system based on data analysis results and prediction suggestions to help managers make scientific and effective operational decisions.

[0074] The present invention also provides real-time monitoring and status display: the data middle platform system provides real-time monitoring and status display functions, and displays various key indicators and system status in the form of dashboards, charts or maps, so that managers can understand the real-time values, trends and warning values ​​of various indicators, as well as the operating status and abnormal conditions of the system at a glance. The data middle platform provides comprehensive reporting and analysis functions, conducts in-depth analysis and visual presentation of the collected data, and can generate various customized reports, statements and charts to help managers fully understand the operation and evaluate the changes and trends of various indicators.

[0075] Among them, the present invention provides a rich variety of visualization charts to provide visualization charts for different businesses and different scene display needs, including regular charts and special-shaped charts, with built-in multiple color schemes and visualization settings, to help users simply and quickly realize data visualization and multi-scenario data display needs.

[0076] Furthermore, for the tasks performed in the analysis phase: after performing data quality measurement, generate detailed statistics on the error ratio of data quality for each rule, and evaluate the overall data quality status based on the detailed statistical data by business line.

[0077] Remediation measures to improve data quality include: reasonable adjustment of business processing procedures, selection of appropriate business system or interface improvement suggestions, and reasonable adjustment of business processing procedures.

[0078] Among them, for the reasonable adjustment of business processing flow, the project team found in the early stage of the difference analysis and data quality analysis of the data of each business system that some abnormal data was caused by the imperfect and incomplete business process of gas. Although the missing or unqualified data can be partially solved by data supplementation or other methods, from the perspective of long-term development of gas, the business process should be reasonably adjusted to provide a sustainable solution. It can be handled from the following aspects: 1. According to the business meaning and source of the difference data, formulate the corresponding business plan for data generation, design the corresponding processing flow on the basis of full communication with the relevant business departments, and determine the access to these data. 2. On the basis of clarifying the corresponding processing flow, determine the business system to access these data items. Factors such as the collection time of the data item and the lag time difference with the actual business to generate the information need to be considered. 3. For important business data items, it is necessary to implement the key responsibilities such as the accuracy of the data in the business system to the main person in charge of the relevant business department and establish a strict performance appraisal system (the time of discovering the problem, the discovery of the root cause of the problem, and the timeliness and rationality of the solution) and supplemented by strict operating procedures to effectively and permanently ensure the qualified data content and corresponding business significance. 4. When determining the improvement or adjustment plan for the business data system for missing special data items, the impact on the existing business system should be comprehensively considered. For data whose impact on existing information and application is easier to control, the main thing is to choose the time and location of data generation and storage; for data items that are comprehensively valued by multiple information sources, there are often multiple options to choose from, and an operational solution can be formulated based on the understanding and operation process of relevant business personnel. In short, specific business data items are standardized and set with concise and clear data values ​​and accurate definition descriptions (data dictionaries), and data integration and management in the entire process of data processing in each business system are clearly analyzed and presented.

[0079] Select appropriate business system or interface improvement suggestions. Data improvement usually includes supplementing data through business system improvements and re-entering data into the data warehouse; data quality improvement usually includes business system and data warehouse data cleansing, business system and data warehouse improvements, business process improvements, etc.

[0080] To this end, it is necessary to comprehensively consider the data quality improvement plan from five dimensions: data difference, data quality, IT system, business line and data characteristics, and work closely with relevant business departments to implement different improvement suggestions for different data classifications. The goal is to meet the requirements of data quality improvement while meeting time requirements and cost-effectiveness principles. It can be considered from the following aspects: 1. Data type: static data refers to data that does not participate in business logic processing (such as approval) or data logic processing (such as summary calculation); dynamic data refers to data that participates in business logic processing (such as approval) or data logic processing (such as summary calculation). 2. Data priority: such as high, medium, low priority, or a finer level. 3. Data relevance: such as isolated data and high, medium, low relevance data, or a finer level. 4. Time dimension: new data and historical data. The different scopes of source system improvements caused by different dimensions of data difference and different dimensions of data quality will be described in the following content. Suggestions for business system improvement: Business system improvement will affect the interface and data platform changes, and business systems usually have real-time requirements. Therefore, the quality improvement or cleaning of the least data items should be classified into this category. Even for the data items classified into this category, under the premise of meeting the data quality improvement requirements, it is also necessary to consider the most suitable solution according to the time requirements and cost-effectiveness principle. This can be considered from the following aspects: 1. Suggestions for business system data supplementation: Usually, suggestions for data supplementation include improving the business system and directly supplementing the data platform. Among them, business system improvement can be divided into the following solutions: supplementing data to the business system database through the ETL process of the external data source; adding a supplement interface and adding data items to the supplement interface; modifying the business interface involved in the data item and adding data items to the business interface; modifying the entire logical processing flow involved in the data item and adding data items to each processing link; building a new business system, improving the application system, and collecting data required by supervision. The cost and difficulty of the above various improvement solutions for the business system are gradually increasing. At the same time, the above various improvements require the establishment of supporting processes, positions, etc., and the cost and difficulty are also increasing step by step. Therefore, data supplementation and business system improvement are the relationship between goals and actions. Only when the business system improvement design fully considers the data supplementation needs and solutions can the pressure and difficulty of business system improvement be reduced. 2. Design of business system data quality improvement plan: The design of data quality improvement plan is essentially the subsequent execution stage of data quality assessment. The data quality improvement plan design method is divided into four steps: determine the direction of data quality improvement, define data supplementation tasks, define the implementation plan of data supplementation tasks, and implement the data supplementation plan. Determine the direction of data quality improvement: According to the data quality reasons obtained from the data quality cause analysis, find out the corresponding improvement direction. The improvement direction corresponds to the cause analysis, and is also divided into four categories: business standards / requirements, processes, technical means, and personnel responsibilities.During the project execution, the type, improvement points and requirements of data quality improvement directions will be determined based on the specific data quality reasons. Define data improvement tasks: Data quality improvement tasks mainly focus on data cleaning and re-entry, system process improvement, strengthening data standardization, and standardizing personnel operations. Based on the improvement direction determined in the data quality analysis task and in combination with the specific rules for data quality problems, define a data remediation task list, and define the goals and specific remediation requirements for each data remediation task.

[0081] The smart gas system provided in the present invention can carry out intelligent upgrade and transformation of the existing gas network management through information technology and means, so as to achieve the purpose of improving the production and operation efficiency of the enterprise and ensuring the safety of the production and operation of the enterprise. On the basis of existing resources, it combines smart terminals, communication technology, cloud computing and other technologies to realize the electronicization and intelligence of the entire production and operation process and the standardization, standardization, process and institutionalization of management.

[0082] The smart gas system provided by the present invention realizes production automation, intelligent operation, scientific management decision-making, information resourceization and networking operation. It can drive the smooth operation of the entire business process of the enterprise with a perfect information flow and information sharing mechanism, improve work efficiency, ensure the monitoring and command of each business link by the decision-making management system, and realize macro decision-making and planning guidance through information analysis and data mining.

[0083] Furthermore, if Figure 2 As shown, based on the above-mentioned standard management method for gas data, the present invention also provides a standard management system for gas data, wherein the standard management system for gas data includes:

[0084] The data preprocessing module 51 is used to obtain raw gas data and preprocess the raw gas data to obtain target gas data;

[0085] The data quality analysis module 52 is used to determine the gas quality requirement and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result;

[0086] The intelligent decision processing module 53 is used to determine the abnormal gas data in the gas data quality analysis result, and perform intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

[0087] Furthermore, if Figure 3 As shown, based on the above standard management method and system for gas data, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0088] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a standard management program 40 for gas data is stored on the memory 20, and the standard management program 40 for gas data can be executed by the processor 10, thereby realizing the standard management method for gas data in the present application.

[0089] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the standard management method of the gas data.

[0090] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0091] In one embodiment, when the processor 10 executes the standard management program 40 of the gas data in the memory 20, the following steps are implemented:

[0092] Acquire raw gas data, and preprocess the raw gas data to obtain target gas data;

[0093] Determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result;

[0094] Abnormal gas data in the gas data quality analysis result is determined, and intelligent decision processing is performed on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

[0095] The obtaining of raw gas data and preprocessing of the raw gas data to obtain target gas data specifically includes:

[0096] Determine the target gas data center, and obtain the original gas data of each subsystem module in the target gas data center;

[0097] The raw gas data is preprocessed to obtain target gas data, wherein the preprocessing includes data cleaning processing, outlier removal processing, missing value filling processing, data format conversion processing and normalization processing.

[0098] The determining of the gas quality requirement and performing data quality analysis on the target gas data according to the gas quality requirement to obtain the gas data quality analysis result specifically includes:

[0099] Determining gas quality requirements, wherein the gas quality requirements include preset data quality analysis dimensions, preset data quality analysis rules, and preset data quality analysis indicators;

[0100] A gas data analysis model is determined according to the gas quality requirement, and data quality analysis is performed on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result.

[0101] The step of performing data quality analysis on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result specifically includes:

[0102] Acquire historical gas data, and construct a gas data training set based on the historical gas data;

[0103] Determine a preset neural network model, and perform model training and model fine-tuning on the preset neural network model according to the gas data training set to obtain a gas data analysis model;

[0104] The target gas data is input into the gas data analysis model, and a gas data quality analysis result is output.

[0105] The determining of abnormal gas data in the gas data quality analysis result and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement specifically includes:

[0106] Determining abnormal gas data in the gas data quality analysis result and determining a decision database;

[0107] Searching the decision database according to the abnormal gas data to obtain an intelligent decision method corresponding to the abnormal gas data;

[0108] The abnormal gas data is processed by intelligent decision-making according to the intelligent decision-making method to obtain target quality gas data that meets the gas quality requirement.

[0109] The step of determining abnormal gas data in the gas data quality analysis result and performing intelligent decision processing on the abnormal gas data to obtain a target quality gas number that meets the gas quality requirement further includes:

[0110] Setting a data quality detection cycle, and monitoring and processing target quality gas data according to the data quality detection cycle and the preset data quality analysis index;

[0111] When the target quality gas data is abnormal, intelligent decision-making is performed.

[0112] The step of determining abnormal gas data in the gas data quality analysis result and performing intelligent decision processing on the abnormal gas data to obtain a target quality gas number that meets the gas quality requirement further includes:

[0113] The target quality gas data is visualized to obtain a visualization chart, wherein the visualization chart includes a conventional chart, a special-shaped chart, a dashboard and a map.

[0114] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a standard management program for gas data, and when the standard management program for gas data is executed by a processor, the steps of the standard management method for gas data as described above are implemented.

[0115] In summary, the present invention provides a standard management method for gas data and related equipment, the method comprising: obtaining original gas data, and preprocessing the original gas data to obtain target gas data; determining gas quality requirements, and performing data quality analysis on the target gas data according to the gas quality requirements to obtain gas data quality analysis results; determining abnormal gas data in the gas data quality analysis results, and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirements. The present invention collects multi-source gas data and performs data quality analysis on the multi-source gas data according to gas quality requirements, so as to not only perform standardized management on the data, but also identify abnormal data in the multi-source gas data. At the same time, by determining corresponding intelligent decisions, the processing efficiency of abnormal data can be effectively improved to ensure the data quality of the gas data.

[0116] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0117] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0118] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A standard management method for gas data, characterized in that: The standard management methods for gas data include: Acquire raw gas data, and preprocess the raw gas data to obtain target gas data; Determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result; Abnormal gas data in the gas data quality analysis result is determined, and intelligent decision processing is performed on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

2. The standard management method for gas data according to claim 1, characterized in that: The obtaining of raw gas data and preprocessing of the raw gas data to obtain target gas data specifically includes: Determine the target gas data center, and obtain the original gas data of each subsystem module in the target gas data center; The raw gas data is preprocessed to obtain target gas data, wherein the preprocessing includes data cleaning processing, outlier removal processing, missing value filling processing, data format conversion processing and normalization processing.

3. The standard management method for gas data according to claim 1, characterized in that: The determining of the gas quality requirement and performing data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result specifically includes: Determining gas quality requirements, wherein the gas quality requirements include preset data quality analysis dimensions, preset data quality analysis rules, and preset data quality analysis indicators; A gas data analysis model is determined according to the gas quality requirement, and data quality analysis is performed on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result.

4. The standard management method for gas data according to claim 3 is characterized in that: The performing data quality analysis on the target gas data according to the gas data analysis model to obtain a gas data quality analysis result specifically includes: Acquire historical gas data, and construct a gas data training set based on the historical gas data; Determine a preset neural network model, and perform model training and model fine-tuning on the preset neural network model according to the gas data training set to obtain a gas data analysis model; The target gas data is input into the gas data analysis model, and a gas data quality analysis result is output.

5. The standard management method for gas data according to claim 1, characterized in that: The determining of abnormal gas data in the gas data quality analysis result and performing intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement specifically includes: Determining abnormal gas data in the gas data quality analysis result and determining a decision database; Searching the decision database according to the abnormal gas data to obtain an intelligent decision method corresponding to the abnormal gas data; The abnormal gas data is processed by intelligent decision-making according to the intelligent decision-making method to obtain target quality gas data that meets the gas quality requirement.

6. The standard management method for gas data according to claim 3, characterized in that: The method further comprises: determining abnormal gas data in the gas data quality analysis result, and performing intelligent decision processing on the abnormal gas data to obtain a target quality gas number that meets the gas quality requirement, and then further comprising: Setting a data quality detection cycle, and monitoring and processing target quality gas data according to the data quality detection cycle and the preset data quality analysis index; When the target quality gas data is abnormal, intelligent decision-making is performed.

7. The standard management method for gas data according to claim 1, characterized in that: The method further comprises: determining abnormal gas data in the gas data quality analysis result, and performing intelligent decision processing on the abnormal gas data to obtain a target quality gas number that meets the gas quality requirement, and then further comprising: The target quality gas data is visualized to obtain a visualization chart, wherein the visualization chart includes a conventional chart, a special-shaped chart, a dashboard and a map.

8. A standard management system for gas data, characterized in that: The standard management system for gas data includes: A data preprocessing module, used to obtain raw gas data and preprocess the raw gas data to obtain target gas data; A data quality analysis module, used to determine the gas quality requirement, and perform data quality analysis on the target gas data according to the gas quality requirement to obtain a gas data quality analysis result; The intelligent decision processing module is used to determine the abnormal gas data in the gas data quality analysis result, and perform intelligent decision processing on the abnormal gas data to obtain target quality gas data that meets the gas quality requirement.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a standard management program for gas data stored in the memory and executable on the processor. When the standard management program for gas data is executed by the processor, the steps of the standard management method for gas data as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a standard management program for gas data, and when the standard management program for gas data is executed by a processor, the steps of the standard management method for gas data according to any one of claims 1 to 7 are implemented.

Images