Data traceability analysis and application system and method based on power grid data medium station

Abstract

The invention provides a data traceability analysis and application system and method based on a power grid data medium station, and relates to the technical field of power big data analysis and application, the system comprises a data development part and a system development part, and the data development part comprises a power data index development module and a power data analysis module; and the system development part comprises a data product deployment module, a data product component software development module, an application software management development module, an external service software development module and a system daily management module. The method improves the utilization efficiency of data resources in data, promotes the fusion of internal and external data, and strengthens the correlation research of power data and social economy. According to the invention, data and model bases are provided for research of power services, virtual power plants, load management, micro-grids and the like.

Description

Technical Field

[0001] The present invention relates to the field of power big data analysis and application technology, and in particular to a data source tracing analysis and application system and method based on a power grid data center. Background Art

[0002] With the widespread application of digital technology, the vitality of enterprises in the field of data elements has been continuously enhanced, data application scenarios have become increasingly rich, and the integration of digital and real development has accelerated. In the field of power big data, data has become an important resource to support the improvement of the company's customer service quality, the green and low-carbon transformation of local energy, and the high-quality development of the economy and society. Domestic and foreign power companies have begun to explore business innovations in energy and power big data in smart cities, smart homes, integrated energy services, power big data finance, etc., transforming from the traditional provision of energy commodities to the provision of energy big data services, connecting the upstream and downstream of the industrial chain, expanding the ecological boundaries, and creating new growth points. For example, the State Grid Corporation of China analyzes user data to provide financial customers with the required support data to help them control risks. Suzhou Power Supply Company has provided authorized enterprise data indicators through in-depth business docking for many years, continuously iterated and updated the power customer indicator system, and provided power customer data traceability, integration, and analysis services for customers such as governments and financial institutions.

[0003] Outside the power system, related companies such as Alibaba have rich experience in building data middle platforms and developing big data applications. Alibaba has gradually built a complete data middle platform business organization through the practice of unified sharing and utilization of data at the group level, and based on its own successful practice, it has formed the Alibaba Cloud data middle platform solution with OneData as the core methodology, which has played a positive role in learning from and promoting the data element value utilization practices of peers such as Meituan.

[0004] Foreign energy and power companies also have many outstanding practices in digital transformation. For example, Tokyo Electric Power Company is committed to transforming from providing products to providing services, and from single services to comprehensive services. It optimizes and adjusts its organizational structure by building four platforms: transmission and distribution platform, infrastructure platform, energy platform and data platform. Germany's E.ON Group has established a partnership with the US company Sight Machine to promote intelligent quality control systems, predict electricity demand and improve production efficiency.

[0005] Although the application of power big data has achieved initial results, there are still some problems in the efficient use of data resources in the data center, the integration of internal and external data, and the research on the correlation mechanism between power data and economic, social and corporate development.

[0006] Inefficient use of data resources in the data middle platform: Although the data middle platform has gathered a large amount of data, how to efficiently use these data resources and tap their potential value remains a challenge.

[0007] Difficulty in integrating internal and external data: Due to the diverse sources and formats of data, how to effectively integrate internal and external data and improve data quality and availability is an important issue currently faced.

[0008] The research on the correlation mechanism between power data and economic, social and corporate development is weak: At present, the research on the correlation mechanism between power data and economic, social and corporate development is not in-depth enough, making it difficult to give full play to the role of power data in decision support, risk management and other aspects.

[0009] Consulting services and market-oriented power data value-added service products are not perfect: consulting services for provincial, municipal, district and town governments are not perfect, and there are relatively few replicable and popularizable market-oriented power data value-added service products, which makes it difficult to meet the market's diversified demand for power big data.

[0010] Therefore, it is necessary to strengthen the basic capabilities of data middle platform development and establish a data middle platform-related application development model to provide data and model basis for power monitoring services and research on virtual power plants, load management, microgrids, etc., while promoting power data to empower high-quality economic and social development. Summary of the invention

[0011] To this end, an embodiment of the present invention provides a data traceability analysis and application system and method based on a power grid data middle station, which is used to solve the problems of insufficient efficient utilization of data resources in the data middle station in the prior art, insufficient integration of internal and external data, and weak research on the correlation mechanism between power data and economic and social development.

[0012] In order to solve the above problems, an embodiment of the present invention provides a data tracing analysis and application system based on a power grid data center, the system comprising:

[0013] The data development part includes the power data indicator development module and the power data analysis module.

[0014] The power data indicator development module is used for the development and synchronization of various indicators within the indicator system; the power data analysis module is used for data collection, data preprocessing, data mining, data statistics and analysis, and data storage management;

[0015] The system development part includes data product deployment module, data product component software development module, application software management development module, external service software development module and system daily management module.

[0016] The data product deployment module is used to build and deploy the server development environment to ensure that the server meets the development conditions; the data product component software development module is used to develop RDS database data acquisition and data synchronization software; the application software management development module is used to develop management software for internal use; the external service software development module is used to develop external service software to realize database data display and query functions, as well as external data interface and data docking services; the system daily management module is used for daily management of the system.

[0017] Preferably, the power data indicator development module is used for developing and synchronizing various indicators within the indicator system, wherein enterprise indicator data is developed from two dimensions, including based on the electricity user number and based on the enterprise name.

[0018] Preferably, the data collection specifically includes:

[0019] Use the data map function of the data center to find and apply for the required data tables;

[0020] After obtaining the data table, perform data verification to ensure that the data meets the requirements and the timeliness of the data table;

[0021] Automatically set the time nodes for data collection and publish the collection tasks to the production environment of the data center;

[0022] The collected data includes electricity user information, substation line information, daily electricity consumption, monthly electricity consumption, load electricity consumption, heavy load and overload information, electricity bill information, abnormal electricity consumption information, etc. At the same time, it is also necessary to collect the data of the association table between data tables.

[0023] Preferably, the data preprocessing specifically includes:

[0024] Use database language and algorithms to classify, deduplicate, and clean data to remove invalid and erroneous data;

[0025] The data relationship between the associated data tables is preprocessed to form a data wide table set.

[0026] Preferably, the data mining specifically includes:

[0027] Through the algorithm, we search for the information of electricity users and the relationship between electricity consumption information, and find the internal connection and rules between the data;

[0028] Data mining is accomplished using a variety of methods, including statistics, online analytical processing, information retrieval, expert systems, and pattern recognition;

[0029] Establish the association between data tables and mine the association between data and base stations.

[0030] Preferably, data statistics and analysis specifically include:

[0031] The collected data are processed and analyzed using data acquisition and statistical analysis program tools and methods.

[0032] Preferably, the application software management development module is used for the development of management software for internal use, including a basic function management module, a business management module, a customer management module and a log management module, wherein the basic function management module is used for user authority and role management of the system; the business management module is used for report template management and authorization letter review statistics for various types of customers; the customer management module is used to manage service customer information; and the log management module is used to collect log statistics.

[0033] Preferably, the external service software development module is used to develop external service software, realize database data display and query functions, as well as external data interface and data docking services, including customer login module, interface service module, client service module, and log management module.

[0034] The embodiment of the present invention further provides a data tracing analysis and application method based on a power grid data middle station, which adopts the above-mentioned data tracing analysis and application system based on a power grid data middle station, and specifically includes:

[0035] Based on the data center DATAWorks, RDS database and the basic original tables of electricity consumption big data, a data wide table set is formed through data collection, data preprocessing, data mining, data statistics and analysis, and data storage management;

[0036] According to the indicator system, various indicators are developed and synchronized in the data center;

[0037] Synchronize data in the local intranet database and develop corresponding system management platform and customer service system.

[0038] Preferably, the method further comprises:

[0039] Facing the State Grid intranet, the encrypted power data calculation results of industries and enterprises are retrieved from the data middle-end server, transmitted to the data server through the intranet, and a wide table of all data analysis is generated;

[0040] For external customers, relevant data services are provided through enterprise authorization. The front-end server obtains the authorization letter. After internal approval, the front-end server generates a whitelist. The data analysis server extracts related data according to the whitelist and authorization indicators and generates an encrypted data package.

[0041] For external customers, relevant data services are provided through dedicated lines. The customer's intranet sends authorized power desensitized data retrieval instructions to the front-end server. The front-end server retrieves the desensitized data and transmits it to the customer service end through a dedicated line in the form of an encrypted data packet interface. After the customer logs into the system, he or she obtains relevant indicator data through a secret key.

[0042] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0043] (1) Improve the efficient use of data resources: Through the data development part, especially the power data indicator development module and the power data analysis module, the development and synchronization of various indicators within the indicator system as well as the collection, preprocessing, mining, statistics and analysis of data are realized, thereby improving the utilization efficiency of data resources in the data center.

[0044] (2) Promote the full integration of internal and external data: The system development part includes modules such as data product deployment, component software development, application software management development, and external service software development. These modules work together to not only meet internal data management needs, but also realize data interfaces and data docking services with the outside world through the external service software development module, thereby promoting the full integration of internal and external data.

[0045] (3) Strengthening the research on the correlation between power data and social economy: This invention aims to strengthen the basic capacity building of data middle platform development and establish a data middle platform related application development model, which provides a data and model basis for conducting power monitoring services and research on virtual power plants, load management, microgrids, etc., thereby strengthening the research on the correlation mechanism between power data and social and economic development. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0047] Figure 1 A block diagram of a data tracing analysis and application system based on a power grid data center provided in an embodiment;

[0048] Figure 2 The present invention is a flowchart of a data traceability analysis and application method based on a power grid data center provided in an embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Embodiment 1

[0051] In order to solve the problems of insufficient efficient use of data resources in the data center in the existing technology, insufficient integration of internal and external data, and weak research on the correlation mechanism between power data and economic and social development. Figure 1 As shown, an embodiment of the present invention provides a data tracing analysis and application system based on a power grid data center, the system comprising:

[0052] The data development part includes the power data indicator development module and the power data analysis module.

[0053] The power data indicator development module is used for the development and synchronization of various indicators within the indicator system; the power data analysis module is used for data collection, data preprocessing, data mining, data statistics and analysis, and data storage management;

[0054] The system development part includes data product deployment module, data product component software development module, application software management development module, external service software development module and system daily management module.

[0055] The data product deployment module is used to build and deploy the server development environment to ensure that the server meets the development conditions; the data product component software development module is used to develop RDS database data acquisition and data synchronization software; the application software management development module is used to develop management software for internal use; the external service software development module is used to develop external service software to realize database data display and query functions, as well as external data interface and data docking services; the system daily management module is used for daily management of the system.

[0056] It can be seen from the above technical scheme that the present invention provides a data traceability analysis and application system based on the power grid data middle platform, which realizes the development and in-depth analysis of power data indicators through the data development part, and constructs a comprehensive system including a data product deployment module, a data product component software development module, an application software management development module, an external service software development module and a system daily management module through the system development part. The system aims to strengthen the basic capacity building of the data middle platform, promote the efficient use of data resources and the full integration of internal and external data, and provide solid data and model support for power service-related research and applications.

[0057] In this embodiment, the data development part includes an electric power data indicator development module and an electric power data analysis module. First, the electric power data indicator development module is used for the development and synchronization of various indicators within the indicator system, among which the enterprise indicator data is developed from two dimensions, including two categories based on the electricity user number and the enterprise name. This module can cover and synchronously update various electric power data indicators within the indicator system to ensure the comprehensiveness and timeliness of the data. The functions of the electric power data indicator development module are shown in Table 1 below.

[0058] Table 1

[0059]

[0060] Furthermore, the power data analysis module is used for data collection, data preprocessing, data mining, data statistics and analysis, and data storage management, providing users with detailed and accurate power data analysis results. The functions of the power data analysis module are shown in Table 2 below.

[0061] Among them, data collection specifically includes: finding and applying for the required data tables through the data map function of the data center. After obtaining the data table, perform data verification to ensure that the data meets the requirements and the timeliness of the data table. Automatically set the time node for collecting data, and publish the collection task to the production environment of the data center. The collected data includes electricity user information, station line information, daily electricity consumption, monthly electricity consumption, load electricity consumption, heavy load and overload information, electricity bill information, abnormal electricity consumption information, etc. At the same time, it is also necessary to collect the association table data between data tables.

[0062] Data preprocessing specifically includes: using database language and algorithms to classify, remove duplicates, and clean data to remove invalid and erroneous data. Relating data relationships between data tables, preprocessing, and forming a data wide table set.

[0063] Data mining specifically includes: searching for the relationship between electricity user information and electricity consumption information through algorithms, and discovering the internal connections and patterns between data. Data mining is achieved using a variety of methods, including statistics, online analysis and processing, intelligence retrieval, expert systems, and pattern recognition. Establishing the relationship between data tables, and mining the relationship between data and base stations.

[0064] Data statistics and analysis specifically include: using data collection and statistical analysis program tools and methods, such as descriptive statistics (mean, median, standard deviation, frequency distribution), inferential statistics (variance analysis, regression analysis, correlation analysis), cluster analysis, classification and prediction, etc., to process and analyze the collected data.

[0065] Table 2

[0066]

[0067]

[0068] In this embodiment, the system development part includes a data product deployment module, a data product component software development module, an application software management development module, an external service software development module and a system daily management module.

[0069] The data product deployment module is used to build and deploy the server development environment to ensure that the server meets the development conditions. Such as server implementation installation and construction environment, test server, disk array optimization, national grid and client project deployment, remote device software installation, plug-in debugging and external access. This module is responsible for the construction and deployment of hardware facilities, providing a solid hardware foundation for the smooth operation of data products. The functions of the data product deployment module are shown in Table 3 below.

[0070] Table 3

[0071]

[0072]

[0073] Furthermore, the data product component software development module is used for the development of RDS database data acquisition and data synchronization software. This module is responsible for developing database data acquisition and data synchronization software to ensure accurate data transmission and synchronization. The functions of the data product component software development module are shown in Table 4 below.

[0074] Table 4

[0075]

[0076] Furthermore, the application software management and development module is used for the development of management software for internal use, including the basic function management module, the business management module, the customer management module and the log management module. The basic function management module is used for the user authority and role management of the system; the business management module is used for report template management for various types of customers, as well as the audit and statistics of authorization letters; the customer management module is used to manage service customer information; and the log management module is used to count logs. From the perspective of business data, the State Grid intranet retrieves the calculation results of industry and enterprise encrypted power data from the data middle-end server, transmits them to the data server through the intranet, and generates all data analysis wide tables. The application software management and development module improves the management efficiency and convenience within the system. The functions of the application software management and development module are shown in Table 5 below.

[0077] Table 5

[0078]

[0079]

[0080] Furthermore, the external service software development module is used to develop external service software (client), realize the database data display and query functions, as well as external data interface and data docking services, including customer login module, interface service module, client service module, and log management module. The client is oriented to the end users of the system, and provides relevant data services through enterprise authorization. The front-end server obtains the authorization letter. After internal approval, the front-end server generates a whitelist, and the data analysis server extracts related data according to the whitelist and authorization indicators to generate encrypted data packets. For external customers, relevant data services are provided through dedicated lines. The customer intranet sends authorized power desensitized data retrieval instructions to the front-end server. The front-end server retrieves the desensitized data and transmits it to the customer service end through a dedicated line in the form of an encrypted data packet interface. After the customer logs in to the system, the relevant indicator data is obtained through the secret key. The external service software development module enables external users to easily access and query power data, expanding the scope of data application. The functions and effects of the external service software development module are shown in Table 6 below.

[0081] Table 6

[0082]

[0083] Furthermore, the system daily management module is used for the daily management of the system. Specifically, it manages the system's customers, user permissions, and system controls on a daily basis to ensure that the system can continue to operate normally. The system daily management service includes the system's user permission role maintenance service, new electricity user review service, system basic maintenance service, etc. This module ensures the stable operation and daily maintenance of the system. The functions of the system daily management module are shown in Table 7 below.

[0084] Table 7

[0085]

[0086] Embodiment 2

[0087] like Figure 2 As shown, the present invention provides a data tracing analysis and application method based on a power grid data middle station, which is used to adopt the data tracing analysis and application system based on a power grid data middle station of the above-mentioned embodiment 1, and specifically includes:

[0088] Based on the data center DATAWorks, RDS database and the basic original tables of electricity consumption big data, a data wide table set is formed through data collection, data preprocessing, data mining, data statistics and analysis, and data storage management;

[0089] According to the indicator system, various indicators are developed and synchronized in the data center;

[0090] Synchronize data in the local intranet database and develop corresponding system management platform and customer service system.

[0091] Furthermore, the method also includes: facing the State Grid intranet, retrieving the encrypted power data calculation results of the industry and enterprise from the data middle-end server, transmitting them to the data server through the intranet, and generating all data analysis wide tables. Facing external customers, relevant data services are provided through enterprise authorization, the front-end server obtains the authorization letter, and after internal approval, the front-end server generates a whitelist, and the data analysis server extracts related data according to the whitelist and authorization indicators to generate an encrypted data packet. Facing external customers, relevant data services are provided through dedicated lines. The customer intranet sends an authorized power desensitized data retrieval instruction to the front-end server, and the front-end server retrieves the desensitized data and transmits it to the customer service end through a dedicated line in the form of an encrypted data packet interface. After the customer logs in to the system, the relevant indicator data is obtained through a secret key.

[0092] The data traceability analysis and application method based on the power grid data middle station of this embodiment adopts the aforementioned data traceability analysis and application system based on the power grid data middle station. Therefore, the specific implementation method of the data traceability analysis and application method based on the power grid data middle station can be seen in the implementation example part of the data traceability analysis and application system based on the power grid data middle station in the previous text. In order to avoid redundancy, it will not be repeated here.

[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0094] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A data tracing analysis and application system based on a power grid data center, characterized in that: include: The data development part includes the power data indicator development module and the power data analysis module. The power data indicator development module is used for the development and synchronization of various indicators within the indicator system; the power data analysis module is used for data collection, data preprocessing, data mining, data statistics and analysis, and data storage management; The system development part includes data product deployment module, data product component software development module, application software management development module, external service software development module and system daily management module. The data product deployment module is used to build and deploy the server development environment to ensure that the server meets the development conditions; the data product component software development module is used to develop RDS database data acquisition and data synchronization software; the application software management development module is used to develop management software for internal use; the external service software development module is used to develop external service software to realize database data display and query functions, as well as external data interface and data docking services; the system daily management module is used for daily management of the system.

2. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The power data indicator development module is used for the development and synchronization of various indicators within the indicator system, among which enterprise indicator data is developed from two dimensions, including based on the electricity user number and based on the enterprise name.

3. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The data collection specifically includes: Use the data map function of the data center to find and apply for the required data tables; After obtaining the data table, perform data verification to ensure that the data meets the requirements and the timeliness of the data table; Automatically set the time nodes for data collection and publish the collection tasks to the production environment of the data center; The collected data includes electricity user information, substation line information, daily electricity consumption, monthly electricity consumption, load electricity consumption, heavy load and overload information, electricity bill information, abnormal electricity consumption information, etc. At the same time, it is also necessary to collect the data of the association table between data tables.

4. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The data preprocessing specifically includes: Use database language and algorithms to classify, deduplicate, and clean data to remove invalid and erroneous data; The data relationship between the associated data tables is preprocessed to form a data wide table set.

5. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The data mining specifically includes: Through the algorithm, we search for the information of electricity users and the relationship between electricity consumption information, and find the internal connection and rules between the data; Data mining is accomplished using a variety of methods, including statistics, online analytical processing, information retrieval, expert systems, and pattern recognition; Establish the association between data tables and mine the association between data and base stations.

6. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: Data statistics and analysis specifically include: The collected data are processed and analyzed using data acquisition and statistical analysis program tools and methods.

7. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The application software management development module is used for the development of management software for internal use, including a basic function management module, a business management module, a customer management module and a log management module. The basic function management module is used for user authority and role management of the system; the business management module is used for report template management and authorization letter review and statistics for various types of customers; the customer management module is used to manage service customer information; and the log management module is used to perform log statistics.

8. The data tracing analysis and application system based on the power grid data middle station according to claim 1 is characterized in that: The external service software development module is used to develop external service software, realize database data display and query functions, as well as external data interface and data docking services, including customer login module, interface service module, client service module, and log management module.

9. A data tracing analysis and application method based on a power grid data center, characterized in that: The method adopts the data tracing analysis and application system based on the power grid data middle station according to any one of claims 1 to 8, and specifically includes: Based on the data center DATAWorks, RDS database and the basic original tables of electricity consumption big data, a data wide table set is formed through data collection, data preprocessing, data mining, data statistics and analysis, and data storage management; According to the indicator system, various indicators are developed and synchronized in the data center; Synchronize data in the local intranet database and develop corresponding system management platform and customer service system.

10. The data tracing analysis and application method based on the power grid data middle station according to claim 9 is characterized in that: The method further comprises: Facing the State Grid intranet, the encrypted power data calculation results of industries and enterprises are retrieved from the data middle-end server, transmitted to the data server through the intranet, and a wide table of all data analysis is generated; For external customers, relevant data services are provided through enterprise authorization. The front-end server obtains the authorization letter. After internal approval, the front-end server generates a whitelist. The data analysis server extracts related data according to the whitelist and authorization indicators and generates an encrypted data package. For external customers, relevant data services are provided through dedicated lines. The customer's intranet sends authorized power desensitized data retrieval instructions to the front-end server. The front-end server retrieves the desensitized data and transmits it to the customer service end through a dedicated line in the form of an encrypted data packet interface. After the customer logs into the system, he or she obtains relevant indicator data through a secret key.

Images