A method, system, medium, and processor for rapid data encapsulation and sharing in power systems.

Abstract

This application discloses a method for rapid encapsulation and sharing of power system data, specifically including the following steps: connecting to various data sources storing power data to collect power data and form raw data; aggregating, processing, and merging various business power data in the raw data to form basic data; classifying and organizing the basic data, establishing data indexes, and packaging data from different data sources through parameter configuration to generate data packages stored in a local directory on a server or on a distributed server, forming data products for user access; establishing a unified data visualization view to receive various types of data access requests from users, and then calling various professional data backends to provide data product access services to the outside world. This research addresses data access technologies that shield the diversity of underlying data for business scenarios, and provides a unified service interface definition for data sources of the same type, simplifying the user's familiarization process and reducing the complexity of connecting to each data source.

Description

Technical Field

[0001] This invention relates to the field of power data interaction technology, and in particular to a method, system, medium, and processor for rapid encapsulation and sharing of power system data. Background Technology

[0002] With the continuous development of energy facilities and technologies, smart grids have become one of the main development directions for future energy systems. A smart grid is a power system with information technology at its core. It has thoroughly upgraded and transformed all aspects of the traditional power grid, including power transmission, distribution, and supply, making the entire power grid system more intelligent, more efficient, safer, and more reliable.

[0003] In response to the large-scale consumption of renewable energy and the challenges brought about by the construction of new power systems, the number and scale of various intelligent power system business scenarios are constantly increasing. This involves the development and training of a large number of AI models for intelligent control, operation error prevention, and renewable energy prediction. In the process, a massive amount of detection, control, and shared power data needs to be processed.

[0004] As the number and scale of various intelligent business scenarios implemented in the power system continue to increase, the cross-business needs between different systems are also increasing. The process of users manually connecting to each data source to become familiar with it is becoming increasingly cumbersome, and the complexity of the work and the time and labor costs required are increasing dramatically. At present, there is no corresponding method to automate the data interaction and sharing operations between various intelligent business scenarios in the power system.

[0005] Therefore, there is a need for a method, system, medium, and processor for rapid encapsulation and sharing of power system data. Summary of the Invention

[0006] To address the lack of a corresponding method in existing technologies to automate data interaction and sharing operations between various intelligent power business scenarios, this invention provides a method, system, medium, and processor for rapid data encapsulation and sharing in power systems. This enables data encapsulation and interactive sharing between various intelligent power business scenarios. The specific technical solution is as follows:

[0007] A method for rapid encapsulation and sharing of power system data specifically includes the following steps:

[0008] It connects to various data sources that store power data, collects power data and assigns timestamps, and forms raw data after compliance and integrity checks;

[0009] The power data from various business operations in the raw data are aggregated, processed, and merged to form basic data;

[0010] The basic data is classified and organized, and a data index is established. Data from different data sources is packaged and processed through parameter configuration, generating data packages that are stored in the local directory of the server or on a distributed server, forming data products for users to access and use.

[0011] Establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions, receiving various types of data access requests from users, and then calling various professional data back-ends to provide data product access services to the outside world.

[0012] Furthermore, based on the diversity of power data, the raw data is stored and processed using different data storage frameworks for different data structures.

[0013] Furthermore, the use of different data storage frameworks for storage processing for different data structures includes the following steps:

[0014] For the processing of structured data, a combination of relational databases and spatial databases is used to achieve high efficiency, real-time performance, and security in data storage and access.

[0015] For the processing of unstructured data, NoSQL databases and object storage are used to achieve high efficiency, real-time performance, and security in data storage and access.

[0016] Furthermore, the data sources include various data centers, data warehouses, or cloud platforms.

[0017] Furthermore, when receiving various types of data access requests from users, different users are granted different data access control permissions; the data access control permissions are set in fine granularity according to data classification; the permission scope includes invisible, browseable, and downloadable, and the corresponding time limit scope includes long-term, short-term, and temporary.

[0018] Furthermore, the aggregation, processing, and fusion processes include data cleaning and transformation, data verification and integration, and data fusion, which significantly improves the quality and accuracy of the data.

[0019] Furthermore, the compliance and integrity checks refer to periodic dataset quality checks based on data update frequency to ensure data quality, specifically including the following check steps:

[0020] Incremental data checks ensure that new data is generated in each cycle;

[0021] Data fluctuation checks ensure that incremental data remains within a reasonable range;

[0022] Check the null value rate of key fields to ensure that the null value rate of key fields is within a reasonable range;

[0023] The uniqueness of primary keys or composite primary keys is checked to ensure that no duplicate or dirty data is generated.

[0024] A power system data rapid encapsulation and sharing system, applied to the above-described power system data rapid encapsulation and sharing method, includes:

[0025] The data acquisition module is used to connect to various data sources that store power data, collect power data and assign timestamps, and form raw data after compliance and integrity checks;

[0026] The data processing module is used to aggregate, process, and merge the various business power data in the raw data to form basic data;

[0027] The data encapsulation module is used to classify and organize basic data, establish data indexes, package data from different data sources through parameter configuration, generate data packets and store them in the local directory of the server or on a distributed server, forming data products for users to access and use.

[0028] The data sharing module is used to establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions. It receives various types of data access requests from users and then calls various professional data back-ends to provide data product access services to the outside world.

[0029] A computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described method for rapid encapsulation and sharing of power system data.

[0030] A processor for running a program, wherein the program executes the above-described method for rapid encapsulation and sharing of power system data.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] For cognitive, decision-making, and forecasting business scenarios in the power sector, this study investigates data access technologies that shield the diversity of underlying data. It provides a unified service interface definition for similar data sources, and the platform implements differentiated requests for various middleware, simplifying the process of users becoming familiar with each data source and reducing the complexity of their work. Attached Figure Description

[0033] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0034] Figure 1 A flowchart illustrating a method for rapid data encapsulation and sharing in power systems;

[0035] Figure 2 This is a schematic diagram of the structure of a power system data rapid encapsulation and sharing system. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0038] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0039] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0040] Example 1

[0041] like Figure 1 The diagram shows a flowchart of a method for rapid encapsulation and sharing of power system data. The entire method divides the storage partition into a raw data area, a basic data area, a service product area, and a shared support area; specifically, it includes the following steps:

[0042] S1: Connects to various data sources that store power data, collects power data and assigns timestamps, performs compliance and integrity checks, and then stores the raw data in the raw data area.

[0043] Furthermore, the data sources include various data centers, data warehouses, or cloud platforms. That is, the raw data area mainly stores various externally connected observation and detection power status data, power equipment data, and data from various intelligent business scenarios. The raw data area does not modify the data, but only performs compliance and integrity checks. A unified timestamp is assigned when the raw data is collected to mark the time sequence of data collection.

[0044] Furthermore, based on the diversity of power data, different data storage frameworks are used in the raw data area to store and process the raw data for different data structures.

[0045] Furthermore, the use of different data storage frameworks for different data structures includes the following steps:

[0046] For the processing of structured data, a combination of relational databases and spatial databases is used to achieve high efficiency, real-time performance, and security in data storage and access.

[0047] For the processing of unstructured data, NoSQL databases and object storage are used to achieve high efficiency, real-time performance, and security in data storage and access.

[0048] Furthermore, the compliance and integrity checks refer to periodic dataset quality checks based on data update frequency to ensure data quality, specifically including the following check steps:

[0049] Incremental data checks ensure that new data is generated in each cycle;

[0050] Data fluctuation checks ensure that incremental data remains within a reasonable range;

[0051] Check the null value rate of key fields to ensure that the null value rate of key fields is within a reasonable range;

[0052] The uniqueness of primary keys or composite primary keys is checked to ensure that no duplicate or dirty data is generated.

[0053] Furthermore, timestamps are assigned when collecting power data to mark the time of data collection, so as to facilitate the analysis of the changing trends of power distribution and supply status in various regions.

[0054] Furthermore, given the vast geographical coverage of the power system, meteorological thresholds are set to monitor meteorological conditions in various regions, while power data in areas experiencing extreme weather is updated in real time. Simultaneously, curves can be fitted to the changes in power data, and the derivative of these curves can be calculated to obtain the degree of change in power data over time. For slowly changing power data, the collection frequency can be reduced, while for rapidly changing power data, the collection frequency can be increased to provide shared data that accurately reflects the state of the power grid. Although the collection frequency differs across regions, the collection time can be uniformly set.

[0055] S2: Aggregate, process, and merge the various business power data in the original data to form basic data storage in the basic data area.

[0056] Furthermore, the aggregation, processing, and fusion processes include data cleaning and transformation, data verification and integration, and data fusion, which significantly improve the quality and accuracy of the data.

[0057] Furthermore, the aggregation, processing, and fusion process also includes sequentially performing data decoding, format conversion, and quality inspection operations.

[0058] Data fusion and processing refers to the collection, integration, analysis, and processing of data from different sources and of different types to improve data accuracy, completeness, and usability. In smart grids, data fusion and processing technologies play a crucial role because the data sources involved in smart grids are very extensive, including data from the grid's own telemetry, remote control, and remote signaling systems, as well as user electrical appliance and environmental data, and even external environmental data sources such as weather forecasts and economic data.

[0059] The process of convergence, processing, and fusion includes the following steps:

[0060] Data acquisition and storage are the foundation of data fusion and processing. They enable the collection, storage, archiving, and management of data, ensuring its accuracy, integrity, and usability.

[0061] Data preprocessing refers to the processes performed on data before it is merged and processed, including cleaning, imputing missing values, handling outliers, and reducing noise. Preprocessing effectively improves data quality and ensures the accuracy and effectiveness of subsequent operations.

[0062] Data fusion is the process of integrating and matching data from different data sources to form a consistent data structure and format. This facilitates unified analysis and modeling of multi-source data, providing fundamental support for subsequent data processing.

[0063] Data mining and intelligent analysis refer to the in-depth analysis and modeling of data to extract useful information and knowledge, providing support for business planning, decision-making, and operational optimization.

[0064] S3: Basic data is categorized and organized, such as reorganizing it according to the needs of various professional fields, establishing data indexes, and packaging data from different data sources through parameter configuration. This data is then stored in the server's local directory or on a distributed server, i.e., in the service product area, forming data products for user access. In other words, the service product area is responsible for providing external data product services, primarily storing product data processed by service-oriented products in various professional fields within the system.

[0065] In practice, the process of packaging data from different data sources through parameter configuration includes the following steps:

[0066] Before encapsulating data, it's essential to first determine its type and attributes. Data types can be basic types, such as integers, floating-point numbers, and strings, or user-defined types, such as structures and classes. Data attributes describe the characteristics and state of the data. By defining the data type and attributes, we can clarify the data's storage and access methods, laying the foundation for subsequent data encapsulation.

[0067] Designing data interfaces and methods involves considering the data's usage scenarios and requirements, and adhering to the principle of encapsulation. This means hiding the internal implementation details of the data and exposing only the necessary interfaces for external use. The interface design should be concise and clear, providing essential functionality, and following established naming conventions and parameter rules.

[0068] Data encapsulation refers to grouping data and related methods together based on data type and attributes. When encapsulating data, appropriate data structures and algorithms are needed to improve data access and operational efficiency. Simultaneously, data security and integrity must be considered, protecting data legitimacy through access control and data validation.

[0069] Test data encapsulation is a crucial step in ensuring the quality of encapsulated data. When encapsulating test data, it's necessary to write corresponding test cases to comprehensively test the data interfaces and methods, ensuring their correctness and stability. Simultaneously, boundary conditions and exceptional situations must be considered to improve the robustness and fault tolerance of the data.

[0070] Optimizing data encapsulation is a continuous process of improvement. When optimizing data encapsulation, consider using more efficient data structures and algorithms, improving interface design and implementation, reducing data storage and access overhead, and enhancing data performance and scalability. Furthermore, data encapsulation can be adjusted and extended according to actual needs to meet diverse business requirements.

[0071] Data encapsulation is a technique that combines data and operations on that data. By defining the data type and attributes, designing data interfaces and methods, implementing and testing data encapsulation, and optimizing the encapsulation process, secure access and manipulation of data can be achieved. Data encapsulation not only improves data security and integrity but also provides concise interfaces for other programs or modules, thereby enhancing code reusability and maintainability. Therefore, mastering data encapsulation techniques is an essential skill for every programmer.

[0072] S4: Establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions, receiving various types of data access requests from users, and then calling various professional data back-ends to provide data product access services to the outside world.

[0073] Furthermore, when receiving various types of data access requests from users, different data access control permissions are granted to different users; the data access control permissions are set in fine granularity according to data classification; the permission scope includes invisible, browseable, and downloadable, and the corresponding time limit scope includes long-term, short-term, and temporary.

[0074] At the same time, different data access control permissions and data sharing access interface permissions are granted to different users to complete cross-system and cross-industry data sharing, provide shared access to data from different professional fields, and realize a transparent data access process for users.

[0075] When setting data access control permissions in a fine-grained manner according to data categories, implementing fine-grained data access control requires consideration of data sensitivity and user permissions. By combining measures such as authentication, authorization management, audit logs, and data encryption, it can be ensured that only authorized users can access sensitive data, and user access behavior is recorded to prevent unauthorized access. Specifically, this includes the following steps:

[0076] First, authentication is the first step in ensuring that only authorized users can access data. This is typically done through methods such as username and password, fingerprint recognition, or facial recognition. After authentication, the system further checks the user's permissions to ensure that they can only access authorized data.

[0077] Secondly, authorization management is crucial for controlling user access permissions. Through role management, permission management, and access control lists (ACLs), it can be ensured that users can only access data they are authorized to access. For example, in Java, access modifiers such as public, protected, and private can be used to control the visibility of classes, methods, and variables.

[0078] Audit logs are used to monitor and record user access behavior, enabling the timely detection and handling of abnormal access activities. Audit logs can record information such as user access time, content accessed, and access results, which helps in tracking and investigating potential security issues.

[0079] Data encryption is a crucial means of protecting sensitive data. By encrypting the storage and transmission of sensitive data, unauthorized access can be prevented. In Java, built-in encryption libraries such as Cipher and MessageDigest can be used to implement data encryption.

[0080] Suppose we have an `employees` table. We can create a strategy function to restrict users to only seeing employee records within their own department. This is achieved using a PL / SQL function that returns a `WHERE` clause that restricts users to seeing only employee records within their department.

[0081] Finally, data sensitivity is also a crucial factor in achieving fine-grained control. Data sensitivity refers not only to the level of confidentiality of data but also to a person's keen awareness of data. By improving users' data sensitivity and business understanding, data can be better understood and utilized, thereby enhancing the effectiveness of data access control.

[0082] In addition to the functions mentioned above, there is also a shared support area, which mainly stores the support environment data required for the operation of the business system, including metadata and system operation and maintenance management data.

[0083] Example 2

[0084] A power system data rapid encapsulation and sharing system, applied to the above-described power system data rapid encapsulation and sharing method, includes:

[0085] The data acquisition module is used to connect to various data sources that store power data, collect power data and assign timestamps, and form raw data after compliance and integrity checks;

[0086] The data processing module is used to aggregate, process, and merge the various business power data in the raw data to form basic data;

[0087] The data encapsulation module is used to classify and organize basic data, establish data indexes, package data from different data sources through parameter configuration, generate data packets and store them in the local directory of the server or on a distributed server, forming data products for users to access and use.

[0088] The data sharing module is used to establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions. It receives various types of data access requests from users and then calls various professional data back-ends to provide data product access services to the outside world.

[0089] Example 3

[0090] A computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described method for rapid encapsulation and sharing of power system data.

[0091] Example 4

[0092] A processor for running a program, wherein the program executes the above-described method for rapid encapsulation and sharing of power system data.

[0093] This application discloses a method for rapid encapsulation and sharing of power system data, specifically including the following steps: connecting to various data sources storing power data, collecting power data and assigning timestamps, and forming raw data after compliance and integrity checks; aggregating, processing, and merging various business power data in the raw data to form basic data; classifying and organizing the basic data, establishing data indexes, packaging data from different data sources through parameter configuration, generating data packets stored in a local directory on a server or on a distributed server, forming data products for user access; establishing a unified data visualization view, providing data cataloging, information display, retrieval, download, subscription, and distribution functions, receiving various types of data access requests from users, and then calling various professional data backends to provide data product access services externally. Targeting the cognitive, decision-making, and forecasting business scenarios in the power sector, this research studies data access technologies that shield the diversity of underlying data, supports a unified service interface definition for similar data sources, and allows the platform to implement differentiated requests from various middleware, simplifying the user's familiarization process and reducing the complexity of connecting to each data source.

[0094] Those skilled in the art will recognize that the units of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0095] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical functional division. In actual implementation, there may be other division methods, such as multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored.

[0096] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0097] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for rapid encapsulation and sharing of power system data, characterized in that, Specifically, the following steps are included: It connects to various data sources that store power data, collects power data and assigns timestamps, and forms raw data after compliance and integrity checks; The power data from various business operations in the raw data are aggregated, processed, and merged to form basic data; The basic data is classified and organized, and a data index is established. Data from different data sources is packaged and processed through parameter configuration, generating data packages that are stored in the local directory of the server or on a distributed server, forming data products for users to access and use. Establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions, receiving various types of data access requests from users, and then calling various professional data back-ends to provide data product access services to the outside world.

2. The method for rapid encapsulation and sharing of power system data according to claim 1, characterized in that, Based on the diversity of power data, the raw data is stored and processed using different data storage frameworks for different data structures.

3. The method for rapid encapsulation and sharing of power system data according to claim 2, characterized in that, The method of using different data storage frameworks for different data structures includes the following steps: For the processing of structured data, a combination of relational databases and spatial databases is used to achieve high efficiency, real-time performance, and security in data storage and access. For the processing of unstructured data, NoSQL databases and object storage are used to achieve high efficiency, real-time performance, and security in data storage and access.

4. The method for rapid encapsulation and sharing of power system data according to claim 1, characterized in that, The data sources include various data centers, data warehouses, or cloud platforms.

5. The method for rapid encapsulation and sharing of power system data according to claim 1, characterized in that, When receiving various types of data access requests from users, different users are granted different data access control permissions; the data access control permissions are set in fine granularity according to data classification; the permission scope includes invisible, browseable, and downloadable, and the corresponding time limit scope includes long-term, short-term, and temporary.

6. The method for rapid encapsulation and sharing of power system data according to claim 1, characterized in that, The aforementioned aggregation, processing, and fusion processes include data cleaning and transformation, data verification and integration, and data fusion, which significantly improves the quality and accuracy of the data.

7. The method for rapid encapsulation and sharing of power system data according to claim 1, characterized in that, The compliance and integrity checks refer to periodic dataset quality checks based on data update frequency to ensure data quality. Specifically, these checks include the following steps: Incremental data checks ensure that new data is generated in each cycle; Data fluctuation checks ensure that incremental data remains within a reasonable range; Check the null value rate of key fields to ensure that the null value rate of key fields is within a reasonable range; The uniqueness of primary keys or composite primary keys is checked to ensure that no duplicate or dirty data is generated.

8. A power system data rapid encapsulation and sharing system, characterized in that, The method for rapid encapsulation and sharing of power system data according to any one of claims 1 to 7 includes: The data acquisition module is used to connect to various data sources that store power data, collect power data and assign timestamps, and form raw data after compliance and integrity checks; The data processing module is used to aggregate, process, and merge the various business power data in the raw data to form basic data; The data encapsulation module is used to classify and organize basic data, establish data indexes, package data from different data sources through parameter configuration, generate data packets and store them in the local directory of the server or on a distributed server, forming data products for users to access and use. The data sharing module is used to establish a unified data visualization view, providing data cataloging, information display, retrieval and download, subscription and distribution functions. It receives various types of data access requests from users and then calls various professional data back-ends to provide data product access services to the outside world.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the power system data fast encapsulation and sharing method according to any one of claims 1 to 7.

10. A processor, characterized in that, The processor is used to run a program, wherein the program executes the power system data rapid encapsulation and sharing method according to any one of claims 1 to 7.

Images