A new energy station time sequence data access system and method
By employing a data processing method that combines receiving, adapting, and distributing modules in new energy power stations with tagging of measuring points and indicator tables, efficient storage and retrieval of multi-source data were achieved, solving the problems of expanded and redundant data, and reducing maintenance costs and network pressure.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 山东华科信息技术有限公司
- Filing Date
- 2024-01-11
- Publication Date
- 2026-08-04
AI Technical Summary
Existing technologies face challenges in accessing multi-source data in new energy power plants, including difficulties in expansion, increased project burden due to improper handling of redundant data, and high maintenance costs.
It employs a receiving module, an adaptation module, a distribution module, and a data storage module. Through tag processing of measurement point tables, indicator tables, and measurement point-indicator relationship tables, combined with the sequence number management of time series databases, it achieves unified data storage and retrieval, and supports heterogeneous data source processing and expansion.
It improves the storage and retrieval efficiency of time-series databases, reduces maintenance costs, solves the problem of handling redundant data, and reduces network pressure and the burden of later maintenance.
Smart Images
Figure CN117874031B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a system and method for storing and retrieving time-series data of new energy power plants. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] New energy power plants constantly generate a large amount of monitoring and environmental data, which is of great significance for the operation management, equipment maintenance, and energy management of these power plants. The time-series database has been specifically optimized for the characteristics and application scenarios of time-series data, including storage, querying, and data deletion.
[0004] The data sources for the new energy power stations on the project are different, including direct reporting from the power stations, forwarding from the power generation group, and transmission from the production control area. The key difference is that the measurement point formats are very different. At the same time, as the amount of data increases, it has a significant impact on the performance of accessing the time series database on the project.
[0005] Currently, the method used for multi-source data access is to establish a unified data access address protocol for data from different systems, and then store the data in the corresponding database after parsing based on this protocol. On the one hand, it is troublesome to expand if new data sites are added later. On the other hand, if there is redundant data, it is either discarded directly or the original data is mixed together, which increases the burden on the project. Summary of the Invention
[0006] To address at least one of the technical problems mentioned above, this invention provides a time-series data storage and retrieval system and method for new energy power plants. This system improves the storage and retrieval efficiency of projects using a time-series database as the storage medium, supports the expansion of the time-series database, solves the problem of heterogeneous data source assimilation, reduces later maintenance costs, and finally solves the problem of redundant data preservation. It avoids simply discarding redundant data or mixing redundant data with useful data, thus reducing the project burden.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] The first aspect of this invention provides a time-series data storage and retrieval system for new energy power plants, comprising:
[0009] The receiving module is configured to acquire message data directly transmitted and forwarded by new energy power plants from different sources;
[0010] The adaptation module is configured to convert the parsed message data into an adaptation data file, label the measurement points in the file with indicator tags and unconfigured tags according to the pre-configured measurement point table, indicator table and measurement point indicator relationship table, encapsulate and forward to the distribution module.
[0011] The distribution module is configured to: determine the timing library serial number based on the serial number corresponding to the site code, and find the address information of the timing library from the timing information table;
[0012] The data storage module is configured to write the data processed by the adaptation module into the corresponding timing information table according to the address information of the found timing library.
[0013] Furthermore, the receiving module does not actively poll the data reported by each heterogeneous data source, but instead opens multiple interfaces in advance to inform each heterogeneous data source end, so that each data source end can actively adapt to the receiving module.
[0014] Furthermore, the system also includes a transmission module and a parsing module, wherein the transmission module is configured to transmit message data to the parsing module;
[0015] The parsing module is configured to translate the message content and parse out the indicator information and data information contained in the message data.
[0016] Furthermore, in the adaptation module, the step of labeling the measurement points with index tags and unconfigured tags according to the pre-configured measurement point table, index table, and measurement point index relationship table includes:
[0017] Define the measurement point table, index table, and measurement point-index relationship table in the relational database;
[0018] Import the monitoring data of each station into the monitoring point table. When a new monitoring point is added, update the corresponding monitoring point in the monitoring point table.
[0019] In the indicator table, corresponding indicators are set according to the specific needs of the project, serving as the KEY value for a certain monitoring item;
[0020] A relationship table between measurement points and indicators is obtained by associating the measurement point table and the indicator table.
[0021] Based on the measurement point index relationship table, label the measurement points in the data file with index labels and unconfigured labels respectively.
[0022] Furthermore, in the adaptation module, the measurement point index relationship table is read, and the measurement point data reported by the station is labeled with index tags. For measurement points not found in the measurement point index relationship table, an unconfigured tag is added, and finally, they are all encapsulated in a data file.
[0023] Furthermore, in the distribution module, the time series libraries used in the project are numbered in advance, the site codes are obtained from the relational database and converted into unique serial numbers, and the corresponding storage time series library serial number i is calculated as i = serial number % (n-1)n>1 and is an integer. i is recorded in the site table of the relational database, and the time series library information is obtained based on i.
[0024] Furthermore, the system also includes a reading module, which is configured to:
[0025] First, query the site table in the relational database using the site code to obtain the time series database code and the power generation group code. Then, read the time series database configuration table using the time series database code to obtain the IP address and module port of the time series database.
[0026] Then, query the measurement point index relationship table, use the index as the query condition, obtain the measurement point set from the measurement point index table, and then combine it with the station code to retrieve the measurement point of a certain index from the measurement point table;
[0027] Finally, the measurement point data is obtained by querying the information from the time series library and the measurement point information.
[0028] A second aspect of the present invention provides a method for accessing time-series data at a new energy power station, comprising the following steps:
[0029] Acquire message data directly transmitted and forwarded by new energy power plants from different sources;
[0030] The parsed message data is converted into an adaptation data file. The measurement points in the file are then labeled with indicator tags and unlabeled tags according to the pre-configured measurement point table, indicator table, and measurement point indicator relationship table, and then encapsulated.
[0031] The timing library serial number is determined based on the serial number corresponding to the site code, and the address information of the timing library is found from the timing information table.
[0032] The adapted data is written into the corresponding timing information table according to the address information of the found timing library.
[0033] Compared with the prior art, the beneficial effects of the present invention are:
[0034] 1. This invention stores and retrieves message data from multi-source heterogeneous renewable energy power plants that are directly transmitted and forwarded, using a time-series database as the storage medium. Measurement points are tagged with either indicator labels or unlabeled labels according to pre-configured measurement point tables, indicator tables, and measurement point-indicator relationship tables, and then encapsulated. The time-series database serial number is determined based on the serial number corresponding to the power plant code, and the address information of the time-series database is found from the time-series information table. The adapted data is then written into the corresponding time-series information table according to the found time-series database address information. This invention supports the expansion of the time-series database, solves the problem of heterogeneous data source assimilation, and reduces subsequent maintenance costs.
[0035] 2. In this invention, the receiving module does not actively poll for data reported by each heterogeneous data source. Instead, it pre-opens multiple interfaces to inform each heterogeneous data source, allowing each data source to proactively adapt to the receiving module. This avoids polling each source, reducing maintenance costs and network pressure.
[0036] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0037] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0038] Figure 1 This is a data flow diagram of each module in the new energy power station time-series data storage and retrieval system provided in this embodiment of the invention;
[0039] Figure 2 This is a schematic diagram of the adapter module structure provided in an embodiment of the present invention;
[0040] Figure 3 This is a schematic diagram of the query module structure provided in an embodiment of the present invention.
[0041] Figure 4 This is a schematic diagram of the overall process of a method for storing and retrieving time-series data at a new energy power station, provided by an embodiment of the present invention. Detailed Implementation
[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0043] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0045] Example 1
[0046] like Figure 1 As shown, this embodiment provides a time-series data storage and retrieval system for new energy power stations, including:
[0047] The receiving module is configured to acquire data packets directly transmitted and forwarded by new energy power plants from different sources;
[0048] The transmission module is configured to transmit data packets to the parsing module;
[0049] The parsing module is configured to parse messages into data files that the adaptation module can use.
[0050] The adaptation module is configured to: read the data file, execute logic, read the measurement point index relationship table, add index tags to the measurement point data reported by the station, add an unconfigured tag to the measurement points not found in the measurement point index relationship table, and finally encapsulate them in the data file.
[0051] The distribution module is configured to: write data packets into the corresponding time series table of the corresponding time series library, obtain the time series library number according to the station code, then calculate the time series library IP and module port, and locate the time series library table where the storage data is located according to the power generation group code of the station;
[0052] The storage module is configured to store the data reported by the field station into the time series database.
[0053] In this embodiment, each source end first agrees on the module port and message format; the source ends include each power station, power generation group, and production control area agreeing on the receiving module port and message format.
[0054] In the receiving module, the receiving module does not actively poll for data reported by each heterogeneous data source. Instead, it pre-opens multiple interfaces to inform each heterogeneous data source, allowing each data source to proactively adapt to the receiving module. By not polling each source, the receiving module reduces maintenance costs and network pressure.
[0055] Each heterogeneous data source forwards data through various routes according to the communication protocol before it reaches the receiving module;
[0056] The problem of heterogeneous data sources is solved by an adaptation module. For example, the active power measurement point corresponding to power generation group_power station code_active for power station A, and the active power measurement point corresponding to power station B is power station name_power; the adaptation module unifies the same indicators.
[0057] like Figure 2 As shown in the illustration, in one embodiment, the adapter module is responsible for parsing the data file into storable data, specifically including:
[0058] (1) First, define the measurement point table, index table, and measurement point index relationship table in the relational database;
[0059] (2) Import the monitoring data of each station into the monitoring point table. When a monitoring point is added, update the corresponding monitoring point in the monitoring point table.
[0060] (3) Set corresponding indicators according to the specific needs of the project, and use them as the KEY value of a certain monitoring item. The set indicators shall follow the principle of standardization and use the setting rules agreed upon by the project. For example, use the first letter of the Chinese Pinyin agreed upon by the project. If there are the same, add Arabic numerals after it.
[0061] (4) Obtain the measurement point index relationship table by associating the obtained measurement point table and index table;
[0062] The rule for associating the measurement point table and the index table is as follows: based on the needs of the project to be analyzed, the corresponding measurement points of the corresponding stations are associated with the indexes.
[0063] (5) According to the measurement point index relationship table, mark the measurement points in the data file with index labels and unconfigured labels respectively. Mark the measurement point data reported by the station with index labels. For measurement points not found in the measurement point index relationship table, mark them with unconfigured labels. Finally, encapsulate them in the data file.
[0064] As one example, the distribution module is designed to address the problem of low retrieval efficiency in single-database, single-table storage.
[0065] Add a site table to the relational database and convert the site code into a unique serial number.
[0066] The project has n time series libraries, numbered S0, S1, ..., Si.
[0067] The database is divided according to the serial number. The formula is i = serial number % (n-1) and n>1 is an integer. This determines which time series database the site corresponds to, and i is recorded in the site table.
[0068] Based on the power generation group code of the source station, there is a table named after the power generation group code in the corresponding time series database. By using the power generation group code field in the station table, the specific sequence number of the station table in the corresponding time series database can be located.
[0069] According to the above algorithm, there is one time series library left unused, which is used to store the measurement point data that the adaptation module has tagged without configuration. This data is not used in the project, but needs to be retained for future use.
[0070] As one example, such as Figure 3 As shown, the system also includes a reading module, which is configured as a separately located module integrated into the business system.
[0071] First, query the site table in the relational database using the site code to obtain the time series database code and the power generation group code. Then, read the time series database configuration table using the time series database code to obtain the IP address and module port of the time series database.
[0072] Then, query the measurement point index relationship table, use the index as the query condition, obtain the measurement point set from the measurement point index table, and then combine it with the station code to retrieve the measurement point of a certain index from the measurement point table;
[0073] Finally, the measurement point data can be retrieved using the information from the time series library and the measurement point information.
[0074] Example 2
[0075] like Figure 4 As shown in the figure, this embodiment provides a method for storing and retrieving time-series data of new energy power stations, including the following steps:
[0076] Step 1: Agree on the receiving module port and message format with each station, power generation group, and production control area.
[0077] Step 2: The receiving module opens its interface, the source requests the service, and sends a message to the receiving module.
[0078] Optionally, the receiving module does not poll each source end, reducing maintenance costs and network pressure.
[0079] Step 3: After receiving source data packets intermittently, the receiving module directly transmits them to the parsing module through the forwarding interface. The parsing module parses the packets according to the protocol to form a data file and forwards it to the adaptation module.
[0080] Step 4: The adaptation module is responsible for parsing the data file into storable data, specifically including:
[0081] Step 401: First, define the measurement point table, index table, and measurement point index relationship table in the relational database;
[0082] Step 402: Import the monitoring data of each station into the monitoring point table. When a monitoring point is added, update the corresponding monitoring point in the monitoring point table.
[0083] Step 403: Set corresponding indicators according to the specific needs of the project, which will serve as the KEY value for a certain monitoring item. The indicators set should follow the principle of standardization and use the setting rules agreed upon by the project. For example, use the first letters of the Chinese Pinyin agreed upon by the project. If there are duplicates, add Arabic numerals after them.
[0084] Step 404: Obtain the measurement point index relationship table by associating the obtained measurement point table and index table;
[0085] Table 1 is the relational database measurement point table, Table 2 is the relational database index table, and Table 3 is the relational database measurement point index table.
[0086] Table 1 Relationship Database Measurement Points
[0087] Auto-incrementing primary key Measurement point coding Notes Station Code
[0088] Table 2 Relationship Database Indicators
[0089] Auto-incrementing primary key Indicator coding Notes
[0090] Table 3. Relationship Database Measurement Indicators
[0091] Measurement point primary key Indicator primary key
[0092] Step 5: Transfer the completed data file to the distribution center.
[0093] Step 6: Query the relational database station table. If the time series database serial number server_number can be found, proceed to step 8. If the time series database serial number is empty, proceed to step 7.
[0094] Step 7: Assign time series libraries used in the project in advance, S0, S1, ..., Si. Obtain the site code from the relational database and convert it into a unique serial number. Calculate the corresponding storage time series library serial number i = serial number % (n-1), where n>1 and is an integer. Record i in the site table of the relational database. The time series library information can be obtained based on i.
[0095] Table 1 is the relational database site table, and Table 2 is the relational database time series database information table.
[0096] Table 1 Relationship Database Station List
[0097] Station Code Station Name unique serial number Time series library serial number Power generation group
[0098] Table 2
[0099] Serial Number IP address port username password
[0100] Step 8: Measurement points with indicator tags are stored in the corresponding time series databases based on the time series database information obtained in the above steps. Measurement points without tags are uniformly stored in the time series database with serial number n. The table is named after the power generation group.
[0101] Step 9: When querying measurement point data, use the indicator as the query condition to obtain the measurement point set from the measurement point indicator table. Then, combine it with the station code to retrieve the corresponding station measurement point from the measurement point table. Using this measurement point and combining it with the time series database information, the data stored in the time series database can be retrieved.
[0102] Example 3
[0103] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the new energy power station time-series data access method described in Embodiment 2.
[0104] Example 4
[0105] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the new energy power station time-series data access method described in Embodiment 2.
[0106] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0107] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0110] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0111] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A time-series data storage and retrieval system for new energy power stations, characterized in that, include: The receiving module is configured to acquire message data directly transmitted and forwarded by new energy power plants from different sources; The receiving module does not actively poll the data reported by each heterogeneous data source, but instead opens multiple interfaces in advance to inform each heterogeneous data source end, so that each data source end can actively adapt to the receiving module. The adaptation module is configured to: convert the parsed message data into an adaptation data file, and label the measurement points in the file with indicator tags and unconfigured tags respectively according to the pre-configured measurement point table, indicator table, and measurement point indicator relationship table, and then encapsulate and forward it to the distribution module; the adaptation module reads the measurement point indicator relationship table, labels the measurement point data reported by the station with indicator tags, and labels the measurement points not found in the measurement point indicator relationship table with unconfigured tags, and finally encapsulates them in a unified data file; The distribution module is configured to: determine the time series library serial number based on the serial number corresponding to the site code, and find the address information of the time series library from the time series information table; in the distribution module, the time series libraries used in the project are numbered in advance, the site code is obtained from the relational database and converted into a unique serial number, and the corresponding storage time series library serial number i = serial number % (n-1) is calculated, where n>1 and is an integer, n is the number of time series libraries on the project, i is recorded in the site table of the relational database, and the time series library information is obtained based on i; Specifically, based on the power generation group code of the source station, there is a table named after the power generation group code in the corresponding time series library. By using the power generation group code field in the station table, the specific serial number of the station table in the corresponding time series library is located. According to the above formula for storing the time series library serial number, the remaining nth time series library is unused and is used to store the measurement point data with unconfigured tags by the adaptation module. The measurement point data with unconfigured tags is retained for subsequent use. The data storage module is configured to write the data processed by the adaptation module into the corresponding timing information table according to the address information of the found timing library.
2. The new energy power station time-series data storage and retrieval system as described in claim 1, characterized in that, The system further includes a transmission module and a parsing module, wherein the transmission module is configured to transmit message data to the parsing module; The parsing module is configured to translate the message content and parse out the indicator information and data information contained in the message data.
3. The new energy power station time-series data storage and retrieval system as described in claim 1, characterized in that, In the adaptation module, the step of labeling the measurement points with index tags and unconfigured tags according to the pre-configured measurement point table, index table, and measurement point index relationship table includes: Define the measurement point table, index table, and measurement point-index relationship table in the relational database; Import the monitoring data of each station into the monitoring point table. When a new monitoring point is added, update the corresponding monitoring point in the monitoring point table. In the indicator table, corresponding indicators are set according to the specific needs of the project, serving as the KEY value for a certain monitoring item; A relationship table between measurement points and indicators is obtained by associating the measurement point table and the indicator table. Based on the measurement point index relationship table, label the measurement points in the data file with index labels and unconfigured labels respectively.
4. The new energy power station time-series data storage and retrieval system as described in claim 1, characterized in that, The system also includes a reading module, which is configured to: First, query the site table in the relational database using the site code to obtain the time series database code and the power generation group code. Then, read the time series database configuration table using the time series database code to obtain the IP address and module port of the time series database. Then, query the measurement point index relationship table, use the index as the query condition, obtain the measurement point set from the measurement point index table, and then combine it with the station code to retrieve the measurement point of a certain index from the measurement point table; Finally, the measurement point data is obtained by querying the information from the time series library and the measurement point information.
5. A method for storing and retrieving time-series data at a new energy power station, characterized in that, Includes the following steps: Acquire message data directly transmitted and forwarded by new energy power stations from different sources; specifically, do not actively poll the data reported by each heterogeneous data source, but instead open multiple interfaces in advance, inform each heterogeneous data source end, and let each data source end actively adapt. The parsed message data is converted into an adaptation data file. The measurement points are then labeled with indicator tags and unlabeled tags according to the pre-configured measurement point table, indicator table, and measurement point indicator relationship table, and then encapsulated. Specifically, the measurement point indicator relationship table is read, and the measurement point data reported by the station is labeled with indicator tags. For measurement points not found in the measurement point indicator relationship table, an unlabeled tag is added, and finally, they are all encapsulated in the data file. The timing library serial number is determined based on the serial number corresponding to the site code, and the address information of the timing library is found from the timing information table. Specifically, the timing libraries used in the project are numbered in advance, the site code is obtained from the relational database and converted into a unique serial number, and the corresponding storage timing library serial number i is calculated as i = serial number % (n-1), where n>1 and is an integer, n is the number of timing libraries in the project, i is recorded in the site table of the relational database, and the timing library information is obtained based on i. Specifically, based on the power generation group code of the source station, there is a table named after the power generation group code in the corresponding time series library. By using the power generation group code field in the station table, the specific serial number of the station table in the corresponding time series library is located. According to the above formula for storing the time series library serial number, the remaining nth time series library is unused and is used to store the measurement point data with unconfigured tags by the adaptation module. The measurement point data with unconfigured tags is retained for subsequent use. The adapted data is written into the corresponding timing information table according to the address information of the found timing library.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for accessing time-series data of a new energy power station as described in any one of claims 1-5.
7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for accessing time-series data of a new energy power station as described in any one of claims 1-5.