Three-level storage method suitable for industrial control data
By combining the three-level storage methods of system cache, real-time database and historical database, the problems of slow writing speed, slow query speed and limited storage capacity in massive high-frequency industrial control data storage are solved, and high-frequency real-time acquisition and persistent storage of industrial control data are realized.
Patent Information
- Application Number
- CN202411863596.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-25
AI Technical Summary
When traditional databases face massive high-frequency industrial control data, the writing speed is slow and the query speed is slow, which cannot meet the needs of high-frequency data writing and real-time call acquisition. At the same time, the storage capacity is limited, which cannot meet the needs of large-capacity data storage.
The three-level storage method is adopted, combined with system cache, real-time database and historical database, and through cache monitoring object services and data dump object services, real-time write, fast query and long-term storage of industrial control data is realized.
It realizes high-frequency real-time acquisition and persistent storage of industrial control data, improves data writing and query speed, meets the needs of large-capacity storage, and overcomes the shortcomings of traditional databases.
Smart Images
Figure CN119884182A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to a three-level storage method suitable for industrial control data. Background Art
[0002] In the current era of automation, informatization, and intelligence, the role of data is becoming increasingly prominent. Since the development of industry, the technological content and automation level have been significantly improved, and the collection and utilization of various types of industrial control data generated in the production process have received more and more attention. The rapid development of the industrial Internet has also led to the rapid development of digital twin technology, industrial equipment remote monitoring and fault diagnosis technology. The development of these technologies has led to an increasing demand for the storage of industrial control data and higher requirements.
[0003] For industrial enterprises, the storage of industrial control data needs to ensure the accuracy and integrity of the data and have the requirement of large-capacity data storage. Therefore, various industrial control applications must ensure that the generated data can be recorded and stored efficiently and accurately, and also need to support the real-time application of high-frequency data. The existing solution is mainly to directly store industrial control data through traditional databases, which can meet the data storage needs to a certain extent.
[0004] However, the high frequency and large volume of industrial control data determine that direct storage through traditional databases cannot solve the problem of high-frequency data writing and meet the needs of industrial control applications for real-time call acquisition of high-frequency data. Therefore, the existing technology in this field has the following problems or shortcomings: First, when faced with massive high-frequency data, the traditional database has a slow writing speed and cannot meet the needs of high-frequency data writing; second, when faced with massive high-frequency data, the traditional database has a slow query speed and cannot meet the needs of industrial control applications for real-time call acquisition of high-frequency data; finally, when faced with massive high-frequency data, the traditional database has limited storage capacity and cannot meet the needs of large-capacity data storage. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a three-level storage method suitable for industrial control data. This method overcomes the defects of traditional industrial control data storage, and is based on a data storage strategy that combines system cache, real-time database and historical database. It can effectively record and store various types of industrial control data collected from data sources, and has the ability to provide real-time and high-frequency industrial control data acquisition requirements, thereby realizing the persistent storage requirements of industrial control data.
[0006] To solve the above technical problems, the present invention is applicable to a three-level storage method for industrial control data, comprising the following steps: Step 1: Initialize the system cache pool and create a cache queue that is sorted by object ID and time. Step 2: Collect and obtain industrial control data of each industrial control data measurement point, and mark the data measurement point as an object, cache it into the system cache pool, and mark the changed industrial control data object as changed; Step 3: Create and start a cache monitoring object service. The cache monitoring object service scans the industrial control data changes of the data measurement points in the cache pool in real time. Step 4: The cache monitoring object service batch updates the industrial control data of the changed data measurement points into the real-time database, and marks the data measurement points as updated; Step 5: Create and start a data dump object service, which periodically scans the data storage status of the real-time database; Step 6: The data dump object service periodically scans the real-time database, and dumps the industrial control data of the data measurement points that exceed the set data status range, data storage time range, and storage data tag list range into the historical database; Step 7: After completing the industrial control data dump operation, the data dump object service deletes the dumped industrial control data from the real-time database.
[0007] Furthermore, the industrial control data in the system cache pool provides real-time industrial control data applications for industrial control data measurement points, and the industrial control data in the real-time database and the historical database provide historical industrial control data applications for industrial control data measurement points.
[0008] Since the three-level storage method for industrial control data of the present invention adopts the above technical scheme, that is, the method is based on cache storage, real-time storage, historical storage, cache monitoring object service, and data dump object service, the industrial control system data acquisition module obtains the latest industrial control data from each data measurement point, and pushes the latest industrial control data collected into the system cache storage in real time according to the cache storage rules, and updates the data measurement points of industrial control data changes; the cache monitoring object service scans the industrial control data of all data measurement points in real time, reads the changed industrial control data, and stores them in batches in real time into the real-time database, and marks the data measurement points as updated; the data dump object service reads the industrial control data that meets the rules in the real-time database according to the configured dump rules, and stores them in the historical database according to the dump rules, and deletes the industrial control data that has been dumped in the real-time database, completing the three-level storage processing process of industrial control data. The method overcomes the defects of traditional industrial control data storage, based on the data storage strategy combining system cache, real-time database and historical database, effectively records and stores various types of industrial control data collected from data sources, and has the acquisition requirements of providing real-time high-frequency industrial control data, and realizes the persistent storage requirements of industrial control data. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments: Figure 1 The present invention is a flowchart of a three-level storage method for industrial control data. DETAILED DESCRIPTION
[0010] Implementation example Figure 1 As shown, the three-level storage method for industrial control data of the present invention includes the following steps: Step 1: Initialize the system cache pool and create a cache queue that is sorted by object ID and time. Step 2: Collect and obtain industrial control data of each industrial control data measurement point, and mark the data measurement point as an object, cache it into the system cache pool, and mark the changed industrial control data object as changed; Step 3: Create and start a cache monitoring object service. The cache monitoring object service scans the industrial control data changes of the data measurement points in the cache pool in real time. Step 4: The cache monitoring object service batch updates the industrial control data of the changed data measurement points into the real-time database, and marks the data measurement points as updated; Step 5: Create and start a data dump object service, which periodically scans the data storage status of the real-time database; Step 6: The data dump object service periodically scans the real-time database, and dumps the industrial control data of the data measurement points that exceed the set data status range, data storage time range, and storage data tag list range into the historical database; Step 7: After completing the industrial control data dump operation, the data dump object service deletes the dumped industrial control data from the real-time database.
[0011] Preferably, the industrial control data in the system cache pool provides real-time industrial control data applications for industrial control data measurement points, and the industrial control data in the real-time database and the historical database provide historical industrial control data applications for industrial control data measurement points.
[0012] The particularity of the industrial control environment is that the industrial control data of a large number of data measurement points changes rapidly, reaching an update frequency of milliseconds or even higher. The high-speed access to this part of data is I / O intensive. At the same time, industrial control data also has the characteristics of simple structural rules. Most of the data information is nothing more than name or ID identification, value, and timestamp. The data is normally stored based on streaming, and only needs to be inserted at the end of the data queue. Usually, deletion, modification, and other operations are not performed.
[0013] This method realizes the storage of industrial control data through a strategy combining system cache, real-time database and historical database, and supports high-frequency application of industrial control data. The real-time industrial control data of the data measurement point is stored in the cache space created by the system to ensure the fastest access speed; at the same time, by creating a cache monitoring object service for the cache space, the changed industrial control data is transferred to the real-time database, and the industrial control data is sorted by point and time using the characteristics of the real-time database, supporting the query and retrieval of new data by industrial control data applications, and cooperating with the data dump object service to retrieve the data of the real-time database regularly, and the data that exceeds the set data status range, data storage time range, storage data tag list and other ranges is dumped into the historical database. The historical database can use the database system or a binary file in a custom format, striving to minimize the space occupied by the data unit and maximize the query speed.
[0014] This method effectively separates the different storage features of industrial control data through a three-level storage strategy of cache, real-time database, and historical database, and distinguishes the real-time, historical and other data application requirements of industrial control data, thereby improving the response speed of industrial control data and effectively taking both into account. It is an ideal industrial control data storage method.
Claims
1. A three-level storage method suitable for industrial control data, characterized in that The steps include: Step 1: Initialize the system cache pool and create a cache queue that is sorted by object ID and time. Step 2: Collect and obtain industrial control data of each industrial control data measurement point, and mark the data measurement point as an object, cache it into the system cache pool, and mark the changed industrial control data object as changed; Step 3: Create and start a cache monitoring object service. The cache monitoring object service scans the industrial control data changes of the data measurement points in the cache pool in real time. Step 4: The cache monitoring object service batch updates the industrial control data of the changed data measurement points into the real-time database, and marks the data measurement points as updated; Step 5: Create and start a data dump object service, which periodically scans the data storage status of the real-time database; Step 6: The data dump object service periodically scans the real-time database, and dumps the industrial control data of the data measurement points that exceed the set data status range, data storage time range, and storage data tag list range into the historical database; Step 7: After completing the industrial control data dump operation, the data dump object service deletes the dumped industrial control data from the real-time database.
2. The three-level storage method for industrial control data according to claim 1 is characterized in that: The industrial control data in the system buffer pool provide real-time industrial control data applications for industrial control data measurement points, and the industrial control data in the real-time database and the historical database provide historical industrial control data applications for industrial control data measurement points.