A multi-module integrated real-time database system

Abstract

This invention relates to a data modeling technology, specifically a multi-module integrated real-time database system. It includes multiple parallel magnetic core memories, each comprising a synchronization host that establishes unidirectional communication with multiple application systems via a filtering channel controller. The magnetic core memories also include a memory database that interacts with the synchronization host. The synchronization host is also connected to multiple data interfaces for real-time data acquisition and supports multiple real-time data acquisition sources. By constructing a distributed memory database using the synchronization host, memory database, and filtering channel controller, a large amount of real-time data is first received and stored in memory. The memory data is then instantiated in a process-by-process manner, forming a system architecture with excellent scalability. The overall system performance can be adjusted by adding a corresponding number of magnetic core memories according to actual application requirements.

Description

Technical Field

[0001] This invention relates to a data modeling technology, specifically a multi-module integrated real-time database system. Background Technology

[0002] Real-time database systems are supporting software for developing real-time control systems, data acquisition systems, CIMS systems, and more. In the process industry, real-time database systems are widely used for control system monitoring, advanced control, and optimization, providing real-time data services and various data management functions for enterprise production management and scheduling, data analysis, decision support, and remote online browsing. Real-time databases have become the foundational data platform for enterprise informatization, directly collecting and acquiring various data from the enterprise's operational processes in real time, and transforming them into effective public information for various business operations. This meets the needs of enterprise production management, process monitoring, and business management for the integrity, consistency, and secure sharing of real-time information, and can establish a bridge for information communication between enterprise automation systems and management information systems. It helps various professional management departments within an enterprise utilize this critical real-time information to improve operational efficiency in production and sales.

[0003] A key characteristic of real-time databases is real-time performance, encompassing both data real-time performance and transaction real-time performance. Data real-time performance refers to the update cycle of on-site I / O data; as a real-time database, data real-time performance cannot be ignored. Generally, data real-time performance is primarily constrained by on-site equipment, especially for older systems. Transaction real-time performance refers to the speed at which the database processes transactions. It can be event-triggered or timed-triggered. Event-triggered events are immediately scheduled upon occurrence, allowing for immediate processing but consuming significant system resources; timed-triggered events, on the other hand, acquire scheduling rights within a defined timeframe. A complete real-time database, considering both system stability and real-time performance, must provide both scheduling methods simultaneously.

[0004] The data sources for real-time databases vary depending on the industry and type of enterprise. Generally, the main data sources include DCS control systems, control systems built with configuration software and PLCs, Supervisory Control and Data Acquisition (SCADA) systems, relational database systems, data directly connected to hardware devices, and data manually entered through a human-machine interface. Based on the acquisition method, data can be categorized as follows: standard OPC method supporting the OPC protocol, standard DDE communication method supporting the DDE protocol, standard MODBUS communication method supporting the MODBUS protocol, ODBC communication method via the ODBC protocol, proprietary communication method written using APIs, and method using proprietary protocol drivers for devices, etc.

[0005] There are many mature real-time databases on the market, but currently, they still have two technical shortcomings: they are expensive and the technology is outdated.

[0006] To elaborate, the most mainstream real-time database products currently available are Industrial SQL, PI, and IP21 (InfoPlus.21). The prices of these three database products are as follows:

[0007] brand base price Interface fee PI $100,000 $6,000 per interface Info Plus.21 $110,000 to $210,000 $10,000 per interface Industrial SQL $65,000 $1200 per access point

[0008] Regarding technical issues, traditional real-time databases primarily serve systems with traditional client / server architectures and have weaker support for browser / server architecture systems.

[0009] The client / server architecture separates the client (usually a program with a graphical user interface) from the server. Each instance of client software can send requests to a server or application server. The browser / server architecture, unlike the client / server architecture, does not require the client to install special software; only a browser is needed. The browser interacts with the database through a web server, allowing it to work easily on different platforms.

[0010] At the same time, the B / S architecture has unparalleled advantages over the C / S architecture in terms of system maintainability, UI friendliness, and multi-device synchronization.

[0011] On the other hand, traditional real-time databases can only run on Windows systems and do not support operating systems such as Linux and UNIX.

[0012] Based on the above reasons, and taking into account the sustainability of the overall water conservancy system, this invention proposes a real-time database system integrating multiple modules. Summary of the Invention

[0013] The purpose of this invention is to provide a multi-module integrated real-time database system to solve the problems mentioned in the background art.

[0014] To achieve the above objectives, the present invention provides the following technical solution:

[0015] A multi-module integrated real-time database system includes multiple parallel magnetic core memories. Each magnetic core memory includes a synchronization host. The synchronization host establishes a one-way communication relationship with multiple application systems through a filter channel controller to achieve synchronous data forwarding.

[0016] A hierarchical relationship is formed between multiple application systems that communicate with the same filter channel controller, and the hierarchical relationship is constructed through multi-level real-time data output terminals.

[0017] The magnetic core storage also includes a memory database, which interacts with the synchronization host and sends data to multiple transmission and conversion modules;

[0018] The synchronization host is also connected to multiple data interfaces for real-time data collection and supports multiple real-time data collection sources.

[0019] The multi-module integrated real-time database system described above: multiple data interfaces form a multi-level sampling end to form a multi-level sampling strategy. The multi-level sampling end supports multiple data types, and the data interfaces also include web service interfaces that support external access.

[0020] The multi-module integrated real-time database system described above: the transmission and conversion module integrates a data compression algorithm, and the multi-module integrated real-time database system supports connection to an OPC server and supports real-time data query;

[0021] The multi-module integrated real-time database system is equipped with a historical data query module to enable the system to support historical data query services.

[0022] The multi-module integrated real-time database system described above: the transmission conversion module is connected to the backup and recovery system to support data backup and recovery, and the backup and recovery system is connected to a multi-threaded recovery tool.

[0023] The multi-module integrated real-time database system described above: the real-time data output terminal includes at least one fast computing unit, which is used to quickly convert data into a signal with a predetermined resolution.

[0024] The multi-module integrated real-time database system described above has the following characteristics: the single-node data tag capacity of the transmission conversion module and the data interface is 100,000, and the event throughput of the transmission conversion module and the data interface per unit time is 100,000 / second.

[0025] The multi-module integrated real-time database system described above includes multiple integration ports for integrating multiple magnetic core memories to deploy more magnetic core memories in the system.

[0026] Compared with the prior art, the beneficial effects of the present invention are: by constructing a distributed memory database through the synchronous host, memory database and filtering channel controller set in the present invention, a large amount of real-time data is first received and stored in memory, and the memory data is instantiated in process to form a system architecture with excellent scalability. The overall performance of the system can be increased by adding a corresponding number of magnetic core memories according to the actual application requirements.

[0027] The real-time database in this invention can achieve a higher information collection frequency, thus obtaining a larger amount of data, and can handle data in different scenarios with hierarchical processing. Attached Figure Description

[0028] Figure 1 This is a structural framework diagram of a multi-module integrated real-time database system.

[0029] Figure 2 This is a technical functional diagram of a real-time database system that integrates multiple modules. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0031] Please see Figures 1-2 As an embodiment of the present invention, the multi-module integrated real-time database system includes multiple parallel magnetic core storage (CS), which is developed based on the cross-platform language Java and can run on platforms such as Linux, UNIX, and Windows.

[0032] Considering factors such as security and cost-effectiveness, it is preferred to run on the Linux platform.

[0033] The magnetic core memory includes a CS sync master, which establishes a one-way communication relationship with multiple application systems through a filter controller to achieve synchronous data forwarding;

[0034] A hierarchical relationship is formed between the multiple application systems that communicate with the same filter channel controller;

[0035] Specifically, multiple application systems that communicate with the same filter channel controller form a hierarchical relationship through a multi-level real-time data publisher (FP). Each level has a different frequency, and multi-frequency sampling data forwarding can be performed through the multi-level real-time data publisher to meet the different needs of third-party application systems. Moreover, the hierarchical relationship between the filter channel controller and the application systems can form a multi-channel data distribution with buffering.

[0036] The real-time data output terminals are divided into n levels (1, 2, 3...n), and each level of the real-time data output terminal corresponds to a different application system. The filtering channel controller transmits the data sequentially through the multi-level real-time data output terminals.

[0037] The magnetic core storage also includes a memory database (Redis), which interacts with the synchronization host and sends data to multiple transmission and conversion modules;

[0038] The transmission and conversion module converts data from the in-memory database into relational databases, non-relational databases, and files. It supports various relational and non-relational databases, including but not limited to Oracle, MySQL, SQL Server, and MongoDB. This allows the entire system to function not only as a data acquisition platform but also as a distribution platform. The transmission and conversion module can be flexibly configured with any type of database instance. In offline situations, it automatically enables temporary local data storage to prevent cached data overflow and loss. After the transmission and conversion module recovers, tools can be used to quickly restore and replenish data using multi-threaded methods. During downtime, local storage is compressed and archived; after recovery, data is decompressed and retrieved, ensuring uninterrupted and lossless data collection. Online tools automatically align data across multiple channels to ensure data consistency across storage systems.

[0039] Furthermore, the MySQL database in this invention has an online migration and backup function, and the migration and backup speed is adjustable to ensure that it will not slow down the normal operation of the main system. The segmented and speed-based migration and archiving does not affect the host speed.

[0040] The biggest advantage of this system framework is its excellent scalability. In theory, as long as the hardware supports it, multiple Core Storages can be deployed, easily improving overall performance.

[0041] The synchronization host is also connected to multiple data interfaces, including the most mainstream and common OPCserver data interface, as well as other data interfaces (Data Server, abbreviated as DS) besides OPC, for real-time data collection, and can support multiple real-time data collection sources.

[0042] The distributed memory database is constructed by the synchronous host, memory database and filtering channel controller set in this invention. It first receives a large amount of real-time data and stores it in memory, and then instantiates the memory data in processes to form a system architecture with excellent scalability. The overall performance of the system can be increased by adding a corresponding number of magnetic core memories according to the actual application requirements.

[0043] The real-time database in this invention can achieve a higher information collection frequency, thus obtaining a larger amount of data, and can handle data in different scenarios with hierarchical processing.

[0044] As a further aspect of the present invention, multiple data interfaces form a multi-level sampling end to form a multi-level sampling strategy. The multi-level sampling end supports multiple data types (single and double precision floating point, 1 / 2 / 4 / 8 byte integer, string, boolean, etc.).

[0045] The data interface also includes a web service interface that supports external access.

[0046] Because of the adoption of a multi-level sampling strategy, the data interface in this invention can be adapted to data collection in a wide range of fields, and the web service interface enables the system in this invention to support external access, making its functions more diversified.

[0047] As a further embodiment of the present invention, the transmission conversion module integrates a data compression algorithm, including lossless compression and lossy compression; and the real-time database system integrated by the multi-modules supports connection with an OPC server and supports real-time data query.

[0048] The multi-module integrated real-time database system is equipped with a historical data query module to enable the system to support historical data query services.

[0049] Because the compression algorithm is integrated into the transmission conversion module, local data can be compressed and temporarily stored after a system interruption, without occupying system space. After the system recovers, the compressed data can be released to fill in the data during the interruption, thus achieving continuous and uninterrupted data collection.

[0050] As a further embodiment of the present invention, the transmission conversion module is connected to the backup and recovery system to support data backup and recovery, and the backup and recovery system is connected to a multi-threaded recovery tool.

[0051] When the transmission conversion module is offline, the backup and recovery system can automatically start local temporary data storage to form a cached data protection function.

[0052] Once the transmission conversion module is online, the multi-threaded recovery tool is used to quickly restore the data and replenish the cached data from the offline state.

[0053] When the system is shut down, the multi-threaded recovery tool can archive the compressed data stored locally and decompress the archived data after restarting.

[0054] As a further embodiment of the present invention, the real-time data output terminal includes at least one fast computing unit, which is used to quickly convert data into a signal with a predetermined resolution.

[0055] By using a fast computing unit, the data signal achieves millisecond-level latency during transmission, enabling a more efficient and faster response.

[0056] As a further embodiment of the present invention, the single-node data tag capacity of the transmission conversion module and the data interface is 100,000, and the event throughput of the transmission conversion module and the data interface per unit time is 100,000 / second.

[0057] The use of larger capacity single-node data tags enables the system to handle larger data inputs and outputs, resulting in faster and more stable overall operation and thus improved overall system performance.

[0058] As a further embodiment of the present invention, the multi-module integrated real-time database system includes multiple integration ports, which are used to integrate multiple magnetic core memories to deploy more magnetic core memories in the system.

[0059] The integrated port allows for expansion within the system framework of this invention, enabling the addition or removal of magnetic core memory deployments based on application scenarios, within the limits of hardware load.

[0060] The above embodiments are exemplary and not restrictive. Therefore, any technical solutions that can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention are included within the scope of the present invention.

Claims

1. A multi-module integrated real-time database system, comprising multiple parallel magnetic core memories, characterized in that: The magnetic core memory includes a synchronization host, which establishes a one-way communication relationship with multiple application systems through a filter channel controller to achieve synchronous data forwarding; A hierarchical relationship is formed between multiple application systems that communicate with the same filter channel controller, and the hierarchical relationship is constructed through multi-level real-time data output terminals. The magnetic core storage also includes a memory database, which interacts with the synchronization host and sends data to multiple transmission and conversion modules; The synchronization host is also connected to multiple data interfaces for real-time data collection and supports multiple real-time data collection sources.

2. The multi-module integrated real-time database system according to claim 1, characterized in that, Multiple data interfaces form a multi-level sampling end to form a multi-level sampling strategy. The multi-level sampling end supports multiple data types, and the data interfaces also include a web service interface that supports external access.

3. The multi-module integrated real-time database system according to claim 1, characterized in that, The transmission conversion module integrates a data compression algorithm, and the real-time database system integrated by the multiple modules supports connection to an OPC server and supports real-time data query. The multi-module integrated real-time database system is equipped with a historical data query module to enable the system to support historical data query services.

4. The multi-module integrated real-time database system according to claim 3, characterized in that, The transmission conversion module is connected to the backup and recovery system to support data backup and recovery, and the backup and recovery system is connected to the multi-threaded recovery tool.

5. A multi-module integrated real-time database system according to claim 1, characterized in that, The real-time data output terminal includes at least one fast computing unit, which is used to quickly convert data into a signal with a predetermined resolution.

6. A multi-module integrated real-time database system according to claim 2, characterized in that, The single-node data tag capacity of the transmission conversion module and the data interface is 100,000, and the event throughput of the transmission conversion module and the data interface per unit time is 100,000 / second.

7. The multi-module integrated real-time database system according to any one of claims 1-6, characterized in that, The multi-module integrated real-time database system includes multiple integration ports, which are used to integrate multiple magnetic core memories to deploy more magnetic core memories in the system.

Images