A real-time library system

Abstract

The application discloses a novel real-time library system, comprising: a panoramic data modeling module for generating a panoramic object model dictionary through a model configuration tool; a flexible control strategy module for analyzing network services transmitted to the real-time library, and flexibly controlling and selecting links of internal network channels of the real-time library; a storage and retrieval module for performing hash calculation on data in the real-time library by using an improved Fibonacci hash algorithm, and performing data storage and retrieval according to hash values; an interface module for providing a real-time library client network interface and a server local interface; a shared memory management module for allocating memory space in the real-time library according to the panoramic model dictionary and loading a data source; and a message notification mechanism module for notifying changes of data attributes in real time and publishing system messages. The application can support ten million-level large-capacity data, and improves overall reliability, stability and security of the system.

Description

Technical Field

[0001] This invention relates to a real-time database system, belonging to the field of data processing technology for power monitoring systems. Background Technology

[0002] Real-time databases have always been a core operating module of power monitoring systems, and their efficient and stable operation directly determines the performance indicators of the entire system. Traditionally, real-time databases in power systems need to be deployed on each machine node, placing high demands on machine performance and resulting in high operating loads and enormous hardware maintenance costs. Currently, with the development of distributed technologies, real-time databases no longer need to be deployed on every machine node. While distributed technologies have alleviated the symptoms of high load and low performance to some extent, they cannot solve deeper problems such as large capacity per node and low configuration of workstations.

[0003] With the rise of thin client models and big data and other new technologies, there is an urgent need for a new type of real-time library to achieve large-capacity distributed processing. However, the capacity of traditional real-time libraries is in the millions, which is difficult to meet the current market demand. At present, it is necessary to achieve a real-time library with a capacity of tens of millions through technological breakthroughs to improve the processing capacity of a single node. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a real-time database system. Through technologies such as panoramic data modeling, flexible network control, large-capacity data retrieval, and remote high-efficiency access, it achieves a real-time database with a capacity of tens of millions, improving system reliability while significantly reducing workstation maintenance costs and hardware / software performance indicators.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical means:

[0006] This invention proposes a real-time library system, comprising:

[0007] The panoramic data modeling module is used to generate a panoramic object model dictionary based on power monitoring system data through a model configuration tool.

[0008] The flexible control strategy module is used to parse the network services transmitted to the real-time library and to perform flexible control and link selection on the internal network channels of the real-time library.

[0009] The storage and retrieval module is used to perform hash calculations on the data in the real-time database using an improved Fibonacci hash algorithm, and to store and retrieve data based on the hash values.

[0010] The interface module provides a real-time library client network interface and a server local interface. The real-time library client network interface is connected to the flexible control strategy module, and the server local interface is connected to the shared memory management module.

[0011] The shared memory management module is used to allocate memory space in the real-time library and load data sources based on the panoramic model dictionary;

[0012] The message notification mechanism module is used to send notifications to clients or applications in the power monitoring system based on changes in objects in the real-time database, and also to publish system messages.

[0013] Furthermore, the panoramic object model dictionary includes table information of multiple objects in the power monitoring system and field information of each table.

[0014] Furthermore, the flexible control strategy module includes multiple information transmission methods: broadcast, intra-node, point-to-point, and customized methods.

[0015] Furthermore, when the flexible control strategy module selects the internal network channel link in real time, it automatically selects an idle channel for data transmission.

[0016] Furthermore, in the storage and retrieval module, the optimal hash golden section point of the improved Fibonacci hash algorithm is selected through multiple simulation experiments. Based on the optimal hash golden section point, the hash value corresponding to each object ID in the power monitoring system is calculated using the improved Fibonacci hash algorithm.

[0017] Furthermore, in the storage and retrieval module, when storing data, the hash value corresponding to the object ID is matched with the hash table in the shared memory of the real-time library. The hash table includes a collision pool linked list and a non-collision pool linked list. If the same hash value exists in the hash table, the object ID is stored in the collision pool linked list; otherwise, the object ID is stored in the non-collision pool linked list.

[0018] Furthermore, in the storage and retrieval module, during data retrieval, the hash value corresponding to the ID of the object to be retrieved is matched against the non-conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the non-conflict pool linked list, and the object ID corresponding to that hash value in the non-conflict pool linked list is consistent with the ID of the object to be retrieved, then the hash value and its corresponding data are retrieved. If no identical hash value is found in the non-conflict pool linked list, the hash value corresponding to the ID of the object to be retrieved is matched against the conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the conflict pool linked list, further retrieval is performed based on the ID of the object to be retrieved. When both the hash value and the object ID are consistent, the corresponding data is retrieved.

[0019] Furthermore, the operations of the shared memory management module include:

[0020] Based on the existing capacity configuration number of each table in the panoramic object model dictionary, request the size of shared memory in the real-time library;

[0021] Based on the table and field information in the panoramic object model dictionary, request real-time tablespace size from shared memory;

[0022] Load table data from the commercial database of the power monitoring system into a real-time table in shared memory.

[0023] Furthermore, in the power monitoring system, the client accesses the real-time database through the real-time database client network interface, and performs batch data addition, deletion, modification, and querying on the real-time database through the real-time database client network interface.

[0024] Furthermore, in the shared memory management module, object data is managed based on object paging. An object paging stores object data under the same object type. The object paging includes object type, attribute list, number of objects, and numerical matrix. When the number of objects under the same object type in the real-time library exceeds the maximum capacity of the object paging, a new object paging is created to manage the object data.

[0025] The following advantages can be obtained by adopting the above technical means:

[0026] This invention proposes a real-time database system. Through technologies such as power system panoramic data modeling, flexible network soft bus control strategies, large-capacity data retrieval and storage, an improved Fibonacci hash algorithm, real-time database shared memory management, remote access architecture, and message notification mechanisms, it enables a large-capacity distributed real-time database, effectively improving its data processing capabilities and reducing its operation and maintenance costs. The flexible control strategy allows for flexible control of network transmission within the real-time database, improving network transmission efficiency. The improved Fibonacci hash algorithm is used for hash calculations, followed by data retrieval and storage, improving data retrieval and storage efficiency and supporting tens of millions of data entries. Shared memory management rationally allocates real-time database memory, reducing resource waste.

[0027] This invention's real-time database system only needs to be deployed on the server side, with client nodes accessing the server remotely via the network. It features flexible network control and other characteristics. The client nodes only require low-configuration settings to achieve the processing capabilities of traditional high-configuration nodes, greatly alleviating the operation and maintenance costs of power companies, enabling high-efficiency operation of the power system's real-time database, and optimizing energy allocation efficiency, production operation efficiency, and enterprise management efficiency. Attached Figure Description

[0028] Figure 1 This is a system block diagram of a real-time library system according to the present invention. Detailed Implementation

[0029] The technical solution of the present invention will be further described below with reference to the accompanying drawings:

[0030] This invention proposes a real-time library system, such as... Figure 1 As shown, the system mainly includes a panoramic data modeling module, a flexible control strategy module, a storage and retrieval module, an interface module, a shared memory management module, and a message notification mechanism module. The system of this invention achieves a real-time database with a capacity of tens of millions through efficient indexing technology, panoramic modeling technology, and shared memory management technology, which greatly improves the processing capacity of a single node.

[0031] The panoramic data modeling module is primarily used to generate a panoramic object model dictionary based on power monitoring system data using model configuration tools, and also prepares for future model upgrades. The most important objects in the power grid dispatch automation system are power equipment, such as switches, disconnectors, transformers, and busbars. These power devices are also the main monitoring objects of the power monitoring system. This invention uses existing model configuration tools to create object models according to project needs, configure model parameters, create tables corresponding to the objects, and create the fields belonging to those tables, ultimately obtaining a panoramic object model dictionary. In this embodiment, the panoramic object model dictionary includes table information for multiple objects in the power monitoring system and field information for each table.

[0032] In this embodiment of the invention, the panoramic data modeling module includes object attribute modeling and data partitioning modeling. The invention determines object attributes based on object information defined in the field acquisition equipment. The modeling steps can be simplified as follows: determining object attributes and their data types based on the field equipment point table; creating an object type table through the data modeling module; and creating object attributes through the data modeling module. The invention also determines the number of data partitions based on the field operating environment. Object attribute modeling can determine the descriptive characteristics and relationships of different objects; data partitioning modeling helps different server nodes load different data partitions, achieving distributed loading on the server side.

[0033] The flexible control strategy module is used as a proxy for network transmission within the real-time library. Its main function is to parse network services transmitted to the real-time library, decouple the target addresses of these services, and perform flexible control and link selection for the internal network channels. In this embodiment, the flexible control strategy module includes multiple information transmission methods: broadcast, intra-node, point-to-point, and customized methods. During data transmission, it autonomously selects an idle channel for data transmission, achieving network data diversion.

[0034] In a real-time database system, when a primary and backup network switch occurs, the original backup network becomes the primary network and high-speed network, while the original primary network becomes the backup network and slow network, ensuring stable transmission of network data and issuance of commands.

[0035] In this invention, the real-time library is deployed on the server side. Client nodes remotely access the server via the network. When making a network service request, the network service client process packages the target service identifier and request information and sends it to the host node of the server where the target service resides. The real-time library process on the target server host node is responsible for delivering the data packet to the target service object based on the local service registration information. The target service object unpacks the data, completes the network service request processing, packages the processing result, converts the source address information of the request information into the target address information, and sends the result back to the client process.

[0036] The storage and retrieval module is used to implement an efficient indexing mechanism, providing optimal algorithms to support the retrieval and storage of tens of millions of data entries. Specifically, it utilizes an improved Fibonacci hash algorithm to perform hash calculations on the data in the real-time database, and then stores and retrieves data based on the hash values.

[0037] This invention uses a hash table set up in a real-time library to store hash values. Since the hash table in the real-time library is built in shared memory, the memory space occupied by the hash table is pre-calculated and allocated. Therefore, the total number of nodes in the hash table is fixed. Among large-capacity objects, hash values ​​obtained after hashing object IDs inevitably collide. Therefore, the hash table includes two sets of memory linked lists: a collision pool linked list and a non-collision pool linked list. The space for colliding nodes cannot be obtained through dynamic memory allocation; therefore, resolving collisions is crucial for implementing a hash table in shared memory.

[0038] In the storage and retrieval module, this invention adjusts the coefficients of the improved Fibonacci hash algorithm through multiple simulation experiments to obtain the coefficients with the least collision, which is the golden section point of the optimal hash of the improved Fibonacci hash algorithm. The golden section point can effectively reduce ID collisions between large-capacity objects. Based on the golden section point of the optimal hash, the improved Fibonacci hash algorithm is used to calculate the hash value corresponding to each object ID in the power monitoring system. Then, based on the hash value, data is stored and retrieved, which enables optimal management of the collision pool and non-collision pool, and achieves fast retrieval and storage of large-capacity data.

[0039] In the storage and retrieval module, when storing data, the hash value corresponding to the object ID is matched with the hash table in the shared memory of the real-time library. If the same hash value exists in the hash table, the object ID is stored in the collision pool linked list, and the object data is stored in the corresponding table in the real-time library. Otherwise, the object ID is stored in the non-collision pool linked list.

[0040] In the storage and retrieval module, when performing data retrieval, the hash value corresponding to the ID of the object to be retrieved is matched with the non-conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the non-conflict pool linked list, and the object ID corresponding to that hash value in the non-conflict pool linked list is consistent with the ID of the object to be retrieved, then the hash value and its corresponding data are retrieved. If the same hash value does not exist in the non-conflict pool linked list, the hash value corresponding to the ID of the object to be retrieved is matched with the conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the conflict pool linked list, further retrieval is performed based on the ID of the object to be retrieved. When the hash value and the object ID are consistent, the corresponding data is retrieved.

[0041] The interface module provides a real-time library client network interface and a server-side local interface. The real-time library client network interface is connected to the flexible control strategy module, and the server-side local interface is connected to the shared memory management module, providing application processes with an interface to access the real-time library. In power grid dispatch automation systems, various static parameters, dynamic parameters, and real-time data related to power equipment are stored in different tables. Obtaining these parameters and real-time data for a single power device requires querying many tables, which is cumbersome and inefficient for applications. This invention provides an object-oriented real-time library access interface for power equipment. Complex operations such as cross-table queries can be implemented internally within the interface, simplifying many operations and avoiding the time-consuming and inefficient operations caused by external applications.

[0042] In the power monitoring system, the client accesses the real-time database through the real-time database client network interface, and adds, deletes, modifies, and queries single objects or batches of data in the real-time database through the real-time database client network interface.

[0043] The shared memory management module is used to create and manage the real-time library. It allocates memory space in the real-time library based on the panoramic model dictionary and loads the data source. The real-time tables in the real-time library include structure and data; the structure corresponds to the model, and the data corresponds to its source. Loading the shared memory of the real-time library is divided into loading the commercial library module and loading the mapping background module. On the first startup, the commercial library module is selected; on subsequent startups, the mapping file loading mode is selected. After loading is complete, the real-time library runs normally.

[0044] In the shared memory management module, the operations of allocating memory space and loading data sources include:

[0045] Based on the existing capacity configuration of each table in the panoramic object model dictionary, a shared memory size is requested in the real-time library; based on the table and field information in the panoramic object model dictionary, a real-time tablespace size is requested from the shared memory; table data from the commercial library of the power monitoring system is loaded into the real-time table in the shared memory, thus realizing the loading of the data source.

[0046] In the shared memory management module, object data is managed based on object paging. An object page stores object data under the same object type. An object page includes object type, attribute list, number of objects, and numerical matrix. When the number of objects under the same object type in the real-time library exceeds the maximum capacity of the object page, a new object page is created to manage the object data.

[0047] The system of this invention can also persist and version control the real-time library. Based on the shared memory persistence algorithm, this invention maps the memory data of the real-time library to a hard disk file to achieve persistence and records the real-time library version number. The real-time library version number is a set of numbers recorded in shared memory. After the real-time library is persisted, the real-time library version number is written to a separate file to distinguish the real-time libraries corresponding to different models. When the on-site model is adjusted or changed, the real-time library version number is incremented.

[0048] The message notification mechanism module is primarily used to send notifications to clients or applications in the power monitoring system based on changes in objects in the real-time database. It is also used to publish system messages. The real-time database can automatically and proactively publish changes to registered objects to client applications.

[0049] The real-time library can implement three types of message notifications: record-level notifications, domain-level timed notifications, and table-level notifications. Record-level notifications are triggered when an object record changes, domain-level notifications are triggered when an object attribute changes, and table-level notifications are triggered when a table changes.

[0050] In this embodiment of the invention, the application uses a message notification mechanism comprising three steps: registering a notification, processing a notification, and canceling a notification. Registering a notification includes three types: specifying a change in a similar object, specifying a change in a specific attribute of a similar object, and specifying a change in a specific attribute of a particular object ID. Processing a notification involves the real-time library listening for all registered notifications and sending notifications that meet the requirements to the application. Canceling a notification means the application cancels a registered notification, and the real-time library no longer listens for changes to that notification.

[0051] Compared with existing technologies, to ensure the stable and efficient operation of the real-time library on server nodes, this invention solves key technical challenges such as panoramic, flexible, and large-capacity operation. It provides panoramic data modeling for power systems, flexible control strategies for network soft buses, retrieval and storage of large-capacity data, an improved Fibonacci hash algorithm, shared memory management for the real-time library, remote access architecture, and message notification mechanism. This enables the realization of a large-capacity distributed real-time library, improves the overall reliability, stability, and security of the system, enhances the data processing capabilities of the real-time library, and reduces the operation and maintenance costs of the real-time library.

[0052] This invention's real-time database system only needs to be deployed on the server side, enabling workstations to operate in a thin client mode. Client nodes remotely access the server via the network, featuring flexible network control and significantly reducing workstation maintenance costs and hardware / software performance. Client nodes, requiring only low configuration, can achieve the processing capabilities of traditional high-configuration nodes, greatly alleviating power company maintenance costs and enabling high-efficiency operation of the power system's real-time database, optimizing energy allocation efficiency, production operation efficiency, and enterprise management efficiency.

[0053] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0054] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0055] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0056] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0057] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A real-time library system, characterized in that, include: The panoramic data modeling module is used to generate a panoramic object model dictionary based on power monitoring system data through a model configuration tool. The flexible control strategy module is used to parse the network services transmitted to the real-time library and to perform flexible control and link selection on the internal network channels of the real-time library. Among them, flexible control refers to automatically selecting an idle channel for data transmission when selecting links on the internal network channels of the real-time library. The storage and retrieval module is used to perform hash calculations on the data in the real-time database using an improved Fibonacci hash algorithm, and to store and retrieve data based on the hash values. The interface module provides a real-time library client network interface and a server local interface. The real-time library client network interface is connected to the flexible control strategy module, and the server local interface is connected to the shared memory management module. The shared memory management module is used to allocate memory space in the real-time library and load data sources based on the panoramic model dictionary; The message notification mechanism module is used to send notifications to clients or applications in the power monitoring system based on changes in objects in the real-time database, and also to publish system messages.

2. The real-time library system according to claim 1, characterized in that, The panoramic object model dictionary includes table information of multiple objects in the power monitoring system and field information of each table.

3. The real-time library system according to claim 1, characterized in that, The flexible control strategy module includes multiple information transmission methods: broadcast, intra-node, point-to-point, and customized methods.

4. A real-time library system according to claim 1, characterized in that, In the storage and retrieval module, the optimal hashing golden section point of the improved Fibonacci hash algorithm was selected through multiple simulation experiments. Based on the optimal hashing golden section point, the hash value corresponding to each object ID in the power monitoring system was calculated using the improved Fibonacci hash algorithm.

5. A real-time library system according to claim 4, characterized in that, In the storage and retrieval module, when storing data, the hash value corresponding to the object ID is matched with the hash table in the shared memory of the real-time library. The hash table includes a collision pool linked list and a non-collision pool linked list. If the same hash value exists in the hash table, the object ID is stored in the collision pool linked list; otherwise, the object ID is stored in the non-collision pool linked list.

6. A real-time library system according to claim 4, characterized in that, In the storage and retrieval module, when performing data retrieval, the hash value corresponding to the ID of the object to be retrieved is matched with the non-conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the non-conflict pool linked list, and the object ID corresponding to that hash value in the non-conflict pool linked list is consistent with the ID of the object to be retrieved, then the hash value and its corresponding data are retrieved. If the same hash value does not exist in the non-conflict pool linked list, the hash value corresponding to the ID of the object to be retrieved is matched with the conflict pool linked list in the shared memory of the real-time library. If the same hash value is found in the conflict pool linked list, further retrieval is performed based on the ID of the object to be retrieved. When the hash value and the object ID are consistent, the corresponding data is retrieved.

7. A real-time library system according to claim 1, characterized in that, The operations of the shared memory management module include: Based on the existing capacity configuration number of each table in the panoramic object model dictionary, request the size of shared memory in the real-time library; Based on the table and field information in the panoramic object model dictionary, request real-time tablespace size from shared memory; Load table data from the commercial database of the power monitoring system into a real-time table in shared memory.

8. A real-time library system according to claim 1, characterized in that, In the power monitoring system, the client accesses the real-time database through the real-time database client network interface, and performs batch data addition, deletion, modification and query of the real-time database through the real-time database client network interface.

9. A real-time library system according to claim 1, characterized in that, In the shared memory management module, object data is managed based on object paging. An object paging stores object data under the same object type. Object paging includes object type, attribute list, number of objects, and numerical matrix. When the number of objects of the same object type in the real-time database exceeds the maximum capacity of the object pagination, a new object pagination is created to manage object data.

Images