Distributed storage method and system of real-time database, electronic equipment and storage medium

By employing a distributed storage method in the power grid dispatching system, combining sharding, replication, and centralized storage, and establishing an index to enable global network and intra-chip shared memory access, the problem of data scale expansion in the power grid dispatching system is solved, improving processing efficiency and the integrity of the power grid model.

CN121009140APending Publication Date: 2025-11-25NR ELECTRIC CO LTD +1

Patent Information

Application Number
CN202410653636.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-23
Publication Date
2025-11-25

AI Technical Summary

Technical Problem

Traditional single-machine centralized storage architecture cannot meet the access requirements of massive data in power grid dispatching systems. Existing sharding technology destroys the integrity of the power grid model and is not suitable for local shared access to the real-time power grid database, resulting in low processing efficiency.

Method used

A distributed storage approach is adopted, which stores the contents of the real-time database on multiple physical nodes in three ways: sharding, replication, and centralization, based on the power public information model. Corresponding indexes are established to realize global network access and on-chip shared memory access interfaces, taking into account both the local integrity of the power grid model and processing efficiency.

Benefits of technology

It improves the horizontal scalability and processing efficiency of the power grid dispatching system, maintains the local integrity of the power grid model, and supports efficient parallel processing and remote access.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121009140A_ABST
    Figure CN121009140A_ABST
Patent Text Reader

Abstract

The invention provides a distributed storage method and system of a real-time database, electronic equipment and a storage medium, and relates to the technical field of power grid dispatching automation systems. The distributed storage method of the real-time database comprises the following steps: constructing the real-time database according to an electric power public information model of a monitored power grid; setting a plurality of fragments for storing data in the plurality of physical nodes; the first content in the real-time database is stored in the corresponding fragment in the multiple fragments in a fragment storage mode; respectively storing the second content in the real-time database into a plurality of fragments in a copy storage mode; and storing the third content in the real-time database into the first fragment in the plurality of fragments in a centralized storage manner. While the data fragmentation is realized, the local integrity of the on-chip power grid model is maintained, and the on-chip application can still perform efficient processing in a local shared memory mapping file mode based on the fragmentation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of power grid dispatch automation system technology, and more specifically, to a distributed storage method and system for real-time databases, electronic devices, and storage media. Background Technology

[0002] In the field of power grid dispatch automation systems, the real-time requirements for data acquisition and control are very high. Existing domestic power grid dispatch automation systems such as PCS9000, D5000, and DF8003 store their power grid models and real-time data in memory-based real-time databases to meet real-time processing requirements.

[0003] To ensure the processing speed of core applications, real-time databases are typically deployed on application servers, such as... Figure 1 As shown, the application can directly access the real-time database using a local shared memory-mapped library file. This local access method, which does not go through the network, can improve the access speed by two orders of magnitude. The human-computer interface program, on the other hand, accesses the real-time database remotely through a conventional network interface.

[0004] However, with the expansion of the power grid, the number of data points connected to the power grid control system has increased from hundreds of thousands to millions or tens of millions. Traditional single-machine centralized storage architectures for real-time databases cannot meet the needs of massive data access. Clustered distributed real-time databases based on database sharding technology have begun to be applied in the field of power monitoring and dispatching. For example, sharded clustered databases such as MongoDB and Redis generally hash the records in the table by key or group them according to set rules to form data shards. The data in each shard is non-overlapping, and the shards of the same database are stored on multiple physical nodes. When processing data read and write requests, the network interface layer's positioning algorithm forwards the request to the node where the data is located.

[0005] Due to the existence of local shared access methods, existing general database sharding technology is not suitable for direct application in real-time power grid databases: First, sharding will destroy the integrity of the power grid model. Although sharding by substation or geographical region can ensure the integrity of equipment and measurement models within a substation, each substation needs to have complete table data for parameters such as voltage level definitions and measurement type definitions, as well as dictionary tables, to ensure the integrity of data references within the substation. Second, for data involving cross-sharding or global processing, such as topology and lines, the data size is not large. To facilitate local memory mapping processing by the processing program, centralized storage should still be used instead of sharding.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] To address at least one of the aforementioned problems, this application proposes a distributed storage method and system for real-time databases, as well as an electronic device and storage medium.

[0008] According to a first aspect of this application, at least one embodiment of this application provides a distributed storage method for a real-time database, comprising: constructing a real-time database based on a power public information model of a monitored power grid; setting up multiple shards for storing data in multiple physical nodes; storing a first content of the real-time database into a corresponding shard among the multiple shards in a sharded storage manner; storing a second content of the real-time database into the multiple shards respectively in a replicated storage manner; and storing a third content of the real-time database into a first shard among the multiple shards in a centralized storage manner.

[0009] For example, in some embodiments of this application, the first content includes a table of equipment models and a table of measurement models for substations in the monitored power grid; the second content includes a configuration table of the monitored power grid, which includes a power grid parameter table and a dictionary table; and the third content includes a global topology table and a line table of the monitored power grid.

[0010] For example, in some embodiments of this application, the method further includes: establishing an index for the real-time database, including: generating corresponding target table identifiers according to the storage methods of the first content, the second content, and the third content to establish a first index; generating target data identifiers according to the sequence numbers of the shards where the first content, the second content, and the third content are stored to establish a second index; and generating target storage location identifiers according to the positions of the first content, the second content, and the third content in the stored shards to establish a third index.

[0011] For example, in some embodiments of this application, a global network access interface is used to access the real-time database, wherein: obtaining the content of the real-time database based on an access request according to an index includes: accessing real-time data within a corresponding shard according to the access request, wherein the access request includes: a target table identifier, a target data identifier, and a target storage location identifier; generating query results based on the accessed real-time data; and returning the query results to the sending end of the access request.

[0012] For example, in some embodiments of this application, accessing real-time data within a corresponding shard according to the access request includes: querying the first index based on the target table identifier to determine the storage method of the data to be accessed; querying the second index based on the target data identifier to determine the shard where the data to be accessed is stored, including: when the storage method of the data to be accessed is sharded storage, querying the second index based on the target data identifier carried in the access request to obtain the shard sequence number of the data to be accessed, and locating the shard of the sequence number; when the storage method of the data to be accessed is replicated storage, determining the type of the access request; when the type of the access request is a query request, locating any one of the plurality of shards; when the type of the access request is a modification request, locating the plurality of shards; when the storage method of the data to be accessed is centralized storage, locating the first shard; and querying the third index based on the target storage location identifier to determine the storage location of the data to be accessed in the data area within the shard.

[0013] For example, in some embodiments of this application, the method of accessing the real-time database using an on-chip shared memory access interface includes: accessing on-chip data based on a complete on-chip plant model, querying the third index according to the target storage location identifier, determining the storage location of the data to be accessed in the data area within the segment, and directly reading the real-time data at the storage location of the data area through a local memory-mapped file.

[0014] According to a second aspect of this application, at least one embodiment of this application provides a distributed storage system for a real-time database, used to execute the distributed storage method as described in any one of the first aspects, characterized in that it includes: a database creation module, used to construct a real-time database based on a power public information model of a monitored power grid; a shard creation module, used to set up multiple shards for storing data in multiple physical nodes; a content sorting module, used to store a first content in the real-time database into a corresponding shard among the multiple shards in a sharded storage manner; store a second content in the real-time database into the multiple shards respectively in a replicated storage manner; and store a third content in the real-time database into a first shard among the multiple shards in a centralized storage manner.

[0015] For example, in some embodiments of this application, it further includes: an index creation module, used to create an index for the real-time database, including: generating corresponding target table identifiers according to the storage methods of the first content, the second content, and the third content to create a first index; and being used to generate target data identifiers according to the sequence numbers of the shards where the first content, the second content, and the third content are stored to create a second index; and being used to generate target storage location identifiers according to the positions of the first content, the second content, and the third content in the stored shards to create a third index;

[0016] The data query module accesses the real-time database using either a global network access interface or an on-chip shared memory access interface. This includes: retrieving the content of the real-time database based on an access request and an index; accessing real-time data within a corresponding slice based on the access request, where the access request includes a target table identifier, a target data identifier, and a target storage location identifier; generating query results based on the accessed real-time data; and returning the query results to the sender of the access request.

[0017] According to a third aspect of this application, at least one embodiment of this application provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the method as described in any one aspect of the first application.

[0018] According to a fourth aspect of this application, at least one embodiment of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described in any one of the first aspects.

[0019] This application provides a distributed storage method and system for real-time databases, as well as electronic devices and storage media. It offers three table storage methods: centralized, replicated, and sharded, and three indexes for fast data retrieval. Based on these three storage and indexing methods, it implements two access interfaces: a global network access interface and an on-chip shared memory access interface. Balancing the horizontal scalability and processing efficiency of the real-time database, different distributed storage methods are adopted for different tables according to the characteristics of power grid model data and business programs. While implementing data sharding, the local integrity of the power grid model within each shard is maintained. Business programs based on the complete substation model within each shard can still perform efficient parallel processing using local shared memory mapped files. The network interface layer masks the differences between various storage methods through routing rules, providing a logical full table view of the real-time database, facilitating remote access for human-machine interfaces such as displays and curves.

[0020] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0021] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application and are not intended to limit the scope of this application.

[0022] Figure 1 This diagram illustrates existing technologies for accessing real-time databases via local shared memory-mapped files and network methods.

[0023] Figure 2 A flowchart illustrating a distributed storage method for a real-time database according to an exemplary embodiment;

[0024] Figure 3 A schematic diagram illustrating a distributed storage of a real-time database according to an exemplary embodiment is shown.

[0025] Figure 4 This is a schematic diagram of the first index described in the embodiment;

[0026] Figure 5 This is a schematic diagram of the second index described in the embodiment;

[0027] Figure 6 This is a schematic diagram of the third index described in the embodiment;

[0028] Figure 7 A flowchart illustrating the access request processing of a global network access interface in an exemplary embodiment is shown.

[0029] Figure 8 A flowchart illustrating an exemplary embodiment of a method for locating real-time database content based on an access request is provided.

[0030] Figure 9 A schematic diagram of a distributed storage system for a real-time database, illustrating an exemplary embodiment;

[0031] Figure 10 This diagram illustrates the structure of an electronic device provided in this application. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. In these cases, well-known structures, methods, devices, implementations, materials, or operations will not be shown or described in detail.

[0034] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content, operations, or steps, nor do they necessarily need to be performed in the described order. For example, some operations or steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0035] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0036] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0037] Figure 2 A flowchart illustrating a distributed storage method for a real-time database according to an exemplary embodiment is shown.

[0038] like Figure 2 As shown, the distributed storage method for real-time databases includes steps S201-S203.

[0039] In step S201, a real-time database is constructed based on the power public information model of the monitored power grid.

[0040] According to the example embodiment, a real-time database is constructed based on the power grid's public information model. The real-time database includes various power grid information model object tables such as regions, intervals, substations, loads, transformers, lines, buses, telemetry, remote signaling, topology islands, reference voltages, and operating parameters.

[0041] In step S202, multiple shards for storing data are set up in multiple physical nodes.

[0042] In step S203, the first content in the real-time database is stored in the corresponding shards of multiple shards in a sharded storage manner; the second content in the real-time database is stored in the multiple shards in a replicated storage manner; and the third content in the real-time database is stored in the first shard of multiple shards in a centralized storage manner.

[0043] According to the example embodiment, the first content includes equipment model tables and measurement model tables for substations in the monitored power grid; objects stored in the tables are grouped by their respective substations or physical regions, with objects from the same substation or physical region stored in the same partition. The second content includes a configuration table for the monitored power grid, which includes a power grid parameter table and a dictionary table; each partition stores a complete copy of the table data. The third content includes a global topology table and a line table for the monitored power grid, which are stored only in the first partition.

[0044] According to some embodiments, the method for determining whether a table is stored in sharded, replicated, or centralized storage is based on the data size, model object characteristics, and business program characteristics.

[0045] According to the power public information model specification, the model tables of the real-time database can be classified into categories such as area tables, equipment tables, measurement tables, parameter tables, dictionary tables, line tables, and topology tables.

[0046] Among them, the capacity of equipment tables and measurement tables increases with the expansion of the power grid, and the processing capacity of a single machine may be insufficient, so there is a need for sharding. Therefore, the tables corresponding to equipment and measurement in the plant are stored in a sharded storage method: equipment and measurement in the same plant are stored in the same shard, and plants in the same geographical area are stored in the same shard.

[0047] The configuration table includes a parameter table and a dictionary table, which are standardized configuration information in the power grid model, such as voltage level definition tables and global operating parameter tables. To ensure the integrity of model references within each segment, the parameter table and dictionary table need to be stored in each segment; therefore, the configuration table is stored using a copied storage method.

[0048] For services requiring global calculations or cross-site processing, such as global topology, line peer-to-peer generation, and multi-source processing, model data from multiple sites are involved. On the one hand, the data volume of such services is not large, so a single machine can handle it. On the other hand, if sharded storage is used to distribute the data across multiple machines, it will be difficult for the business program to aggregate the data for processing, which will reduce the processing efficiency of the business program. Therefore, topology, line tables, etc. are stored in a centralized storage method in a designated dedicated shard.

[0049] According to an example embodiment, the method further includes building an index for the real-time database:

[0050] According to the example embodiment, a corresponding target table identifier is generated based on the storage method of the first content, the second content, and the third content to establish a first index; a target data identifier is generated based on the sequence number of the shard where the first content, the second content, and the third content are stored to establish a second index; and a target storage location identifier is generated based on the position of the first content, the second content, and the third content in the stored shard to establish a third index.

[0051] like Figure 3 As shown, the first and second indexes are stored independently and used to locate the shard where the data resides. The third index is stored within each shard and used to locate the storage location of the data within the data area of ​​that shard.

[0052] like Figure 4 , Figure 5 and Figure 6 As shown, by querying the first, second, and third indexes based on the primary key of the real-time database, the corresponding real-time data can be located. The primary key of the real-time database uses a three-segment numbering system, such as "1:2:3". The first segment on the left is the identifier of the target table where the data resides. Querying the first index based on the target table identifier determines the table's storage method. The middle segment is the identifier of the target data. Querying the second index based on the target data identifier locates the shard where the data resides; data with the same target data identifier are distributed within the same shard. The third segment indicates the target storage location identifier of the data within the shard. Querying the third index based on the target storage location identifier determines the data's position within the shard.

[0053] For example, the offset of the data in the data area within the fragment can be obtained through the third index, and the data storage location can be obtained by adding the offset to the starting address of the data area.

[0054] According to some embodiments, the index is not limited to a three-segment number representation, but can also be any other data ID that can determine unique content, as long as it can be ensured that the storage method of the table in the real-time database, the data shard, and the storage location within the data shard can be located based on the data ID and the first, second, and third indexes.

[0055] According to the example embodiment, this application uses a global network access interface to access the real-time database. Based on the first, second, and third indexes, the access request of the global network access interface is used to locate and access the data.

[0056] According to the example embodiment, such as Figure 7 As shown, based on the index, the content of the real-time database is retrieved according to the access request, including steps S2051-S2053:

[0057] In step S2051, real-time data within the corresponding slice is accessed according to the access request.

[0058] In step S2052, query results are generated based on the accessed real-time data.

[0059] In step S2053, the query result is returned to the sending end of the access request.

[0060] According to the example embodiment, the information carried in the access request includes: target table identifier, target data identifier, and target storage location identifier.

[0061] According to the example embodiment, the types of access requests include query requests and modification requests. A query request refers to a request to read real-time data. Modification requests include: insert requests, update requests, and delete requests. An insert request refers to a request to insert a new record into the real-time data. An update request refers to a request to update existing real-time data. A delete request refers to a request to delete existing real-time data records.

[0062] like Figure 8 As shown, accessing real-time data within the corresponding shard according to the access request includes: determining the storage method of the data to be accessed based on the target table identifier; determining the shard where the data to be accessed is stored based on the target data identifier; and determining the storage location of the shard where the data to be accessed is located based on the target storage location identifier.

[0063] Specifically, the first index is queried based on the target table identifier to determine the storage method of the data to be accessed.

[0064] Query the second index based on the target data identifier to determine the shard where the data to be accessed is stored:

[0065] When the storage method of the data to be accessed is fragmented storage, the second index is queried according to the target data identifier carried in the access request to obtain the fragment sequence number where the data to be accessed is located, and the fragment of the sequence number is located.

[0066] When the storage method of the data to be accessed is replicated storage, determine the type of access request: when the access request is a data query request, randomly locate the shard with any sequence number; when the access request is a data modification request, locate all shards.

[0067] When the storage method for the data to be accessed is centralized storage, the system locates the first shard where the centralized storage content is located.

[0068] The third index is queried based on the target storage location identifier to determine the storage location of the data to be accessed in the data area within the shard.

[0069] According to some embodiments, applications such as human-machine interfaces send access requests through the network interface layer, which is located above the storage layer and the logical access layer. The application passes in the target table identifier, the target data identifier, and the target storage location identifier. The network interface layer locates the data fragment through the index, combines the corresponding fragment data, and returns the processing result.

[0070] The network interface layer shields applications from the differences in real-time database storage methods, so applications do not need to care about the underlying storage method and location of the data, thereby improving the convenience and efficiency of data access.

[0071] For example, when the access request is a single table point-to-point access request, that is, the access request only carries a target table identifier, a target data identifier, and a target storage location identifier, the storage method of the data to be accessed can be determined according to step S2051, the partition where it is located can be determined according to the target data identifier, and the data can be accessed according to the target storage location identifier.

[0072] For example, when an access request includes a data modification request for replicated storage, if the method request type in step S2051 is replicated storage, it is necessary to locate all shards and apply the modification request to the replicated storage content in all shards.

[0073] For example, when the access request is a single-table multi-point access request for allocating storage tables, that is, the access request only carries a target table identifier stored in the sharded storage mode and multiple target data identifiers and target storage location identifiers contained therein, according to the sharded storage mode in step S2051, the shard number of each target data identifier is calculated, the target data identifier IDs are grouped according to the shard number to obtain the set of shards to be queried, and then the list of target storage location identifiers to be queried for each shard is determined. The data corresponding to the target storage location identifiers is queried in batches in each shard; finally, the interface layer aggregates the data queried from each shard into a data table and returns it to the sending end of the access request.

[0074] Furthermore, when the access request is a mixed request involving multiple tables and locations (i.e., the target table identifier carried in the access request covers at least two of the three storage methods: sharding, replication, and centralized storage): For centralized storage tables, the query needs to be performed on the first shard containing the content in centralized storage; for sharded storage tables, the query needs to be performed on the corresponding shard; for replicated storage content: when the mixed request is a query request, to reduce the number of shards accessed, any shard containing the content to be accessed in centralized storage or sharded storage can be randomly selected for access; when the mixed request is a data modification request, the modification request needs to be performed on all shards. The union of the shards located for the content in the above three storage methods is taken, access requests are initiated to these shards, and finally, the response results are merged and returned to the sending end of the access request.

[0075] Furthermore, when returning the query results to the sending end, this application can provide a logical real-time full table view of the database by merging the query results of each shard, so as to facilitate remote access to human-computer interaction services such as screens and curves.

[0076] According to the example embodiment, this application also uses an on-chip shared memory access interface to access the real-time database. A schematic diagram of the on-chip shared memory access interface is shown below. Figure 1 As shown, the acquisition and control SCADA business programs and real-time library are deployed on the same physical node in segments. Based on the complete plant model within the segment, the real-time data within the segment is accessed through local memory-mapped files. The third index is queried according to the target storage location identifier to read the data corresponding to the target storage location identifier within the segment. The real-time data of the storage location of the data area is read through the local memory-mapped file, without the need to access other segment data through network interfaces.

[0077] For example, such as Figure 3 As shown in the shard 2-N diagram, on-chip application processes access the shard real-time database using local memory-mapped files. By querying the third index within the shard, the target storage location can be located to identify the corresponding model data. The on-chip shard storage table contains the equipment and measurement models of the on-chip plant, while the replicated storage table contains the basic parameters associated with the models. The plant model within each shard is partially complete. On-chip application processes only need to perform data acquisition and control processing based on their local shards, without needing to access data from other shards via network interfaces, thus improving the processing efficiency of such business applications. Furthermore, this approach allows application processes in each shard to perform business processing in parallel, improving the parallel processing capabilities of the business applications.

[0078] The first shard containing data stored using a centralized storage method can also access real-time data within the corresponding shard using local memory-mapped files. The third index is queried based on the target storage location identifier, and the corresponding record's data is determined by reading the target storage location identifier within the shard. For cases involving a small amount of cross-shard data, it is also necessary to access other shard data using a network interface.

[0079] For example, such as Figure 3 The image shows shard number 1, which is the shard containing content stored in a centralized storage manner. Application processes within this shard access the data in the centralized storage using local memory-mapped files. By querying the third index within the shard, the target storage location can be identified, indicating the corresponding data. The centralized storage content includes topology islands and line tables, while the replicated storage content contains basic parameters related to the model and dictionary definition tables. Application processes can efficiently access data within the centralized storage shard locally. However, when data needs to be accessed from different shards, a global network access interface is required to access the data across shards. For example, when dealing with data from other shards, such as shards associated with the ends of a line, only the network interface needs to be called to retrieve the data; the business application does not need to concern itself with the shard where the data resides.

[0080] This application provides a distributed storage method for real-time databases, offering three table storage methods: centralized, replicated, and sharded, as well as three indexes for fast data retrieval. Based on these three storage and indexing methods, it implements two access interfaces: a global network access interface and an on-chip shared memory access interface. Balancing the horizontal scalability and processing efficiency of real-time databases, different distributed storage methods are adopted for different tables according to the characteristics of power grid model data and business programs. While implementing data sharding, the local integrity of the power grid model within each shard is maintained. Business programs based on the complete substation model within each shard can still perform efficient parallel processing using local shared memory mapped files. The network interface layer masks the differences between various storage methods through routing rules, providing a logical full table view of the real-time database, facilitating remote access for human-machine interfaces such as video feeds and curves.

[0081] Figure 9 A schematic diagram of a distributed storage system for a real-time database, illustrating an exemplary embodiment, is shown.

[0082] like Figure 9 As shown, the distributed storage system of the real-time database includes a database creation module 801, a shard creation module 802, and a content sorting module 803.

[0083] The database creation module 801 is used to construct a real-time database based on the power public information model of the monitored power grid. The shard creation module 802 is used to set up multiple shards for storing data in multiple physical nodes. The content sorting module 803 is used to store the first content in the real-time database into the corresponding shards of the multiple shards in a sharded storage manner; to store the second content in the real-time database into the multiple shards in a replicated storage manner; and to store the third content in the real-time database into the first shard of the multiple shards in a centralized storage manner.

[0084] According to some embodiments, the distributed storage system of the real-time database also includes an index creation module 804.

[0085] The index creation module 804 is used to create indexes for the real-time database. It generates corresponding target table identifiers based on the storage methods of the first, second, and third contents to create a first index; it also generates target data identifiers based on the sequence numbers of the shards where the first, second, and third contents are stored to create a second index; and it generates target storage location identifiers based on the positions of the first, second, and third contents within their respective shards to create a third index.

[0086] According to some embodiments, the distributed storage system of the real-time database also includes a data query module 805.

[0087] The data query module 805 accesses the real-time database using either a global network access interface or an on-chip shared memory access interface. The data query module 805 retrieves the content of the real-time database based on the index and the access request, including:

[0088] The data query module 805 is used to access real-time data within the corresponding segment according to the access request; it is also used to generate query results based on the accessed real-time data; and return the query results to the sending end of the access request.

[0089] The data query module 805 is used to query the first index based on the target table identifier to determine the storage method of the data to be accessed.

[0090] The data query module 805 is used to query the second index based on the target data identifier to determine the shard where the data to be accessed is stored, including: when the storage method of the data to be accessed is sharded storage, querying the second index based on the target data identifier carried in the access request to obtain the shard sequence number of the data to be accessed and locating the shard with the sequence number; when the storage method of the data to be accessed is replicated storage, determining the type of access request; when the type of access request is a query request, locating any one of the multiple shards; when the type of access request is a modification request, locating multiple shards; when the storage method of the data to be accessed is centralized storage, locating the shard with sequence number 1.

[0091] The data query module 805 is also used to query the third index based on the target storage location identifier to determine the storage location of the data to be accessed in the data area within the shard.

[0092] The data query module 805 is also used to access the real-time database using the on-chip shared memory access interface, including: accessing on-chip data based on the complete on-chip plant model, querying the third index according to the target storage location identifier, determining the storage location of the data to be accessed in the data area within the segment, and directly reading the real-time data of the storage location of the data area through a local memory-mapped file without transmitting the accessed data over the network.

[0093] The distributed storage system for real-time databases is used to execute the distributed storage method for real-time databases as described above, so the distributed storage method for real-time databases will not be elaborated here.

[0094] Figure 10 This diagram illustrates the structure of an electronic device provided in this application.

[0095] See Figure 10 , Figure 10An electronic device is provided, including a processor and a memory. The memory stores computer instructions, which, when executed by the processor, cause the processor to perform the computer instructions to achieve the following: Figure 2 , Figure 7 and Figure 8 The method and its detailed scheme are shown.

[0096] It should be understood that the above-described device embodiments are merely illustrative, and the device disclosed in this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0097] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0098] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor or chip can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, on-chip cache, off-chip memory, and storage can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0099] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments disclosed herein. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0100] This application also provides a non-transitory computer storage medium storing a computer program, which, when executed by multiple processors, causes the processors to perform actions such as... Figure 2 , Figure 7 and Figure 8 The method and its detailed scheme are shown.

[0101] It should be clearly understood that this application describes how specific examples are formed and used, but this application is not limited to any details of these examples. Rather, based on the teachings of the disclosure of this application, these principles can be applied to many other embodiments.

[0102] Furthermore, it should be noted that the above figures are merely illustrative representations of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0103] Exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A distributed storage method for a real-time database, characterized in that, include: A real-time database is constructed based on the power public information model of the monitored power grid; Configure multiple shards across multiple physical nodes for storing data; The first content in the real-time database is stored in the corresponding shard among the multiple shards in a sharded storage manner; The second content in the real-time database is stored in the multiple shards by means of replication; The third content in the real-time database is stored in the first shard among the multiple shards in a centralized storage manner.

2. The distributed storage method as described in claim 1, characterized in that, The first content includes the equipment model table and measurement model table of the power plants and substations in the monitored power grid; The second content includes a configuration table of the monitored power grid, which includes a power grid parameter table and a dictionary table; The third content includes the global topology table and line table of the monitored power grid.

3. The distributed storage method as described in claim 1, characterized in that, Also includes: Establishing an index for the real-time database includes: Based on the storage methods of the first content, the second content, and the third content, corresponding target table identifiers are generated to establish a first index; Based on the sequence numbers of the fragments stored in the first content, the second content, and the third content, target data identifiers are generated respectively to establish a second index; Based on the positions of the first content, the second content, and the third content in the stored fragments, target storage location identifiers are generated respectively to establish a third index.

4. The distributed storage method as described in claim 3, characterized in that, Also includes: Accessing the real-time database via a global network access interface, wherein: based on the index and the access request, the content of the real-time database is retrieved, including: According to the access request, access the real-time data in the corresponding shard. The access request includes: target table identifier, target data identifier, and target storage location identifier. Generate query results based on the accessed real-time data; The query results are returned to the sender of the access request.

5. The distributed storage method as described in claim 4, characterized in that, The step of accessing real-time data within the corresponding slice according to the access request includes: Query the first index based on the target table identifier to determine the storage method of the data to be accessed; Querying the second index based on the target data identifier to determine the shard where the data to be accessed is stored includes: When the storage method of the data to be accessed is fragmented storage, the second index is queried according to the target data identifier carried by the access request to obtain the fragment number where the data to be accessed is located, and the fragment of the fragment number is located. If the storage method for the data to be accessed is copy storage, determine the type of the access request; If the access request is a query request, locate any one of the multiple shards; If the access request is a modification request, the location is determined to be one of the multiple shards; If the storage method for the data to be accessed is centralized storage, then locate the first fragment; The third index is queried based on the target storage location identifier to determine the storage location of the data to be accessed in the data area within the fragment.

6. The distributed storage method as described in claim 3, characterized in that, Also includes: Accessing the real-time database using the on-chip shared memory access interface includes: In-chip data access is performed based on the complete in-chip plant model. The third index is queried according to the target storage location identifier to determine the storage location of the data to be accessed in the data area within the chip. Real-time data of the storage location of the data area is read through a local memory-mapped file.

7. A distributed storage system for a real-time database, used to execute the distributed storage method as described in any one of claims 1-6, characterized in that, include: The database creation module is used to build a real-time database based on the power public information model of the monitored power grid; The sharding creation module is used to set up multiple shards for storing data across multiple physical nodes; The content sorting module is used to store the first content in the real-time database into the corresponding fragments in the plurality of fragments in a fragmented storage manner; The second content in the real-time database is stored in the multiple shards by means of replication; The third content in the real-time database is stored in the first shard among the multiple shards in a centralized storage manner.

8. The distributed storage system as described in claim 7, characterized in that, Also includes: An index creation module is used to create an index for the real-time database, including: generating corresponding target table identifiers based on the storage methods of the first content, the second content, and the third content to create a first index; generating target data identifiers based on the sequence numbers of the shards where the first content, the second content, and the third content are stored to create a second index; and generating target storage location identifiers based on the positions of the first content, the second content, and the third content in the stored shards to create a third index. The data query module accesses the real-time database using either a global network access interface or an on-chip shared memory access interface, including: Used to retrieve the content of the real-time database based on an index and an access request, including: Based on the access request, access the real-time data within the corresponding slice. The access request includes: The system includes a target table identifier, a target data identifier, and a target storage location identifier; it is also used to generate query results based on the accessed real-time data; and to return the query results to the sending end of the access request.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Fragment information storage method of distributed real-time memory database

    CN107609143A

  • Data processing method and device, electronic equipment and storage medium

    CN117555906A

Cited By

  • Data processing method and device for distributed database, equipment, storage medium and program product

    CN121542356A