Real-time database data access method and system

Through metadata, the synchronization and integrity of database nodes are judged, and the load balancing algorithm and concurrent access to historical data segments are used to solve the problem of low efficiency in real-time database data access and achieve efficient and complete data access.

CN117216351BActive Publication Date: 2025-08-19XIAN TPRI POWER PLANT INFORMATION TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311190224.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-14
Publication Date
2025-08-19
Estimated Expiration
2043-09-14

AI Technical Summary

Technical Problem

Existing real-time databases are inefficient in data access, especially when querying large sections of historical data, and fail when data cross-disappearance of database nodes, and synchronous operation takes time.

Method used

Through metadata, the data synchronization and integrity of database nodes are judged, the load balancing algorithm is used to select database nodes, and the historical data segments are accessed concurrently, and the complete data is aggregated to optimize the data access process.

Benefits of technology

Improve data access efficiency, ensure data integrity, avoid waste of time waiting for synchronization to be completed, adapt to multiple query methods, and improve system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117216351B_ABST
    Figure CN117216351B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of electrical signal data processing and discloses a real-time database data access method and system, comprising: obtaining a time interval, measurement point name, and query method of historical data to be accessed, sent by a client; querying, through metadata, whether the data of two database nodes in the time interval of the historical data to be accessed are synchronized; if synchronization is achieved, selecting a database node using a load balancing algorithm to access the historical data; if synchronization is not achieved, confirming, through metadata, the database node with complete data in the interval and accessing the historical data from the database node with complete data; if two database nodes have overlapping data missing in the time interval of the historical data to be accessed, concurrently accessing the historical data from the two database nodes to aggregate and form complete historical data; and pushing the accessed historical data to the client. The present invention improves data access efficiency and ensures data integrity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of electric signal data processing, and in particular to a real-time database data access method and system. Background Art

[0002] Today's industrial production environment, especially in the Internet of Things fields such as electricity, petrochemicals, and transportation, has increasingly higher requirements for the integrity and reliability of time series data. The high availability of real-time databases provides a great degree of reliability guarantee. Compared with stand-alone real-time databases, they can ensure service availability and data integrity in the event of a single database node service failure.

[0003] In a high-availability model, a real-time database is typically divided into primary and standby database nodes. If a primary database node experiences an unexpected downtime, the standby node is switched to the standby node or directly promoted to primary to continue providing database services. If either or both database nodes experience anomalies over a period of time, data on the two nodes may become inconsistent. This data is then compared and synchronized when the two nodes are operating normally. When accessing data, if synchronization has completed, the traditional strategy is to query data directly on any connected database node or to select a specific node for querying based on a policy. During synchronization, the query is first performed on the primary database node. If the query succeeds, a success signal and the query data are returned. If the query fails, a failure signal is returned and the query is switched to the standby node. If both database nodes fail, the client or API determines the handling strategy. The traditional strategy is to wait for synchronization between the two database nodes before querying again. This approach ensures the reliability and integrity of the query data, but it is not as efficient. If data is missing from both database nodes during the data access period, the query will fail. With many synchronization operations, the waiting time is long, making it difficult to query data quickly. In scenarios where large amounts of historical data need to be queried, such as when a reporting system performs statistical calculations, this will affect a series of subsequent operations. Summary of the Invention

[0004] In response to the problem of low efficiency of real-time database data access in the above-mentioned prior art, the purpose of the present invention is to propose a real-time database data access method and system, which improves data access efficiency and ensures data integrity by modifying the data access method and optimizing the data access process.

[0005] In order to achieve the above object, the present invention has the following technical solutions:

[0006] A first aspect of the present invention is to provide a real-time database data access method, comprising the following steps:

[0007] Obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client;

[0008] Use metadata to query whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, a load balancing algorithm is used to select a database node to access the historical data;

[0009] Under the condition of incomplete synchronization, metadata is used to confirm the database nodes with complete data within the interval, and historical data is accessed from the database nodes with complete data;

[0010] If two database nodes have overlapping data missing in the time interval of the historical data to be accessed, the historical data is accessed from the two database nodes concurrently and aggregated to form complete historical data;

[0011] Push historical data after access to the client.

[0012] As a further improvement of the present invention, the query mode includes sample values, interpolation values or statistical values, the sample values include all sample values and some sample values that meet the query conditions, the interpolation values include equal data interval interpolation or equal time interval interpolation, and the statistical values include maximum value, minimum value, arithmetic mean value, weighted mean value or cumulative value;

[0013] Under the synchronization condition, a load balancing algorithm is used to select a database node to access historical data. The load balancing algorithm includes a polling method, a minimum connection method, a system load method or a comprehensive dynamic method.

[0014] As a further improvement of the present invention, the metadata records the latest historical data timestamp before the abnormal interruption of historical data of each measuring point, and multiple latest historical data timestamps before the abnormal interruption of historical data of each measuring point are saved.

[0015] As a further improvement of the present invention, the timestamp of the latest historical data before the abnormal interruption of the historical data of each measuring point is saved multiple times, including:

[0016] During data synchronization, data is synchronized starting from the timestamp of the latest historical data before the abnormal interruption of the earliest historical data. When the data of the two database nodes are consistent, the synchronization of the historical data segment starting with the timestamp is completed, the timestamp is deleted, and the next historical data segment is synchronized.

[0017] As a further improvement of the present invention, the condition for whether the data is synchronized is that the data of the two database nodes in the time interval of the historical data to be accessed are consistent, including:

[0018] When the timestamps of the latest historical data before the abnormal interruption of the historical data of the accessed measurement points in the metadata of the two database nodes are both greater than the end time of the data access time interval, the data in the historical data time interval to be accessed is consistent; when the timestamp of a database node falls within the data access time interval, the data in the historical data time interval to be accessed is inconsistent, and the data of the database nodes whose timestamps do not fall within the data access time interval are complete.

[0019] As a further improvement of the present invention, the condition for cross-data missing is that the data of the two database nodes in the time interval of the historical data to be accessed are incomplete, and the metadata of the two database nodes contain one or more latest historical data timestamps before the abnormal interruption of the historical data of the accessed measurement points.

[0020] As a further improvement of the present invention, the specific steps of the method of concurrently accessing historical data from two database nodes are as follows:

[0021] S41: Compare the metadata of the two database nodes, and within the accessed time interval, the timestamps of the latest historical data before the abnormal interruption of the historical data of the earliest two accessed measurement points occur. On the database node with the larger timestamp, select the historical data from the start time of the accessed time interval to the end time stamp as the first historical data segment;

[0022] S42: in another database node, selecting historical data from the end timestamp of the first database node to the end timestamp of the latest historical data before an abnormal interruption occurs in the historical data of the next accessed measurement point of this database node as the second historical data segment;

[0023] S43: alternately selecting until a database node selects the last historical data segment whose end time of the accessed time interval is the last segment;

[0024] S44: Access historical data segments concurrently at two database nodes, access only the historical data segments selected by the current database node, and finally aggregate the historical data segments into complete historical data.

[0025] A second aspect of the present invention provides a real-time database data access system, comprising:

[0026] The acquisition module is used to obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client;

[0027] The synchronous access module is used to query the metadata whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, the load balancing algorithm is used to select a database node to access the historical data;

[0028] The unsynchronized access module is used to confirm the database nodes with complete data within the interval through metadata under the condition of uncompleted synchronization, and access historical data from the database nodes with complete data;

[0029] Cross-access module, used to access historical data from two database nodes concurrently if data in the time interval of the historical data to be accessed is missing from two database nodes, and aggregate to form complete historical data;

[0030] Push module pushes historical data after access to the client.

[0031] The third aspect of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the real-time database data access method when executing the computer program.

[0032] A fourth aspect of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the real-time database data access method is implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention adds metadata to the structure of a high-availability real-time database, and judges the data synchronization and data integrity of the two database nodes by the latest historical data timestamp before the abnormal interruption of the historical data of each measuring point stored in the metadata. At the same time, it does not clearly distinguish between the master and slave database nodes. When the data of the two database nodes are complete, the system load can be balanced by the load balancing algorithm to improve the system performance; when the data of the two database nodes are incomplete, the complete historical data can be aggregated by concurrently accessing the historical data segments, making full use of the system concurrency capability, without waiting for the data synchronization of the two database nodes to be completed, thereby improving data access efficiency and ensuring data integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 A flow chart of a real-time database data access method provided by the present invention;

[0037] Figure 2 is a flow chart of an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of a data interleaving missing conditional access method according to an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of a second data interleaving missing conditional access method according to an embodiment of the present invention.

[0040] Figure 5 A real-time database data access system provided by the present invention;

[0041] Figure 6 This is a schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0042] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer and more specific, the present invention is further described in detail below in conjunction with specific embodiments and accompanying drawings. The present invention can also be implemented or applied through different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0044] The first object of the present invention is to provide a real-time database data access method, comprising the following steps:

[0045] Obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client;

[0046] Use metadata to query whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, a load balancing algorithm is used to select a database node to access the historical data;

[0047] Under the condition of incomplete synchronization, metadata is used to confirm the database nodes with complete data within the interval, and historical data is accessed from the database nodes with complete data;

[0048] If two database nodes have overlapping data missing in the time interval of the historical data to be accessed, the historical data is accessed from the two database nodes concurrently and aggregated to form complete historical data;

[0049] Push historical data after access to the client.

[0050] This method assumes that no data is lost due to simultaneous anomalies in the two database nodes or a server power outage. If data is still missing after synchronization, it is not covered by this method. This invention addresses the problem of low data access efficiency in real-time databases and proposes a real-time database data access method that modifies the data access method, optimizes the data access process, improves data access efficiency, and ensures data integrity.

[0051] See also Figure 1 , an embodiment of the present invention provides a real-time database data access method, comprising the following steps:

[0052] Step S11: The client sends the time interval of the historical data to be accessed, the name of the measurement point and the query method;

[0053] Specifically, the query methods include sample values, interpolation, and statistical values. Sample values include all sample values and some sample values that meet the query conditions. Interpolation includes equal data interval interpolation and equal time interval interpolation. Statistical values include maximum value, minimum value, arithmetic mean, weighted mean, and cumulative value.

[0054] The query types of the present invention include sample values, interpolation values, and statistical values, which meet various data access requirements and have strong adaptability.

[0055] Exemplarily, the query methods are all applicable to this method, wherein some sample values that meet the query conditions can be limited by expressions such as quality, numerical values, etc., such as `quality==1`, `value>=100.0`, etc.; equal data interval interpolation is performed by specifying the number of query data to obtain interpolation; equal time interval interpolation is performed by specifying the time interval to obtain interpolation.

[0056] Step S12: The real-time database data access module queries the metadata to see whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. If synchronization is required, a database node is selected to access the data. A load balancing algorithm is used to select a database node to access the historical data.

[0057] Specifically, under synchronous conditions, the load balancing algorithm used by the data access module can be configured as polling mode, minimum connection method, system load mode, and comprehensive dynamic mode; if the comprehensive dynamic mode is selected, the load balancing algorithm will be dynamically selected based on performance-related parameters such as system concurrency, load, and number of database connections.

[0058] For example, assuming that there are multiple concurrent data accesses, the server system loads of the two database nodes are not much different during the access, but the number of connections is quite different. At this time, the load balancing algorithm with the minimum number of connections is selected, and data access is performed on the database node with a small number of connections, and the corresponding database node increases the corresponding number of access connections; if there are enough access operations and the number of connections is equal to the load, it can also be dynamically switched to polling mode to reduce complexity and make full use of system resources.

[0059] Applying a load balancing algorithm can effectively avoid concurrent data access on the same database node, which can lead to performance degradation and reduced efficiency.

[0060] Step S13: If synchronization is not completed, use metadata to confirm the database node with complete data within the interval, and access historical data from the database node with complete data;

[0061] Specifically, the metadata in step S12 and step S13 records the latest timestamp of historical data before abnormal interruption of historical data of each measuring point. Multiple latest timestamps of historical data before abnormal interruption of historical data of each measuring point can be saved.

[0062] During data synchronization, data is synchronized starting from the timestamp of the latest historical data before the abnormal interruption of the earliest historical data. When the data of the two database nodes are consistent, the synchronization of the historical data segment starting with the timestamp is completed, the timestamp is deleted, and the next historical data segment is synchronized.

[0063] For example, if database node 1 experiences two abnormal interruptions, two timestamps will be saved. Database node 2 will synchronize data with database node 1. After synchronizing the first segment of historical data, the timestamp of the first interruption will be deleted, indicating that this segment of data has been synchronized and can be accessed.

[0064] See also Figure 2 , the data synchronization condition in the data access is that the data of the two database nodes in the time interval of the historical data to be accessed are consistent. When the timestamps of the latest historical data before the abnormal interruption of the historical data of the accessed measurement points in the metadata of the two database nodes are both greater than the end time of the data access time interval, the data in the time interval of the historical data to be accessed are consistent; when the timestamp of a database node falls within the data access time interval, the data in the time interval of the historical data to be accessed are inconsistent, and the data of the database nodes whose timestamps do not fall within the data access time interval are complete;

[0065] For example, the time interval of the historical data to be accessed is from "2023-06-01 12:00:00" to "2023-06-02 12:00:00". The historical data of the accessed measurement point in the metadata of database node 1 happens to be interrupted at "2023-06-01 18:00:00". The timestamp of the latest historical data before the abnormal interruption is "2023-06-01 18:00:00", which is less than the end time "2023-06-02 12:00:00". Therefore, the data is not synchronized, and the data in database node 1 is incomplete. The process goes to step S13 to access the data from the database node with complete data.

[0066] Step S14: If two database nodes have overlapping missing data in the time interval of the historical data to be accessed, access the historical data from the two database nodes concurrently and aggregate them to form complete historical data;

[0067] Specifically, the data cross-missing condition is that the data of the two database nodes in the time interval of the historical data to be accessed are incomplete, and the metadata of the two database nodes contain the latest historical data timestamp before the abnormal interruption of the historical data of one or more accessed measurement points;

[0068] In the above example, if the process proceeds to step S13 and the interruption timestamp of database node 2 is also less than the end time "2023-06-02 12:00:00", the data of database node 2 is also incomplete and there is data cross-missing. At this time, the process proceeds to step S14, and the metadata is first compared to confirm the access time period of the two database nodes to the historical data.

[0069] Specifically, the steps for concurrently accessing historical data from two database nodes are as follows:

[0070] Step S141: Compare the metadata of the two database nodes. Within the accessed time interval, the timestamps of the latest historical data before the abnormal interruption of the historical data of the two earliest accessed measurement points are compared. On the database node with the larger timestamp, select the historical data from the start time of the accessed time interval to the end time stamp as the first historical data segment.

[0071] Step S142: at another database node, selecting historical data from the end timestamp of the first database node to the end timestamp of the latest historical data before an abnormal interruption occurs in the historical data of the next accessed measurement point of this database node as the second historical data segment;

[0072] Step S143: alternately select until a database node selects the last historical data segment whose end time of the accessed time interval is the last segment;

[0073] Specifically, two access time intervals are given as examples: In the first case, see Figure 3 , two database nodes have only one interrupt timestamp falling within the interval, and they concurrently access a historical data segment respectively; in the second case, see Figure 4 , two interruption timestamps of database node 1 fall within the interval, and one interruption timestamp of database node 2 falls within the interval. During concurrent access, one historical data segment of database node 1 and two historical data segments of database node 2 are accessed. And so on. In all cases, data access integrity is guaranteed as long as high-availability data is not lost.

[0074] Step S144: the data access module concurrently accesses historical data segments on the two database nodes, accesses only the historical data segments selected by the current database node, and finally aggregates the historical data segments into complete historical data;

[0075] Step S15: The data access module pushes the accessed historical data to the client.

[0076] This invention has been applied to a independently developed high-availability real-time database. After deploying and enabling high-availability database nodes, data can be accessed through the above-described embodiments. Currently, it is primarily used in plant-level information monitoring systems for large thermal power plants. High-availability deployment improves the integrity and reliability of massive industrial time-series data storage. Application of this invention can effectively improve data retrieval performance in high-availability real-time databases and ensure data access stability in data processing applications.

[0077] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0078] like Figure 5 As shown, the present invention also provides a real-time database data access system, comprising:

[0079] The acquisition module is used to obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client;

[0080] The synchronous access module is used to query the metadata whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, the load balancing algorithm is used to select a database node to access the historical data;

[0081] The unsynchronized access module is used to confirm the database nodes with complete data within the interval through metadata under the condition of uncompleted synchronization, and access historical data from the database nodes with complete data;

[0082] Cross-access module, used to access historical data from two database nodes concurrently if data in the time interval of the historical data to be accessed is missing from two database nodes, and aggregate to form complete historical data;

[0083] Push module pushes historical data after access to the client.

[0084] Exemplarily, in the synchronous access module, the metadata records the latest historical data timestamp before the abnormal interruption of the historical data of each measuring point, and multiple latest historical data timestamps before the abnormal interruption of the historical data of each measuring point are saved, including:

[0085] During data synchronization, data is synchronized starting from the timestamp of the latest historical data before the abnormal interruption of the earliest historical data. When the data of the two database nodes are consistent, the synchronization of the historical data segment starting with the timestamp is completed, the timestamp is deleted, and the next historical data segment is synchronized.

[0086] Exemplarily, in the synchronous access module, the condition for whether data is synchronized is that the data of the two database nodes in the time interval of the historical data to be accessed is consistent, including:

[0087] When the timestamps of the latest historical data before the abnormal interruption of the historical data of the accessed measurement points in the metadata of the two database nodes are both greater than the end time of the data access time interval, the data in the historical data time interval to be accessed is consistent; when the timestamp of a database node falls within the data access time interval, the data in the historical data time interval to be accessed is inconsistent, and the data of the database nodes whose timestamps do not fall within the data access time interval are complete.

[0088] Exemplarily, in the cross-access module, the condition for data cross-missing is that the data of the two database nodes in the time interval of the historical data to be accessed are incomplete, and the metadata of the two database nodes contain one or more latest historical data timestamps before the abnormal interruption of the historical data of the accessed measurement points.

[0089] As an optional embodiment, in the cross access module, the specific steps of concurrently accessing historical data from two database nodes are as follows:

[0090] S41: Compare the metadata of the two database nodes, and within the accessed time interval, the timestamps of the latest historical data before the abnormal interruption of the historical data of the earliest two accessed measurement points occur. On the database node with the larger timestamp, select the historical data from the start time of the accessed time interval to the end time stamp as the first historical data segment;

[0091] S42: in another database node, selecting historical data from the end timestamp of the first database node to the end timestamp of the latest historical data before an abnormal interruption occurs in the historical data of the next accessed measurement point of this database node as the second historical data segment;

[0092] S43: alternately selecting until a database node selects the last historical data segment whose end time of the accessed time interval is the last segment;

[0093] S44: Access historical data segments concurrently at two database nodes, access only the historical data segments selected by the current database node, and finally aggregate the historical data segments into complete historical data.

[0094] like Figure 6 As shown, the third object of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the real-time database data access method when executing the computer program.

[0095] A fourth object of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the real-time database data access method is implemented.

[0096] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A real-time database data access method, comprising the following steps: Obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client; Use metadata to query whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, a load balancing algorithm is used to select a database node to access the historical data; Under the condition of incomplete synchronization, metadata is used to confirm the database nodes with complete data within the interval, and historical data is accessed from the database nodes with complete data; If two database nodes have overlapping data missing in the time interval of the historical data to be accessed, the historical data is accessed from the two database nodes concurrently and aggregated to form complete historical data; Push historical data after access to the client; The specific steps of the method of concurrently accessing historical data from two database nodes are as follows: S41: Compare the metadata of the two database nodes, obtain the latest historical data timestamps before the abnormal interruption of the historical data of the two earliest accessed measurement points within the accessed time interval, and select the historical data from the start time of the accessed time interval to the end time stamp as the first historical data segment on the database node with the larger timestamp; S42: in another database node, selecting historical data from the end timestamp of the first database node to the end timestamp of the latest historical data before an abnormal interruption occurs in the historical data of the next accessed measurement point of this database node as the second historical data segment; S43: Alternate selection until any database node selects the last historical data segment whose end time of the accessed time interval is the last segment; S44: Access historical data segments concurrently at two database nodes, access only the historical data segments selected by the current database node, and finally aggregate the historical data segments into complete historical data.

2. A real-time database data access method according to claim 1, characterized in that: The query mode includes sample value, interpolation or statistical value. Sample value includes all sample values and some sample values that meet the query conditions. Interpolation includes equal data interval interpolation or equal time interval interpolation. Statistical value includes maximum value, minimum value, arithmetic mean value, weighted mean value or cumulative value. Under the synchronization condition, a load balancing algorithm is used to select a database node to access historical data. The load balancing algorithm includes a polling method, a minimum connection method, a system load method or a comprehensive dynamic method.

3. A real-time database data access method according to claim 1, characterized in that: The metadata records the latest historical data timestamp before the abnormal interruption of historical data of each measuring point, and multiple latest historical data timestamps before the abnormal interruption of historical data of each measuring point are saved.

4. A real-time database data access method according to claim 3, characterized in that: The latest historical data timestamps before the abnormal interruption of the historical data of each measuring point are saved multiple times, including: During data synchronization, data is synchronized starting from the timestamp of the latest historical data before the abnormal interruption of the earliest historical data. When the data of the two database nodes are consistent, the synchronization of the historical data segment starting with the timestamp is completed, the timestamp is deleted, and the next historical data segment is synchronized.

5. A real-time database data access method according to claim 1, characterized in that: The condition for whether the data is synchronized is that the data of the two database nodes in the time interval of the historical data to be accessed are consistent, including: When the timestamps of the latest historical data before the abnormal interruption of the historical data of the accessed measurement points in the metadata of the two database nodes are both greater than the end time of the data access time interval, the data in the time interval of the historical data to be accessed are consistent; When a database node timestamp falls within the data access time interval, the data in the historical data time interval to be accessed is inconsistent, and the database node data whose timestamp does not fall within the data access time interval is complete.

6. A real-time database data access method according to claim 1, characterized in that: The conditions for data cross-missing are that the data of the two database nodes in the time interval of the historical data to be accessed are incomplete, or the metadata of the two database nodes contain one or more latest historical data timestamps before the abnormal interruption of the historical data of the accessed measurement points.

7. A real-time database data access system, characterized in that: include: The acquisition module is used to obtain the time interval, measurement point name and query method of the historical data to be accessed sent by the client; The synchronous access module is used to query the metadata whether the data of the two database nodes in the time interval of the historical data to be accessed are synchronized. Under the synchronization condition, the load balancing algorithm is used to select a database node to access the historical data; The unsynchronized access module is used to confirm the database nodes with complete data within the interval through metadata under the condition of uncompleted synchronization, and access historical data from the database nodes with complete data; Cross-access module, used to access historical data from two database nodes concurrently if data in the time interval of the historical data to be accessed is missing from two database nodes, and aggregate to form complete historical data; Push module, pushes historical data after access to the client; The specific steps of the method of concurrently accessing historical data from two database nodes are as follows: S41: Compare the metadata of the two database nodes, obtain the latest historical data timestamps before the abnormal interruption of the historical data of the two earliest accessed measurement points within the accessed time interval, and select the historical data from the start time of the accessed time interval to the end time stamp as the first historical data segment on the database node with the larger timestamp; S42: in another database node, selecting historical data from the end timestamp of the first database node to the end timestamp of the latest historical data before an abnormal interruption occurs in the historical data of the next accessed measurement point of this database node as the second historical data segment; S43: Alternate selection until any database node selects the last historical data segment whose end time of the accessed time interval is the last segment; S44: Access historical data segments concurrently at two database nodes, access only the historical data segments selected by the current database node, and finally aggregate the historical data segments into complete historical data.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the real-time database data access method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the real-time database data access method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Metadata searching method, device and equipment and computer readable storage medium

    CN113010476A

  • Method and system for synchronizing historical data of database, electronic equipment and storage medium

    CN114416718A