Real-time database data acquisition dual-computer redundancy method and system

Abstract

The invention discloses a real-time database data acquisition dual-computer redundancy method and system, and the method achieves the automatic cooperation and high-efficiency switching between a main node and a standby node through the construction of a bidirectional communication structure among the main acquisition node, the standby acquisition node and an instruction service. When the main node has a fault, the standby node can seamlessly take over the acquisition task according to the instruction, and the problem of data interruption caused by a single-point fault is effectively avoided. And meanwhile, a timing data synchronization and difference comparison mechanism is adopted, so that the consistency of configuration and database files between the host and the backup is ensured, and abnormal collection caused by data mismatching during switching is prevented. Besides, the switching instruction is subjected to OFF, PRE and ON three-stage state control, and is matched with a delay stop mechanism and instruction queue sequential processing, so that the stability and continuity of the switching process are further ensured, and system chaos caused by sudden disconnection or instruction conflict is avoided. In conclusion, according to the scheme, the fault recovery efficiency and the operation stability of the system are improved, and the redundancy configuration and maintenance process is remarkably simplified.

Description

Technical Field

[0001] This application relates to the technical field of dual-machine redundancy for real-time database data acquisition, and in particular to a method and system for dual-machine redundancy for real-time database data acquisition. Background Technology

[0002] As key industries such as industrial automation, energy management, and traffic dispatching increasingly demand real-time data and system stability, real-time database data acquisition systems are gradually becoming core infrastructure in industrial informatization. Especially in scenarios like power monitoring, intelligent manufacturing, and environmental sensing, acquisition systems need to operate continuously and acquire critical field data; any data interruption could lead to serious consequences. Therefore, building highly reliable and available real-time data acquisition systems has become an important direction for technological development in the industry.

[0003] Currently, most traditional real-time database data acquisition systems adopt a single-machine deployment architecture, relying on a single device to complete data acquisition, processing, and storage tasks. To address system stability issues, some solutions improve reliability through hardware redundancy (such as dual power supplies and RAID disk arrays), software retry mechanisms for abnormalities, or regular data backups. However, these methods still cannot effectively eliminate the risk of single points of failure. Once the acquisition host experiences hardware failure or software crashes, acquisition will be interrupted and cannot be recovered in real time. In addition, some solutions that achieve redundancy through cluster deployment have problems such as complex configuration, high maintenance costs, and difficulties in synchronizing the primary and backup states, resulting in long primary / backup switchover times, poor reliability, and weak data consistency. Especially in actual operation, due to the lag in backup machine configuration and weak synchronization mechanisms, acquisition logic errors or data loss are easily caused, seriously affecting the stable operation of the system.

[0004] Therefore, in real-time database data acquisition systems, problems such as intolerable single point of failure, high latency of master-slave switching, complex synchronization configuration, and poor data consistency have become technical bottlenecks that urgently need to be addressed in the current system reliability design. Summary of the Invention

[0005] This application provides a method, system, computer equipment, and storage medium for real-time database data acquisition with dual-machine redundancy, aiming to solve problems such as intolerable single point of failure, high master-slave switching latency, complex synchronization configuration, and poor data consistency in real-time database data acquisition systems.

[0006] A first aspect is a method for real-time database data acquisition with dual-machine redundancy, the method comprising:

[0007] Deploy a primary data acquisition node and a backup data acquisition node. The primary data acquisition node performs data acquisition tasks under normal conditions, while the backup data acquisition node is in standby mode.

[0008] Configure the command service to establish bidirectional GRPC communication connections with both the primary and backup data acquisition nodes.

[0009] The main data acquisition node periodically synchronizes local configuration files and database files to the command service, creating multiple versions of data copies.

[0010] The backup data acquisition node periodically pulls the latest data file corresponding to the primary data acquisition node from the command service and replaces the local file;

[0011] The command service continuously receives the running status information of the primary acquisition node and the backup acquisition node, and sends a switchover command to the backup acquisition node after detecting that the primary acquisition node has been disconnected for more than a preset time.

[0012] The switching instructions include three states: OFF, PRE, and ON. The OFF instruction keeps the standby acquisition node in a standby state; the PRE instruction starts the acquisition process on the standby acquisition node but does not push data; and the ON instruction executes the acquisition and data push task on the standby acquisition node.

[0013] After receiving the switching instruction, the backup acquisition node determines whether to execute the data download task based on its current status and data synchronization status, starts the acquisition process, and performs the corresponding switching behavior.

[0014] Once the primary data acquisition node reconnects and pushes a normal operating status update, the instruction service sends a termination notification to the backup data acquisition node and delays the execution of the data acquisition process shutdown operation.

[0015] Optionally, in the above scheme, the main acquisition node performs data synchronization including:

[0016] Check if there is a difference between the most recent modification time and the last synchronization time of the file to be synchronized;

[0017] If modifications are made, a synchronization operation is performed, and the synchronized version is recorded in the command service;

[0018] If no changes are made, skip this synchronization task.

[0019] In the above scheme, optionally, the data file includes a configuration file and a database file. The configuration file is appsetting.json, and the database file includes HanaTech.ProCollector.db, HanaTech.ProCollector.db-shm, and HanaTech.ProCollector.db-wal.

[0020] In the above scheme, optionally, the GRPC bidirectional communication connection is used to transmit the following information: interface machine name, interface machine prefix, hostname, and IP address;

[0021] Whether to enable redundancy, redundancy number, and redundancy identifier;

[0022] Total number of interfaces, total number of tags, and total number of data collection processes.

[0023] In the above scheme, optionally, the OFF, PRE and ON switching instructions are issued sequentially by the instruction service according to the queue order. When a new switching instruction arrives, the previous unfinished instructions are automatically removed from the queue.

[0024] Optionally, under the above scheme, if the backup data acquisition node is currently in the data acquisition process, the "normal shutdown" command is sent to the data acquisition process in batches after a 3-minute delay to terminate the data acquisition task.

[0025] In the above scheme, optionally, under the PRE command, the backup acquisition node first checks the data synchronization status. If the synchronization fails, it executes the data download task and then starts the acquisition process. The acquisition process does not perform data push.

[0026] Optionally, under the ON command, the backup data acquisition node performs data acquisition and data push after the acquisition process starts, and the archiving service is restored synchronously.

[0027] Optionally, in the above scheme, the instruction service issues a PRE instruction 60 seconds after detecting a disconnection of the main acquisition node, issues an ON instruction 90 seconds after the disconnection, and notifies the backup acquisition node to stop acquisition after the main acquisition node recovers, delaying the acquisition process for 3 minutes.

[0028] Secondly, a real-time database data acquisition dual-machine redundancy system, the system comprising:

[0029] An instruction service module is used to establish a bidirectional GPRC communication connection with the primary and backup acquisition nodes, receive status information, and issue switching instructions.

[0030] The main data acquisition node is used to periodically upload configuration and database files to the command service;

[0031] The backup acquisition node is used to periodically pull data files uploaded by the main acquisition node and execute acquisition tasks;

[0032] The system controls the transfer of data acquisition responsibilities between the primary and backup servers based on switching commands, and implements status monitoring and a delayed stop mechanism.

[0033] Compared with the prior art, this application has at least the following beneficial effects:

[0034] Based on further analysis and research of existing technical problems, this application recognizes that real-time database data acquisition systems suffer from intolerable single-point failures, high master-slave switching latency, complex synchronization configurations, and poor data consistency. By constructing a bidirectional communication structure among the master acquisition node, the backup acquisition node, and the command service, automatic collaboration and efficient switching between master and backup nodes are achieved. When the master node fails, the backup node seamlessly takes over the acquisition task according to instructions, effectively avoiding data interruption caused by single-point failures. Simultaneously, a timed data synchronization and difference comparison mechanism is adopted to ensure the consistency of configurations and database files between the master and backup nodes, preventing acquisition anomalies due to data mismatches during switching. Furthermore, switching commands are controlled through three levels of states (OFF, PRE, ON), combined with a delayed stop mechanism and sequential processing of the command queue, further ensuring a stable and continuous switching process and avoiding system chaos caused by sudden disconnections or command conflicts. In summary, this solution not only improves the system's fault recovery efficiency and operational stability but also significantly simplifies redundant configuration and maintenance processes, effectively solving key problems in existing technologies such as single-point failure risk, long recovery time, complex configuration, and poor data consistency. Attached Figure Description

[0035] Figure 1 A flowchart illustrating a method for real-time database data acquisition with dual-machine redundancy, provided in one embodiment of this application;

[0036] Figure 2 This is a schematic diagram of a dual-machine redundant system architecture for real-time database data acquisition provided in one embodiment of this application;

[0037] Figure 3 A schematic diagram of real-time database data acquisition status monitoring with dual-machine redundancy provided in one embodiment of this application;

[0038] Figure 4 A schematic diagram of the interface management interface for real-time database data acquisition with dual-machine redundancy provided in one embodiment of this application;

[0039] Figure 5 A data flow diagram of a method for real-time database data acquisition with dual-machine redundancy provided in one embodiment of this application;

[0040] Figure 6 This is a schematic diagram of a dual-machine redundant master data acquisition process for real-time database data acquisition, provided in one embodiment of this application.

[0041] Figure 7 This is a schematic diagram of the backup data acquisition process of a real-time database data acquisition dual-machine redundancy method provided in one embodiment of this application;

[0042] Figure 8This is a schematic diagram of the instruction service flow for a method of real-time database data acquisition with dual-machine redundancy provided in one embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "comprising," "including," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0045] In one embodiment, such as Figure 1 As shown, a method for real-time database data acquisition with dual-machine redundancy is provided, including the following steps:

[0046] Deploy a primary data acquisition node and a backup data acquisition node. The primary data acquisition node performs data acquisition tasks under normal conditions, while the backup data acquisition node is in standby mode.

[0047] Configure the command service to establish bidirectional GRPC communication connections with both the primary and backup data acquisition nodes.

[0048] The main data acquisition node periodically synchronizes local configuration files and database files to the command service, creating multiple versions of data copies.

[0049] The backup data acquisition node periodically pulls the latest data file corresponding to the primary data acquisition node from the command service and replaces the local file;

[0050] The command service continuously receives the running status information of the primary acquisition node and the backup acquisition node, and sends a switchover command to the backup acquisition node after detecting that the primary acquisition node has been disconnected for more than a preset time.

[0051] The switching instructions include three states: OFF, PRE, and ON. The OFF instruction keeps the standby acquisition node in a standby state; the PRE instruction starts the acquisition process on the standby acquisition node but does not push data; and the ON instruction executes the acquisition and data push task on the standby acquisition node.

[0052] After receiving the switching instruction, the backup acquisition node determines whether to execute the data download task based on its current status and data synchronization status, starts the acquisition process, and performs the corresponding switching behavior.

[0053] Once the primary data acquisition node reconnects and pushes a normal operating status update, the instruction service sends a termination notification to the backup data acquisition node and delays the execution of the data acquisition process shutdown operation.

[0054] This embodiment designs a three-layer structure system of "main acquisition node + backup acquisition node + instruction service" to build an acquisition architecture with fault tolerance, automatic switching and efficient data synchronization. It effectively solves the problems of single point of failure, untimely recovery, complex configuration and poor data consistency in the existing single-machine deployment.

[0055] In terms of deployment structure, a dual-node architecture with one primary and one backup node is adopted. The primary acquisition node (i.e., the host, type Master) performs normal data acquisition tasks, while the backup acquisition node (i.e., the standby machine, type Slave) is in hot standby mode, ready to take over the acquisition function of the primary node at any time. Both types of nodes must fill in the same interface machine name and interface machine prefix during system registration to ensure the consistency of their acquired data primary keys, guaranteeing that the data structure remains unchanged after primary / backup switchover and avoiding tag conflicts or data loss. Regarding the communication mechanism, both the primary and backup nodes establish a bidirectional communication connection with the command service via the gRPC protocol. The primary and backup nodes periodically upload their own operating status to the command service. The command service, as a central coordination and management module, monitors the connection status and health indicators (such as the total number of acquisition processes, total number of tags, number of interfaces, etc.) of the primary and backup nodes in real time and has the ability to independently determine the host's disconnection status. Once the host's abnormal disconnection reaches a set time threshold (e.g., 60 seconds or 90 seconds), the system can automatically trigger the standby machine takeover process without manual intervention, significantly improving the real-time performance and controllability of system recovery.

[0056] To ensure data consistency, the main data acquisition node synchronizes the current database files and configuration files, including `appsetting.json` and multiple SQLite-formatted database files (such as `.db`, ​​`.db-shm`, and `.db-wal`), to the command service every predetermined time interval (e.g., every minute). A file update time determination mechanism is used during synchronization; a new version is only uploaded if the last modification time of a file is later than the last synchronized record, avoiding duplicate transmission of unchanged files and reducing bandwidth waste and system load. A maximum of 20 synchronized versions are retained to ensure that a switchback can revert to the most recent stable version.

[0057] The standby data acquisition node periodically retrieves the corresponding database and configuration files from the command service via polling and overwrites the old local versions, thus maintaining configuration consistency with the primary data acquisition node and laying a solid data foundation for taking over data acquisition tasks. If network jitter or file synchronization failures occur between the primary and standby nodes, the standby machine can still execute data acquisition tasks based on the last successful local configuration, thereby enhancing the system's fault tolerance.

[0058] In terms of primary / standby switchover control, the command service supports three switchover commands: OFF (standby), PRE (ready), and ON (enabled), corresponding to the idle, semi-takeover, and full takeover states of the standby node, respectively. This three-level switchover design enables a "gradual switchover": after a 60-second disconnection, a PRE command is sent, and the standby node starts the data acquisition process but does not push data; after a 90-second disconnection, an ON command is sent, and the standby node is officially enabled and performs data push operations. Once the primary node comes back online and its status returns to normal, the command service sends an OFF command to the standby node and delays for 3 minutes to shut down the data acquisition process, ensuring uninterrupted data acquisition during the primary node's recovery period and avoiding data aborts due to momentary misjudgments, achieving a truly seamless switchover.

[0059] Furthermore, all switchover commands are processed sequentially through a queue. If a new command is issued before a preceding command has been completed, invalid commands can be automatically cleared according to a preset strategy to avoid inconsistencies in primary and backup states caused by command concurrency or chaos. This "command queuing + state verification" strategy ensures the atomicity and consistency of the switchover process.

[0060] In summary, the technical solution described in this embodiment has the following technical effects:

[0061] Eliminates single point of failure risk: Primary and backup nodes automatically collaborate, allowing backup nodes to quickly take over in case of primary node failure; shortens recovery time: through automatic monitoring and switching mechanisms, primary-backup switchover time is controlled within 90 seconds; simplifies deployment and configuration: primary and backup nodes share the same interface machine name and prefix, avoiding manual configuration differences; ensures data consistency: through a timed synchronization mechanism of the database and configuration files, it ensures that the data collection logic remains unchanged after the switchover; improves system intelligence and stability: introduces GRPC communication and queue command mechanisms, supporting status awareness and gradual switching; enhances system scalability: this method naturally supports expansion to advanced deployment structures such as one primary with multiple backups and cross-backup.

[0062] In this embodiment, the data synchronization process of the main acquisition node includes:

[0063] Check if there is a difference between the most recent modification time and the last synchronization time of the file to be synchronized;

[0064] If modifications are made, a synchronization operation is performed, and the synchronized version is recorded in the command service;

[0065] If no changes are made, skip this synchronization task.

[0066] In this embodiment, the data files include a configuration file and a database file. The configuration file is appsetting.json, and the database files include HanaTech.ProCollector.db, HanaTech.ProCollector.db-shm, and HanaTech.ProCollector.db-wal.

[0067] In this embodiment, the GRPC bidirectional communication connection is used to transmit the following information: interface machine name, interface machine prefix, hostname, and IP address;

[0068] Whether to enable redundancy, redundancy number, and redundancy identifier;

[0069] Total number of interfaces, total number of tags, and total number of data collection processes.

[0070] In this embodiment, the three switching instructions, OFF, PRE, and ON, are issued sequentially by the instruction service according to the queue order. When a new switching instruction arrives, the previous unfinished instructions are automatically removed from the queue.

[0071] In this embodiment, under the OFF command, if the backup data acquisition node is currently in the data acquisition process, a "normal shutdown" command will be sent to the data acquisition process in batches after a 3-minute delay to terminate the data acquisition task.

[0072] In this embodiment, under the PRE command, the backup acquisition node first checks the data synchronization status. If the synchronization fails, it executes the data download task and then starts the acquisition process. The acquisition process does not push data.

[0073] In this embodiment, under the ON command, the standby data acquisition node performs data acquisition and data push after the acquisition process is started, and the archiving service is restored synchronously.

[0074] In this embodiment, the instruction service issues a PRE instruction 60 seconds after detecting a disconnection at the primary acquisition node, an ON instruction 90 seconds after the disconnection, and notifies the backup acquisition node to stop acquisition after the primary acquisition node recovers, delaying acquisition for 3 minutes.

[0075] This embodiment achieves automatic collaboration and efficient switching between primary and backup nodes by constructing a bidirectional communication structure among the primary acquisition node, backup acquisition node, and command service. When the primary node fails, the backup node seamlessly takes over the acquisition task according to instructions, effectively avoiding data interruption caused by single-point failures. Simultaneously, a timed data synchronization and difference comparison mechanism ensures the consistency of configuration and database files between the primary and backup nodes, preventing acquisition anomalies due to data mismatches during switching. Furthermore, switching commands are controlled by a three-level state control (OFF, PRE, ON), combined with a delayed stop mechanism and sequential processing of the command queue, further ensuring a stable and continuous switching process and preventing system chaos caused by sudden disconnections or command conflicts. In summary, this solution not only improves the system's fault recovery efficiency and operational stability but also significantly simplifies redundant configuration and maintenance processes, effectively solving key problems in existing technologies such as single-point failure risk, long recovery time, complex configuration, and poor data consistency.

[0076] In one embodiment, traditional real-time database data acquisition systems typically employ a standalone deployment model, relying on a single device to complete data acquisition, processing, and storage. Such systems usually ensure stability through the following methods: hardware redundancy (such as disk arrays and power supply redundancy); software-level retry mechanisms for exceptions; and regular data backups.

[0077] Single point of failure risk: Hardware failure or software crash in a single-machine deployment will interrupt data acquisition, affecting real-time performance. Long recovery time: Manual intervention is required to restart or switch over after a failure; recovery time is unpredictable and may result in data loss. Complex configuration: Existing redundancy solutions (such as cluster deployment) require additional configuration for load balancing and state synchronization, leading to high maintenance costs. Poor data consistency: Switching to a standby machine may cause acquisition anomalies or data conflicts due to configuration inconsistencies.

[0078] This embodiment ensures uninterrupted data acquisition during hardware failures through an automatic primary / standby switchover mechanism; it provides standardized interface machine configuration templates, allowing primary and standby machines to share key parameters (such as interface machine name and prefix); it synchronizes primary machine data to the command service periodically, and the standby machine downloads and updates as needed; and it enables rapid fault detection and recovery based on GPRC bidirectional communication and health status monitoring.

[0079] The system architecture of this embodiment is as follows: Figure 2 As shown, a dual-machine redundant configuration is adopted, with one master and one slave data acquisition node. The master node is responsible for routine data acquisition, while the slave node is on standby. The command service acts as the control center, communicating with the master and slave nodes via gRPC to manage status monitoring and switchover commands. The data synchronization mechanism involves the master node synchronizing configuration files and database files to the command service every minute, while the slave node periodically pulls and updates these files.

[0080] This system adds redundant configuration information, including: interface machine type, master number configuration, setting the interface machine type to Master, filling in the interface machine name, interface machine prefix, hostname, IP, and port number, and submitting it for ProDBManager review;

[0081] Configure the backup data acquisition system, setting the interface machine type to Slave. Enter the interface machine name, interface machine prefix, hostname, IP address, and port number, then submit it for ProDBManager review. After submission, a master-slave relationship is established between the interface machine and the primary data acquisition system using the interface machine name. The interface machine prefix will ultimately be based on the primary data acquisition system. If the corresponding primary data acquisition system does not exist or fails review after submission, the review will automatically fail.

[0082] After the data synchronization task starts, it establishes two-way communication with the instruction service and starts a polling task. It synchronizes information from the host to the instruction service once every minute and saves the 20 most recent versions according to the synchronization time.

[0083] During synchronization, consider comparing the most recent update time of the file to be synchronized with the last synchronization time. If there is no change, then do not synchronize.

[0084] After the backup data acquisition task is started, it establishes two-way communication with the command service, starts a polling task, and downloads the corresponding data file from the command service every minute to replace the local data file.

[0085] The primary and backup servers connect via gRPC command service, each pushing its operational status to the command service. Status information includes: interface name, interface prefix, hostname, IP address, whether redundancy is enabled, redundancy number, redundancy identifier, total number of interfaces, total number of tags, and total number of data collection processes.

[0086] Primary / standby switchover: The standby machine awaits the switchover command for the instruction service.

[0087] After receiving commands, the command service must execute them sequentially to prevent multiple commands from being executed simultaneously. For example, if a PRE command arrives but gets stuck at the data synchronization verification node, and then an ON command arrives, all commands preceding the OFF command can be removed from the queue.

[0088] OFF - Standby command: After receiving the backup data, save or update the local switching command to OFF.

[0089] If a local data collection process is running, a "normal shutdown" process command will be sent to the data collection process in batches after a 3-minute delay to complete the data collection and stop.

[0090] The 3-minute delay is to allow time for the main data acquisition process and ensure that data acquisition is not interrupted.

[0091] PRE - Prepare command. After receiving the backup data, save or update the local switch command to PRE.

[0092] First, check if the data synchronization was successful. If not, check if the local configuration file has the latest version. If not, execute the download task to update the local file with the latest data. Otherwise, the data collection process cannot be started.

[0093] The standby machine activates the acquisition process and sends a PRE switchover command to the acquisition process.

[0094] The data collection process is proceeding normally, but the archiving service is not pushing data.

[0095] The ON command enables data acquisition. After the backup machine receives the data, it saves or updates the local switching command to ON. First, it checks if data synchronization was successful. If not, it checks if the local configuration file has the latest version. If not, it executes a download task to update the local file with the latest data. Otherwise, the acquisition process cannot start. If the backup machine does not have an acquisition process, it enables the acquisition process and sends the ON switching command to it. The acquisition process acquires data normally, but the archiving service pushes data.

[0096] Data acquisition status monitoring, such as Figure 3 As shown, the extended instruction service client information display adds client types (archive, ProDBManager, data acquisition) and displays them according to client type.

[0097] For data acquisition, data is grouped and displayed according to the interface machine name, making it convenient to view primary and backup information.

[0098] Primary / standby failover: Command service performs status checks

[0099] If the primary machine disconnects for 60 seconds, the backup machine is notified to prepare for a switchover (the data collection process starts, but no push notifications are sent). If the disconnection continues for 90 seconds, the backup machine is notified to perform the actual switchover, and data collection and push notifications resume normally.

[0100] The primary machine reconnects and waits for the primary machine to push the data collection status. Once the status is normal, it notifies the backup machine to stop data collection. After receiving the notification, the backup machine delays for 3 minutes before stopping the data collection.

[0101] Interface machine management interface as follows Figure 4 As shown, the following redundant configuration information is added: primary port, redundant IP, redundant port, standby status, switch (standby, pre-switching, switched), and switch status (incomplete / completed).

[0102] Add command logs: (command identifier, command value, issuance time, feedback time, execution status, host status (online / offline), standby status (online / offline)).

[0103] Master data acquisition process: After startup, establish a connection with the command service and periodically push health status (number of interfaces, number of tags, process status). Execute data synchronization tasks to ensure that the command service has the latest configuration.

[0104] Data acquisition process: Receive OFF / PRE / ON commands from the instruction service and start the acquisition process as needed.

[0105] The PRE command only collects data without pushing it, while the ON command takes over all functions.

[0106] Command service process: The standby machine is triggered with a PRE command after 60 seconds of disconnection and an ON command after 90 seconds; after the host machine reconnects, it notifies the standby machine to stop data collection after a 3-minute delay.

[0107] Primary / backup consistency: Interface machine names and prefixes must be consistent; configuration changes only take effect on the primary machine.

[0108] Status monitoring interface: Real-time display of primary and backup IPs, switchover status, and command logs;

[0109] This embodiment uses a three-level command system (OFF / PRE / ON) to control the standby machine's gradual takeover, avoiding data interruption. It only compares differences in file update times, reducing unnecessary synchronization overhead. Upon receiving an OFF command, the standby machine stops after a 3-minute delay, ensuring uninterrupted data collection until the primary machine fully recovers. Command services process switchover requests sequentially to prevent concurrent command conflicts.

[0110] In this embodiment, the primary / standby switchover time is ≤90 seconds, reducing the risk of data loss by over 90%. Redundancy configuration is completed through a graphical interface, requiring no code modification. Future expansion to a one-primary-multiple-standby or mutually primary / standby architecture is supported. Command logs and status monitoring interfaces provide complete fault tracing capabilities.

[0111] like Figure 5 As shown, the primary and backup servers establish bidirectional GRPC communication with the command service. The primary server database data is synchronized to the command service through scheduled tasks. A primary health check mechanism is established, and the command service directs the primary and backup switchover. The primary and backup servers periodically push their health status to the command service.

[0112] like Figure 7 As shown in the figure, in a preferred embodiment, the startup and state transition process of the backup acquisition node mainly includes five steps: initialization startup, status reporting, data synchronization, instruction reception and switchover execution, as detailed below:

[0113] Once the standby data acquisition node starts up, it first establishes a two-way communication connection with the command service via the gRPC protocol, and periodically uploads its own operating status (including the number of interfaces, data acquisition process status, primary / standby type, redundancy flag, etc.) to the command service. Simultaneously, the standby node also downloads the database and configuration files synchronized from the primary data acquisition node from the command service at a fixed polling period (e.g., every 60 seconds) to ensure that its local configuration remains up-to-date and to prepare data for any potential primary / standby failover operations.

[0114] The system uses the local switchover status command cache as a criterion for judgment. If the current local cache is in an OFF state (i.e., standby state), the standby node will not perform any data collection and will remain idle. During this period, the standby node continuously waits for the command service to issue new switchover commands. Upon receiving the PRE (ready) status command, the standby node first verifies whether the current synchronization status is successful. If it fails, it will immediately download the latest configuration file and database file from the command service, complete the synchronization, and then start the data collection process.

[0115] When in PRE state, the standby node's data acquisition process is in a "data acquisition enabled, data not pushed" mode. This means the acquisition module can acquire and process data normally, but does not archive or push data to avoid conflicts between the primary and standby nodes. This state is primarily used for data monitoring and synchronization verification during the transition period.

[0116] If an ON (enabled) status command is received, the system will check again whether data synchronization is complete. After confirming that there are no errors, the acquisition process will be started and the system will enter the "acquisition + data push" mode, taking over the work responsibilities of the main acquisition node and achieving redundant takeover.

[0117] In addition, if an OFF command is received in any state, the standby node will execute a "delayed stop mechanism", that is, wait for a preset time (such as 180 seconds) before sending "normal shutdown" commands to the data collection process in batches, ensuring that the master node has full preparation time after it comes back online, and ensuring a smooth and uninterrupted switchover process.

[0118] Through this process, the system achieves dynamic state management and phased behavior control from standby to takeover, ensuring that the standby node can take over in an orderly, safe, and on-demand manner when the master node fails, thereby improving the overall fault tolerance and high availability of the system.

[0119] like Figure 8As shown in the figure, in one embodiment, the startup and primary / backup status determination process of the instruction service in the system mainly includes steps such as instruction service initialization, primary data acquisition node disconnection detection, disconnection duration determination, and instruction issuance control. This process realizes continuous monitoring and dynamic response to the operating status of the primary acquisition node, and automatically issues different levels of switching instructions to the backup node based on a preset disconnection time threshold, ensuring that the entire system can quickly switch over when the primary node malfunctions, avoiding data acquisition interruption.

[0120] Specifically, the process begins with the startup of the command service. After the service goes online, it first establishes a bidirectional GRPC communication connection with the primary and backup data acquisition nodes, continuously receiving operational status information reported by each node. Under normal circumstances, the primary data acquisition node will periodically send health status heartbeat packets to the command service as a sign that it is online.

[0121] When the command service detects that the primary data acquisition node has not continuously pushed status information within a preset time interval (e.g., 10 seconds), it determines that the primary data acquisition node may have lost connection and initiates the disconnection monitoring mechanism. If the disconnection lasts for no more than 60 seconds, the command service issues an OFF command to the standby data acquisition node, instructing it to remain in standby mode and not take over the data acquisition task.

[0122] If the disconnection time reaches 60 seconds but does not exceed 90 seconds, the system considers the master node to have a moderate fault, which has the possibility of recovery but is risky. Therefore, the instruction service sends a PRE instruction to the standby acquisition node to start the acquisition process but not push data, in preparation for takeover.

[0123] If the disconnection time exceeds 90 seconds, the system determines that the main acquisition node is in a state of serious abnormality or complete disconnection. At this time, the instruction service immediately sends an ON command to the backup acquisition node, requiring it to take over the acquisition tasks of the main acquisition node, including data acquisition and data push, to ensure that the entire system acquisition link is not interrupted.

[0124] This process employs a phased judgment and gradual switching mechanism to avoid accidental switching when the main acquisition node experiences short-term fluctuations, and to quickly activate backup resources when a true disconnection occurs. This not only enhances the system's fault tolerance but also optimizes resource utilization efficiency, effectively improving the system's continuity and stability.

[0125] In one embodiment, a dual-machine redundant system for real-time database data acquisition is provided, comprising the following program modules:

[0126] An instruction service module is used to establish a bidirectional GPRC communication connection with the primary and backup acquisition nodes, receive status information, and issue switching instructions.

[0127] The main data acquisition node is used to periodically upload configuration and database files to the command service;

[0128] The backup acquisition node is used to periodically pull data files uploaded by the main acquisition node and execute acquisition tasks;

[0129] The system controls the transfer of data acquisition responsibilities between the primary and backup servers based on switching commands, and implements status monitoring and a delayed stop mechanism.

[0130] This embodiment provides a complete dual-machine redundant data acquisition system architecture, including a primary acquisition node, a backup acquisition node, an instruction service module, and related communication mechanisms. During deployment, the primary and backup nodes are registered and verified through ProDBManager, binding the interface machine name and prefix to ensure data consistency. The GRPC channel is used for bidirectional communication; status uploading and instruction issuance are both implemented through this channel.

[0131] The primary node performs data acquisition and periodically synchronizes data files, while the standby node remains in standby mode, dynamically switching over via a command service. During system operation, the command service continuously monitors the primary and standby status, dynamically issuing PRE, ON, or OFF commands based on health check results to achieve automated primary / standby switching, fault tolerance, and recovery. The system supports a graphical interface displaying node status, historical command logs, synchronization status, and other information, facilitating maintenance personnel's understanding of the overall system operation. Due to its standardized structure and decoupled modules, the system boasts excellent scalability, supporting more complex application scenarios such as multi-node deployment and primary / standby switching.

[0132] The specific implementation details of each module can be found in the above description of the dual-machine redundancy method for real-time database data acquisition, and will not be repeated here.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for real-time database data acquisition with dual-machine redundancy, characterized in that, The method includes: Deploy a primary data acquisition node and a backup data acquisition node. The primary data acquisition node performs data acquisition tasks under normal conditions, while the backup data acquisition node is in standby mode. Configure the command service to establish bidirectional GRPC communication connections with both the primary and backup data acquisition nodes. The main data acquisition node periodically synchronizes its local configuration files and database files to the command service, creating multiple versions of data copies. The backup data acquisition node periodically pulls the latest data file corresponding to the primary data acquisition node from the command service and replaces the local file; The command service continuously receives the running status information of the primary acquisition node and the backup acquisition node, and sends a switchover command to the backup acquisition node after detecting that the primary acquisition node has been disconnected for more than a preset time. The switching instructions include three states: OFF, PRE, and ON. The OFF instruction keeps the standby acquisition node in a standby state; the PRE instruction starts the acquisition process on the standby acquisition node but does not push data; and the ON instruction executes the acquisition and data push task on the standby acquisition node. After receiving the switching instruction, the backup acquisition node determines whether to execute the data download task based on the current status and data synchronization status, starts the acquisition process, and performs the corresponding switching behavior. Once the primary data acquisition node reconnects and pushes a normal operating status update, the instruction service sends a termination notification to the backup data acquisition node and delays the execution of the data acquisition process shutdown operation.

2. The method according to claim 1, characterized in that, The main data acquisition node performs data synchronization, including: Check if there is a difference between the most recent modification time and the last synchronization time of the file to be synchronized; If modifications are made, a synchronization operation is performed, and the synchronized version is recorded in the command service; If no changes are made, skip this synchronization task.

3. The method according to claim 1, characterized in that, The data files include a configuration file and a database file. The configuration file is appsetting.json, and the database files include HanaTech.ProCollector.db, HanaTech.ProCollector.db-shm, and HanaTech.ProCollector.db-wal.

4. The method according to claim 1, characterized in that, The GRPC bidirectional communication connection is used to transmit the following information: interface machine name, interface machine prefix, hostname, and IP address; Whether to enable redundancy, redundancy number, and redundancy identifier; Total number of interfaces, total number of tags, and total number of data collection processes.

5. The method according to claim 1, characterized in that, The OFF, PRE, and ON switching instructions are issued sequentially by the instruction service according to the queue order. When a new switching instruction arrives, the previous unfinished instructions are automatically removed from the queue.

6. The method according to claim 1, characterized in that, Under the OFF command, if the backup data acquisition node is currently in the data acquisition process, a "normal shutdown" command will be sent to the data acquisition process in batches after a 3-minute delay to terminate the data acquisition task.

7. The method according to claim 1, characterized in that, Under the PRE command, the backup acquisition node first checks the data synchronization status. If synchronization fails, it executes the data download task and then starts the acquisition process. The acquisition process does not push data.

8. The method according to claim 1, characterized in that, Under the ON command, the standby data acquisition node performs data acquisition and data push after the acquisition process starts, and the archiving service is restored synchronously.

9. The method according to claim 1, characterized in that, The instruction service issues a PRE instruction 60 seconds after detecting a disconnection at the primary acquisition node, and an ON instruction 90 seconds after the disconnection. After the primary acquisition node recovers, it notifies the backup acquisition node to stop acquisition after a 3-minute delay.

10. A dual-machine redundant system for real-time database data acquisition, characterized in that, include: An instruction service module is used to establish a bidirectional GPRC communication connection with the primary and backup acquisition nodes, receive status information, and issue switching instructions. The main data acquisition node is used to periodically upload configuration and database files to the command service; The backup acquisition node is used to periodically pull data files uploaded by the main acquisition node and execute acquisition tasks; The system controls the transfer of data acquisition responsibilities between the primary and backup servers based on switching commands, and implements status monitoring and a delayed stop mechanism.

Images