Data acquisition method, apparatus, system, device and storage medium

Abstract

The present application belongs to the technical field of data processing. Disclosed are a data acquisition method, an apparatus, a system, a device and a storage medium, which are applied to standby servers. A plurality of data acquisition servers are connected to at least one standby server by means of data buses, the number of the standby servers is less than the number of the data acquisition servers, and the standby servers respectively store registration information of the plurality of data acquisition servers and configuration information between the plurality of data acquisition servers and corresponding devices to be acquired. The method comprises: acquiring operating state data of each data acquisition server; on the basis of the operating state data, determining an abnormal server; and loading registration information of the abnormal server, and, on the basis of configuration information between the abnormal server and a corresponding device to be acquired, establishing a communication connection with said corresponding device, so as to replace the abnormal server for data acquisition. A plurality of hosts share one or more standby machines, so as to control the number of standby machines, thereby avoiding resource waste, and improving reliability.

Description

Data acquisition methods, devices, systems, equipment and storage media

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202411982313.1, filed on December 30, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of data processing technology, and in particular to a data acquisition method, apparatus, system, device, and storage medium. Background Technology

[0004] Data acquisition servers are typically configured with dual redundancy, meaning each data acquisition server is equipped with a backup server. This data acquisition mode has high hardware costs and a large workload for installation and deployment. In actual applications, most backup servers are only used as backups and do not perform data acquisition services, resulting in wasted server resources. Furthermore, reliability is only achieved through dual-machine hot standby; if the backup server fails, it cannot be quickly replaced when the data acquisition server malfunctions, affecting the completion of data acquisition tasks and resulting in poor reliability.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a data acquisition method, apparatus, system, device and storage medium, which aims to solve the technical problem of insufficient reliability of data acquisition servers using dual redundancy configuration in the prior art.

[0007] In a first aspect, this application provides a data acquisition method applied to a backup server in a data acquisition system. The data acquisition system includes a data acquisition server and backup servers. Multiple data acquisition servers are connected to at least one backup server via a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores registration information of the multiple data acquisition servers and configuration information between the multiple data acquisition servers and the corresponding device to be acquired. The data acquisition method includes the following steps:

[0008] Obtain the operational status data of each data acquisition server;

[0009] Based on the aforementioned operational status data, identify the abnormal server;

[0010] The system loads the registration information of the abnormal server and establishes a communication connection with the corresponding device to be collected based on the configuration information between the abnormal server and the device to be collected, so as to replace the abnormal server in collecting data.

[0011] In this solution, multiple data acquisition servers share a backup server. The number of backup servers does not increase with the number of data acquisition servers, effectively controlling the number of backup servers, avoiding resource waste, reducing hardware costs, and alleviating the workload of installation and deployment. At the same time, the data acquisition server can connect to multiple backup servers. Even if a backup server is not working properly, other backup servers can be used to quickly switch between primary and backup servers when the data acquisition server fails, automatically taking over the data acquisition task of the failed server and improving reliability.

[0012] In some embodiments, the method further includes: synchronizing the collected data of the plurality of data acquisition servers in real time based on the data bus.

[0013] The data bus connecting the backup server and the data acquisition server enables real-time synchronization of data from all data acquisition servers. This allows the backup server to maintain real-time synchronization with the data acquisition servers and seamlessly switch over in case of server failure. It can replace the failed server to provide complete data to the user, avoiding data loss, and does not increase the communication burden on the devices being acquired. This ensures the normal operation of the data acquisition service and guarantees that the user can obtain the required data in real time, further improving reliability.

[0014] In some embodiments, after loading the registration information of the abnormal server and establishing a communication connection with the corresponding device to be collected based on the configuration information between the abnormal server and the corresponding device to be collected, in order to replace the abnormal server in data collection, the method further includes: stopping the replacement of the abnormal server in data collection when the abnormal server recovers; synchronizing the collected data of the abnormal server and the newly collected data obtained during the replacement process to the abnormal server, so that the abnormal server can reconnect to the corresponding device to be collected and collect data.

[0015] By monitoring the operation of each data acquisition server in real time, and promptly stopping and replacing the service of an abnormal server after it recovers, the newly acquired data and all historical data acquired by the abnormal server are synchronized to the abnormal server so that it can re-acquire data. This ensures that all servers currently collecting data have complete data, avoids data loss, guarantees the normal operation of data acquisition tasks, and ensures that users can obtain the data they need in real time, thus improving reliability.

[0016] In some embodiments, before the step of loading the registration information of the abnormal server, the method further includes: when the abnormal server has not been replaced, performing the step of loading the registration information of the abnormal server; and when the abnormal server has been replaced, returning to the step of obtaining the running status data of each data acquisition server.

[0017] By monitoring the data acquisition server, if the abnormal server has been replaced by another backup server, no other actions will be taken to avoid resource waste caused by repeated replacement. The operation of all data acquisition servers will continue to be monitored in real time. If the abnormal server has not been replaced, a master-slave switch will be performed in a timely manner to automatically take over the data acquisition task of the abnormal server and improve reliability.

[0018] In some embodiments, the step of determining abnormal servers based on the operating status data includes: identifying data acquisition servers whose operating status data meets preset abnormal conditions as abnormal servers.

[0019] The current abnormal server is identified by analyzing the operational status data of the data acquisition server. If the operational status data meets the preset abnormal conditions, it indicates that the data acquisition server is an abnormal server. This allows for timely primary / backup switching, automatically taking over the data acquisition tasks of the abnormal server, ensuring the normal operation of the data acquisition tasks, and improving reliability.

[0020] In some embodiments, the operating status data includes at least the most recent heartbeat time, and the step of obtaining the operating status data of each data acquisition server includes: receiving heartbeat information sent by each data acquisition server; and determining the most recent heartbeat time of each data acquisition server based on the heartbeat information of each data acquisition server.

[0021] The heartbeat mechanism determines the real-time operating status of the data acquisition server. If the backup server can receive the heartbeat from the data acquisition server normally, it means that the data acquisition server is operating normally. If it does not receive the heartbeat from the data acquisition server normally, it means that the data acquisition server is abnormal. This allows for accurate identification of the abnormal server, enabling timely primary / backup switching and automatic takeover of the data acquisition task from the abnormal server, ensuring the normal operation of the data acquisition task and improving reliability.

[0022] In some embodiments, each device to be acquired uses an independent communication connection, and the method further includes: obtaining the connection status of an existing communication connection; when the connection status of the existing communication connection is disconnected, re-establishing the communication connection with the corresponding device to be acquired to resume data acquisition.

[0023] By setting up independent communication connections for each device to be collected and using independent communication processes to transmit data, the stability and reliability of data transmission can be guaranteed, interference caused by resource contention in multi-threaded mode can be avoided, the overall concurrency performance can be improved, and the communication connection status of each device to be collected can be monitored. Once an abnormal connection occurs, it will be automatically restored to ensure the reliability of data collection.

[0024] Secondly, to achieve the above objectives, this application also proposes a data acquisition device, which includes:

[0025] The data acquisition module is used to acquire the operating status data of each data acquisition server;

[0026] The status determination module is used to determine abnormal servers based on the running status data;

[0027] The data acquisition replacement module is used to load the registration information of the abnormal server and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server in data acquisition.

[0028] Thirdly, to achieve the above objectives, this application also proposes a data acquisition system, which includes a data acquisition server and a backup server. Multiple data acquisition servers are connected to at least one backup server via a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores the registration information of the multiple data acquisition servers and the configuration information between the multiple data acquisition servers and the corresponding device to be acquired.

[0029] The data acquisition server is used to acquire data from the device to be acquired;

[0030] The backup server is used to obtain the operating status data of each data acquisition server, determine the abnormal server based on the operating status data, load the registration information of the abnormal server, and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server in data acquisition.

[0031] Fourthly, to achieve the above objectives, this application also proposes a data acquisition device, which includes: a memory, a processor, and a data acquisition program stored in the memory and executable on the processor, the data acquisition program being configured to implement the steps of the data acquisition method described above.

[0032] Fifthly, to achieve the above objectives, this application also proposes a storage medium storing a data acquisition program, which, when executed by a processor, implements the steps of the data acquisition method described above.

[0033] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0034] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0035] Figure 1 is a schematic flowchart of a data acquisition method according to some embodiments of this application;

[0036] Figure 2 is a schematic diagram of the overall architecture of the data acquisition method in some embodiments of this application;

[0037] Figure 3 is a schematic diagram of the application scenario of the data acquisition method of some embodiments of this application;

[0038] Figure 4 is another flowchart illustrating a data acquisition method according to some embodiments of this application;

[0039] Figure 5 is a schematic diagram of data synchronization in a data acquisition method according to some embodiments of this application;

[0040] Figure 6 is a schematic flowchart of some embodiments of this application;

[0041] Figure 7 is a schematic diagram of a data acquisition device according to some embodiments of this application.

[0042] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0043] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0045] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0046] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0047] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0048] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0049] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0050] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0051] Currently, data acquisition servers are generally configured with dual redundancy, meaning each data acquisition server is equipped with a backup server. For example, if 10 data acquisition servers each have one backup server, then 10 backup servers are needed, for a total of 20 servers. This data acquisition model is not only costly and involves a large workload for installation and deployment, but also requires significant storage space. In practical applications, most backup servers are only used as backups and do not perform data acquisition services, resulting in a waste of server resources.

[0052] Since each data acquisition server is equipped with a backup server, the reliability is only achieved through dual-machine hot standby. If the backup server of the data acquisition server fails, it cannot be quickly replaced when the data acquisition server malfunctions, which affects the completion of the data acquisition task and results in insufficient reliability.

[0053] Therefore, in this embodiment of the application, multiple data acquisition servers share a backup server. The number of backup servers does not increase with the number of data acquisition servers, which can effectively control the number of backup servers, avoid resource waste, reduce hardware costs, and alleviate the workload of installation and deployment. At the same time, the data acquisition server can connect to multiple backup servers. Even if a backup server cannot be used normally, other backup servers can be used to quickly switch between primary and backup servers when the data acquisition server is abnormal, automatically taking over the data acquisition task of the abnormal server and improving reliability.

[0054] It should be noted that the embodiments described in this application are applied to data acquisition scenarios, especially high-capacity data acquisition scenarios, such as data acquisition for gigawatt-hour-level energy storage power stations. A gigawatt-hour-level energy storage power station may have up to 1 million battery cells, requiring the acquisition of up to 10 million data points. A single ordinary data acquisition server can only support a maximum of 1 million data points. To meet the data acquisition needs of a gigawatt-hour-level energy storage power station, at least 10 data acquisition servers are required to jointly handle the data acquisition task. If a traditional method is used to configure backup servers, each data acquisition server would be equipped with one backup server, requiring at least 10 backup servers, totaling 20 servers. As the required data volume increases, the number of servers will increase at a rate of 2 times, resulting in resource waste and high costs. Furthermore, if a backup server fails, it cannot be replaced when the corresponding data acquisition server malfunctions, easily affecting the smooth progress of the acquisition task and resulting in insufficient reliability. The data acquisition method described in this application can effectively control the number of backup servers, avoid resource waste, reduce hardware costs, alleviate installation and deployment workload, and ensure the reliability of each data acquisition server.

[0055] This application addresses the technical problem of insufficient reliability when using a dual-redundant configuration for the data acquisition server, and proposes a data acquisition method. Referring to Figure 1, in this example, the data acquisition method includes:

[0056] Step S10: Obtain the operating status data of each data acquisition server.

[0057] It should be noted that the execution entity in this embodiment is the backup server in the data acquisition system. The backup server runs a data acquisition program, which performs primary / backup switching to replace the faulty server for data acquisition. This embodiment does not impose any restrictions on this.

[0058] In this embodiment, the data acquisition system includes a data acquisition server and a backup server. Referring to Figure 2, multiple data acquisition servers are connected to at least one backup server via a data bus. That is, multiple data acquisition servers can share a single backup server; for example, 10 data acquisition servers can share one backup server. Alternatively, multiple data acquisition servers can share multiple (two or more) backup servers; for example, 10 data acquisition servers can share three backup servers. Both the data acquisition server and the backup server are servers with data acquisition capabilities, such as SCADA (Supervisory Control and Data Acquisition) data acquisition servers. This embodiment does not specifically limit the specific type of server. The data acquisition server can be considered the host machine used during the data acquisition process, and the backup server can be considered a standby machine used during the data acquisition process. The number of backup servers can be set according to the actual needs of different scenarios, and usually will not exceed the number of data acquisition servers; that is, the number of backup servers is less than the number of data acquisition servers.

[0059] Each data acquisition server connects to a certain number of devices to be acquired. The number of devices connected is usually determined by the data processing capacity of the data acquisition server. For example, if the maximum data acquisition volume that a data acquisition server can support is 1 million points, then 1 million points is taken as the data processing capacity of the data acquisition server. After connecting the devices to be acquired, the total data acquisition volume of the data acquisition server cannot exceed 1 million points. Assuming that each device to be acquired generates 100,000 data points, then the number of devices to be acquired connected to the data acquisition server cannot exceed 10. The devices to be acquired are those that need to use the data acquisition server for data acquisition. Different scenarios usually have different devices to be acquired. For example, when acquiring data from a gigawatt-hour-level energy storage power station, the devices to be acquired are usually a Battery Monitoring and Management System (BMS). This embodiment does not specifically limit this.

[0060] For example, referring to Figure 3, the data acquisition system of the gigawatt-hour energy storage power station includes multiple acquisition subsystems (acquisition subsystem 1 to acquisition subsystem n). Each acquisition subsystem is equipped with a data acquisition server (data acquisition host). Each data acquisition server is connected to multiple devices to be acquired (BMS1 to BMSm). All data acquisition servers share multiple backup servers (data acquisition backup machines).

[0061] Generally, a data acquisition server collects data from the device being acquired. If the data acquisition server malfunctions and cannot collect data normally, a primary / backup switch is performed, with a backup server taking over data collection to ensure continued data acquisition. In this embodiment, since all data acquisition servers share a backup server, it is not necessary to configure a backup server for each data acquisition server. The number of backup servers does not increase with the number of data acquisition servers, effectively controlling the number of backup servers, avoiding resource waste, reducing the total number of servers required, reducing hardware costs, and reducing installation and deployment workload.

[0062] For example, if the total amount of data to be collected is 10 million points, and each data acquisition server can support 1 million data points, then the number of data acquisition servers is 10. If a traditional dual-redundancy configuration (one primary and one backup) is adopted, one backup server needs to be configured for each of the 10 data acquisition servers, requiring a total of 10 backup servers. In this case, the total number of servers required is 20. If the solution of this embodiment is adopted, one shared backup server is configured, and the total number of servers required is 11. Compared with the 20 servers required by the traditional solution, the cost can be reduced by 45%, and a large amount of cabinet and installation and deployment labor costs are also saved.

[0063] The operational status of a data acquisition server refers to its running condition, such as normal or abnormal. A normal status typically indicates the server is functioning correctly, while an abnormal status usually indicates malfunction, such as a component failure; however, this is not a specific limitation. Since each backup server connects to multiple data acquisition servers, their operational status needs to be monitored in real time to promptly identify any malfunctioning servers and initiate a timely primary / backup switchover.

[0064] The operational status data of the data acquisition server is relevant data that characterizes the server's operating status. Based on this data, the current operational status of the data acquisition server can be determined, thereby identifying whether the server is malfunctioning. This operational status data is typically acquired in real-time by a backup server.

[0065] Step S20: Identify the abnormal server based on the running status data.

[0066] It should be noted that an abnormal server is a server that is malfunctioning, specifically a data acquisition server whose operating status is abnormal. This embodiment uses operating status data to determine the current abnormal server.

[0067] As an example, the steps for identifying abnormal servers based on operational status data include: identifying data acquisition servers whose operational status data meets preset abnormal conditions as abnormal servers.

[0068] Preset anomaly conditions are pre-defined criteria used to identify abnormal servers. Generally, if the operational status data of a data acquisition server meets the preset anomaly conditions, it indicates that the data acquisition server's operational status data is abnormal, and the server is operating abnormally; in this case, the data acquisition server's operational status is abnormal, and it can be considered an abnormal server. Conversely, if the operational status data of a data acquisition server does not meet the preset anomaly conditions, it indicates that the data acquisition server's operational status data is normal, and the server is operating normally; in this case, the data acquisition server's operational status is normal, and it is not an abnormal server. This process allows us to identify the current abnormal server among all data acquisition servers.

[0069] The current abnormal server is identified by analyzing the operational status data of the data acquisition server. If the operational status data meets the preset abnormal conditions, it indicates that the data acquisition server is an abnormal server. This allows for timely primary / backup switching, automatically taking over the data acquisition tasks of the abnormal server, ensuring the normal operation of the data acquisition tasks, and improving reliability.

[0070] Step S30: Load the registration information of the abnormal server, and establish a communication connection with the corresponding device to be collected based on the configuration information between the abnormal server and the corresponding device to be collected, so as to replace the abnormal server in data collection.

[0071] It should be noted that each data acquisition server has corresponding registration information, which is the server's basic information, such as server name, server number, IP (Internet Protocol) address, physical address, etc. This embodiment does not impose specific limitations on this. Generally speaking, different data acquisition servers have different registration information, that is, the registration information of each data acquisition server is unique. The corresponding data acquisition server can be found through the registration information, and the corresponding registration information can also be found through the data acquisition server.

[0072] In this embodiment, the backup server stores the registration information of multiple data acquisition servers. That is, the backup server stores the registration information of all data acquisition servers. After an abnormal server is detected, its corresponding registration information can be found based on the abnormal server, and the registration information of the abnormal server can be loaded so that the backup server can start the data acquisition task, complete the master-slave switch, and continue to collect data in place of the abnormal server.

[0073] Configuration information refers to the configuration of parameters related to establishing a communication connection. Data acquisition servers typically store the configuration information of the devices to be acquired, enabling them to establish communication connections, transmit data, and obtain the required data. In this embodiment, each data acquisition server only stores the configuration information of the devices to be connected, thus maintaining the CPU (Central Processing Unit) and memory usage of the data acquisition server at a reasonable level, while also ensuring that data sent from all devices to be acquired can be processed in a timely manner.

[0074] Understandably, the backup server connects to multiple data acquisition servers, all of which could potentially become malfunctioning. The backup server should be able to replace any malfunctioning data acquisition server. Therefore, the backup server stores the configuration information between multiple data acquisition servers and their corresponding devices to be acquired; that is, the backup server stores the configuration information for all devices to be acquired. Upon detecting a malfunctioning data acquisition server, the backup server can use the stored configuration information to find the configuration information between the malfunctioning server and the device to be acquired, thereby establishing a communication connection with the device and retrieving the corresponding data from the device in place of the malfunctioning server.

[0075] Since each data acquisition server in this embodiment can connect to multiple backup servers, that is, each data acquisition server can be configured with multiple backup servers, i.e., one primary and multiple backups, if one of the multiple backup servers cannot be used normally, other backup servers can be used to quickly switch between primary and backup, automatically taking over the data acquisition task of the abnormal server, thereby improving reliability.

[0076] Understandably, the reliability of each data acquisition server (host) can be calculated using the formula: R = 1 - x n

[0077] In the formula, R represents reliability, n represents the number of backup servers configured for the data acquisition server, and x represents the probability that the server is working normally.

[0078] For example, assuming the probability of the server working normally is 90%, in the traditional solution (one primary and one backup), one data acquisition server is configured with one backup server, and the reliability is constant at 99%, which cannot be further improved. However, in this solution (one primary and multiple backups), one data acquisition server is configured with two backup servers, and the reliability R is 99.9%. One data acquisition server is configured with three backup servers, and the reliability R is 99.99%. As the number of backup servers increases, the reliability can be continuously improved.

[0079] As can be seen, this embodiment can increase the number of backup servers as needed to further improve the reliability of the data acquisition server.

[0080] Furthermore, the probability of all data acquisition servers failing simultaneously can be calculated using the following formula: P = y m

[0081] In the formula, P represents the probability that all data acquisition servers will fail simultaneously, m represents the number of data acquisition servers, and y represents the probability that a data acquisition server will fail.

[0082] For example, if the probability of the data acquisition server failing is 10%, the probability of all 5 servers failing simultaneously is 0.1%. 5 =0.00001, one or more backup machines can ensure that the reliability of SCADA system data acquisition reaches the "five nines" standard, guaranteeing disaster recovery capability.

[0083] In this embodiment, by using multiple data acquisition servers to share a backup server, the number of backup servers does not increase with the number of data acquisition servers. This effectively controls the number of backup servers, avoids resource waste, reduces hardware costs, and alleviates the workload of installation and deployment. At the same time, a data acquisition server can connect to multiple backup servers. Even if a backup server is not working properly, other backup servers can be used to quickly switch between primary and backup servers when a data acquisition server malfunctions, automatically taking over the data acquisition task of the malfunctioning server and improving reliability.

[0084] As an example, the operational status data includes at least the most recent heartbeat time. The steps for obtaining the operational status data of each data acquisition server include: receiving heartbeat information sent by each data acquisition server; and determining the most recent heartbeat time of each data acquisition server based on the heartbeat information of each data acquisition server.

[0085] It should be noted that heartbeat information is a message sent by the data acquisition server to the backup server, usually sent at a certain time interval, such as 1 second. This embodiment does not specify a particular time limit. The most recent heartbeat time is the difference between the time when the heartbeat information was last received and the current time. The most recent heartbeat time can be calculated based on the time when the heartbeat information was received and the current time. For example, if the most recent heartbeat time is 5 seconds ago, then the most recent heartbeat time is 5 seconds.

[0086] The preset time threshold is the threshold of the most recent heartbeat time. The specific value is usually determined according to the period of heartbeat information transmission. For example, if the period of heartbeat information transmission is 1 second, the preset time threshold is set to 2 seconds. This embodiment does not limit this.

[0087] In this embodiment, the operating status data includes at least the most recent heartbeat time. If the most recent heartbeat time of the data acquisition server is greater than a preset time threshold, it indicates that the heartbeat information of the data acquisition server is not being sent according to the set period, and the heartbeat information transmission of the data acquisition server can be considered abnormal. If the most recent heartbeat time of the data acquisition server is less than or equal to the preset time threshold, it indicates that the heartbeat information of the data acquisition server is being sent according to the set period, and the heartbeat information transmission of the data acquisition server can be considered normal.

[0088] For example, assuming the preset time threshold is 2 seconds, if the most recent heartbeat time of data acquisition server A is 5 seconds, then the most recent heartbeat time is greater than the preset time threshold, and the heartbeat information transmission of data acquisition server A is abnormal. If the most recent heartbeat time of data acquisition server B is 1 second, then the most recent heartbeat time is less than the preset time threshold, and the heartbeat information transmission of data acquisition server B is normal.

[0089] Since preset abnormal conditions are usually set based on abnormal operation conditions, the preset abnormal conditions include at least the most recent heartbeat time being greater than the preset time threshold. The most recent heartbeat time and the preset time threshold can be used to determine whether the heartbeat information is sent abnormally. Based on other set conditions, the operation status data can be comprehensively judged to see if it meets the preset abnormal conditions, and then it can be determined whether the data acquisition server is an abnormal server.

[0090] In practical implementation, if the running status data only uses the most recent heartbeat time, the preset abnormal condition is that the most recent heartbeat time is greater than a preset time threshold. If the most recent heartbeat time of the data acquisition server is greater than the preset time threshold, then the running status data of the data acquisition server is considered to meet the preset abnormal condition. At this time, the data acquisition server that sends heartbeat information abnormally is an abnormal server. If the most recent heartbeat time of the data acquisition server is less than or equal to the preset time threshold, then the running status data of the data acquisition server is considered not to meet the preset abnormal condition.

[0091] The heartbeat mechanism determines the real-time operating status of the data acquisition server. If the backup server can receive the heartbeat from the data acquisition server normally, it means that the data acquisition server is operating normally. If it does not receive the heartbeat from the data acquisition server normally, it means that the data acquisition server is abnormal. This allows for accurate identification of the abnormal server, enabling timely primary / backup switching and automatic takeover of the data acquisition task from the abnormal server, ensuring the normal operation of the data acquisition task and improving reliability.

[0092] In some embodiments, to avoid data loss, data from the data acquisition server can be synchronized in real time.

[0093] As an example, referring to Figure 4, the steps before obtaining the operational status data of each data acquisition server include:

[0094] Step S01: Based on the data bus, synchronize the collected data from multiple data acquisition servers in real time.

[0095] It should be noted that the collected data refers to the data that has been collected by each data collection server.

[0096] Understandably, the backup server connects to the corresponding data acquisition server via a private data bus, enabling real-time synchronization of data collected by all data acquisition servers. Upon detecting an abnormal server among the data acquisition servers, the backup server replaces the abnormal server for data acquisition. Because the backup server has already synchronized the collected data from the abnormal server, it ensures that no anomalies occur during the primary / backup switchover process due to data loss, achieving a seamless switchover.

[0097] It should be understood that when the data acquisition server is running normally, the user application obtains data from the data acquisition server in real time. After an abnormal server is detected in the data acquisition server, the backup server replaces the abnormal server to collect data. Since the backup server has synchronized the collected data of the abnormal server, the backup server can provide the user with the complete collected data of the abnormal server, ensuring that the user can query the data they need in real time, maximizing availability and stability, and also providing a good user experience.

[0098] The data bus connecting the backup server and the data acquisition server enables real-time synchronization of data from all data acquisition servers. This allows the backup server to maintain real-time synchronization with the data acquisition servers and seamlessly switch over in case of server failure. It can replace the failed server to provide complete data to the user, avoiding data loss, and does not increase the communication burden on the devices being acquired. This ensures the normal operation of the data acquisition service and guarantees that the user can obtain the required data in real time, further improving reliability.

[0099] In practical implementation, referring to Figure 5, when the data acquisition server is running normally (without a primary / backup switch), the user application obtains / queries data from the data acquisition server, and the backup server keeps synchronized with the real-time data of all data acquisition servers through the data bus; when the data acquisition server malfunctions (after a primary / backup switch), the backup server has synchronized the real-time data of the malfunctioning server, thus ensuring that the application will not malfunction due to the loss of real-time data during the switch, achieving seamless switching.

[0100] Furthermore, as an example, referring to Figure 4, after identifying the abnormal server, the steps of loading the abnormal server's registration information and establishing a communication connection with the corresponding data collection device based on the configuration information between the abnormal server and the corresponding data collection device to replace the abnormal server in data collection also include:

[0101] Step S41: When the abnormal server recovers to normal, stop replacing the abnormal server to collect data;

[0102] It should be noted that "restoring a faulty server to normal" refers to the server's operating status changing from an abnormal state to a normal state. The operating status of a faulty server can be monitored by other backup servers, and this embodiment does not specifically limit this.

[0103] It is understandable that although the abnormal server is abnormal, some of the causes of the abnormality can be resolved with simple repairs. For example, if the operating status enters an abnormal state due to a power outage, the operating status can be restored to normal after the power is reconnected. At this time, the abnormal server can be put back into use. Therefore, when the abnormal server returns to normal, the backup server needs to promptly cancel its replacement of the abnormal server, that is, the backup server stops replacing the abnormal server for data collection.

[0104] Step S42: Synchronize the collected data of the abnormal server and the newly collected data obtained during the replacement process to the abnormal server so that the abnormal server can reconnect to the corresponding device to be collected for data collection.

[0105] It should be noted that the newly collected data refers to the data newly collected by the backup server during the replacement process.

[0106] Understandably, after the faulty server resumes normal operation, all data previously collected by the faulty server (already collected data) and newly collected data from the backup server (newly collected data) are synchronized to the faulty server via the data bus. This ensures the faulty server has complete data, preventing data loss, and also guarantees that users can retrieve the required data from the recovered faulty server. The recovered faulty server then re-establishes a communication connection with the corresponding data collection device to begin data collection.

[0107] By monitoring the operational status of each data acquisition server in real time, and promptly stopping and replacing the service of an abnormal server after it recovers, the newly acquired data and all historical data acquired by the abnormal server are synchronized to the abnormal server so that it can re-acquire data. This ensures that all servers currently collecting data have complete data, avoids data loss, guarantees the normal operation of data acquisition tasks, and ensures that users can obtain the required data in real time, thus improving reliability.

[0108] In some embodiments, when the data acquisition server malfunctions, it is necessary to avoid repeatedly replacing it with a backup server.

[0109] As an example, the step of loading the registration information of the abnormal server includes, before the step of loading the registration information of the abnormal server, if the abnormal server has not been replaced; and if the abnormal server has been replaced, returning to the step of obtaining the running status of each data acquisition server in real time.

[0110] It should be noted that since all data acquisition servers can connect to multiple backup servers, when there is an abnormal server among these data acquisition servers, multiple backup servers can replace the abnormal server to perform data acquisition. It is possible that more than one backup server will take over. Therefore, in this embodiment, when an abnormal server is determined to exist, it is first determined whether the abnormal server has been replaced by another backup server. If the abnormal server has been replaced by another backup server, no other action is taken, and the operating status of each data acquisition server continues to be monitored. If the abnormal server has not been replaced by another backup server, the registration information of the abnormal server is loaded, a master-slave switch is performed, and the abnormal server takes over to perform data acquisition.

[0111] For example, suppose data acquisition server 1 and data acquisition server 2 share backup server A and backup server B. If backup server A finds that data acquisition server 1 is an abnormal server, it first determines whether backup server B has replaced data acquisition server 1 to perform data acquisition. If it has, it will not replace data acquisition server 1 and will continue to monitor the operating status of data acquisition server 1 and data acquisition server 2. If backup server B has not replaced data acquisition server 1, then backup server A will replace data acquisition server 1 to perform data acquisition.

[0112] By monitoring the data acquisition server, if the abnormal server has been replaced by another backup server, no other actions will be taken to avoid resource waste caused by repeated replacement. The operation of all data acquisition servers will continue to be monitored in real time. If the abnormal server has not been replaced, a master-slave switch will be performed in a timely manner to automatically take over the data acquisition task of the abnormal server and improve reliability.

[0113] Furthermore, if there are multiple backup servers that can be switched between primary and backup, the most suitable backup server should be selected to replace the faulty server.

[0114] As an example, when the abnormal server has not been replaced, the switching scheduling policy is obtained; when the switching scheduling policy is a health check switching policy, the health score corresponding to the health indicator is determined; when the health score meets the preset switching conditions, the step of loading the registration information of the abnormal server is executed.

[0115] It should be noted that the switching scheduling strategy is the strategy for determining the most suitable backup server among multiple backup servers, and includes at least the manual switching strategy and the health check switching strategy.

[0116] The manual switching strategy determines the most suitable backup server based on the user's selection. In other words, the most suitable backup server is manually selected by the user, and the selected backup server performs the step of loading the registration information of the abnormal server.

[0117] The health check switchover strategy determines the most suitable backup server based on the results of health checks, typically selecting the backup server with the best health status. Health checks usually involve comprehensively scoring set health indicators, and the final score is the health score, which is the result of the health check. These health indicators include at least CPU utilization, memory utilization, and network connectivity status, and can be flexibly adjusted according to actual conditions; this embodiment does not impose specific limitations on these. Generally, a higher health score indicates a better health status, and a lower health score indicates a worse health status. Therefore, the backup server with the best health status, i.e., the backup server with the highest health score, can be selected for primary / backup switchover.

[0118] The preset switching conditions are the conditions set for a primary / standby switchover, namely, the health score is the highest score. In other words, when the health score of the standby server is higher than that of other standby servers, the standby server will replace the abnormal server to collect data.

[0119] In practice, a switchover management server can be set up in the backup server to perform health checks on the backup server and generate the corresponding health score.

[0120] By managing the backup server switchover, when there are multiple backup servers, the backup server with the best health status can be quickly and accurately identified for primary / backup switching, taking over the data collection task from the abnormal server, ensuring data collection efficiency and improving reliability.

[0121] In some embodiments, it is necessary to ensure a normal connection between the server and the device to be collected during the data acquisition process.

[0122] As an example, the data acquisition method also includes: obtaining the connection status of the existing communication connection; and when the connection status of the existing communication connection is disconnected, re-establishing the communication connection with the corresponding device to be acquired in order to resume data acquisition.

[0123] In this embodiment, each device to be collected uses an independent communication connection. That is, the data acquisition server sets up an independent communication connection for each connected device. After the backup server replaces the faulty server, it also sets up an independent communication connection for each connected device. As can be seen, each device to be collected transmits data independently, thereby ensuring the stability and reliability of the system, avoiding interference caused by resource competition in multi-threaded mode, and improving the overall concurrency performance.

[0124] It should be noted that the current state of the communication connection is the connection status, which can generally be divided into two situations: disconnected state and normal communication state. The disconnected state means that the communication connection is broken, which means that data cannot be transmitted normally and the required data cannot be collected from the device to be collected. The normal communication state means that the communication connection is normal, which means that data can be transmitted normally and the required data can be collected from the device to be collected.

[0125] The existing communication connection refers to the established communication connection between the backup server and the device to be collected. If the existing communication connection is in a disconnected state, it means that the backup server and the device to be collected cannot transmit data normally, and the backup server cannot collect the required data from the device. If the existing communication connection is in a normal communication state, it means that the backup server and the device to be collected can transmit data normally, and the backup server can collect the required data from the device.

[0126] Understandably, when the data acquisition server is running normally, it monitors the connection status of the communication connections of each device to be acquired. If the connection status is disconnected, it will be restored immediately. Therefore, after the backup server takes over the data acquisition from the faulty server, the backup server monitors the connection status of the existing communication connections. Once the connection status of the existing communication connection is disconnected, it will be re-established immediately to restore the normal connection status, so as to ensure the normal progress and reliability of data acquisition.

[0127] For example, the backup server C has two existing communication connections: existing communication connection 1 is the communication connection established between the backup server C and the device to be collected 1, and existing communication connection 2 is the communication connection established between the backup server C and the device to be collected 2. The backup server C monitors the connection status of existing communication connection 1 and existing communication connection 2. If the connection status of existing communication connection 1 is disconnected, the backup server C re-establishes the communication connection with the device to be collected 1 and resumes data collection. If the connection status of existing communication connection 2 is disconnected, the backup server C re-establishes the communication connection with the device to be collected 2 and resumes data collection. If the connection status of existing communication connection 1 and existing communication connection 2 is normal, no other action is taken, and the monitoring of the connection status continues.

[0128] By setting up independent communication connections for each device to be collected and using independent communication processes to transmit data, the stability and reliability of data transmission can be guaranteed, interference caused by resource contention in multi-threaded mode can be avoided, the overall concurrency performance can be improved, and the communication connection status of each device to be collected can be monitored. Once an abnormal connection occurs, it will be automatically restored to ensure the reliability of data collection.

[0129] Referring to Figure 6, this embodiment proposes the following scheme to illustrate the implementation process of the data acquisition method, specifically:

[0130] The backup server monitors the operating status of all data acquisition servers in real time. If there is no abnormal server among the data acquisition servers, it continues to monitor the operating status of the data acquisition servers. If there is an abnormal server among the data acquisition servers, it determines whether the abnormal server has been replaced. If it has not been replaced by another backup server, it loads the registration information of the abnormal server, establishes a communication connection with the device to be acquired on the abnormal server using the stored configuration information, and performs data acquisition in place of the abnormal server. If it has been replaced by another backup server, it continues to monitor the operating status of the data acquisition servers.

[0131] This application also proposes a data acquisition device.

[0132] In this embodiment, multiple data acquisition servers are connected to at least one backup server via a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores the registration information of multiple data acquisition servers and the configuration information between the multiple data acquisition servers and the corresponding device to be acquired.

[0133] Referring to Figure 7, in this example, the data acquisition device includes:

[0134] The data acquisition module 10 is used to acquire the operating status of each data acquisition server;

[0135] The status determination module 20 is used to determine the abnormal server based on the running status data;

[0136] The data acquisition replacement module 30 is used to load the registration information of the abnormal server when an abnormal server exists, and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server in data acquisition.

[0137] In the technical solution of this application embodiment, multiple data acquisition servers share a backup server. The number of backup servers does not increase with the increase of the number of data acquisition servers, which can effectively control the number of backup servers, avoid resource waste, reduce hardware costs, and alleviate the workload of installation and deployment. At the same time, the data acquisition server can connect to multiple backup servers. Even if a backup server cannot be used normally, other backup servers can be used to quickly switch between primary and backup servers when the data acquisition server is abnormal, automatically taking over the data acquisition task of the abnormal server and improving reliability.

[0138] This application also proposes a data acquisition system, which includes a data acquisition server and a backup server. Multiple data acquisition servers are connected to at least one backup server through a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores the registration information of multiple data acquisition servers and the configuration information between the multiple data acquisition servers and the corresponding device to be acquired.

[0139] The data acquisition server is used to acquire data from the device to be acquired;

[0140] The backup server is used to obtain the operating status of each data acquisition server, determine the abnormal server based on the operating status data, load the registration information of the abnormal server, and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server to perform data acquisition.

[0141] This application also proposes a data acquisition device, which includes: a memory, a processor, and a data acquisition program stored in the memory and executable on the processor, the data acquisition program being configured to implement the steps of the data acquisition method described above.

[0142] This application also provides a storage medium storing a data acquisition program, which, when executed by a processor, implements the steps of the data acquisition method described above.

[0143] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0144] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0145] In addition, for technical details not described in detail in this embodiment, please refer to the data acquisition method provided in any embodiment of this application, which will not be repeated here.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A data acquisition method, applied to a backup server in a data acquisition system, wherein, The data acquisition system includes a data acquisition server and backup servers. Multiple data acquisition servers are connected to at least one backup server via a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores registration information of the multiple data acquisition servers and configuration information between the multiple data acquisition servers and the corresponding device to be acquired. The data acquisition method includes: Obtain the operational status data of each data acquisition server; Based on the aforementioned operational status data, identify the abnormal server; The system loads the registration information of the abnormal server and establishes a communication connection with the corresponding device to be collected based on the configuration information between the abnormal server and the device to be collected, so as to replace the abnormal server in collecting data.

2. The method as described in claim 1, wherein, The method further includes: Based on the data bus, the collected data from the multiple data acquisition servers are synchronized in real time.

3. The method as described in claim 2, wherein, After loading the registration information of the abnormal server and establishing a communication connection with the corresponding data collection device based on the configuration information between the abnormal server and the corresponding data collection device, in order to replace the abnormal server in data collection, the method further includes: When the abnormal server recovers, stop replacing the abnormal server to collect data; The collected data from the abnormal server, as well as the newly collected data acquired during the replacement process, are synchronized to the abnormal server so that the abnormal server can reconnect to the corresponding device to be collected for data collection.

4. The method of claim 1, wherein, The step of loading the registration information of the abnormal server also includes: If the abnormal server is not replaced, execute the step of loading the registration information of the abnormal server; When the abnormal server has been replaced, return to the step of obtaining the running status data of each data acquisition server.

5. The method of claim 1, wherein, The step of determining the abnormal server based on the operational status data includes: The data acquisition server whose operating status data meets the preset abnormal conditions is designated as the abnormal server.

6. The method of claim 5, wherein, The operational status data includes at least the most recent heartbeat time, and the steps for obtaining the operational status data of each data acquisition server include: Receive heartbeat information sent by each data acquisition server; Based on the heartbeat information of each data acquisition server, the most recent heartbeat time of each data acquisition server is determined.

7. The method according to any one of claims 1 to 6, wherein, Each device to be acquired uses an independent communication connection, and the method further includes: Get the connection status of an existing communication connection; When the existing communication connection is in a disconnected state, the communication connection with the corresponding device to be acquired is re-established to resume data acquisition.

8. A data acquisition device, wherein, The data acquisition device includes: The data acquisition module is used to acquire the operating status data of each data acquisition server; The status determination module is used to determine abnormal servers based on the running status data; The data acquisition replacement module is used to load the registration information of the abnormal server and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server in data acquisition.

9. A data acquisition system, wherein, The system includes a data acquisition server and a backup server. Multiple data acquisition servers are connected to at least one backup server via a data bus. The number of backup servers is less than the number of data acquisition servers. Each backup server stores the registration information of the multiple data acquisition servers and the configuration information between the multiple data acquisition servers and the corresponding device to be acquired. The data acquisition server is used to acquire data from the device to be acquired; The backup server is used to obtain the operating status data of each data acquisition server, identify abnormal servers based on the operating status data, load the registration information of the abnormal server, and establish a communication connection with the corresponding data acquisition device based on the configuration information between the abnormal server and the corresponding data acquisition device, so as to replace the abnormal server in data acquisition.

10. A data acquisition device, wherein, The device includes: a memory, a processor, and a data acquisition program stored in the memory and executable on the processor, the data acquisition program being configured to implement the steps of the data acquisition method as described in any one of claims 1 to 7.

11. A storage medium, wherein, The storage medium stores a data acquisition program, which, when executed by a processor, implements the steps of the data acquisition method as described in any one of claims 1 to 7.

Images