Acquisition gateway load peak shifting transfer method for stable acquisition period

Abstract

The invention provides an acquisition gateway load off-peak transfer method for a stable acquisition period, and belongs to the technical field of the Internet of Things, the acquisition gateway load off-peak transfer method for the stable acquisition period is used for transferring an acquisition gateway load between different servers and ensuring the acquisition period to be stable, and the acquisition gateway load off-peak transfer method for the stable acquisition period can be used for transferring the acquisition gateway load between different servers. The main process is as follows: receiving and collecting a gateway load transfer demand; synchronizing the time of the plurality of servers by using a time synchronization algorithm; determining a target server of load transfer by using a load transfer negotiation algorithm; according to the method, an acquisition gateway load transfer negotiation and peak shifting method is designed, system operation data are analyzed by using the method, and a load transfer scheme is obtained, so that peak shifting and transfer of the acquisition gateway load are realized on the premise of ensuring the stability of an acquisition period; and the load bearing capacity of the hardware system in the parallel state of the multiple acquisition gateways is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular, relates to a peak-shifting method for a collection gateway load facing a stable collection cycle. Background Art

[0002] With the continuous improvement of industrial automation and intelligence, the application of IoT technology in industrial environments has become increasingly widespread, especially in the field of data collection. IoT devices can monitor and collect various data on the production line in real time, which is essential for optimizing production processes, improving efficiency and ensuring the safety of equipment operation. As the number of IoT devices increases, the scale of data collection also expands accordingly, which requires more efficient data processing and management systems to cope with the pressure of data collection and transmission.

[0003] In order to effectively manage and process this huge amount of data, a distributed deployment strategy is usually adopted to deploy multiple data collection services on multiple servers to improve the service capacity of the entire system. However, this deployment method also brings new challenges. Since data collection services have special peak loads, when the load peaks of multiple collection services that can be processed under normal circumstances are unexpectedly aligned, they will require far more processing power than the average required by the collection service during operation, which may cause collection delays in the entire data collection service, thereby affecting the integrity and accuracy of production data.

[0004] In the face of similar distributed deployment scenarios, the Internet industry has developed some load balancing and failover technologies. Through these technologies, when a server fails, tasks can be automatically transferred to other healthy servers to ensure the availability of specific services. However, these load scheduling algorithms designed to maximize throughput are not fully applicable to industrial environments. Load scheduling in industrial environments needs to pay more attention to real-time processing and synchronization of data. Therefore, developing gateway load transfer algorithms that are more suitable for industrial environments to meet the special needs of industrial data collection has become a technical problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a method for staggered load transfer of collection gateways for a stable collection cycle, which solves the problem that the current commonly used load transfer algorithms in the Internet industry ignore the requirements for real-time processing and precise synchronization of data in the industrial Internet of Things environment. At the same time, it improves resource utilization and optimizes the reliability of the overall system.

[0006] The present invention is achieved in that: The present invention provides a collection gateway load peak shifting method for a stable collection cycle, wherein the collection gateway load shifting method is used to transfer the collection gateway load between different servers and ensure the stability of the collection cycle. The main process is as follows: Step 1: Receive the load transfer demand of the acquisition gateway; Step 2: Use the time synchronization algorithm to synchronize the time of multiple servers; Step 3: Use the load transfer negotiation algorithm to determine the target server for load transfer; Step 4: Use the load peak shifting algorithm to determine the operation method of the gateway load after the transfer.

[0007] On the basis of the above technical solution, the load peak shifting method of the acquisition gateway for a stable acquisition cycle of the present invention can also be improved as follows: The acquisition gateway is a software program that runs on a computer that meets the hardware requirements, is initialized according to the configuration file, establishes a connection with the sensor device according to the device connection information in the configuration file, and performs a specific data acquisition task. The main process of the acquisition task includes generating acquisition commands or sensor device control commands, acquiring the original data of the corresponding device or controlling the configuration or status of the corresponding device, parsing the returned original data or status data according to the agreed format, and transmitting the parsed data through the agreed network protocol.

[0008] The computer that meets the hardware requirements refers to a computer that uses a CPU with an X86, AMD64, or ARM instruction set and runs a Windows or Linux operating system.

[0009] Furthermore, the acquisition gateway load refers to the occupation of the hardware resources of the server on which the acquisition gateway is located when executing the acquisition task, including CPU occupation, memory occupation and network occupation.

[0010] The servers include three types: running servers, hot backup servers and idle servers. To implement the "collection gateway load peak shifting method", there needs to be at least one running server, plus 0 to any number of hot backup servers and idle servers.

[0011] The running server is a server that runs a collection gateway under normal circumstances, on which one or more collection gateways run, and each collection gateway performs its own collection task.

[0012] The hot standby server is a server running a collection gateway in a hot standby state, and is used to reduce the time consumption of transferring the load from the collection gateway on the running server.

[0013] An idle server is a server that does not run a collection gateway. It is used to ensure the normal operation of the collection gateway when the collection gateway load exceeds the hardware resource upper limit of the running server and the hot backup server.

[0014] The time synchronization algorithm runs on a specific acquisition gateway, which provides services to other acquisition gateways through the network. The service return value is the difference between the timestamp sent by the requester and the local timestamp.

[0015] The load transfer negotiation algorithm determines the target gateway according to the load transfer request of the transfer-out gateway. The transfer-out gateway refers to the collection gateway that needs to transfer the load, and the target gateway refers to the collection gateway that has not executed the collection task.

[0016] The load transfer negotiation algorithm includes the following steps: Step 1: The outgoing gateway broadcasts a load transfer request to all target gateways; Step 2: After receiving the load transfer request, the target gateway determines whether to accept the load of the collection gateway based on the current server resource occupancy and the pre-collected operation resource occupancy data, and returns the judgment result to the outgoing gateway; Step 3: The outgoing gateway randomly selects a target gateway as the load transfer object from all target gateways that agree to receive the load.

[0017] The load transfer request refers to the resource occupancy data collected by the collection gateway when the current collection task is running, including the maximum CPU occupancy and required time when the collection gateway is initialized, and the maximum CPU occupancy and required time under formal operation.

[0018] The "judgment" operation in step 2 is to compare whether the remaining available resources are greater than the resources required by the target gateway. The specific values ​​of the required resources here can be from the following two sources: 1. If a certain collection task in the pre-collected running resource occupancy data can be located according to the received load transfer request data, the located pre-collected data is used for comparison and judgment.

[0019] 2. If the data of the received load transfer request cannot be located to a certain collection task in the pre-collected running resource occupancy data, the data carried in the load transfer request is used for comparison and judgment.

[0020] The judgment operation in step 2 of the load negotiation algorithm refers to comparing whether the remaining available resources of the server where the current target gateway is running are greater than the resources required for the target gateway to run. The resource data required for the target gateway to run comes from the following two sources: Source 1: If a collection task in the pre-collected running resource occupancy data can be located according to the received load transfer request data, the located pre-collected resource data is used as the data source; Source 2: If a certain collection task in the pre-collected running resource occupancy data cannot be located according to the data of the received load transfer request, the resource data carried in the load transfer request is used as the data source.

[0021] The load peak shifting algorithm is that the target gateway calculates the high resource occupancy peak shifting solution according to the load transfer request sent by the outgoing gateway, and returns the solution to the target gateway. The high resource occupancy peak shifting solution refers to the situation that multiple acquisition gateways can run at the same time. The load peak shifting algorithm includes the following steps: Step 1: The load transfer gateway sends the real-time resource occupancy data in operation to the target gateway through the network according to the agreed data format; Step 2: The target gateway forwards the real-time resource occupancy data of the outgoing gateway to the load peak shifting service running on the current server; Step 3: The load peak shifting service calculates whether there are peak shifting solutions with high resource usage for different acquisition gateways based on the resource usage on the current server, and returns the result to the target gateway; Step 4: The target gateway decides whether to accept the gateway load based on the result. If not, it requests the outgoing gateway to perform load negotiation again.

[0022] The pre-collected resource usage data refers to the resource usage collected by the collection gateway when executing different collection tasks, including the maximum CPU usage and time required when the collection gateway is initialized, and the maximum CPU usage and time required during formal operation.

[0023] Compared with the prior art, the method for peak-shifting the load of a collection gateway for a stable collection cycle provided by the present invention has the following beneficial effects: Improve the stability of the collection cycle: By dynamically transferring the collection gateway load, the stability of the collection cycle is ensured to avoid collection cycle fluctuations caused by uneven load distribution; Optimize resource utilization: through the load peak shifting algorithm, reasonably allocate server resources, improve the utilization of server hardware resources, and reduce resource waste; Optimize the load transfer process: Through the time synchronization algorithm and load transfer negotiation algorithm, the load transfer process is simplified and the interference to the collection cycle during the load transfer process is stabilized; Enhanced system scalability: The combination of hot backup servers and idle servers enhances system scalability and enables flexible response to changes in the collection gateway load. Improve system reliability: By combining pre-collected resource occupancy data with real-time resource occupancy data, the reliability of the system is improved, ensuring the accuracy and stability of load transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0025] Figure 1 This is an example diagram of basic information of the operating environment of a method for peak-shifting the load of a collection gateway for a stable collection cycle in an embodiment of the present invention; Figure 2 is an interaction timing diagram of a load transfer negotiation algorithm in an embodiment of the present invention; Figure 3 It is an interactive timing diagram of a load peak shifting algorithm with a peak shifting solution in an embodiment of the present invention; Figure 4 It is an interaction timing diagram of a load peak shifting algorithm without a peak shifting solution in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0027] In this embodiment, a specific implementation process of a collection gateway load peak shifting method for a stable collection cycle is described. The method uses a time synchronization algorithm, a load negotiation algorithm, and a load peak shifting algorithm to ensure the stability of the collection cycle when the collection gateway load is dynamically transferred between different servers.

[0028] The acquisition gateway is a software program that runs on a computer that meets the hardware requirements. It is initialized according to the configuration file, establishes a connection with the sensor device according to the device connection information in the configuration file, and performs specific data acquisition tasks. The main process of the acquisition task includes generating acquisition commands or sensor device control commands, collecting the original data of the corresponding device or controlling the configuration or status of the corresponding device, parsing the returned original data or status data according to the agreed format, and transmitting the parsed data through the agreed network protocol. The acquisition gateway load refers to the occupation of the hardware resources of the server on which the acquisition gateway is located when executing the acquisition task, including CPU occupation, memory occupation, and network occupation.

[0029] In this embodiment, if Figure 1As shown, the server includes three types: running server, hot standby server and idle server. The capital letters AG are used to uniquely identify an independent computer. The lowercase letters plus numbers a1-a4, b1, c1-c3, d1-d2, e1-e4, f1 are used to uniquely identify a collection gateway program. The running server is a server that runs the collection gateway under normal circumstances. One to multiple collection gateways are run on it, and each collection gateway performs different collection tasks. The hot standby server is a server that runs the collection gateway in the hot standby state, which is used to reduce the time spent on transferring the load from the collection gateway on the running server. The idle server is a server that does not run the collection gateway. When the collection gateway load exceeds the upper limit of the hardware resources of the running server and the hot standby server, the idle server is used to ensure the normal operation of the collection gateway.

[0030] In this embodiment, if Figure 1 As shown in the figure, the time synchronization algorithm runs on a specific collection gateway. In this figure, the time synchronization service runs on the collection gateway program b1. The collection gateway provides services to other collection gateways through the network. The service return value is the difference between the timestamp sent by the requester and the local timestamp. The specific implementation steps are as follows: Step 1: Randomly select a collection gateway to start the time synchronization service and listen to time synchronization requests from other collection gateways.

[0031] Step 2: Other collection gateways send a time synchronization request to a specific collection gateway, and the request includes a local timestamp.

[0032] Step 3: After receiving the request, the specific collection gateway records the local timestamp and calculates the difference between the timestamp sent by the requester and the local timestamp.

[0033] Step 4: The specific collection gateway returns the calculated difference to the requesting collection gateway.

[0034] In this embodiment, if Figure 2 As shown in the figure, the role of the load transfer negotiation algorithm is to determine the target gateway based on the load transfer request of the outgoing gateway. The outgoing gateway refers to the collection gateway that needs to transfer the load, and the target gateway refers to the collection gateway that has not performed the collection task. The main process of the load transfer negotiation algorithm is as follows: Step 1: The outgoing gateway broadcasts a load transfer request to all target gateways, and the request contains the resource occupancy data of the current acquisition gateway.

[0035] Step 2: After receiving the load transfer request, the target gateway determines whether to agree to receive the collection gateway load based on the current server resource occupancy and the pre-collected operation resource occupancy data, and returns the judgment result to the outgoing gateway.

[0036] Step 3: The outgoing gateway randomly selects a target gateway as the load transfer object from all target gateways that agree to receive the load.

[0037] Since the outgoing gateway sends the load transfer request to other target gateways in the form of broadcast, the multiple target gateways will return the load transfer approval requests in a certain order due to the order in which they receive the messages and the different speeds at which they execute the code. The outgoing gateway will select the target gateway that returns the unified load transfer request first as the subsequent target gateway and discard the load transfer approval request messages received later.

[0038] In this embodiment, the role of the load peak-shifting algorithm is to calculate whether there is a peak-shifting scheme that can stagger the moments of high performance requirements for different acquisition gateways during operation, based on the result of the load negotiation algorithm, when it is known that the remaining computing performance of the system can support the operation of the transferred acquisition gateway, according to the resource occupancy of the server where the target gateway is currently located, and return this scheme to the target gateway.

[0039] like Figure 3 As shown in the figure, it is the main process of the load negotiation algorithm in the case of peak shifting scheme: Step 1: The outgoing gateway sends the real-time resource occupancy data in operation to the target gateway through the network according to the agreed data format.

[0040] Step 2: The target gateway forwards the real-time resource occupancy data of the outgoing gateway to the load shifting service running on the current server.

[0041] Step 3: The load peak shifting service calculates whether there are high resource occupancy peak shifting solutions for different acquisition gateways based on the resource occupancy on the current server, and returns the result to the target gateway.

[0042] Step 4: The target gateway receives the load of the collection gateway according to the returned peak-shifting plan, executes the collection task, and notifies the outgoing gateway to agree to receive the load of the collection gateway.

[0043] like Figure 4 As shown in the figure, it is the main process of the load negotiation algorithm when there is no peak-shifting scheme: Step 1: The outgoing gateway sends the real-time resource occupancy data in operation to the target gateway through the network according to the agreed data format.

[0044] Step 2: The target gateway forwards the real-time resource occupancy data of the outgoing gateway to the load shifting service running on the current server.

[0045] Step 3: The load peak shifting service calculates whether there are high resource occupancy peak shifting solutions for different acquisition gateways based on the resource occupancy on the current server, and returns the result to the target gateway.

[0046] Step 4: Since there is no available peak-shifting solution, the target gateway refuses to receive the gateway load and notifies the outgoing gateway to re-execute the load negotiation operation.

[0047] Through the above embodiments, the present invention provides a method for peak-shifting the load of a collection gateway for a stable collection cycle. By combining a time synchronization algorithm, a load negotiation algorithm and a load peak-shifting algorithm, the collection gateway load ensures the stability of the collection cycle when performing load transfer according to actual needs, optimizes the utilization of server resources, and enhances the scalability and reliability of the system.

Claims

1. A method for peak-shifting the load of a collection gateway for a stable collection cycle, characterized in that: The acquisition gateway load transfer method is used to transfer the acquisition gateway load between different servers and ensure the stability of the acquisition cycle. The main process of the acquisition gateway load transfer method is as follows: Step 1: Receive the load transfer demand of the acquisition gateway; Step 2: Use the time synchronization algorithm to synchronize the time of multiple servers; Step 3: Use the load transfer negotiation algorithm to determine the target server for load transfer; Step 4: Use the load peak shifting algorithm to determine the operation method of the gateway load after the transfer.

2. According to the method for peak-shifting load of a collection gateway for a stable collection cycle according to claim 1, it is characterized in that: The acquisition gateway is a software program that runs on a computer that meets hardware requirements, is initialized according to a configuration file, establishes a connection with a sensor device according to device connection information in the configuration file, and executes a specific data acquisition task.

3. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 2 is characterized in that: The main process of the collection task is as follows: Step 1: Generate acquisition commands or sensor device control commands; Step 2: Collect the original data of the corresponding device, or control the configuration or status of the corresponding device; Step 3: Parse the returned raw data or status data according to the agreed format; Step 4: Transmit the parsed data via the agreed network protocol.

4. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 3 is characterized in that: The collection gateway load refers to the occupation of the hardware resources of the server where the collection gateway is located when the collection gateway executes the collection task, including CPU occupation, memory occupation and network occupation.

5. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 4 is characterized in that: The servers include three types, namely, running servers, hot standby servers and idle servers; The operation server is a server that runs a collection gateway under normal circumstances. One to multiple collection gateways are run on the operation server, and each collection gateway performs different collection tasks; The hot standby server is a server running a collection gateway in a hot standby state, and is used to reduce the time consumption of transferring the load from the collection gateway on the running server. The hot standby state refers to a state in which the collection gateway has been initialized according to the configuration file, but has not established a connection with the sensor device; The idle server is a server that does not run the acquisition gateway. When the acquisition gateway load exceeds the upper limit of the hardware resources of the running server and the hot backup server, or when the acquisition demand is elastically expanded, the idle server is used to ensure the normal operation of the acquisition gateway; The implementation of the peak-shifting method for the acquisition gateway load facing the stable acquisition cycle requires at least one running server, and any number of hot backup servers and idle servers.

6. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 5, characterized in that: The time synchronization algorithm runs on a specific acquisition gateway, and the acquisition gateway provides services to other acquisition gateways through the network. The service returns the difference between the timestamp sent by the requester and the local timestamp.

7. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 6, characterized in that: The load transfer negotiation algorithm is used to determine the target gateway according to the load transfer request of the transfer-out gateway; the transfer-out gateway refers to the collection gateway that needs to transfer the load; the target gateway refers to the collection gateway that has not performed the collection task; The main process of the load transfer negotiation algorithm is: Step 1: The outgoing gateway broadcasts a load transfer request to all target gateways; Step 2: After receiving the load transfer request, the target gateway determines whether the current server can provide sufficient system resources for the target gateway to run according to the current server resource occupancy and the pre-collected operation resource occupancy data, and returns the judgment result to the outgoing gateway; Step 3: The outgoing gateway randomly selects a target gateway as the load transfer object from all target gateways that agree to receive the load.

8. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 7, characterized in that: The main process of the load peak shifting algorithm is: Step 1: The outgoing gateway sends the real-time resource occupancy data in operation to the target gateway through the network according to the agreed data format; Step 2: The target gateway forwards the real-time resource occupancy data of the outgoing gateway to the load peak shifting service running on the current server; Step 3: The load peak shifting service calculates whether there are peak shifting solutions with high resource usage for different acquisition gateways based on the resource usage on the current server, and returns the result to the target gateway; Step 4: The target gateway decides whether to accept the gateway load based on the result. If not, it requests the outgoing gateway to perform load negotiation again.

9. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 8, characterized in that: The load transfer request refers to the resource occupancy data collected by the collection gateway when the current collection task is running, including the maximum CPU occupancy and required time when the collection gateway is initialized, and the maximum CPU occupancy and required time when it is officially running.

10. The method for peak-shifting the load of a collection gateway for a stable collection cycle according to claim 9, characterized in that: The pre-collected resource occupancy data refers to the resource occupancy collected by the collection gateway when executing different collection tasks, including the maximum CPU occupancy and required time when the collection gateway is initialized, and the maximum CPU occupancy and required time when it is officially running.

Images