An optimization method and system for heartbeat data storage
By adding a local heartbeat cache on the server instance and using asynchronous threads to batch process non-expired heartbeat data, the server pressure problem caused by frequent heartbeat and message changes was solved, resulting in a reduction in the number of heartbeats sent and an improvement in system efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUJIAN TQ ONLINE INTERACTIVE INC
- Filing Date
- 2026-02-25
- Publication Date
- 2026-06-05
Smart Images
Figure CN122160431A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage technology, and in particular to an optimized method and system for storing heartbeat data. Background Technology
[0002] In today's internet systems, there are various apps, and each app may have push notification functionality. The principle behind push notifications is that each client device establishes a long-lived connection with a specific instance of the server. All subsequent messages are pushed through this long-lived connection. To ensure the long-lived connection remains valid and the client stays alive, a heartbeat mechanism is typically used between the client and server. The client periodically sends heartbeat packets to the server, which usually uses a distributed cache to store the heartbeat time information for each device. The heartbeat interval for each device is generally 1-4 minutes, and the heartbeat time needs to be updated synchronously whenever a message is sent or received. Currently, because distributed caching is used to store heartbeat messages, every change in heartbeat or message requires synchronous modification of the remote cache information. As the number of devices increases, the variables related to heartbeats and various messages will grow exponentially. For example, within a heartbeat cycle, one heartbeat packet contains four messages, which requires at least five information changes. If 1 million devices connect, there will be 5 million requests to change heartbeats in a short period of time, which will put some pressure on the server. Summary of the Invention
[0003] The purpose of this invention is to provide an optimized method and system for storing heartbeat data, which can effectively aggregate heartbeat packets and reduce the number of heartbeats sent.
[0004] The technical solution of this invention: An optimized method and system for storing heartbeat data, the method comprising the following steps: Step 1: After the client device establishes a long connection with a certain instance of the server, it needs to allocate a separate block of memory in that instance as a local heartbeat cache to store the heartbeat cache data. Step 2: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Step 3: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
[0005] Furthermore, step 2 is further specified as follows: when the client device receives a heartbeat request or a message change in a long-connection, it is sent to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is changed synchronously.
[0006] Furthermore, step 3 is further specified as follows: set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache; first, scan the heartbeat list page by page; if the heartbeat time has expired, close the long connection of the client device; if the heartbeat time has not expired, collect them uniformly and use a batch upload method to transmit the unexpired heartbeat cache data in batches to the unified cache on the server for persistent storage.
[0007] Furthermore, step 4 is included: setting up a scheduled task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.
[0008] An optimized system for storing heartbeat data, the system comprising a storage module, a sending module, and an asynchronous module; Storage module: After a client device establishes a long connection with a server instance, it needs to allocate a separate memory block in that instance as a local heartbeat cache to store heartbeat cache data; Sending module: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Asynchronous module: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
[0009] Furthermore, the sending module is further specified as follows: when the client device receives a heartbeat request or a message change in a long-connection, it sends the message to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is updated synchronously.
[0010] Furthermore, the asynchronous module is further specified as follows: an asynchronous thread is set up to periodically scan the heartbeat cache data in the local heartbeat cache; first, the heartbeat list is scanned page by page, and if the heartbeat time has expired, the long connection of the client device is closed; if the heartbeat time has not expired, the data is collected uniformly and uploaded in batches to the unified cache on the server for persistent storage.
[0011] Furthermore, it also includes a timed module: set up a timed task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.
[0012] The beneficial effects of this invention are: This invention employs a novel architecture to address this issue. On each access instance on the server side, a new heartbeat information storage cache is added. Each heartbeat and message sent by the client requires a synchronous update of the heartbeat timer in local memory, thus initially aggregating most of the heartbeat counts. Simultaneously, asynchronous threads are used for subsequent special processing. First, heartbeat data is fetched in batches from local memory, initially one record per device. If the heartbeat has expired, the instance can directly disconnect the long-lived connection; otherwise, heartbeat messages are sent in batches to a unified cache for storage. When the cached heartbeat expires, a notification message is synchronously sent back to the long-lived connection instance to disconnect. This approach effectively aggregates heartbeat packets, reducing the number of heartbeats sent. Furthermore, the heartbeat expiration function, previously handled by the unified cache, is now handled separately by the local and unified caches, further reducing data processing volume and improving overall system efficiency.
[0013] This invention can effectively aggregate heartbeat packets, reducing the number of heartbeats sent. Simultaneously, the heartbeat expiration function, previously handled by a unified cache, is now handled separately locally and in a unified cache, further reducing data processing volume and improving overall system efficiency. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a system principle block diagram of the present invention. Detailed Implementation
[0015] The invention will now be further described with reference to the accompanying drawings.
[0016] like Figure 1 This invention provides an optimized method for storing heartbeat data, the method comprising the following steps: Step 1: After the client device establishes a long connection with a certain instance of the server, it needs to allocate a separate block of memory in that instance as a local heartbeat cache to store the heartbeat cache data. Step 2: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Step 3: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
[0017] Step 2 is further specified as follows: when a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is changed synchronously.
[0018] Step 3 is further specified as follows: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache; first, scan the heartbeat list page by page; if the heartbeat time has expired, close the long connection of the client device; if the heartbeat time has not expired, collect them uniformly and use a batch upload method to transfer the unexpired heartbeat cache data to the unified cache on the server for persistent storage.
[0019] It also includes step 4: setting up a scheduled task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.
[0020] The present invention will be further described below with reference to a specific embodiment: An optimized method for storing heartbeat data, the method comprising the following steps: Step 1: After the client device establishes a long connection with a certain instance of the server, it needs to allocate a separate block of memory in that instance as a local heartbeat cache to store the heartbeat cache data. Example: In the old method: within one period of a client, say one minute, five messages will be sent, one of which is a heartbeat and four are other operation commands. At this time, five messages will arrive at a certain instance of the server within this period, and that instance will send five cached messages to the unified cache.
[0021] The new solution of this invention will allocate a special memory space in the instance. At this time, a device will only have 1 record in this memory space within a cycle.
[0022] Step 2: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; When a client device sends a heartbeat request or a message change for a long-lived connection, it sends the message to the local heartbeat cache. The local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time. At the same time, if the client device has message communication, the expiration time in the local heartbeat cache is updated synchronously.
[0023] Example: Key is device ID, value is heartbeat update time. In this case, 5 messages within this period would originally have 5 heartbeat changes, but now, each time only the local cache record inside the instance will be updated, that is, 5 heartbeats become 1 heartbeat.
[0024] Each time a command arrives, the value of the corresponding device ID is updated directly, that is, the update time is updated.
[0025] Step 3: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
[0026] Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache. First, scan the heartbeat list page by page. If the heartbeat time has expired, close the long connection on the client device. If the heartbeat time has not expired, collect them all at once and upload them in batches to the unified cache on the server for persistent storage.
[0027] For example, there is an asynchronous task that sends the device heartbeat information from each instance within each cycle to a unified cache center in batches.
[0028] Step 4: Set up a scheduled task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection on the client device.
[0029] For example, in a scheduled task, scan every cached record in the unified cache. When data with a value earlier than the current time is retrieved from the cache, notify these devices to close the client's long connection.
[0030] like Figure 2 The present invention also provides an optimized system for storing heartbeat data, the system comprising a storage module, a sending module, and an asynchronous module; Storage module: After a client device establishes a long connection with a server instance, it needs to allocate a separate memory block in that instance as a local heartbeat cache to store heartbeat cache data; Sending module: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Asynchronous module: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
[0031] The sending module is further specified as follows: when a heartbeat request or a message change occurs on a long-lived connection, it is sent to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is changed synchronously.
[0032] The asynchronous module is further specified as follows: an asynchronous thread is set up to periodically scan the heartbeat cache data in the local heartbeat cache; first, the heartbeat list is scanned page by page, and if the heartbeat time has expired, the long connection of the client device is closed; if the heartbeat time has not expired, it is collected uniformly and uploaded in batches to the unified cache on the server for persistent storage.
[0033] It also includes a timed module: set up a timed task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.
[0034] In summary, this invention effectively aggregates heartbeat packets, reducing the number of heartbeats sent. Simultaneously, the heartbeat expiration function, previously handled by a unified cache, is now handled separately locally and in a unified cache, further reducing data processing volume and improving overall system efficiency.
[0035] The above description is only a preferred embodiment of the present invention and should not be construed as a limitation of this application. All equivalent changes and modifications made in accordance with the scope of the patent application of the present invention should be covered by the present invention.
Claims
1. An optimized method for storing heartbeat data, characterized in that, The method includes the following steps: Step 1: After the client device establishes a long connection with a certain instance of the server, it needs to allocate a separate block of memory in that instance as a local heartbeat cache to store the heartbeat cache data. Step 2: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Step 3: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache and filter out the heartbeat cache data that has not expired; collect the heartbeat cache data that has not expired into a unified cache for storage.
2. The optimized method for storing heartbeat data according to claim 1, characterized in that, Step 2 is further specified as follows: when a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is changed synchronously.
3. The optimized method for storing heartbeat data according to claim 1, characterized in that, Step 3 is further specified as follows: Set up an asynchronous thread to periodically scan the heartbeat cache data in the local heartbeat cache; first, scan the heartbeat list page by page, and if the heartbeat time has expired, close the long connection of the client device; If the heartbeat time has not expired, it is collected in batches and uploaded to the unified cache on the server for persistent storage.
4. The optimized method for storing heartbeat data according to claim 1, characterized in that, It also includes step 4: setting up a scheduled task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.
5. An optimized system for storing heartbeat data, characterized in that, The system includes a storage module, a sending module, and an asynchronous module; Storage module: After a client device establishes a long connection with a server instance, it needs to allocate a separate memory block in that instance as a local heartbeat cache to store heartbeat cache data; Sending module: When a heartbeat request or a message change occurs on a client device, it is sent to the local heartbeat cache; Asynchronous module: Set up an asynchronous thread to periodically scan the heartbeat cache in the local heartbeat cache and filter out heartbeat cache data that has not expired; Unexpired heartbeat cache data is collected and stored in a unified cache.
6. The optimized system for storing heartbeat data according to claim 5, characterized in that, The sending module is further specified as follows: when a heartbeat request or a message change occurs on a long-lived connection, it is sent to the local heartbeat cache; the local heartbeat cache is in key-value format, where the key is the device ID and the value is the expiration time; at the same time, if the client device has message communication, the expiration time in the local heartbeat cache is changed synchronously.
7. An optimized system for storing heartbeat data according to claim 5, characterized in that, The asynchronous module is further specified as follows: an asynchronous thread is set up to periodically scan the heartbeat cache data in the local heartbeat cache; first, the heartbeat list is scanned page by page, and if the heartbeat time has expired, the long connection of the client device is closed; If the heartbeat time has not expired, it is collected in batches and uploaded to the unified cache on the server for persistent storage.
8. An optimized system for storing heartbeat data according to claim 5, characterized in that, It also includes a timed module: set up a timed task to periodically scan the heartbeat cache data in the unified cache. When expired heartbeat cache data is detected, the corresponding instance will be notified to close the long connection of the client device.