Distributed WebSocket Cluster Message Push Method and System
By employing a two-stage routing query and Redis pipeline mode, the problems of resource waste and high latency in distributed WebSocket systems are solved, enabling efficient and accurate message push and session management, and improving the system's throughput and scalability.
Patent Information
- Application Number
- CN202611632328.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
In distributed WebSocket systems, existing technologies suffer from problems such as resource waste, high network latency, low session query efficiency, and incomplete cleanup of expired sessions. In particular, they are difficult to achieve accurate message push and efficient resource utilization in large-scale cluster scenarios.
A two-stage routing query mechanism is adopted, which registers server node information and session identifiers through Redis, and combines scheduled tasks and Redis pipeline mode to achieve efficient session management and message push, ensuring that messages reach the target client accurately.
It enables precise message delivery, reduces network latency and resource consumption, improves system throughput and query efficiency, and supports high availability and scalability of the cluster.
Smart Images

Figure CN123123429A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, specifically to a distributed WebSocket cluster message push method and system. Background Technology
[0002] WebSocket, as a full-duplex communication protocol, is widely used in real-time message push scenarios such as medical information systems. However, when the business scale is large, a single WebSocket server cannot handle a large number of concurrent connections. It is necessary to deploy the WebSocket service as a cluster consisting of multiple server nodes and distribute the client's connection requests to each server node. In a cluster environment, multiple client connections for the same business object are often scattered across different server nodes. How to accurately push business messages to the server node where the target client is located is an important technical problem faced by distributed WebSocket systems.
[0003] In existing technologies, the more common solutions fall into three categories: The first category is broadcast push, where all WebSocket server nodes subscribe to the same message queue. Each node, upon receiving a message, determines whether the target client is connected to this node. If so, it pushes the message; otherwise, it discards it. The second category is push based on Redis Pub / Sub, which transmits messages between server nodes through Redis's publish / subscribe mechanism. The third category is single-level mapping routing, where Redis only stores the mapping from the business identifier to the server node, allowing the message consumer to directly locate the server node based on this mapping.
[0004] However, the aforementioned existing technologies typically have the following problems:
[0005] First, both broadcast push and push based on Redis Pub / Sub will distribute the same message to all server nodes in the cluster. Each message is processed by all nodes. As the number of nodes increases, the consumption of network bandwidth and computing resources increases linearly, which can easily lead to resource waste.
[0006] Second, the establishment, querying and cleanup of a session all require multiple interactions with Redis. A single session establishment or a single batch cleanup often involves multiple interrelated write or delete commands. Frequent read and write operations on Redis, and the cumulative network round-trip time (RTT) when sending commands one by one, directly affect the system's throughput.
[0007] Third, a single-level mapping scheme that only stores the mapping relationship between business identifiers and server nodes cannot accurately route clients corresponding to a certain business identifier across multiple server nodes; at the same time, the query efficiency of the mapping relationship between session identifiers and server nodes is low, making it difficult to meet the low latency requirements in large-scale cluster scenarios.
[0008] Fourth, the lack of an effective cleanup mechanism for invalid sessions leads to the accumulation of a large amount of invalid mapping data in Redis, which consumes storage resources and affects query efficiency. Summary of the Invention
[0009] The purpose of this invention is to provide a distributed WebSocket cluster message push method and system to solve the problems mentioned in the background art.
[0010] To achieve the above objectives, the present invention provides the following technical solution:
[0011] Distributed WebSocket cluster message push methods include:
[0012] Build a WebSocket server cluster. When each server node starts up, it registers its service IP address and node status information with Redis.
[0013] After the client establishes a WebSocket connection with any WebSocket server node, the node generates a unique session identifier and stores the mapping relationship in batches to Redis. The mapping relationship includes a first mapping relationship from the business identifier set to the session identifier and a second mapping relationship from the session identifier to the server node IP.
[0014] The consumer consumes business messages from the cluster message queue and parses the target business identifier.
[0015] Based on the target service identifier, a two-stage routing query is performed through the first mapping relationship and the second mapping relationship to obtain the target server IP list;
[0016] Based on each IP in the target server IP list, push the business message to the corresponding client;
[0017] Use scheduled tasks to periodically check and clean up failed sessions.
[0018] Furthermore, it registers the service IP address and node status information of this node with Redis, including:
[0019] Store node information in Redis using the server node IP as the key and the node status information as the value, and set an expiration time.
[0020] The node status information includes at least one of the following: node IP, startup time, and current number of connections.
[0021] Furthermore, the session identifier is a globally unique identifier, generated by a combination of the server node IP, timestamp, and random number, or generated using a UUID;
[0022] A service identifier is an identification information that can uniquely identify the service recipient, and includes at least one of the following: user ID, device ID, hospital ID, or department ID.
[0023] Furthermore, the first mapping relationship is stored in Redis in a set structure, with the key being the business identifier and the value being the set of all session identifiers corresponding to that business identifier;
[0024] The second mapping relationship is stored in Redis in the form of a hash structure or key-value pairs, where the key is the session identifier and the value is the server node IP and the session establishment time.
[0025] The Redis pipeline mode is used to perform batch write operations that store the first and second mapping relationships in Redis, and to package multiple Redis commands into a single network request for sending.
[0026] Furthermore, the two-stage routing query includes:
[0027] The first stage of routing involves querying the first mapping relationship stored in Redis based on the business identifier to obtain all session identifiers corresponding to that business identifier.
[0028] The second-stage routing involves querying the second mapping relationship stored in Redis based on the session identifiers obtained in the first-stage routing to obtain the server node IP corresponding to each session identifier.
[0029] The server node IPs obtained in the second stage of routing are deduplicated to obtain a list of target server node IPs that need to be pushed.
[0030] Preferably, pushing business messages to the corresponding clients based on each IP in the target server IP list includes:
[0031] The corresponding server node is called through the HTTP interface. The HTTP interface is an internal interface exposed by each WebSocket server node to receive message push requests. The request parameters of the HTTP interface include at least the target session identifier and the content of the message to be pushed.
[0032] After receiving the HTTP call request, the target server node looks up the corresponding WebSocket session object in its local memory based on the target session identifier;
[0033] If a session object exists in local memory and the session is active, the message is pushed to the client through the WebSocket sending method of the session object.
[0034] If the session object does not exist in local memory or the session has expired, a failure response is returned, and session cleanup is triggered by the caller or a scheduled task.
[0035] Furthermore, it also includes a reliability guarantee mechanism for push notifications:
[0036] When a call to the target server node via the HTTP interface fails, a retry is performed. Once the maximum number of retries is reached, the failure message is recorded and stored in the dead letter queue.
[0037] Record the complete chain of messages from message queue consumption to final push to the client.
[0038] Preferably, periodically checking and cleaning up failed sessions using scheduled tasks includes:
[0039] Set the execution cycle of the scheduled task to periodically scan the session mapping relationships stored in Redis;
[0040] For each session identifier, check the server node IP mapping to see if the WebSocket connection corresponding to the session is still active and identify invalid sessions.
[0041] If the corresponding WebSocket connection has been actively closed by the client, the corresponding server node has crashed or is unreachable, the session has not had any message exchange for more than the preset maximum idle time, or the connection has been confirmed to be interrupted through heartbeat detection, then it is judged as an invalid session.
[0042] The heartbeat detection process includes the client periodically sending heartbeat messages to the server; the server updating the last active time of the session in Redis after receiving the heartbeat message; and a scheduled task scanning the last active time of the session and marking sessions that exceed a preset timeout threshold as invalid.
[0043] For expired sessions, perform expired session cleanup operations;
[0044] For sessions deemed invalid, logs are recorded and alerts are triggered before cleanup.
[0045] Furthermore, the mapping relationship also includes a third mapping relationship from the set of server node IPs to the session identifier, and a fourth mapping relationship from the session identifier to the service identifier;
[0046] The third mapping relationship uses the server node IP as the key and the set of all session identifiers carried by the server node as the value. The fourth mapping relationship is the mapping relationship between session identifiers and business identifiers, using the session identifier as the key and the business identifier associated with the session identifier as the value. The third and fourth mapping relationships are written to Redis in batches using Redis pipeline mode.
[0047] Expired session cleanup operations include:
[0048] Based on the third mapping relationship, query the server node IP information using the session identifier, delete the corresponding session identifier from the set of server node IPs in the third mapping relationship, and delete the corresponding second mapping relationship;
[0049] Based on the fourth mapping relationship, query the service identifier using the session identifier, delete the corresponding session identifier in the service identifier set in the first mapping relationship, and delete the fourth mapping relationship.
[0050] The reason for the failed session is that the corresponding server node has been detected as down or unreachable. Based on the third mapping relationship, using the determined server node IP, all session identifiers under the server node IP are queried, the second mapping relationship corresponding to the session identifier is deleted, and then based on the fourth mapping relationship, the corresponding service identifier is queried, the corresponding fourth mapping relationship is deleted, and based on the queried service identifier, the corresponding session identifier in the service identifier set in the first mapping relationship is deleted.
[0051] A distributed WebSocket cluster message push system, which includes a WebSocket server cluster module, a Redis session storage module, a message consumption module, a two-phase routing query module, an HTTP call module, and a session cleanup module;
[0052] The WebSocket server cluster module contains multiple server nodes, each node is used to manage local sessions, establish WebSocket connections with clients, and expose HTTP push interfaces;
[0053] The Redis session storage module stores the mapping relationship between business identifiers and session identifier sets, the mapping relationship between session identifiers and server node IPs, server node registration information, and supports batch operations in pipeline mode.
[0054] The consumer consumes business messages from the cluster message queue and parses the target business identifier.
[0055] The two-stage routing query module is used to query Redis to obtain a list of session identifiers based on the business identifier, and then query the list of target server node IPs based on the list of session identifiers.
[0056] The HTTP call module is used to call the WebSocket send method of the corresponding server node through the HTTP interface based on the list of target server node IPs;
[0057] The session cleanup module is used to periodically check and clean up invalid session mappings in Redis via scheduled tasks.
[0058] Compared with the prior art, the beneficial effects of the present invention are:
[0059] 1. Two-stage routing lookup is performed through the first and second mapping relationships. The session identifier is queried based on the business identifier, and then the server node IP is queried based on the session identifier. This helps to accurately push messages to the server node where the target client is located, avoiding the resource waste of all nodes processing the same message in broadcast push. Moreover, the two-level mapping design can support multiple client sessions corresponding to the same business identifier and multiple service identifier sessions on the same server node, making the routing granularity flexible and controllable.
[0060] 2. By registering the service IP address and node status information of this node with Redis, the Redis pipeline mode is used in the two high-frequency stages of session establishment and invalid session cleanup. Multiple commands are packaged into a single network request, and multiple network round trips are merged into a single network round trip, which helps to reduce network round trip latency and improve system throughput.
[0061] 3. By using scheduled tasks in conjunction with Redis pipeline mode to batch clean up invalid sessions, it is beneficial to avoid the accumulation of invalid data in Redis, ensure the freshness of the mapping data and query efficiency, and ensure cross-dimensional consistency of the mapping data.
[0062] 4. By adopting the consumer group mode and the Redis pipeline mode, the WebSocket server cluster supports horizontal scaling. New nodes only need to register with Redis to join the cluster. The consumer group mode supports horizontal scaling of the consumer side, and Redis, as a centralized storage, supports cluster deployment. The overall architecture has good high availability and scalability. Attached Figure Description
[0063] Figure 1 This is a schematic diagram of the overall process of the method in this invention;
[0064] Figure 2 This is a block diagram of WebSocket connection establishment and session mapping storage in this invention;
[0065] Figure 3 This is a schematic diagram of the two-stage routing query process in this invention;
[0066] Figure 4 This is a flowchart of the WebSocket connection establishment process in this invention;
[0067] Figure 5 This is the overall system block diagram of the present invention. Detailed Implementation
[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0069] Please see Figures 1-5 In this embodiment of the invention, the distributed WebSocket cluster message push method includes steps S1 to S7;
[0070] Step S1: Build a WebSocket server cluster. When each server node starts up, it registers its service IP address and node status information with Redis.
[0071] This involves deploying a WebSocket server cluster, which contains multiple server nodes, each of which can be deployed independently on different physical machines or containers.
[0072] Step S2: After the client establishes a WebSocket connection with any WebSocket server node, the node generates a unique session identifier and stores the mapping relationship in batches to Redis;
[0073] Among them, the session identifier is a globally unique identifier, generated by combining the server node IP, timestamp, and random number, or generated using UUID; the service identifier is identification information that can uniquely identify the service recipient, including at least one of the following: user ID, device ID, hospital ID, and department ID;
[0074] In the application scenarios of medical information systems, the preferred business identifier is the hospital ID, and one hospital ID can correspond to one or more client sessions;
[0075] The mapping relationship includes a first mapping relationship from the set of business identifiers to the session identifiers and a second mapping relationship from the session identifiers to the server node IP. The first mapping relationship is stored in Redis in a set structure, with the key being the business identifier and the value being the set of all session identifiers corresponding to that business identifier. The second mapping relationship is stored in Redis in a hash structure or in key-value pair form, with the key being the session identifier and the value being the server node IP and the session establishment time.
[0076] In the first mapping relationship, one set of service identifiers corresponds to multiple session identifiers; in the second mapping relationship, one session identifier corresponds to one server node IP.
[0077] Step S3: Use Redis pipeline mode to perform batch write operations to store the first mapping relationship and the second mapping relationship in Redis, and package multiple Redis commands into a single network request for sending.
[0078] During the session establishment phase, in addition to the mapping relationship, metadata such as session creation time and business identifier can also be written and packaged into a single pipeline command. During the scheduled cleanup phase, each time the scheduled task scans and finds a batch of invalid sessions, it packages all deletion operations (DEL and SREM) corresponding to a batch of sessions into a single pipeline command and sends it to Redis.
[0079] By using Redis pipeline mode to perform the above write operations in batches, multiple Redis commands can be packaged into a single network request, which can significantly reduce network round-trip time (RTT) and improve the efficiency of session establishment.
[0080] Step S4: The consumer consumes business messages from the cluster message queue and parses the target business identifier;
[0081] The consumer side is deployed in the form of consumer groups. Each consumer group independently consumes business messages from the topic of the message queue. The business messages are generated by the upstream business system and sent to the message queue. The messages contain at least the target business identifier and the message body to be pushed.
[0082] The consumer uses a batch retrieval method to obtain messages from the message queue in order to improve consumption efficiency. For the batch of messages retrieved, the consumer processes each message in a loop and parses out the target business identifier and message body.
[0083] The consumer group mode supports horizontal scaling of the consumer side, and Redis, as a centralized storage, supports cluster deployment. The overall architecture has good high availability and scalability.
[0084] Step S5: Based on the target service identifier, perform a two-stage routing query through the first mapping relationship and the second mapping relationship to obtain the target server IP list;
[0085] Through a two-stage routing query, precise positioning is achieved from the business dimension to the session dimension, and finally to the server dimension, ensuring that each message is only pushed to the server IP node that actually holds the target client.
[0086] Step S6: Push the business message to the corresponding client according to each IP in the target server IP list;
[0087] Step S7: Periodically check and clean up invalid sessions using a scheduled task;
[0088] Specifically, a scheduled task is used to periodically check the session mapping relationships stored in Redis, and to identify and clean up invalid sessions.
[0089] The WebSocket server cluster supports horizontal scaling; new nodes only need to register with Redis to join the cluster. The consumer group mode supports horizontal scaling of the consumer side. Redis, as a centralized storage, supports cluster deployment, and the overall architecture has good high availability and scalability.
[0090] In one specific deployment implementation, the method is deployed in a regional medical information platform, which deploys a WebSocket cluster consisting of multiple server nodes. Each node is configured, for example, with a 16-core CPU and 32GB of memory. The platform connects to several hospitals, and each hospital has several client devices that need to receive real-time messages, such as doctor workstations, nurse station terminals, patient monitoring devices, and other client devices.
[0091] When each server node starts up, it registers its node information with Redis using its local IP address as the key, sets the validity period to T1 seconds, and renews the node every T2 seconds. T1 is greater than T2 and less than or equal to 60 seconds.
[0092] When a hospital's client device establishes a connection with the WebSocket cluster, the load balancer distributes the request to a specific node. This node generates a UUID as a session identifier and performs the following batch operations using Redis pipeline mode:
[0093] (1) SET session:{sessionId} = {serverIp}, sets the validity period to H hours;
[0094] (2)SADD hospital:{hospitalId} {sessionId}.
[0095] The two commands mentioned above are sent to Redis through a single pipeline request, reducing the network round-trip latency from approximately 2ms (two independent requests) to approximately 0.5ms.
[0096] When a hospital information system needs to send a critical value report to a hospital, the system sends the message to the message queue topic, and the message body contains the target hospital ID and the report content.
[0097] After the consumer receives the message, it performs a two-phase routing lookup:
[0098] Phase 1: Using the hospital ID as the key, execute SMEMBERS hospital:{hospitalId} to retrieve a list of all session identifiers for that hospital, for example, returning 150 session identifiers;
[0099] Phase 2: Execute `GET session:{sessionId}` on each of the 150 session identifiers to obtain the server node IP corresponding to each session. After deduplication, three target server node IPs are obtained.
[0100] Subsequently, the consumer initiates HTTP calls to the three target IPs, requesting each node to push the critical value report to the corresponding client. Upon receiving the HTTP request, each node locates the WebSocket session object in its local memory based on the session identifier and completes the push.
[0101] A scheduled task executes every T3 seconds, scanning all session identifiers and checking their last active time. For sessions that have not had any heartbeats or message interactions for more than T4 seconds, the Redis pipeline mode is used to batch execute the DEL session:{sessionId} and SREM hospital:{hospitalId} {sessionId} operations to complete the cleanup.
[0102] Tests showed that, with an 8-node cluster and 5000 concurrent connections, the above method achieved an average message push latency of less than 100ms, saving approximately 87.5% of network and computing resources compared to the broadcast push solution, and only 3 out of the 8 nodes participated in the push.
[0103] In another specific deployment implementation, a WebSocket cluster is deployed on a financial trading system platform to provide users with real-time market data and transaction confirmation push notifications;
[0104] Unlike deploying a regional medical information platform, this embodiment uses user ID as the business identifier;
[0105] When a user logs in and establishes a WebSocket connection via web or mobile, the server node generates a session identifier and stores it using Redis pipelining mode.
[0106] (1) SET session:{sessionId} = {serverIp};
[0107] (2)SADD user:{userId} {sessionId}.
[0108] A user may log in on multiple devices simultaneously (such as PC and mobile), so one user ID can correspond to multiple session identifiers.
[0109] When the transaction system generates a transaction confirmation message, the message is sent to the message queue. The consumer queries Redis based on the user ID to obtain all session identifiers for that user, and then retrieves the corresponding server node IP based on each session identifier to complete precise push notifications.
[0110] Example 1
[0111] like Figure 1 and Figure 2 As shown, in this embodiment, registering the service IP address and node status information of this node with Redis includes:
[0112] Store node information in Redis using the server node IP as the key and the node status information as the value, and set an expiration time.
[0113] The node status information includes at least one of the following: node IP, startup time, and current number of connections.
[0114] Specifically, each server node registers its information with Redis upon startup. It stores this node information in Redis using the server node IP address as the key and node status information (including node IP address, startup time, current connection count, etc.) as the value, and sets an expiration date.
[0115] When a server node shuts down normally, its information is proactively deleted from Redis; when a server node crashes abnormally, a subsequent scheduled task checks if the node information has timed out and marks it as invalid. This maintains a list of all active server nodes in the cluster, providing a foundation for subsequent session management and message routing.
[0116] like Figure 3 As shown, in this embodiment, the two-stage routing query includes:
[0117] The first stage of routing involves querying the first mapping relationship stored in Redis based on the business identifier to obtain all session identifiers corresponding to that business identifier.
[0118] The second-stage routing involves querying the second mapping relationship stored in Redis based on the session identifiers obtained in the first-stage routing to obtain the server node IP corresponding to each session identifier.
[0119] The server node IPs obtained in the second stage of routing are deduplicated to obtain a list of target server node IPs that need to be pushed.
[0120] In practice, after receiving a business message, the consumer parses out the target business identifier; then, using the set of target business identifiers as the key, it queries the first mapping relationship to obtain the session identifier corresponding to the set of business identifiers.
[0121] If the session identifier is empty, that is, no corresponding session identifier is found, it is determined that there is no corresponding online session for the target business identifier, and the current push process ends;
[0122] If the session identifier is not empty, the second mapping relationship is queried using the session identifier as the key to obtain the server node IP corresponding to each session identifier.
[0123] During the query process, if a corresponding server node IP is not found for a certain session identifier, it means that the mapping data of that session is incomplete. At this time, the session identifier is determined to be an invalid session, and the session identifier is removed from the mapping relationship using Redis pipeline mode.
[0124] The system deduplicates all the server node IPs found to obtain a list of target server node IPs, and retains the correspondence between each server node IP and its corresponding session identifier. Deduplication can prevent the same server node from being called repeatedly, thereby reducing the number of invalid HTTP calls.
[0125] Subsequently, in step S6, the business message is pushed to the corresponding client according to each IP in the target server IP list.
[0126] Two-stage routing lookup helps to accurately push messages to the server node where the target client is located, avoiding the waste of resources in broadcast push where all nodes process the same message. In addition, the two-level mapping design can support multiple client sessions corresponding to the same business identifier and multiple sessions of business identifiers on the same server node, making the routing granularity flexible and controllable.
[0127] like Figure 1 and Figure 2 As shown, in this embodiment, step S6, pushing the service message to the corresponding client according to each IP in the target server IP list, includes:
[0128] Step S61: Call the corresponding server node through the HTTP interface. The HTTP interface is an internal interface exposed by each WebSocket server node for receiving message push requests. The request parameters of the HTTP interface include at least the target session identifier and the content of the message to be pushed.
[0129] Step S62: After receiving the HTTP call request, the target server node searches for the corresponding WebSocket session object in its local memory based on the target session identifier.
[0130] Step S63: If a session object exists in local memory and the session is active, push the message to the client through the WebSocket sending method of the session object;
[0131] In step S64, if the session object does not exist in local memory or the session has expired, a failure response is returned, and session cleanup is triggered by the caller or a scheduled task.
[0132] Optionally, the method also includes a reliability guarantee mechanism for message push:
[0133] When a call to the target server node via the HTTP interface fails, a retry is performed. Once the maximum number of retries is reached, the failure message is recorded and stored in the dead letter queue.
[0134] Record the complete chain of messages from message queue consumption to final push to the client.
[0135] Specifically, each server node uses its HTTP interface exposed to the cluster as the entry point for receiving business messages; failed sessions can be detected immediately and complement periodic cleanup. When the target server node does not have a corresponding WebSocket session object in its local memory, or when the session has expired, it returns a failure response instead of silently discarding the message. This allows for timely detection of undelivered messages and the implementation of retry or compensation measures, making message delivery status verifiable. At the same time, failed sessions trigger failure responses, thereby timely detection and cleanup of failed sessions. This helps shorten the lifespan of failed mappings in Redis, suppresses the accumulation of invalid mapping data, and thus maintains the hit rate and query efficiency of routing queries during long-term system operation.
[0136] Example 2
[0137] Based on Example 1, such as Figure 1 and Figure 2 As shown, in this embodiment, step S7, periodically checking and cleaning up failed sessions using a scheduled task, includes:
[0138] Step S71: Set the execution cycle of the scheduled task, periodically scan the session mapping relationship stored in Redis. The execution cycle of the scheduled task can be set according to the system scale.
[0139] Step S72: For the server node IP mapping corresponding to each session identifier, check whether the WebSocket connection corresponding to the session is still active and identify invalid sessions;
[0140] Step S73: If the corresponding WebSocket connection has been actively closed by the client, the corresponding server node has crashed or is unreachable, the session has exceeded the preset maximum idle time without any message interaction, or the connection has been confirmed to be interrupted through heartbeat detection, then it is determined to be an invalid session.
[0141] The heartbeat detection process includes the client periodically sending heartbeat messages to the server; the server updating the last active time of the session in Redis after receiving the heartbeat message; and a scheduled task scanning the last active time of the session and marking sessions that exceed a preset timeout threshold as invalid.
[0142] The client periodically sends heartbeat messages to the server; after receiving the heartbeat message, the server updates the last active time of the session in Redis; a scheduled task scans the last active time of the sessions and marks sessions that exceed a preset timeout threshold as invalid, such as setting the threshold to 3 times the heartbeat interval;
[0143] Step S74: For expired sessions, perform expired session cleanup operations. Specifically, Redis pipeline mode can be used to perform cleanup operations in batches, which can package multiple delete commands into a single network request, thus improving the efficiency of cleaning up large batches of sessions.
[0144] Step S75: For sessions that are determined to be invalid, log them and trigger an alarm before cleaning them up.
[0145] The above-mentioned timed cleanup mechanism can effectively prevent the accumulation of a large amount of invalid mapping data in Redis, maintain the freshness of session mapping data, and improve routing query efficiency.
[0146] like Figure 1 and Figure 2 As shown, in this embodiment, the mapping relationship also includes a third mapping relationship from the server node IP set to the session identifier and a fourth mapping relationship from the session identifier to the service identifier;
[0147] The third mapping relationship uses the server node IP as the key and the set of all session identifiers carried by the server node as the value. The fourth mapping relationship is the mapping relationship between session identifiers and business identifiers, using the session identifier as the key and the business identifier associated with the session identifier as the value. The third and fourth mapping relationships are written to Redis in batches using Redis pipeline mode.
[0148] Expired session cleanup operations include:
[0149] Based on the third mapping relationship, query the server node IP information using the session identifier, delete the corresponding session identifier from the set of server node IPs in the third mapping relationship, and delete the corresponding second mapping relationship;
[0150] Based on the fourth mapping relationship, query the service identifier using the session identifier, delete the corresponding session identifier in the service identifier set in the first mapping relationship, and delete the fourth mapping relationship.
[0151] The reason for the failed session is that the corresponding server node has been detected as down or unreachable. Based on the third mapping relationship, using the determined server node IP, all session identifiers under that server node IP are queried, and the second mapping relationship corresponding to the session identifier is deleted. Then, based on the fourth mapping relationship, the corresponding service identifier is queried, and the corresponding fourth mapping relationship is deleted. Based on the queried service identifier, the corresponding session identifier in the service identifier set in the first mapping relationship is deleted.
[0152] In practice, when the client establishes a new connection and establishes the first and second mapping relationships, it also adds the counterpart mapping of the first mapping relationship, that is, establishes the fourth mapping relationship, and adds the counterpart mapping of the second mapping relationship, that is, establishes the third mapping relationship.
[0153] In the third mapping relationship, one set of server node IPs corresponds to multiple session identifiers; in the fourth mapping relationship, one session identifier corresponds to one service identifier.
[0154] The scheduled task is triggered according to a preset execution cycle. The first step is to query the third mapping relationship with the server node IP as the entry point to obtain the set of session identifiers corresponding to the server node IP. The second step is to traverse each session identifier in the set and check whether the corresponding WebSocket connection is active.
[0155] When the list to be cleaned is not empty, Redis pipeline mode is used to package multiple cleanup commands corresponding to the associated data of each session identifier in the four mapping relationships in the list into a single network request for batch execution.
[0156] After the cleanup is complete, a cleanup log is recorded, and an alarm is triggered to notify the operations and maintenance personnel according to the preset alarm policy.
[0157] The cleanup process ends when the list of items to be cleaned is empty.
[0158] By establishing a bidirectional mapping, fast bidirectional queries can be performed. The first and second mapping relationships serve as a forward mapping, finding the session identifier and server based on the business identifier; the third and fourth mapping relationships serve as a reverse mapping, finding all relevant session identifiers and business identifiers based on the server, without needing to scan the entire table.
[0159] like Figure 5 As shown, the distributed WebSocket cluster message push system includes a WebSocket server cluster module, a Redis session storage module, a message consumption module, a two-phase routing query module, an HTTP call module, and a session cleanup module.
[0160] The WebSocket server cluster module contains multiple server nodes, each node is used to manage local sessions, establish WebSocket connections with clients, and expose HTTP push interfaces;
[0161] The Redis session storage module stores the mapping relationship between business identifiers and session identifier sets, the mapping relationship between session identifiers and server node IPs, server node registration information, and supports batch operations in pipeline mode.
[0162] The consumer consumes business messages from the cluster message queue and parses the target business identifier.
[0163] The two-stage routing query module is used to query Redis to obtain a list of session identifiers based on the business identifier, and then query the list of target server node IPs based on the list of session identifiers.
[0164] The HTTP call module is used to call the WebSocket send method of the corresponding server node through the HTTP interface based on the list of target server node IPs;
[0165] The session cleanup module is used to periodically check and clean up invalid session mappings in Redis through scheduled tasks. The cleanup operation is performed in batches using Redis pipeline mode.
[0166] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0167] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A distributed WebSocket cluster message push method, characterized in that, include: Build a WebSocket server cluster. When each server node starts up, it registers its service IP address and node status information with Redis. After the client establishes a WebSocket connection with any WebSocket server node, the node generates a unique session identifier and stores the mapping relationship in batches to Redis. The mapping relationship includes a first mapping relationship from the business identifier set to the session identifier and a second mapping relationship from the session identifier to the server node IP. The consumer consumes business messages from the cluster message queue and parses the target business identifier. Based on the target service identifier, a two-stage routing query is performed through the first mapping relationship and the second mapping relationship to obtain the target server IP list; Based on each IP in the target server IP list, push the business message to the corresponding client; Use scheduled tasks to periodically check and clean up failed sessions.
2. The distributed WebSocket cluster message push method according to claim 1, characterized in that, The process of registering the service IP address and node status information of this node with Redis includes: Store node information in Redis using the server node IP as the key and the node status information as the value, and set an expiration time. The node status information includes at least one of the following: node IP, startup time, and current number of connections.
3. The distributed WebSocket cluster message push method according to claim 1, characterized in that, The session identifier is a globally unique identifier, generated by a combination of server node IP, timestamp, and random number, or generated using a UUID; The service identifier is identification information that can uniquely identify the service recipient, including at least one of the following: user ID, device ID, hospital ID, and department ID.
4. The distributed WebSocket cluster message push method according to claim 3, characterized in that, The first mapping relationship is stored in Redis in a set structure, with the key being the business identifier and the value being the set of all session identifiers corresponding to that business identifier; The second mapping relationship is stored in Redis in the form of a hash structure or key-value pairs, where the key is the session identifier and the value is the server node IP and the session establishment time. The Redis pipeline mode is used to perform batch write operations that store the first and second mapping relationships in Redis, and to package multiple Redis commands into a single network request for sending.
5. The distributed WebSocket cluster message push method according to claim 1, characterized in that, The two-stage routing query includes: The first stage of routing involves querying the first mapping relationship stored in Redis based on the business identifier to obtain all session identifiers corresponding to that business identifier. The second-stage routing involves querying the second mapping relationship stored in Redis based on the session identifiers obtained in the first-stage routing to obtain the server node IP corresponding to each session identifier. The server node IPs obtained in the second stage of routing are deduplicated to obtain a list of target server node IPs that need to be pushed.
6. The distributed WebSocket cluster message push method according to claim 1, characterized in that, The step of pushing business messages to the corresponding clients based on each IP in the target server IP list includes: The corresponding server node is called through an HTTP interface. The HTTP interface is an internal interface exposed by each WebSocket server node for receiving message push requests. The request parameters of the HTTP interface include at least the target session identifier and the content of the message to be pushed. After receiving the HTTP call request, the target server node looks up the corresponding WebSocket session object in its local memory based on the target session identifier; If the session object exists in local memory and the session is active, the message is pushed to the client through the WebSocket sending method of the session object; If the session object does not exist in local memory or the session has expired, a failure response is returned, and session cleanup is triggered by the caller or a scheduled task.
7. The distributed WebSocket cluster message push method according to claim 6, characterized in that, It also includes a reliability guarantee mechanism for push notifications: When a call to the target server node via the HTTP interface fails, a retry is performed. Once the maximum number of retries is reached, the failure message is recorded and stored in the dead letter queue. Record the complete chain of messages from message queue consumption to final push to the client.
8. The distributed WebSocket cluster message push method according to claim 1, characterized in that, Periodically checking and cleaning up failed sessions using scheduled tasks includes: Set the execution cycle of the scheduled task to periodically scan the session mapping relationships stored in Redis; For each session identifier, check the server node IP mapping to see if the WebSocket connection corresponding to the session is still active and identify invalid sessions. If the corresponding WebSocket connection has been actively closed by the client, the corresponding server node has crashed or is unreachable, the session has not had any message exchange for more than the preset maximum idle time, or the connection has been confirmed to be interrupted through heartbeat detection, then it is judged as an invalid session. The heartbeat detection process includes the client periodically sending heartbeat messages to the server; the server updating the last active time of the session in Redis after receiving the heartbeat message; and a scheduled task scanning the last active time of the session and marking sessions that exceed a preset timeout threshold as invalid. For expired sessions, perform expired session cleanup operations; For sessions deemed invalid, logs are recorded and alerts are triggered before cleanup.
9. The distributed WebSocket cluster message push method according to claim 8, characterized in that, The mapping relationship also includes a third mapping relationship from the server node IP set to the session identifier, and a fourth mapping relationship from the session identifier to the service identifier; The third mapping relationship uses the server node IP as the key and the set of all session identifiers carried by the server node as the value. The fourth mapping relationship is a mapping relationship between session identifiers and business identifiers, using the session identifier as the key and the business identifier associated with the session identifier as the value. The third and fourth mapping relationships are written to Redis in batches using Redis pipeline mode. Expired session cleanup operations include: Based on the third mapping relationship, query the server node IP information using the session identifier, delete the corresponding session identifier from the set of server node IPs in the third mapping relationship, and delete the corresponding second mapping relationship; Based on the fourth mapping relationship, query the service identifier using the session identifier, delete the corresponding session identifier in the service identifier set in the first mapping relationship, and delete the fourth mapping relationship. The reason for the failed session is that the corresponding server node has been detected as down or unreachable. Based on the third mapping relationship, using the determined server node IP, all session identifiers under the server node IP are queried, the second mapping relationship corresponding to the session identifier is deleted, and then based on the fourth mapping relationship, the corresponding service identifier is queried, the corresponding fourth mapping relationship is deleted, and based on the queried service identifier, the corresponding session identifier in the service identifier set in the first mapping relationship is deleted.
10. A distributed WebSocket cluster message push system, characterized in that, For implementing the distributed WebSocket cluster message push method as described in any one of claims 1-9, the system comprises: The WebSocket server cluster module contains multiple server nodes, each node is used to manage local sessions, establish WebSocket connections with clients, and expose HTTP push interfaces; The Redis session storage module is used to store the mapping relationship between business identifiers and session identifier sets, the mapping relationship between session identifiers and server node IPs, server node registration information, and supports batch operations in pipeline mode. The message consumption module is used to consume business messages from the message queue and parse the target business identifier. The two-stage routing query module is used to query Redis to obtain a list of session identifiers based on the business identifier, and then query the list of target server node IPs based on the list of session identifiers. The HTTP call module is used to call the WebSocket send method of the corresponding server node through the HTTP interface based on the list of target server node IPs; The session cleanup module is used to periodically check and clean up invalid session mappings in Redis via scheduled tasks.
