Data processing method, device and computer-readable storage medium

Abstract

The embodiments of the present application provide a data processing method, device and computer-readable storage medium. The data processing method includes: a routing node receives a service request, the service request includes a user identifier; the routing node determines a first service node from a service node cluster based on the user identifier; if the first service node is unavailable, the routing node caches the service request and determines a second service node from the service node cluster based on the user identifier; after the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends the service request to the second service node so that the second service node processes the service request. By adopting the embodiments of the present application, online scaling can be achieved, and the consistency of data during scaling can be guaranteed, thereby improving user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, and computer-readable storage medium. Background Art

[0002] Most internet services are provided by distributed service nodes. As the scale of internet services expands, more and more service nodes are required, and the cost of equipment investment is also increasing. In a distributed architecture, to reduce resource waste and lower service costs, service nodes with low load can be shut down. For example, in online games that use distributed service node deployment, low-load service nodes can be shut down, which is called scaling down.

[0003] In the prior art, there are two ways to scale down. The first way is to scale down after suspending the business. For example, after the entire online game area is shut down, the service nodes with lower loads are shut down. The second way is to scale down during the execution of the business. For example, when the online game service node is providing services, the service node with lower loads is shut down. However, using the first scaling down method will cause a period of resource waste. For example, when the entire online game area is shut down, the service nodes providing services are idle. Using the second scaling down method will generate disordered window messages, resulting in data errors during the scaling down process. For example, when the online game service node is providing services, the player terminal sends a team request before scaling down, and the service node may return different team information before and after scaling down. Therefore, how to ensure resource utilization when shutting down the service node and data consistency during the scaling down process is a problem that urgently needs to be solved. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, device, and computer-readable storage medium, which can perform online scaling and ensure data consistency during the scaling process, thereby improving user experience.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0006] The routing node receives a service request, wherein the service request includes a user identifier;

[0007] The routing node determines a first service node from a service node cluster according to the user identifier;

[0008] If the first service node is in an unavailable state, the routing node caches the service request and determines a second service node from the service node cluster according to the user identifier;

[0009] After the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends the service request to the second service node, so that the second service node processes the service request.

[0010] In a second aspect, an embodiment of the present application further provides a data processing method, the method comprising:

[0011] The central control node obtains a scaling-down instruction for the first service node;

[0012] The central control node sends a broadcast message to the routing node in response to the scaling-in instruction, where the broadcast message is used to indicate that the first service node is in an unavailable state, so that the routing node caches the service request after determining from the service node cluster that the service node corresponding to the service request is the first service node, and sends the service request to the second service node after the first service node migrates user data to the second service node.

[0013] In a third aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0014] A receiving module, configured to receive a service request, wherein the service request includes a user identifier;

[0015] a determination module, configured to determine a first service node from a service node cluster according to the user identifier;

[0016] a processing module, configured to cache the service request if the first service node is unavailable, and determine a second service node from the service node cluster according to the user identifier;

[0017] The sending module is configured to send the service request to the second service node after the first service node migrates the user data corresponding to the first service node to the second service node, so that the second service node processes the service request.

[0018] In a fourth aspect, an embodiment of the present application provides another data processing device, the device comprising:

[0019] An acquisition module, configured to acquire a scaling-down instruction for the first service node;

[0020] A sending module is used to send a broadcast message to the routing node in response to the scaling-down instruction, where the broadcast message is used to indicate that the first service node is in an unavailable state, so that the routing node caches the service request after determining from the service node cluster that the service node corresponding to the service request is the first service node, and sends the service request to the second service node after the first service node migrates the user data to the second service node.

[0021] In a fifth aspect, an embodiment of the present application provides a routing node, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the data processing method described in the first aspect above.

[0022] In the sixth aspect, an embodiment of the present application provides a central control node, including a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the data processing method described in the second aspect above.

[0023] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first or second aspect above is implemented.

[0024] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method provided in the first or second aspect.

[0025] In an embodiment of the present application, when a routing node receives a service request containing a user identifier sent by a terminal corresponding to a user, the routing node determines a first service node from a service node cluster based on the user identifier. If the first service node is unavailable, the routing node caches the service request and determines a second service node from the service node cluster based on the user identifier. Then, after the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends a service request to the second service node so that the second service node processes the service request. If the first service node is available, the routing node sends the service request to the first service node so that the first service node processes the service request. Thus, compared to the prior art, online scaling can be achieved, reducing resource waste. Moreover, compared to shutting down a service node when the load is too low during business execution, the embodiment of the present application caches service requests corresponding to scaled-down service nodes with too low load. This ensures data consistency during online scaling, achieves lossless scaling, and protects user experience while saving resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1a This is a schematic diagram of the architecture of a data processing system provided in an embodiment of the present application;

[0028] Figure 1b This is a schematic diagram of the architecture of another data processing system provided in an embodiment of the present application;

[0029] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0030] Figure 3 This is a schematic diagram of a hash ring provided in an embodiment of the present application;

[0031] Figure 4 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0032] Figure 5 This is a flow chart of another data processing method provided in an embodiment of the present application;

[0033] Figure 6 This is a schematic diagram of the steps of a data processing method provided in an embodiment of the present application;

[0034] Figure 7 This is a schematic diagram of the steps of another data processing method provided in an embodiment of the present application;

[0035] Figure 8 is a structural diagram of a data processing device provided in an embodiment of the present application;

[0036] Figure 9 is a structural diagram of another data processing device provided in an embodiment of the present application;

[0037] Figure 10 This is a schematic diagram of the structure of a routing node provided in an embodiment of the present application;

[0038] Figure 11 This is a structural diagram of a central control node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0041] Cloud technology is a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identifier, requiring transmission to backend systems for logical processing. Different levels of data will be processed separately, and data from various industries will require a strong system backend, which can only be achieved through cloud computing.

[0042] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to required resources over the internet. Broadly speaking, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to required services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0043] Cloud computing has rapidly grown, driven by the internet, real-time data streams, the diversification of connected devices, and the growing demand for search services, social networks, mobile commerce, and open collaboration. Unlike previous parallel and distributed computing approaches, the emergence of cloud computing will fundamentally revolutionize the entire internet and enterprise management model.

[0044] See Figure 1a , Figure 1a This is a schematic diagram of the architecture of a data processing system provided by an embodiment of the present application. The technical solution of the embodiment of the present application can be Figure 1a The system architecture shown in the example or a similar system architecture is specifically implemented. The system architecture may include a central control node 100 and multiple routing nodes (such as Figure 1aAs shown, it specifically includes routing node 210, routing node 220, routing node 230) and at least one service node in the service node cluster mounted under each routing node (for example, routing node 210 is mounted with service node 310a and service node 310b; routing node 220 is mounted with service node 320a, service node 320b, and service node 320c; routing node 230 is mounted with service node 330a and service node 330b). Among them, when a routing node (for example, routing node 210) receives a service request, it can determine the first service node (for example, service node 330a) to process the service request from the service node cluster based on the user identifier in the service request (for example, the user's role name). Then, the routing node obtains the status of the first service node. If the first service node is in an unavailable state, usually when a service node needs to be scaled down, the service node can be set to an unavailable state, then the routing node caches the service request and determines the second service node (for example, service node 330b) from the service node cluster based on the user identifier. Then, after the user data corresponding to the first service node is migrated to the second service node, the routing node sends a service request to the second service node so that the second service node can process the service request. If the first service node is available, the routing node sends the service request to the first service node so that the first service node can process the service request. Utilizing the method of the embodiment of the present application, when the service node is scaled down, service requests can be cached to ensure the accuracy and consistency of data during the scaling process.

[0045] It should be noted that Figure 1a The system architecture shown includes one central control node, three routing nodes, and seven service nodes. The number of routing nodes and service nodes is for illustration only; any number of routing nodes and service nodes can be deployed based on actual business needs. The central control node can communicate with multiple routing nodes, and the central control node can also communicate with service nodes through routing nodes. Routing nodes can also communicate with each other. Routing nodes can also communicate with their attached service nodes, and they can also communicate with service nodes under other routing nodes through routing nodes. Service nodes can communicate with other service nodes through routing nodes.

[0046] For each node in the system architecture (such as the central control node, routing node, service node, etc.), there is a node identifier corresponding to it, and each node can store the node identifier of each node. Each node can maintain a node identifier list (which can be recorded as a routing table) as shown in the following table, and store the node name and node identifier in the node identifier list. Among them, the node identifier can be an Internet Protocol (IP) address or any other information that can be used to identify the node. The embodiment of the present application does not specifically limit this. As shown in Table 1, the routing table with the IP address as the node identifier is used as an example for explanation.

[0047]

[0048] Table 1 Routing table

[0049] In the embodiments of the present application, routing nodes are included. Each routing node has at least one service node mounted on it. Service nodes can communicate with other nodes through their mounted routing nodes. As shown in Table 1, if a user on service node 310a initiates a request to service node 330a, service node 310a sends the service request to routing node 210. Routing node 210 then sends the service request to routing node 230, which forwards it to service node 330a.

[0050] The central control node 100 may be a management device in the system architecture. The central control node 100 may receive user control commands and, in response to the control commands, manage routing nodes and / or service nodes in the system architecture. For example, the central control node 100 may receive a user instruction to scale down a service node. In response to the instruction, the central control node may send a broadcast message indicating that the service node is unavailable.

[0051] The central control node may be a terminal device having the above functions (such as a smart phone, tablet, wearable device, etc.) or a server with an interactive interface (such as a computer or server with a display screen, etc.).

[0052] The routing node can be a node device used for data forwarding in the system architecture. The routing node is a bridge between the central control node and the service node mounted on the routing node. The central control node and the service node can establish a communication connection through the routing node. For example, Figure 1a In the example, the central control node broadcasts the status information of the service node 330a to the service node 320a. The central control node first sends the status information of the service node 330a to the routing node 220. Then, the routing node 220 sends the status information of the service node 330a to the service node 320a.

[0053] The routing node may be a terminal device (such as a smart phone, tablet, wearable device, etc.) having the above functions or a server (such as a computer or server, etc.) having the above functions.

[0054] A service node can be a node that provides business services to users. In online games, service nodes can provide data computing and storage services to gamers. This data may include movement data of objects in the online game (for example, the distance a game character has moved), scene data in the online game (for example, the map data where the game character is located), and user data in the online game (for example, information about the guild the game character belongs to). This is not limited here.

[0055] The service node may be a terminal device (such as a smart phone, tablet, wearable device, etc.) having the above functions or a server (such as a computer or server, etc.) having the above functions.

[0056] In online games deployed in a distributed architecture, the data processing method of this solution can be used to ensure the consistency and accuracy of data during the scaling process, thereby achieving lossless online scaling.

[0057] refer to Figure 1b The system architecture may include a central control node 100 and multiple routing nodes (such as Figure 1b As shown, it specifically includes routing nodes 211, routing nodes 221, and routing nodes 231, and service clusters including at least one service node mounted under each routing node. In online games, service clusters may include functional clusters and battle clusters. Routing node 211 mounts a functional cluster, which may include matching service nodes, guild service nodes, team service nodes, and scenario service nodes. Routing node 221 mounts battle cluster 1, which may include game service nodes 1 and 2. Routing node 231 mounts battle cluster 2, which may include game service nodes 3 and 4.

[0058] When a routing node (such as routing node 211) receives a service request from a scene service node, it can determine the first service node (such as game service node 1) that processes the service request from the service node cluster based on the user identifier (such as the user's character name) in the service request. Then, the routing node obtains the status of the first service node. If the first service node is in an unavailable state, usually when a service node needs to be scaled down, the service node can be set to an unavailable state, then the routing node caches the service request and determines the second service node (such as game service node 2) from the service node cluster based on the user identifier. Then, after the user data corresponding to the first service node is migrated to the second service node, the routing node sends a service request to the second service node so that the second service node processes the service request. Using the method of the embodiment of the present application, when the service node of the online game is scaled down, the service request can be cached to ensure the accuracy and consistency of the data during the scaling process.

[0059] The following is a detailed description of the method of the embodiment of the present application.

[0060] See Figure 2 , Figure 2 This is a flow chart of a data processing method provided by an embodiment of the present application. Figure 1a-Figure 1b In the system architecture, the method includes:

[0061] Step S201: A routing node receives a service request, where the service request includes a user identifier.

[0062] In some feasible implementations, the routing node may receive a service request and uniquely determine a target user according to a user identifier in the service request.

[0063] The user identifier can uniquely identify the target user. In some feasible implementations, the user identifier can be a user account (e.g., a user's mobile phone number, a user's email address, a user's ID number, etc.), or the user identifier can be a user's role name, which is not limited here.

[0064] Step S202: The routing node determines a first service node from a service node cluster according to the user identifier.

[0065] The service node cluster includes at least one service node. Each routing node is mounted with at least one service node in the service node cluster.

[0066] In one feasible embodiment, the routing node uses a consistent hashing algorithm to map the target user corresponding to the user identifier to a hash ring, and maps each service node included in the service node cluster to the hash ring. The routing node then determines the first service node from the service node cluster based on the target user and the mapping position of each service node on the hash ring.

[0067] In a feasible implementation, the arc of the hash ring may be evenly distributed with 0 to 2 32 -1 hash value. The routing node then calculates the hash value of the target user's user ID and maps the target user onto the hash ring based on the hash value of the target user's user ID. The routing node calculates the hash value of the service node's node ID and maps the service node onto the hash ring based on the hash value of the service node's node ID. The routing node then searches clockwise along the hash ring from the target user's mapped position on the hash ring and selects the first service node found as the first service node.

[0068] The following is an explanation with reference to the accompanying drawings.

[0069] refer to Figure 3 , Figure 3 A schematic diagram of a hash ring is shown. Assume there are 4 service nodes, namely node 1, node 2, node 3, and node 4. Calculate the hash values ​​of the node identifiers of the 4 service nodes and map the 4 nodes on the hash ring, as shown below: Figure 3 As shown in the figure, the hash ring starts from integer 0 and goes clockwise to the largest positive integer. The hash value calculated based on the four node identifiers will definitely fall into a certain point on the hash ring. At this point, we have mapped the service node to the hash ring.

[0070] Assume there are 7 target users, namely user 1, user 2, user 3, user 4, user 5, user 6, and user 7. Calculate the hash values ​​of the user IDs of the 7 target users, and map the 7 users to the hash ring according to the hash values ​​of the user IDs of the target users, as shown below: Figure 3 As shown. So far, we have mapped the target user to the hash ring.

[0071] The consistent hashing algorithm is balanced, or load balancing, and the service requests of target users can be distributed to different service nodes, thereby improving the processing speed of the service nodes and ensuring the user experience. Figure 3 In the hash ring, user 1's request is processed by node 1. Requests from users 2 and 3 are processed by node 2. Requests from users 4, 5, and 6 are processed by node 3. Requests from user 7 are processed by node 4.

[0072] The consistent hashing algorithm is monotonic. When the number of service nodes changes, only the user data of some users needs to be changed. Figure 3 In the hash ring shown, when node 2 is unavailable, the requests of user 2 and user 3 will be processed by node 3, and other target users will not be affected.

[0073] Step S203: If the first service node is in an unavailable state, the routing node caches the service request and determines a second service node from the service node cluster according to the user identifier.

[0074] Service nodes can be divided into two states: available state and unavailable state.

[0075] When a service node is in an available state, it can provide data processing and storage services normally.

[0076] When a service node is in an unavailable state, it cannot provide data processing and storage services. Optionally, when the user determines that the first service node needs to be scaled down, the service node can be set to an unavailable state. Optionally, when the first service node fails, the service node can be placed in an unavailable state.

[0077] In some feasible implementations, the service node may receive a user operation on a button to change the state of the service node. The service node may also receive a control instruction from the central control node and switch the state in response to the control instruction.

[0078] In some feasible implementations, a routing node may include a cache mode and an operating mode. When the routing node is in cache mode, the routing node can cache service requests and forward service requests. When the routing node is in operating mode, the routing node can forward service requests.

[0079] In some feasible implementations, the routing node may receive a control instruction from the central control node, and then the routing node adjusts the mode in response to the control instruction from the central control node.

[0080] In some feasible implementations, if the first service node is unavailable, the routing node caches the service request. The routing node then removes the first service node from the hash ring, obtaining an updated hash ring. The routing node then searches clockwise along the hash ring starting from the target user's mapping location on the updated hash ring, selecting the first service node found as the second service node.

[0081] In other feasible implementations, if the first service node is unavailable, the routing node caches the service request. The routing node then calculates a hash value for the target user's user identifier and maps the target user onto the hash ring. The routing node then calculates hash values ​​for the node identifiers of other service nodes that do not include the first service node and maps these other service nodes that do not include the first service node onto the hash ring. Thus, an updated hash ring is generated. Next, the routing node searches clockwise along the updated hash ring starting from the target user's mapped position on the updated hash ring and selects the first service node found as the second service node.

[0082] Step S204: After the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends the service request to the second service node, so that the second service node processes the service request.

[0083] In some feasible implementations, the first service node receives a notification message sent by the central control node and, in response to the notification message, sends all corresponding user data in the first service node to the second service node. The routing node then sends a service request to the second service node based on a node identifier list (e.g., a routing table) so that the second service node processes the service request.

[0084] In other feasible implementations, the first service node receives a notification message from the central control node and sends all corresponding user data in the first service node to the cloud storage space. The second service node can download some or all of the corresponding user data in the first service node from the cloud storage space. Then, the routing node sends a service request to the second service node based on a node identifier list (e.g., a routing table) so that the second service node processes the service request.

[0085] In some feasible implementations, the routing node sends the service request to the routing node mounted on the second service node according to the node identification list (eg, routing table). The routing node mounted on the second service node then sends the service request to the second service node.

[0086] Optionally, in some feasible implementations, if the first service node is in an available state, the routing node sends a service request to the first service node according to a node identification list (eg, a routing table), so that the first service node processes the service request.

[0087] Optionally, in some feasible implementations, after the first service node migrates the user data corresponding to the first service node to the second service node, the central control node may send an instruction to the routing node to enable the routing node to enter the working mode from the cache mode and update the node identification table (for example, delete the IP address of the service node in the unavailable state from the routing table).

[0088] In an embodiment of the present application, when a routing node receives a service request containing a user identifier sent by a terminal corresponding to a user, the routing node determines a first service node from a service node cluster based on a consistent hashing algorithm. If the first service node is unavailable, the routing node caches the service request and determines a second service node from the service node cluster based on a consistent hashing algorithm. Then, after the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends a service request to the second service node so that the second service node processes the service request. If the first service node is available, the routing node sends the service request to the first service node so that the first service node processes the service request. The embodiment of the present application utilizes a consistent hashing algorithm to schedule user data, which can ensure that only part of the user data is affected during the scaling process. By utilizing routing nodes to cache the service requests of affected users, online scaling can be achieved, and during the scaling process, the data will not be disordered, ensuring the consistency and accuracy of the data and achieving lossless scaling. It protects user experience while saving resources.

[0089] See Figure 4 , Figure 4 This is a flow chart of another data processing method provided by the embodiment of the present application. Figure 1a-Figure 1b In the system architecture, the method includes:

[0090] In step S401, a routing node receives a broadcast message sent by a central control node when obtaining a scaling-down instruction for a first service node. The broadcast message is used to indicate that the first service node is in an unavailable state.

[0091] When the central control node obtains the scaling-down instruction for the first service node, the central control node may broadcast a broadcast message indicating that the first service node is unavailable to the entire network. After receiving the broadcast message, the routing node may determine that the first service node is unavailable.

[0092] Step S402: The routing node receives a user request including a user identifier. For details, please refer to step S201.

[0093] In step S403, the routing node determines a first service node from the service node cluster according to the user identifier. For details, please refer to step S202.

[0094] In step S404, if the first service node is unavailable, the routing node caches the service request and determines a second service node from the service node cluster according to the user identifier.

[0095] Step S405: The routing node receives the clear message sent by the central control node, and sends the clear message to at least one service node mounted thereon.

[0096] In some feasible implementations, the routing node may receive a clear message sent by the central control node (e.g., the routing node receives a control instruction carrying a clear message sent by the central control node). The routing node may then send a clear message (e.g., an empty data packet) to at least one of its attached service nodes.

[0097] For example, in Figure 1a In the system architecture shown, the routing node 210 receives a control instruction carrying a clear message sent by the central control node 100. The routing node 210 then sends an empty data packet to the service node 310a and the service node 310b.

[0098] Step S406: After receiving the reply message sent by the at least one service node in response to the clear message, the routing node sends a notification message to the central control node.

[0099] After receiving a reply message, such as an acknowledgment message (ACK), from at least one service node in response to the clear message, the routing node sends a notification message to the central control node. Then, after receiving the notification message from each routing node, the central control node sends a control instruction to the routing node attached to the first service node. After receiving the control instruction, the routing node attached to the first service node sends an instruction message to the first service node instructing the first service node to perform data migration.

[0100] For example, in Figure 1a In the illustrated system architecture, after receiving the reply messages sent by service nodes 310a and 310b in response to the clear message, routing node 210 sends a notification message to central control node 100. After receiving the notification messages sent by routing nodes 210, 220, and 230, central control node 100 instructs the first service node to migrate data.

[0101] In step S407, after the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends the service request to the second service node so that the second service node processes the service request. This process may refer to step S204 and will not be described in detail here.

[0102] The first service node migrates the user data corresponding to the first service node to the second service node. The routing node then sends the cached service request to the routing node attached to the second service node. The routing node attached to the second service node receives the cached service request and sends it to the second service node, so that the second service node processes the cached service request.

[0103] In some feasible implementations, the routing node mounted by the second service node may be the same as the routing node. The routing node mounted by the second service node may also be different from the routing node. This is not limited here.

[0104] In an embodiment of the present application, when a routing node receives a service request containing a user identifier from a terminal corresponding to a user, the routing node determines a first service node from a service node cluster based on a consistent hashing algorithm. If the first service node is unavailable, the routing node caches the service request and determines a second service node from the service node cluster based on the consistent hashing algorithm. The routing node then receives a clear message from a central control node and sends the clear message to at least one of its mounted service nodes. After receiving a reply message from the at least one service node in response to the clear message, the routing node sends a notification message to the central control node, so that the central control node, after receiving notification messages from all routing nodes, instructs the first service node to perform data migration. After the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends a service request to the second service node, so that the second service node processes the service request. If the first service node is available, the routing node sends the service request to the first service node, so that the first service node processes the service request. In this embodiment of the present application, by using a consistent hashing algorithm to schedule user data, only some user data is affected during the scaling process. Routing nodes are used to cache service requests from affected users and clear affected data across networks before user data migration. This allows for online scaling, ensuring that user data is not corrupted during the scaling process, ensuring data accuracy and consistency, and achieving lossless scaling. This conserves resources while maintaining a positive user experience.

[0105] See Figure 5 , Figure 5 This is a flow chart of another data processing method provided by the embodiment of the present application. Figure 1a-Figure 1b In the system architecture, the method includes:

[0106] Step S501: The central control node obtains a scaling-down instruction for a first service node.

[0107] In some feasible implementations, the user inputs a scaling-down instruction for the first service node through user operation, and the central control node obtains the scaling-down instruction for the first service node.

[0108] Step S502: The central control node sends a broadcast message to each routing node in response to the scaling-down instruction, where the broadcast message is used to indicate that the first service node is in an unavailable state.

[0109] The central control node sends a broadcast message indicating that the first service node is unavailable to each routing node in response to the scaling-down instruction. The routing node can then determine from the service node cluster that the service node corresponding to the service request is the first service node and cache the service request.

[0110] Step S503: The central control node sends a clear message to each routing node.

[0111] The central control node sends a clear message to each routing node, so that each routing node sends the clear message to at least one mounted service node. After each routing node receives a reply message sent by the at least one service node in response to the clear message, the central control node sends a notification message to the central control node.

[0112] Step S504: After receiving the notification message sent by each routing node, the central control node instructs the first service node to perform data migration through the routing node mounted by the first service node.

[0113] After receiving the notification message sent by each routing node, the central control can send a control instruction to the routing node mounted on the first service node, and then the routing node mounted on the first service node responds to the control instruction and sends an indication message to the first service node to instruct the first service node to perform data migration.

[0114] Through the embodiment of the present application, the central control node sends a broadcast message to the routing node in response to the scaling-down instruction, so that the routing node caches the service request after determining from the service node cluster that the service node corresponding to the service request is the first service node in the unavailable state. Then, the central control node sends a clear message to each routing node, so that each routing node sends the clear message to at least one mounted service node, and after receiving the reply message sent by the at least one service node for the clear message, sends a notification message to the central control node. After receiving the notification message sent by each routing node, the central control node instructs the first service node to perform data migration through the routing node mounted by the first service node. In this way, online scaling can be achieved, and during the scaling-down process, the affected user data will not be disordered, and lossless scaling can be achieved. It saves resources while ensuring user experience.

[0115] See Figure 6 , Figure 6 This is a schematic diagram of the steps of a data processing method provided in an embodiment of the present application. Figure 6 As shown, the structure for implementing the method may include a central control node, routing nodes 1 and 2, and service nodes 1, 2, and 3. Service nodes 1 and 2 are mounted on routing node 1, and service node 3 is mounted on routing node 2. The method may simply include the following steps:

[0116] Step S601: The central control node broadcasts to routing node 1 and routing node 2 that the service node 2 is in an unavailable state.

[0117] In step S602, routing node 1 and routing node 2 confirm receipt of the broadcast message indicating that service node 2 is in an unavailable state.

[0118] In step S603, routing node 1 receives the service request and determines, based on the user identifier in the service request, that the service node that processes the service request is service node 2, and routing node 1 caches the service request.

[0119] Step S604: The central control node sends a clear message to routing node 1 and routing node 2.

[0120] In step S605 , routing node 1 sends a clear message to service node 1 and service node 2 , and routing node 2 sends a clear message to service node 3 .

[0121] In step S606 , service node 1 and service node 2 send reply messages of the clear message to routing node 1 , and service node 3 sends a reply message of the clear message to routing node 2 .

[0122] In step S607, routing node 1 and routing node 2 send a reply message of the clear message to the central control node.

[0123] In step S608 , the central control node sends an instruction for instructing data migration to routing node 1 .

[0124] In step S609, routing node 1 sends an instruction for instructing data migration to service node 2.

[0125] In step S610 , service node 2 migrates user data in service node 2 to service node 1 .

[0126] In step S611 , the service node 2 sends a notification message to the routing node 1 to confirm that the user data migration is completed.

[0127] In step S612, routing node 1 sends the cached service request to service node 1, so that service node 1 processes the service request.

[0128] See Figure 7 , Figure 7 This is a schematic diagram of the steps of another data processing method provided in the embodiment of the present application. Figure 7 As shown, the structure for implementing the method may include a central control node, routing nodes 1 and 2, and service nodes 1, 2, and 3. Service nodes 1 and 2 are mounted on routing node 1, and service node 3 is mounted on routing node 2. The method may simply include the following steps:

[0129] Step S701: The central control node broadcasts to routing node 1 and routing node 2 that the service node 2 is in an unavailable state.

[0130] In step S702, routing node 1 and routing node 2 confirm receipt of the broadcast message indicating that service node 2 is in an unavailable state.

[0131] In step S703, routing node 1 receives the service request and determines, based on the user identifier in the service request, that the service node that processes the service request is service node 2. Routing node 1 then caches the service request.

[0132] Step S704: The central control node sends a clear message to routing node 1 and routing node 2.

[0133] In step S705 , routing node 1 sends a clear message to service node 1 and service node 2 , and routing node 2 sends a clear message to service node 3 .

[0134] In step S706 , service node 1 and service node 2 send a reply message of the clear message to routing node 1 , and service node 3 sends a reply message of the clear message to routing node 2 .

[0135] In step S707, routing node 1 and routing node 2 send a reply message of the clear message to the central control node.

[0136] In step S708 , the central control node sends an instruction for instructing data migration to the routing node 2 .

[0137] In step S709 , the routing node 2 sends an instruction for instructing data migration to the service node 3 .

[0138] In step S710 , service node 2 migrates user data in service node 2 to service node 3 .

[0139] In step S711 , the service node 2 sends a notification message to the routing node 1 to confirm that the user data migration is completed.

[0140] Step S712: Routing node 1 sends the cached service request to routing node 2.

[0141] In step S713 , routing node 2 sends the cached service request to service node 3 , so that service node 3 processes the service request.

[0142] An embodiment of the present application provides a data processing method. When a service node is unavailable, the routing node caches the service request. The central control node then sends a clear message to clear the affected service requests across the network. The central control node then instructs the unavailable service node to migrate data. After the data migration is complete, the cached service request is processed by the new server. This method enables online scaling while ensuring data consistency during the scaling process, achieving lossless scaling and conserving resources while ensuring a positive user experience.

[0143] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The device described in the embodiment of the present application includes:

[0144] Receiving module 801, configured to receive a service request, wherein the service request includes a user identifier;

[0145] A determination module 802 is configured to determine a first service node from a service node cluster according to the user identifier;

[0146] The processing module 803 is configured to cache the service request if the first service node is unavailable, and determine a second service node from the service node cluster according to the user identifier;

[0147] The sending module 804 is configured to send the service request to the second service node after the first service node migrates the user data corresponding to the first service node to the second service node, so that the second service node processes the service request.

[0148] Optionally, the device further includes:

[0149] The receiving module 801 is further configured to receive a clear message sent by the central control node;

[0150] The sending module 804 is further configured to send the clear message to at least one mounted service node;

[0151] The sending module 804 is further configured to send a notification message to the central control node after receiving a reply message sent by the at least one service node in response to the clear message.

[0152] Optionally, the determining module 802 includes:

[0153] A mapping module 805 is configured to map the target user corresponding to the user identifier to a hash ring using a consistent hashing algorithm, and to map each service node included in the service node cluster to the hash ring;

[0154] The determination module 802 is further configured to determine a first service node from the service node cluster according to the mapping position of the target user and each service node on the hash ring.

[0155] Optionally, the processing module 803 includes:

[0156] A removal module 806 is configured to remove the first service node from the hash ring to obtain an updated hash ring;

[0157] The determination module 802 is further configured to determine a second service node from the service node cluster according to a mapping position between the target user and each service node on the updated hash ring.

[0158] Optionally, the sending module 804 includes:

[0159] An acquisition module 807 is configured to acquire a routing table, wherein the routing table includes a node identifier of each routing node and a node identifier of at least one service node mounted to each routing node;

[0160] The sending module 804 is further configured to send the service request to the second service node according to the routing table.

[0161] Optionally, the device further includes:

[0162] The receiving module 801 is further configured to receive a broadcast message sent by the central control node when obtaining a scaling-down instruction for the first service node, where the broadcast message is used to indicate that the first service node is in an unavailable state.

[0163] It can be understood that the functions of each functional module of the data processing device of this embodiment can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment and will not be repeated here.

[0164] In an embodiment of the present application, the receiving module 801 receives a service request, and the service request includes a user identifier. Then, the determining module 802 determines the first service node from the service node cluster based on the user identifier. Next, the processing module 803 caches the service request when the first service node is unavailable, and determines the second service node from the service node cluster based on the user identifier. Finally, after the first service node migrates the user data corresponding to the first service node to the second service node, the sending module 804 sends the service request to the second service node, so that the second service node processes the service request. In this way, online scaling can be achieved, and user data will not be confused during the scaling process, ensuring the accuracy and consistency of the data and achieving lossless scaling. It protects user experience while saving resources.

[0165] See also Figure 9 , Figure 9 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application. The device described in an embodiment of the present application includes:

[0166] An acquisition module 901 is configured to acquire a scaling-down instruction for a first service node;

[0167] Sending module 902 is used to send a broadcast message to the routing node in response to the scaling-down instruction, where the broadcast message is used to indicate that the first service node is in an unavailable state, so that the routing node caches the service request after determining from the service node cluster that the service node corresponding to the service request is the first service node, and sends the service request to the second service node after the first service node migrates the user data to the second service node.

[0168] Optionally, the device further includes:

[0169] The sending module 902 is further configured to send a clear message to each routing node, so that each routing node sends the clear message to at least one mounted service node, and after receiving a reply message sent by the at least one service node to the clear message, sends a notification message to the central control node;

[0170] The instructing module 903 is configured to instruct the first service node to perform data migration through the routing node mounted by the first service node after receiving the notification message sent by each routing node.

[0171] It can be understood that the functions of each functional module of the data processing device of this embodiment can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment and will not be repeated here.

[0172] In an embodiment of the present application, the acquisition module 901 acquires a scaling-down instruction for the first service node, and then the sending module 902 sends a broadcast message to the routing node in response to the scaling-down instruction. The broadcast message is used to indicate that the first service node is unavailable, so that the routing node caches the service request after determining from the service node cluster that the service node corresponding to the service request is the first service node, and sends the service request to the second service node after the first service node migrates the user data to the second service node. In this way, online scaling can be achieved, and during the scaling-down process, the affected user data will not be disordered, and lossless scaling can be achieved. This saves resources while ensuring user experience.

[0173] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a routing node provided in an embodiment of the present application. Figure 10 The routing node in this embodiment may include a processor 1001 and a memory 1002. The processor 1001 and the memory 1002 are connected via a bus 1003. The memory 1002 is used to store a computer program including program instructions, and the processor 1001 is used to execute the program instructions stored in the memory 1002.

[0174] In an embodiment of the present application, the processor 1001 performs the following operations by running the executable program code in the memory 1002: receiving a service request, the service request including a user identifier, and then determining a first service node from a service node cluster based on the user identifier. If the first service node is unavailable, caching the service request and determining a second service node from the service node cluster based on the user identifier. Finally, after the first service node migrates the user data corresponding to the first service node to the second service node, the processor 1001 sends the service request to the second service node so that the second service node processes the service request.

[0175] Optionally, the processor 1001 is further configured to:

[0176] Receive the clear message sent by the central control node;

[0177] Sending the clear message to at least one mounted service node;

[0178] After receiving a reply message sent by the at least one service node in response to the clear message, a notification message is sent to the central control node.

[0179] Optionally, the processor 1001 is further configured to:

[0180] The routing node maps the target user corresponding to the user identifier to a hash ring using a consistent hashing algorithm, and maps each service node included in the service node cluster to the hash ring;

[0181] The routing node determines a first service node from the service node cluster according to the target user and a mapping position of each service node on the hash ring.

[0182] Optionally, the processor 1001 is further configured to:

[0183] The routing node removes the first service node from the hash ring to obtain an updated hash ring;

[0184] The routing node determines a second service node from the service node cluster according to a mapping position of the target user and each service node on the updated hash ring.

[0185] Optionally, the processor 1001 is further configured to:

[0186] The routing node obtains a routing table, where the routing table includes a node identifier of each routing node and a node identifier of at least one service node mounted corresponding to each routing node;

[0187] The routing node sends the service request to the second service node according to the routing table.

[0188] Optionally, the processor 1001 is further configured to:

[0189] The routing node receives a broadcast message sent by the central control node when obtaining a scaling-down instruction for the first service node, where the broadcast message is used to indicate that the first service node is in an unavailable state.

[0190] It should be understood that in the embodiments of the present application, the processor 1001 may be a central processing unit (CPU), and the processor 1001 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0191] The memory 1002 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1001. A portion of the memory 1002 may also include a non-volatile random access memory, which may store a routing table, etc.

[0192] In a specific implementation, the processor 1001 and the memory 1002 described in the embodiment of the present application can execute the embodiment of the present application. Figure 2 or Figure 4 The implementation method described in the process of a data processing method provided can also be implemented by the embodiment of the present application. Figure 8 The implementation method described in the data processing device will not be repeated here.

[0193] In an embodiment of the present application, a processor receives a service request, and the service request includes a user identifier. Then, a first service node is determined from a service node cluster based on the user identifier. If the first service node is unavailable, the service request is cached, and a second service node is determined from the service node cluster based on the user identifier. Finally, after the first service node migrates the user data corresponding to the first service node to the second service node, the service request is sent to the second service node so that the second service node processes the service request. In this way, online scaling can be achieved, and user data will not be confused during the scaling process, ensuring the accuracy and consistency of the data and achieving lossless scaling. It protects user experience while saving resources.

[0194] See Figure 11 , Figure 11 This is a schematic diagram of the structure of a central control node provided in an embodiment of the present application. Figure 11 The central control node in the embodiment shown may include: a processor 1101 and a memory 1102. The processor 1101 and the memory 1102 are connected via a bus 1103. The memory 1102 is used to store computer programs, which include program instructions. The processor 1101 is used to execute the program instructions stored in the memory 1102.

[0195] In the embodiment of the present application, the processor 1101 executes the following operations by running the executable program code in the memory 1102: obtaining a scaling-down instruction for the first service node. Then, in response to the scaling-down instruction, sending a broadcast message to the routing node, wherein the broadcast message is used to indicate that the first service node is unavailable, so that the routing node, after determining from the service node cluster that the service node corresponding to the service request is the first service node, caches the service request, and after the first service node migrates user data to the second service node, sends the service request to the second service node.

[0196] Optionally, the processor 1101 is further configured to:

[0197] Sending a clear message to each routing node, so that each routing node sends the clear message to at least one mounted service node, and after receiving a reply message sent by the at least one service node to the clear message, sends a notification message to the central control node;

[0198] After receiving the notification message sent by each routing node, the routing node mounted by the first service node instructs the first service node to perform data migration.

[0199] It should be understood that in the embodiments of the present application, the processor 1101 may be a central processing unit (CPU), and the processor 1001 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0200] The memory 1102 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1101. A portion of the memory 1102 may also include a non-volatile random access memory, which may store a routing table, etc.

[0201] In a specific implementation, the processor 1101 and the memory 1102 described in the embodiment of the present application can execute the embodiment of the present application. Figure 5 The implementation method described in the process of a data processing method provided can also be implemented by the embodiment of the present application. Figure 9 The implementation method described in another data processing device will not be repeated here.

[0202] In an embodiment of the present application, the processor obtains a scaling-down instruction for the first service node, and then sends a broadcast message to the routing node in response to the scaling-down instruction. The broadcast message is used to indicate that the first service node is unavailable, so that after the routing node determines from the service node cluster that the service node corresponding to the service request is the first service node, it caches the service request, and after the first service node migrates the user data to the second service node, it sends the service request to the second service node. In this way, online scaling can be achieved, and during the scaling-down process, the affected user data will not be disordered, and lossless scaling can be achieved. This saves resources while ensuring user experience.

[0203] The present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the above-mentioned data processing method embodiment can be executed. Figure 2 or Figure 4-Figure 5 The steps performed in .

[0204] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the computer device performs the above-mentioned data processing method embodiment. Figure 2 or Figure 4-Figure 5 The steps performed in .

[0205] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0206] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of this application are still within the scope covered by the application.

Claims

1. A data processing method, characterized in that: The method comprises: The routing node receives a broadcast message sent by the central control node when obtaining a scaling-down instruction for the first service node, where the broadcast message is used to indicate that the first service node is in an unavailable state; The routing node receives a service request, where the service request includes a user identifier; The routing node determines a first service node from a service node cluster according to the user identifier; If it is determined according to the broadcast message that the first service node is in an unavailable state, the routing node caches the service request and determines a second service node from the service node cluster according to the user identifier; The routing node receives the data migration instruction sent by the central control node, and sends the data migration instruction to the first service node, where the data migration instruction is used to instruct the first service node to migrate corresponding user data to the second service node; After the first service node migrates the user data corresponding to the first service node to the second service node, the routing node sends the service request to the second service node, so that the second service node processes the service request.

2. The method according to claim 1, wherein Each routing node is mounted with at least one service node in the service node cluster. If the first service node is in an unavailable state, the method further includes: The routing node receives a clear message sent by the central control node; The routing node sends the clear message to at least one mounted service node; After receiving the reply message sent by the at least one service node to the clear message, the routing node sends a notification message to the central control node, so that after receiving the notification message sent by each routing node, the central control node instructs the first service node to perform data migration through the routing node mounted by the first service node.

3. The method according to claim 1 or 2, wherein: The routing node determines a first service node from a service node cluster according to the user identifier, including: The routing node maps the target user corresponding to the user identifier to a hash ring using a consistent hashing algorithm, and maps each service node included in the service node cluster to the hash ring; The routing node determines a first service node from the service node cluster according to the target user and a mapping position of each service node on the hash ring.

4. The method according to claim 3, wherein The routing node determines a second service node from the service node cluster according to the user identifier, including: The routing node removes the first service node from the hash ring to obtain an updated hash ring; The routing node determines a second service node from the service node cluster according to a mapping position of the target user and each service node on the updated hash ring.

5. The method according to claim 1, wherein The routing node sending the service request to the second service node includes: The routing node obtains a routing table, where the routing table includes a node identifier of each routing node and a node identifier of at least one service node mounted corresponding to each routing node; The routing node sends the service request to the second service node according to the routing table.

6. A data processing method, characterized in that: The method comprises: The central control node obtains a scaling-down instruction for the first service node; The central control node sends a broadcast message to the routing node in response to the scaling-down instruction, where the broadcast message indicates that the first service node is in an unavailable state, so that the routing node, after determining from the service node cluster that the service node corresponding to the service request is the first service node, caches the service request and determines a second service node from the service node cluster according to the user identifier; The central control node sends a data migration instruction to the routing node so that the routing node sends the data migration instruction to the first service node. The data migration instruction is used to instruct the first service node to migrate the corresponding user data to the second service node, and after the first service node migrates the corresponding user data to the second service node, send the service request to the second service node.

7. The method according to claim 6, wherein Each routing node is mounted with at least one service node in the service node cluster. After the central control node sends a broadcast message to the routing node in response to the scaling-down instruction, the method further includes: The central control node sends a clear message to each routing node, so that each routing node sends the clear message to at least one mounted service node, and after receiving a reply message sent by the at least one service node in response to the clear message, sends a notification message to the central control node; After receiving the notification message sent by each routing node, the central control node instructs the first service node to perform data migration through the routing node mounted by the first service node.

8. A data processing device, characterized in that: The device includes a module for implementing the data processing method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the data processing method according to any one of claims 1 to 5, or the data processing method according to any one of claims 6 to 7.

Images